World: r4wp

I think this is just a misunderstanding. Kaj, would you be happy 
with a simple language change, for now, that descibes the current 
state of external unicode support?

Yes, I think it's very dangerous to claim that Red has Unicode and 
Latin-1 support

We could say they are supported internally, but I/O is TBD. But I'm 
all for a simple text change that solves this for now.

Well, the console Latin1 claim is wrong, other than that, there are 
no claims that Red supports Latin1.

Gregg: that's already what is claimed at the moment, but it seems 
to not be a very communicable message.

Kaj: I do take into account all observations, and especially from 
people that work on it and with it since the beginning. I just can't 
do miracles and implement in a night features that take weeks.

Where is the exact text in question?

I'm not asking that of you, and I'm working on it myself

So it's probably better to not mention Unicode at all, for now (which 
is what I understand Kaj suggesting).

I'm good with that. I'm American so I don't care about any of this. 
;-)

Not really. Mentioning internal features as such is fine. It's not 
even true that I/O is to be done, because I support it

Unicode support was claimed on Twitter

Finally had the chance to look over this:

For: print read 

http://syllable.org", do you feed string/load with an UTF-8 input 
even on Windows?"

Yes, syllable.org is UTF-8

Looks like the newer Beagleboards are using regular ARM CPU (no more 
Cortex-M series), so Red should run just fine on them:

http://techcrunch.com/2013/04/23/the-beaglebone-black-is-a-new-single-board-computer-that-can-brew-beer/

Thanks Doc, I might go ahead and create that group and just do repost 
on RED status updates via Linked in as new news is posted.  Does 
RED have an "official Logo" yet?

Nope, as we haven't found someone skilled enough yet to convert the 
Tower of HanoÏ draft into a real logo.

You can use the one I'm using for #red_lang on Twitter.

I'd like to know what your development environment for Red/System 
is especially that used by Doc.
In the meantime...
	My development environment:
	OS: Windows 7
	Editor: Notepad++ v5.0.3
	  not the latest version but I prefer this one,

   using "Python" as the language for syntax highlithing and coloring

   (there is not a "REBOL" language) and used mainly for the folding 
   on indentation

   (I can not live without folding, and folding on indentation makes 
   life even easier)

   I have added a menu item with a keybord shortcut to run REBOL with 
   the currently shown file as argument.

   I have added also a menu item with a keybord shortcut to run my modified 
   version of REBOL-Word-Browser

 At the end of the r/s file I add these lines (mostly taken from Bruno 
 Anselme):
		#if OS = '???? [{
		REBOL []
		appname: "myprog"

  rs-dir: %../Red-master-0.3.2_Bruno/red-system                ; locate 
  here your red-system directory
		dest: rejoin [what-dir appname ]
		print [ "------ Compiling" appname "------" ]
		do/args rs-dir/rsc.r rejoin ["-o " dest " " dest %.reds ]

		print [ "Destination file:" dest ]
		call/wait/show/console dest
		halt
		;}]

 This way pressing the shortcut I can run REBOL that compiles the 
 program I am writing and than starts the program.
	Any suggestion is welcomed!

Nice way to get self-compiled Red/System scripts. :-)

I use Textpad on Win7 for coding. As Textpad does not support Unicode, 
I also use Notepad++ for Red UTF-8 scripts. On Windows, I use PE 
Explorer for reviewing the disassembled code and IDA Pro for debugging 
it (hopefully, this happens rarely). I also use IDA Pro on Linux/IA-32 
and gdb everywhere else. On Mac, I try to use osxdebug, when it's 
not crashing.


I dream about the day when I'll be able to replace all those tools 
with a cross-platform Red[/System] IDE.

I should set up a keyboard shortcut for UltraEdit, but I've been 
focused more on Red Mezz lately, so just have a build script for 
the console, and then I paste things into the console to play. When 
I start doing more real Red compiled work, I will revisit Arnold's 
redcompiler script.

I had the idea of adding a button to it to compile the console and 
1 to launch the console and as such making it more of an IDE/toolbox. 

Pretty busy and going away for a short holiday, no computer.

I did a similar tool a long time ago, for encapping REBOL scripts. 
Maybe we can merge them. 

http://www.rebol.org/view-script.r?script=enlist.r&sid=ghz24nd

ok with me. I had some trouble with the compiler script last time 
I used it. Location of red or rsc compiler is changed by compiler, 
the active directory changes.

I have spend many hours today reviewing again Kaj's claims and looking 
into the code to see what is implemented. There are many public claims 
made by Kaj about Red that I need to correct, so these are my answers:

1) "I found out that not only does Red not support Unicode, it doesn't 
support Latin-1, not even on Windows" Red *does* "support" Unicode, 
Latin-1 "support" was not claimed in Red, except for the console 
script. I've put quotes around support word, because you're mixing 
up internal representation and I/O encoding formats.

2) "The printing backend doesn't fully support Unicode, either." 
It does, as proven by the test suite.

3) "So for most purposes, Red is currently ASCII only" Obviously 
not, it is for your specific purposes. Again, look at the Unicode 
tests.

4) "It was announced last year that Unicode support was implemented 
in a week. What I found first is that Unicode support is useless, 
and now I've found that only ASCII is really supported." Unicode 
support has been implemented exactly as stated in (will get back 
to that point later):
http://www.red-lang.org/2012/09/plan-for-unicode-support.html

5) "Yes, I think it's very dangerous to claim that Red has Unicode 
and Latin-1 support". Red *has* Unicode support, string! and word! 
value support Unicode, input Red scripts are Unicode, PRINT outputs 
Unicode characters. Latin-1 is used as an *internal* encoding format, 
I don't remember ever claiming that "Red supports Latin-1 for I/O" 
except for the console script (which is wrong, I agree). OTOH, I 
do remember thinking about supporting it at the beginning for printing, 
then I found it cumbersome to support in addition to Unicode mode 
and dropped it during the implementation.

So, about the console issue, the runtime lexer is able to parse Latin-1 
input but the input string gets internalized before being passed 
to the lexer using the UTF-8 loader, which chokes on MSDOS console 
incompatible codepages. For the Unix version, the console input being 
in UTF-8 by default, it passes the internalization, but crashes the 
runtime lexer.

So, currently, only 7-bit ASCII is safe to input in the console. 
This limitation has nothing to do with Red implementation or the 
interpreter, it's a console input issue, so generalizing it to whole 
Red is inaccurate and unfair.

Anyway, as I stated many times, the current runtime lexer is a temporary 
implementation, waiting to be replaced by a proper Unicode lexer. 
It shouldn't have survived more than a few weeks, hopefully, it should 
be soon gone. We'll switch then the console to the Windows Unicode 
API to get rid of the codepages hell. In the meantime, I will correct 
the console banner.

Wrt the Unicode plan (my blog entry link above), I would like to 
highlight only one sentence: "Conversion for input and output strings 
will be done on-the-fly between one of the internal representation 
and UTF-8/UTF-16." This is what have been implemented for Red input 
scripts (except from the console), and for outputting text on screen 
with the currently hardwired PRINT output support because the I/O 
sub-system has not been yet implemented in Red. The PRINT backend 
will be rewritten once we get ports/devices support. Also, the "on-the-fly" 
part (no intermediary buffer) should have hinted you that I could 
not implement encoders/decoders before I/O sub-system is done. This 
also means that the current encoding/decoding logic you've implemented 
these last days probably won't be useful for Red's I/O.

Kaj, it seems to me that you were confused by a few things:
- console script banner wrong statement (my fault)

- internal "Latin-1" naming (like in Python's internals) which might 
be misleading (there's no other closer naming in Unicode for one 
byte representation AFAIK, though some people call it "UCS-1", maybe 
we should adopt that too).

- "Unicode support" seems to imply to you that *all* possible Unicode 
encodings have to be supported (with encoders/decoders). It doesn't, 
having just one encoding supporting the full Unicode range (like 
UCS-4) is enough for claiming "Unicode support".

Furthermore:


Red can input some Unicode and print some Unicode. That's enough 
to support the test suite, but mostly useless in real life programs

 I must have missed the point in time when Red was declared beta. 
 :-) AFAICT, Red is not yet ready for real-life programs (Red/System 
 is though).

It's not even true that I/O is to be done, because I support it

 Reading how you put it, I just hope you still remember which version 
 is the official one? ;-)

Really, I don't feel like spending more time on this. Let's agree 
to disagree

I feel the joy in my Red work slipping away from me

I have showered Red with praise on many occasions. Why is it such 
a problem to have some criticism, too?

Well, I don't understand all the fuss. Of course, eventual false 
claims should be corrected, but it feels like selecting one feature, 
knowing we are not there yet, and trying it to push it forward, reshifting 
other priorities? If someone else would wait for dynamic libraries, 
and Doc now switched the focus, would we see another complaints from 
 someone else, that dynamic libraries are not yet fully supported?

If dyn-lib is around the corner, let's better ask Doc, what is imediatelly 
next focus of implementation. IIRC, it should be objects, needed-by 
and followed by I/O. And if Doc says, that more Unicode stuff comes 
with I/O implementation, what timeframe are we talking about? 2-3 
months? Well, not sure how much you are kept stuck with such a timeframe, 
but maybe even you can wait?

OTOH - it seems, that for now, you found/implemented interim solution? 
I believe that if Doc could add full support in 2-3 days, he could 
reshift the priorities, but if it really depends on I/O, then let's 
say - first things first ....

And as for the criticism - it is OK. It is just that we can feel 
you are not happy about the current state of Unicode support, and 
as I know Doc, he would like to please us all, so I can imagine it 
kind of creates some push on him :-)

There is as I read this a different issue. Dock want Red to be as 
complete as posible, Kaj wants it to officially useable. Kaj really 
needs UTF-8 (and or Latin-1) character support, for getting this, 
I guess this has to do with the Syllable operating system amongst 
others.

I would like Red to support time and random functions as natives 
and (Gregg is one of your mezz funcs REJOIN ? I want that too) be 
able to connect to a MySQL database so I can dump PHP for some webdevelopment.

Besdies that we all love to see a VID (like) solution for display 
and creating apps. 

We have to be patient agreed 100% amongst everybody? Where the roadmap 
mentions all things to progress Red, above things are not on that 
list. I want Red to have enough to make it useable in production 
and after that expand, imho that is the way to really attrackt more 
funding/enthousiast programmers and make sure current support does 
not fade/ loose interest.

I think, and hope, the conflict over this will fade soon, and everyone 
will be happy again. We're just all so anxious for the things we 
each need, and Doc has made such great progress that it seems like 
Red should be almost ready to use. I was going to post that REJOIN 
wasn't stable yet, but decided to test it on my latest console build 
and it seems to work great.

rejoin: func [

 "Returns a new string (or same series type as block/1) made from 
 reduced values."
	block [block!]
	/local op
][
	either empty? block: reduce block [block] [
		op: either series? first block [:copy] [:form]
		append op first block next block
	]
]

Why is it such a problem to have some criticism, too?

 There is no problem with that, and believe me, you can't be more 
 critical on Red that I am myself. But I find it really unfair to 
 paint a bad picture of whole Red because some features that are planned 
 are not yet implemented. 


I can't go faster than the music, to get I/O done with required encoders/decoders, 
I need to setup the ports/devices infrastructure. To do that, I need 
objects support done. Also, as shown by my entries on Red Trello 
page, error! (and typeset!) support and getting a Unicode runtime 
lexer are even more prioritary to make Red "more usable for real-world 
apps". Moreover, when you manage a lot of tasks, some of them marked 
as "important" that keep been postponed because of other more urgent 
ones, will at some point become "urgent" themselves. That is what 
is happening with shared libs support, which is a blocker for getting 
Red on Android and iOS. I'll also probably make a Java bridge prototype 
this week before getting back on other Red features.

Almost forgot about the Trello page. Do you use that page as an interim 
planning for what is actually being worked on? If so, and once you 
have few minutes of spare time (I can imagine you don't :-), maybe 
one sentence reference on the Roadmap page would make it a nice addition:

You can find actual tasks we work on on the following Trello page

 .... it would make feel ppl more dynamic about the project, as Roadmap 
 page is rather static. OTOH we can watch Git commits, twitter, but 
 still if one needs to know, what is an actual priority, and as far 
 as Trello page is being updated, it would make some sense. E.g. "Download" 
 addtion was imo very nic simplification for those starting with Red 
 ...