World: r4wp

Just a matter of discerning internal implementation details and user-visible 
external features.

Could we say that internal unicode support exists, but is not available 
for I/O?

Kaj: you're basically saying that all the Unicode tests we wrote 
in the tests suite are lying...which is obviously wrong.

Precisely, yes.

Yes, internal features are of no use to me. I want to write actual 
programs

Again, I'm tired of explaining this. Red can input some Unicode and 
print some Unicode. That's enough to support the test suite, but 
mostly useless in real life programs

More importantly, Red can internally _represent_ Unicode.

Writing actual programs: I would like too, but until I/O is not fully 
implemented in Red, we are limited into what can be done. That's 
why Red is in alpha state.

I'm not contesting that. I just want features to be represented truthfully, 
lest we get slaughtered on the public Internet

Agreed, if we have some misleading naming or feature not implemented 
as advertised, please report them with all the required details for 
me to process it, either here or on the bugtracker. But simply bashing 
Red won't help.

Please, I'm not bashing Red, I'm trying to represent it truthfully. 
I'm trying to protect it from getting bashed. Why would I bash a 
project I spent the last two years of my life on?

It's exactly that I sometimes get the feeling that you're not taking 
my observations seriously that I am afraid for this misrepresentation

I think this is just a misunderstanding. Kaj, would you be happy 
with a simple language change, for now, that descibes the current 
state of external unicode support?

Yes, I think it's very dangerous to claim that Red has Unicode and 
Latin-1 support

We could say they are supported internally, but I/O is TBD. But I'm 
all for a simple text change that solves this for now.

Well, the console Latin1 claim is wrong, other than that, there are 
no claims that Red supports Latin1.

Gregg: that's already what is claimed at the moment, but it seems 
to not be a very communicable message.

Kaj: I do take into account all observations, and especially from 
people that work on it and with it since the beginning. I just can't 
do miracles and implement in a night features that take weeks.

Where is the exact text in question?

I'm not asking that of you, and I'm working on it myself

So it's probably better to not mention Unicode at all, for now (which 
is what I understand Kaj suggesting).

I'm good with that. I'm American so I don't care about any of this. 
;-)

Not really. Mentioning internal features as such is fine. It's not 
even true that I/O is to be done, because I support it

Unicode support was claimed on Twitter

Finally had the chance to look over this:

For: print read 

http://syllable.org", do you feed string/load with an UTF-8 input 
even on Windows?"

Yes, syllable.org is UTF-8

Looks like the newer Beagleboards are using regular ARM CPU (no more 
Cortex-M series), so Red should run just fine on them:

http://techcrunch.com/2013/04/23/the-beaglebone-black-is-a-new-single-board-computer-that-can-brew-beer/

Thanks Doc, I might go ahead and create that group and just do repost 
on RED status updates via Linked in as new news is posted.  Does 
RED have an "official Logo" yet?

Nope, as we haven't found someone skilled enough yet to convert the 
Tower of HanoÏ draft into a real logo.

You can use the one I'm using for #red_lang on Twitter.

I'd like to know what your development environment for Red/System 
is especially that used by Doc.
In the meantime...
	My development environment:
	OS: Windows 7
	Editor: Notepad++ v5.0.3
	  not the latest version but I prefer this one,

   using "Python" as the language for syntax highlithing and coloring

   (there is not a "REBOL" language) and used mainly for the folding 
   on indentation

   (I can not live without folding, and folding on indentation makes 
   life even easier)

   I have added a menu item with a keybord shortcut to run REBOL with 
   the currently shown file as argument.

   I have added also a menu item with a keybord shortcut to run my modified 
   version of REBOL-Word-Browser

 At the end of the r/s file I add these lines (mostly taken from Bruno 
 Anselme):
		#if OS = '???? [{
		REBOL []
		appname: "myprog"

  rs-dir: %../Red-master-0.3.2_Bruno/red-system                ; locate 
  here your red-system directory
		dest: rejoin [what-dir appname ]
		print [ "------ Compiling" appname "------" ]
		do/args rs-dir/rsc.r rejoin ["-o " dest " " dest %.reds ]

		print [ "Destination file:" dest ]
		call/wait/show/console dest
		halt
		;}]

 This way pressing the shortcut I can run REBOL that compiles the 
 program I am writing and than starts the program.
	Any suggestion is welcomed!

Nice way to get self-compiled Red/System scripts. :-)

I use Textpad on Win7 for coding. As Textpad does not support Unicode, 
I also use Notepad++ for Red UTF-8 scripts. On Windows, I use PE 
Explorer for reviewing the disassembled code and IDA Pro for debugging 
it (hopefully, this happens rarely). I also use IDA Pro on Linux/IA-32 
and gdb everywhere else. On Mac, I try to use osxdebug, when it's 
not crashing.


I dream about the day when I'll be able to replace all those tools 
with a cross-platform Red[/System] IDE.

I should set up a keyboard shortcut for UltraEdit, but I've been 
focused more on Red Mezz lately, so just have a build script for 
the console, and then I paste things into the console to play. When 
I start doing more real Red compiled work, I will revisit Arnold's 
redcompiler script.

I had the idea of adding a button to it to compile the console and 
1 to launch the console and as such making it more of an IDE/toolbox. 

Pretty busy and going away for a short holiday, no computer.

I did a similar tool a long time ago, for encapping REBOL scripts. 
Maybe we can merge them. 

http://www.rebol.org/view-script.r?script=enlist.r&sid=ghz24nd

ok with me. I had some trouble with the compiler script last time 
I used it. Location of red or rsc compiler is changed by compiler, 
the active directory changes.

I have spend many hours today reviewing again Kaj's claims and looking 
into the code to see what is implemented. There are many public claims 
made by Kaj about Red that I need to correct, so these are my answers:

1) "I found out that not only does Red not support Unicode, it doesn't 
support Latin-1, not even on Windows" Red *does* "support" Unicode, 
Latin-1 "support" was not claimed in Red, except for the console 
script. I've put quotes around support word, because you're mixing 
up internal representation and I/O encoding formats.

2) "The printing backend doesn't fully support Unicode, either." 
It does, as proven by the test suite.

3) "So for most purposes, Red is currently ASCII only" Obviously 
not, it is for your specific purposes. Again, look at the Unicode 
tests.

4) "It was announced last year that Unicode support was implemented 
in a week. What I found first is that Unicode support is useless, 
and now I've found that only ASCII is really supported." Unicode 
support has been implemented exactly as stated in (will get back 
to that point later):
http://www.red-lang.org/2012/09/plan-for-unicode-support.html

5) "Yes, I think it's very dangerous to claim that Red has Unicode 
and Latin-1 support". Red *has* Unicode support, string! and word! 
value support Unicode, input Red scripts are Unicode, PRINT outputs 
Unicode characters. Latin-1 is used as an *internal* encoding format, 
I don't remember ever claiming that "Red supports Latin-1 for I/O" 
except for the console script (which is wrong, I agree). OTOH, I 
do remember thinking about supporting it at the beginning for printing, 
then I found it cumbersome to support in addition to Unicode mode 
and dropped it during the implementation.

So, about the console issue, the runtime lexer is able to parse Latin-1 
input but the input string gets internalized before being passed 
to the lexer using the UTF-8 loader, which chokes on MSDOS console 
incompatible codepages. For the Unix version, the console input being 
in UTF-8 by default, it passes the internalization, but crashes the 
runtime lexer.

So, currently, only 7-bit ASCII is safe to input in the console. 
This limitation has nothing to do with Red implementation or the 
interpreter, it's a console input issue, so generalizing it to whole 
Red is inaccurate and unfair.

Anyway, as I stated many times, the current runtime lexer is a temporary 
implementation, waiting to be replaced by a proper Unicode lexer. 
It shouldn't have survived more than a few weeks, hopefully, it should 
be soon gone. We'll switch then the console to the Windows Unicode 
API to get rid of the codepages hell. In the meantime, I will correct 
the console banner.

Wrt the Unicode plan (my blog entry link above), I would like to 
highlight only one sentence: "Conversion for input and output strings 
will be done on-the-fly between one of the internal representation 
and UTF-8/UTF-16." This is what have been implemented for Red input 
scripts (except from the console), and for outputting text on screen 
with the currently hardwired PRINT output support because the I/O 
sub-system has not been yet implemented in Red. The PRINT backend 
will be rewritten once we get ports/devices support. Also, the "on-the-fly" 
part (no intermediary buffer) should have hinted you that I could 
not implement encoders/decoders before I/O sub-system is done. This 
also means that the current encoding/decoding logic you've implemented 
these last days probably won't be useful for Red's I/O.

Kaj, it seems to me that you were confused by a few things:
- console script banner wrong statement (my fault)

- internal "Latin-1" naming (like in Python's internals) which might 
be misleading (there's no other closer naming in Unicode for one 
byte representation AFAIK, though some people call it "UCS-1", maybe 
we should adopt that too).

- "Unicode support" seems to imply to you that *all* possible Unicode 
encodings have to be supported (with encoders/decoders). It doesn't, 
having just one encoding supporting the full Unicode range (like 
UCS-4) is enough for claiming "Unicode support".

Furthermore:


Red can input some Unicode and print some Unicode. That's enough 
to support the test suite, but mostly useless in real life programs

 I must have missed the point in time when Red was declared beta. 
 :-) AFAICT, Red is not yet ready for real-life programs (Red/System 
 is though).

It's not even true that I/O is to be done, because I support it

 Reading how you put it, I just hope you still remember which version 
 is the official one? ;-)