World: r4wp

I think Red's USP is, "the performance of C with the elegance of 
REBOL" ... which is attractive to many folks like myself who woundln't 
otherwise venture near a "compiled" language (given the usual ease 
of use trade-off).

Nenad, could you describe a little the structure that's built up 
by the lexer? Are you intending to (at some point) allow for some 
sort of AST-like (if this is not what's generated already) structure 
to be passed back in along with some way of describing the start/end 
of a modified region in order to reduce the parsing that would need 
to be done if the lexer was being called relatively frequently when 
editing a large source string?

Jerry: when I find time to write it. :-) Probably when I get back 
to memory manager code to add the few missing parts, like GC and 
allocation of big memory chunks, that should happen in the next weeks.

AdrianS: the output of the lexer is nested blocks of Red values, 
same as REBOL with its own lexer (LOAD). The AST is not stored anywhere, 
AST nodes are created and consumed on the fly during the compilation. 
So the closest thing to an AST you can get currently is the output 
of the lexer.


For the needs of a code editor, maybe you could just invoke it on 
the currently edited line (though you would need to deal with unmatched 
opening/closing delimiters). I haven't yet though how I will achieve 
it in Red IDE.

This is all far out of my league. Though I had the same feeling when 
I started working with computers many years ago :-) Keep up the good 
work!

It's not as complex as it sounds. Feel free to ask any question about 
it in Rebol School or in ~ChitChat if it's purely Red-oriented. Open 
source projects are an excellent way to share knowledge and I would 
be glad if Red project can help people (including myself) learn something 
new.

Doc and the others:

I hav had some considerations about the RED = REBOL issue and I will 
continue the discussion in the REBOL3 area.

I noticed doc-strings are not available yet for #import?

It's late to talk about merging. It's been 2 years, Red is getting 
mature very soon aspecially after the Red/System part is almost done. 
Don't make any distraction to slow it down. ... Besides, R3 is not 
open yet (where is the code?), Red and Doc are all we got now. Doc's 
got the whole plan for Red, which is a good plan. We donate for that 
plan, not for a merge plan. The latest blog article in Red-lang.org 
made me realize that we might have a complete and mature Red in one 
year.

Jerry: that sounds like a realistic deadline to reach 1.0 release, 
as long as I can keep working full time on Red in 2013. Though, Red 
should be fully usable in a couple of months, all features would 
not be there, it won't run at full speed, but it will be enough to 
be able to build almost any app.

Kaj: the compiler should accept them from #import too...looking into 
it....

Nope, no support has been added for doc-strings in #import, but it 
obviously should be supported. I will add it tomorrow.

Thanks. Will have to wait in my bindings for the next release, though

Doc, how about Red becoming the Rebcode part of R3? IMO that would 
be a nice addition.

So, we get both worlds. If we manage to call R3 code from the Red 
section and vice versa, that would be great. We could use the compiled 
speed for inner loops and let the interpreter do all the non-speed 
relevant things.

Robert, I guess you mean Red/System and not Red? Well, the license 
are compatibles, so if someone wants to include Red/System into R3, 
he doesn't need to ask anyone for permission.

Or the other way around, no? With Kaj's system, you can write R3 
extensions in Red/System, no?

Yep, RedCode for R3 is already here

It could be integrated better, but that would require Red to run 
on R3

Kaj - but it is not like Cyphre creted JIT for R2, which looked like 
REBOL and was just faster. Red/System code surely can't run that 
way - it is for offline stuff, where you first have to compile the 
app, no runtime stuff ...

Cyphre's code also needed to be JITed first. I don't see a fundamental 
difference

Pekr: the difference between AOT and JIT compilation is much thiner 
than you think. Just load Red/System compiler code to your R2 app, 
pass it any source code at runtime, use the link?: no option and 
you get compiled code and related data in form of binary! values...and 
voilà! :-) The rest is same as for Cyphre's JIT, you need a way to 
call native code in memory, something that is hardly possible in 
R2, but maybe Cyphre found a hole to achieve it anyway.

something that is hardly possible in R2
 - not a problem

Ladislav: I was thinking about an internal only solution, I know 
it's possible to do it by using a tiny C code in a shared lib.   
        If you've figured out a way to do it without any external 
dependency, I would be glad to learn how you did it.

I know it's possible to do it by using a tiny C code in a shared 
lib.
 - does that look like a problem?

It breaks the "fit all in one binary" REBOL philosophy, requires 
to compile it for each platform and maintain OS-specific code...Not 
a problem per se, but IMHO a sub-optimal solution, that's why I was 
interested in a possibly purely "internal" solution, in case I would 
have missed it.

The link?: no option doesn't seem to be available yet. Is it in the 
unreleased AVR backend?

It's available but not documented as it is only used by the compiler 
internally for now. You can add it to any of the target definition 
block in %config.r for testing (or create a %custom-targets.r file 
instead). It will put the compiler in an "incremental" mode (it can 
compile incrementally as many source file as you want). Once compilation 
has finished, no file will be generated and compiler state will not 
be reset. You can then inspect the result of the compilation from 
console using:

>> probe system-dialect/compiler/job
>> probe emitter/symbols
>> probe emitter/code-buf
>> probe emitter/data-buf
>> probe system-dialect/compiler/imports


You can basically get most of these data in logs when compiling using 
-v 9 option.

Cool, thanks

R3 extensions break that philosophy already, so go for it :)

Does not linking mean that no binary file is output, or does it also 
mean that addresses in the code are not resolved?

Only the latter.

You can link to something that only exists in RAM.

That can't be right

People overestimate what linking means. It really is as simple as 
a pointer, the same thing that it means in a linked list.

Kaj: that's right. If you want addresses to be resolved, we need 
to add a in-memory linker. It will basically just resolve all the 
addresses. But that would not work as is with the imports that expect 
to be statically linked. So, a custom dynamic loader would be required 
(same requirements as for implementing LOAD/LIBRARY and MAKE ROUTINE!).

Not a big deal to implement though, just time-consuming.

I know what linking is; I wrote a relocating loader in Atari 8-bit 
assembler

I don't mean external symbols that are for the operating system to 
resolve, I mean the internal linking that the Red/System linker, 
presumably %red-system/linker.r, does inside a Red/System compile, 
possibly consisting of multiple sources

To put it concretely, the AVR backend needs to generate a memory 
image, without external symbols because there is no operating system, 
but with all internal addresses resolved. Can that memory image be 
constructed by appending data-buf and code-buf?

Nope, you need to do some processing first. All global addresses 
in code are zeroed by default. The linker fills the blank by calculating 
the right code or data addresses.

It's basically what does linker/resolve-symbol-refs

OK. Any idea when you'll publish the AVR backend? It would be useful 
for reference

You mean the linker backend?

Yes, but anything that's fit for inclusion :-)

It's already published since several months: red-system/formats/Intel-HEX.r

Oh, great :-)

As you will notice, it uses a minimal C-based kernel and some imports 
are linked to it using syscalls.

An Arduino? I'll look into it

Yes, that's a kernel for a Arduino Uno board.