World: r4wp

Good to know, but we'll need to make a last testing pass once the 
merge of 0.3.0 done.

Sure, that's much easier now

Although I'll be away the coming week, so I probably won't be able 
to generate them then

If there's no new issue to fix, we should do the merge this weekend.

Oh, the particular quality of the R3 extension dispatch model that 
makes it well-suited to JIT compiler implementation is that a command 
function contains an indirect reference to the dispatch function, 
and an index integer. When the command is called, the runtime calls 
the dispatch function and passes the integer and a marshalled stack 
frame. For a JIT compiler dispatch function, the index of the command 
can be an index into an array of function pointers or something like 
that, and the dispatch function can just pass the stack frame to 
the appropriate function, then return the results. This means that 
the hard part of JIT compiling - getting the regular runtime to call 
the created functions - is something that you essentially get for 
free with the existing command mechanism.


You could also use the dispatch function to marshall arguments into 
another runtime with a different call model. You could, for instance, 
have a dispatch function that pushes the contents of a marshalled 
stack frame onto a Lua stack and calls Lua functions. Or you could 
do something similar for LLVM functions, or ActiveScripting languages, 
or V8, or ODBC queries, or even Red's JIT.


This all depends on having a good marshalling model in the first 
place that can handle the datatypes you need to support, and it would 
also help if there was a good task-safe callback mechanism that actually 
works (R3's needs a bit of work at the moment). Still, the principle 
is sound.

Interesting, that meshes with the idea that the extension interface 
should be able to function without an OS loader

I get a new error on GTK on Linux when it tries to load the Red-48x48.png 
logo:

(dressed-up-hello-GTK-world:4515): Gtk-CRITICAL **: gtk_container_add: 
assertion `GTK_IS_WIDGET (widget)' failed

But no time left to look into it further

Other than that, Linux x86 still works

It's helpful to make a conceptual distinction between the host interface 
and the extension interface, even though for R3 they are currently 
related to each other and share a lot of the same code.


For the host interface, the host is the OS (more or less) and provides 
an execution environment that the R3 runtime runs on like a program 
(this is all metaphorical, but I'm sure you get it). The OS in this 
case could be something like Windows, Linux, some microkernel, whatever, 
or it could be an application or application plugin like Eclipse, 
Visual Studio, Notepad++, Excel, Firefox, whatever.


For the extension interface, R3 is the OS, the extension-embedded 
module is the program that runs on the OS, and that program calls 
the extension's native code like a library. The program source is 
returned by the extension's RX_Init function, and that program then 
wraps the native library code. The module source is loaded like a 
normal script (slightly hacked after loading to make it a better 
wrapper), so the script could be embedded in binary data along with 
non-Rebol stuff just like with normal scripts. You could even have 
Red and Rebol scripts in the same file (if they use the same embedding 
method) so you the data the init function returns can be like a Red/Rebol 
fat binary, metaphorically.


Given this, Red could either be (or compile) a host for R3; or it 
could be (or compile) a runtime library that implements the same 
host interface as r3lib, making it a drop-in replacement for R3; 
or it could be (or compile) an extension that R3 is a client of, 
returning R3 code that calls calls the compiled Red code; or it could 
be an alternate extension container, for extensions that return both 
Red and R3 code from the same init function, which would call the 
Red code returned, which would in turn call the same native code. 
The two languages could be integrated at any point in the stack, 
along with other languages.

Much of this is already in the Red/R3 bridge

What bothers me about the module script returned by the RX_Init function 
is that it looks like an Interface Description Language, but it isn't, 
because arbitrary REBOL code can be intermixed. While the whole interface 
looks very strongly decoupled, this breaks it because it ties it 
strongly to REBOL if one is not very careful what not to put in the 
module script

It would be much better to have a real IDL for the command descriptions

Most extension module scripts are currently IDL-like, but that is 
only because they aren't (and don't need to be) very ambitious in 
their system integration because they just export a bunch of functions. 
Any native implementation of a port scheme, native dialect, or other 
system enhancement would need Rebol code to integrate that enhancement. 
Doing that in Rebol code is what allows the actual native interface 
to be that simple. It is also what would allow Red wrapper code (which 
could be returned from the same RX_Init function in the same string) 
to use the same native code unchanged, even though Red's runtime 
model is likely to require different integration code.

[rebol [] do something]
[red [] do something]

People who stick to IDL-style extension modules are really underestimating 
the power of the model :)

Yes, there's endless opportunity to make it incompatible with anything 
but R3 :-)

That has been a fabulously successful strategy for Microsoft

I was really careful to make extensions support mixed-language multi-modules. 
For that matter, with my recent patches it even supports same-language 
multi-modules.

Except that in a careless extension, you wouldn't be able to execute 
the module script in Red

Same goes for any module, even the non-native ones. Multi-modules 
are supported in regular scripts too.

Not if you split off the interface into a proper IDL

Ah, but the interface is too simple to need an IDL - make command! 
will do. The extra stuff is for system integration, which is only 
needed when you are doing port schemes, dialects, anything that you 
wouldn't expect to be cross-language compatible anyways, unless you 
explicitly implement a compatible system model. If you're just exporting 
functions then you can implement a simple IDL just by interpreting 
the (cooincidentally the same) module spec code with a very limited 
IDL dialect processor if no Red script wrapper is found.

The same could be said for changes in major versions of the same 
language if they weren't made with deliberate compatibility in mind.

I didn't have Red in mind when I implemented multi-modules, as Red 
didn't exist yet. I was actually thinking of other languages that 
didn't even need to have the same syntax as Rebol.

Carl was thinking of better encapping methods for R3 itself. It works 
for both :)

I think you don't want to understand me, but my time is up

Btw, you can't run applications for Linux on Unix without a compatibility 
library either. Bringing MS into it is just an insult.

Ah, see, you do understand me :-)

I don't expect Red and R3 to have the same system model, because 
if they did there would be no point to having Red at all. Being able 
to have extensions that can integrate into both would be an unusually 
amazing bonus :)

I still don't know if you really don't see it, but I could easily 
define them

For that matter, I expect to make my own Rebol spinoff language that 
will follow a completely different system model than either Red or 
Rebol, and the only reason to do so is because those other languages 
don't cover that situation (otherwise I would have been more active 
in Red so far). Being different justifies their existence; interoperating 
with each other justifies their cooperation :)

I don't expect Red and R3 to be compatible because Red is compiled 
and R3 is interpreted. Both models have their place and their strengths. 
That doesn't mean that they can't be compatible with each other where 
that makes sense, or interoperate with each other where that makes 
sense.

I don't expect Red and R3 to be compatible because Red is compiled 
and R3 is interpreted.
 I hope you like surprizes then. ;-)

I don't expect it, but that doesn't mean that we won't do our best 
to fake it :)

Are you planning to use the direct binding model, and value slots?

Out of curiosity...

Assuming that Red will be compiled (even JIT), the actual semantics 
will be different even if the outside behavior will appear to be 
similar enough that it won't matter for most people, hence the "fake 
it" phrase. It would be a disservice to us if we got a compiler, 
which has definite if minimal disadvantges over an interpreter, without 
getting the advantages of a compiler such as a practical optimizer. 
The behavior of R3 and Red could be quite similar to an outside observer 
that doesn't look closely, but they would require different optimization 
strategies to get the most efficient code. In that way I don't expect 
them to be compatible - they would likely be even less compatible 
than R2 and R3. But that's not really a problem :)

Are you planning to use the direct binding model, and value slots?
 That's already there in Red runtime.

Red runtime library

Sorry for the dumb questions, I've been too busy to participate yet 
:(

Semantics will be very close, probably 99% close, as I've moved from 
a statically typed system to an hybrid static/dynamic typing system.

So, no lexical binding? Or do you plan to infer lexical binding where 
possible, and use direct where not?

So, if all goes well, you'll have the full REBOL model + optimizations. 
There are many ways to optimize and on many different abstraction 
levels. Yet, some atomic operations will remain "unoptimized", like 
series atomic manipulations (maybe we'll figure out a way to optimize 
them too in the future).

Lexical binding will be added when we'll add contexts (in a few weeks), 
along with dynamic binding (yes, you'll be able to have fun with 
BIND again ;-)).

In a couple of days, I'll release the first Red alpha with a blog 
entry to describe it, you'll be able to get a better picture of what's 
in Red already and where it's heading.

So, my module system will end up being of more use to you. I'll be 
sure to write up that TRANSCODE/part option request for R3 in CureCode, 
for your reference.

I ran into the need for that with multi-scripts.

Well, maybe. :-) I must admit I haven't yet dived into R3 module 
system to see if it fits well our needs.