World: r4wp

Part of the motivation of the R3 extension/host API was to isolate 
the extensions and hosts from changes in the underlying data model, 
which makes hosts and extensions really resilient to upgrades of 
R3. It also was designed to let you have a consistent internal execution 
model even in cases where the host has a completely different process/thread 
model.

I think it went a little too far at times, especially the lack of 
marshallers for immediate values that are more than 64 bits internally. 
I've frequently wanted to supplement it with marshallers for the 
other datatypes in R3, particularly the date, time and money types.

The big win is the command dispatch model though, because it basically 
lets you get dispatch to JIT-compiled functions for free. The dispatch 
function can manage changes between execution models completely without 
R3 even noticing. Lua has similar separation, though since it doesn't 
have to support anywhere near as many datatypes it can get away with 
a stack-based interface.

Another interesting side effect of the strong separation is that 
it would be possible to implement the client-side of the extension 
interface in other languages, such as Red, so that it could use R3 
extensions without changes. The REBOL portion of the extension might 
be trickier to implement though, because that code tends to be more 
tied into the actual R3 runtime model. You could write your own wrapper 
code if your system model is different, but the native code could 
be used as-is.

However, the way that the R3 script loader works, you could make 
an extension that has both Red and Rebol scripts in the module data, 
especially if Red uses a similar script-in-a-block embedding method,

That would make it so you could make one extension that can be used 
by both languages, that shares the same native code.

Hi Kaj on my macbook:

Last login: Fri Oct 19 07:21:16 on 
ttys001

 MacBook-van-Arnold-160:~ Arnold$ 
 /Users/Arnold/Downloads/Red\(/System\)\ 
 Testing-dc1b702068063b65/Darwin/Red/hello 
 ; exit;
Hello, world!
§±ÖÁµ, ºÌüµ!
`O}Y, NLu
Dobrý den sv
te
logout

[Proces voltooid]

 /Users/Arnold/Downloads/Red\(/System\)\ Testing-dc1b702068063b65/Darwin/Red/Fibonacci 
 ; exit;

MacBook-van-Arnold-160:~ Arnold$ /Users/Arnold/Downloads/Red\(/System\)\ 
Testing-dc1b702068063b65/Darwin/Red/Fibonacci ; exit;
Fibonacci 35: 9227465
logout

[Proces voltooid]


MacBook-van-Arnold-160:~ Arnold$ /Users/Arnold/Downloads/Red\(/System\)\ 
Testing-dc1b702068063b65/Darwin/Red/empty ; exit;
logout

[Proces voltooid]
I took the programs from the Red tree under Darwin

Oh the Hello World looks very much better than is shown above. Still 
unreadable by me but I recognise different charactersets.

Do you need a print?

screenshot would be fine, or just take picture with your cell phone 
and upload it :-)

http://arnoldvanhofwegen.com/stuff/helloworldRed.jpg

Needed 6 programs to do that! Schermafbeelding (screenshot?) to make 
the picture, Spotlight to find it, Imagewell to change tiff to jpg 
Preview to check Finder to put it in the right directory Filezilla 
to transfer Safari to check (clipboard to transfer the url to here)

I need one HTC sensation, press of a shooter, one button press to 
get jpeg into my email or facebook. Just throw your workflow to the 
trashcan, you can do better nowadays :-)

Brian: thanks for the info.

Arnold, thanks for testing. Those results look good

Could you test the Red/System programs, as well?

If people check out the repository, you get all programs at once 
and you can keep them up to date very simply for new test versions:

http://rebol.esperconsultancy.nl/documentation/how-to-use-Fossil.html

Fossil is already included in Syllable 0.6.7

I've tested to confirm that all examples that apply to Syllable still 
work perfectly

Good to know, but we'll need to make a last testing pass once the 
merge of 0.3.0 done.

Sure, that's much easier now

Although I'll be away the coming week, so I probably won't be able 
to generate them then

If there's no new issue to fix, we should do the merge this weekend.

Oh, the particular quality of the R3 extension dispatch model that 
makes it well-suited to JIT compiler implementation is that a command 
function contains an indirect reference to the dispatch function, 
and an index integer. When the command is called, the runtime calls 
the dispatch function and passes the integer and a marshalled stack 
frame. For a JIT compiler dispatch function, the index of the command 
can be an index into an array of function pointers or something like 
that, and the dispatch function can just pass the stack frame to 
the appropriate function, then return the results. This means that 
the hard part of JIT compiling - getting the regular runtime to call 
the created functions - is something that you essentially get for 
free with the existing command mechanism.


You could also use the dispatch function to marshall arguments into 
another runtime with a different call model. You could, for instance, 
have a dispatch function that pushes the contents of a marshalled 
stack frame onto a Lua stack and calls Lua functions. Or you could 
do something similar for LLVM functions, or ActiveScripting languages, 
or V8, or ODBC queries, or even Red's JIT.


This all depends on having a good marshalling model in the first 
place that can handle the datatypes you need to support, and it would 
also help if there was a good task-safe callback mechanism that actually 
works (R3's needs a bit of work at the moment). Still, the principle 
is sound.

Interesting, that meshes with the idea that the extension interface 
should be able to function without an OS loader

I get a new error on GTK on Linux when it tries to load the Red-48x48.png 
logo:

(dressed-up-hello-GTK-world:4515): Gtk-CRITICAL **: gtk_container_add: 
assertion `GTK_IS_WIDGET (widget)' failed

But no time left to look into it further

Other than that, Linux x86 still works

It's helpful to make a conceptual distinction between the host interface 
and the extension interface, even though for R3 they are currently 
related to each other and share a lot of the same code.


For the host interface, the host is the OS (more or less) and provides 
an execution environment that the R3 runtime runs on like a program 
(this is all metaphorical, but I'm sure you get it). The OS in this 
case could be something like Windows, Linux, some microkernel, whatever, 
or it could be an application or application plugin like Eclipse, 
Visual Studio, Notepad++, Excel, Firefox, whatever.


For the extension interface, R3 is the OS, the extension-embedded 
module is the program that runs on the OS, and that program calls 
the extension's native code like a library. The program source is 
returned by the extension's RX_Init function, and that program then 
wraps the native library code. The module source is loaded like a 
normal script (slightly hacked after loading to make it a better 
wrapper), so the script could be embedded in binary data along with 
non-Rebol stuff just like with normal scripts. You could even have 
Red and Rebol scripts in the same file (if they use the same embedding 
method) so you the data the init function returns can be like a Red/Rebol 
fat binary, metaphorically.


Given this, Red could either be (or compile) a host for R3; or it 
could be (or compile) a runtime library that implements the same 
host interface as r3lib, making it a drop-in replacement for R3; 
or it could be (or compile) an extension that R3 is a client of, 
returning R3 code that calls calls the compiled Red code; or it could 
be an alternate extension container, for extensions that return both 
Red and R3 code from the same init function, which would call the 
Red code returned, which would in turn call the same native code. 
The two languages could be integrated at any point in the stack, 
along with other languages.

Much of this is already in the Red/R3 bridge

What bothers me about the module script returned by the RX_Init function 
is that it looks like an Interface Description Language, but it isn't, 
because arbitrary REBOL code can be intermixed. While the whole interface 
looks very strongly decoupled, this breaks it because it ties it 
strongly to REBOL if one is not very careful what not to put in the 
module script

It would be much better to have a real IDL for the command descriptions

Most extension module scripts are currently IDL-like, but that is 
only because they aren't (and don't need to be) very ambitious in 
their system integration because they just export a bunch of functions. 
Any native implementation of a port scheme, native dialect, or other 
system enhancement would need Rebol code to integrate that enhancement. 
Doing that in Rebol code is what allows the actual native interface 
to be that simple. It is also what would allow Red wrapper code (which 
could be returned from the same RX_Init function in the same string) 
to use the same native code unchanged, even though Red's runtime 
model is likely to require different integration code.

[rebol [] do something]
[red [] do something]

People who stick to IDL-style extension modules are really underestimating 
the power of the model :)

Yes, there's endless opportunity to make it incompatible with anything 
but R3 :-)

That has been a fabulously successful strategy for Microsoft

I was really careful to make extensions support mixed-language multi-modules. 
For that matter, with my recent patches it even supports same-language 
multi-modules.

Except that in a careless extension, you wouldn't be able to execute 
the module script in Red

Same goes for any module, even the non-native ones. Multi-modules 
are supported in regular scripts too.

Not if you split off the interface into a proper IDL

Ah, but the interface is too simple to need an IDL - make command! 
will do. The extra stuff is for system integration, which is only 
needed when you are doing port schemes, dialects, anything that you 
wouldn't expect to be cross-language compatible anyways, unless you 
explicitly implement a compatible system model. If you're just exporting 
functions then you can implement a simple IDL just by interpreting 
the (cooincidentally the same) module spec code with a very limited 
IDL dialect processor if no Red script wrapper is found.

The same could be said for changes in major versions of the same 
language if they weren't made with deliberate compatibility in mind.

I didn't have Red in mind when I implemented multi-modules, as Red 
didn't exist yet. I was actually thinking of other languages that 
didn't even need to have the same syntax as Rebol.

Carl was thinking of better encapping methods for R3 itself. It works 
for both :)

I think you don't want to understand me, but my time is up

Btw, you can't run applications for Linux on Unix without a compatibility 
library either. Bringing MS into it is just an insult.

Ah, see, you do understand me :-)