World: r4wp

In my view, the R3 interface is also motivated by the wish to be 
independent from any operating system loader mechanisms, to discard 
them for Wildman in some unknown future

Red/System extensions will usually be compiled together with the 
Red runtime, so they will be much more flexible. For interfacing 
with precompiled binaries, a more stable interface would be needed

Public API stability: right, that's a good point in favor of a set 
of wrappers on top of current Red runtime API.

A stack-oriented API might be a good candidate for that...

For example, I've noted the alias! issue before. As long as all code 
is compiled together, alias numbers are a good interface, like symbol 
IDs are assigned at runtime in REBOL. But when precompiled code needs 
to communicate they become useless, hence why the R3 interface makes 
efforts to map symbols to known numbers

Alias! type ID are attributed per compilation session, so they are 
not shareable with other binaries.

Anyway, we want to expose Red API to host languages/apps, not Red/System's 
one (at least not for now).

Think PARSE usable from any other language as example. ;-)

The same issue affects Red symbols

Red/System is already easy to expose to other languages, because 
it's C compatible, and you made the dynlib interface. :-) However, 
alias IDs break it

Part of the motivation of the R3 extension/host API was to isolate 
the extensions and hosts from changes in the underlying data model, 
which makes hosts and extensions really resilient to upgrades of 
R3. It also was designed to let you have a consistent internal execution 
model even in cases where the host has a completely different process/thread 
model.

I think it went a little too far at times, especially the lack of 
marshallers for immediate values that are more than 64 bits internally. 
I've frequently wanted to supplement it with marshallers for the 
other datatypes in R3, particularly the date, time and money types.

The big win is the command dispatch model though, because it basically 
lets you get dispatch to JIT-compiled functions for free. The dispatch 
function can manage changes between execution models completely without 
R3 even noticing. Lua has similar separation, though since it doesn't 
have to support anywhere near as many datatypes it can get away with 
a stack-based interface.

Another interesting side effect of the strong separation is that 
it would be possible to implement the client-side of the extension 
interface in other languages, such as Red, so that it could use R3 
extensions without changes. The REBOL portion of the extension might 
be trickier to implement though, because that code tends to be more 
tied into the actual R3 runtime model. You could write your own wrapper 
code if your system model is different, but the native code could 
be used as-is.

However, the way that the R3 script loader works, you could make 
an extension that has both Red and Rebol scripts in the module data, 
especially if Red uses a similar script-in-a-block embedding method,

That would make it so you could make one extension that can be used 
by both languages, that shares the same native code.

Hi Kaj on my macbook:

Last login: Fri Oct 19 07:21:16 on 
ttys001

 MacBook-van-Arnold-160:~ Arnold$ 
 /Users/Arnold/Downloads/Red\(/System\)\ 
 Testing-dc1b702068063b65/Darwin/Red/hello 
 ; exit;
Hello, world!
§±ÖÁµ, ºÌüµ!
`O}Y, NLu
Dobrý den sv
te
logout

[Proces voltooid]

 /Users/Arnold/Downloads/Red\(/System\)\ Testing-dc1b702068063b65/Darwin/Red/Fibonacci 
 ; exit;

MacBook-van-Arnold-160:~ Arnold$ /Users/Arnold/Downloads/Red\(/System\)\ 
Testing-dc1b702068063b65/Darwin/Red/Fibonacci ; exit;
Fibonacci 35: 9227465
logout

[Proces voltooid]


MacBook-van-Arnold-160:~ Arnold$ /Users/Arnold/Downloads/Red\(/System\)\ 
Testing-dc1b702068063b65/Darwin/Red/empty ; exit;
logout

[Proces voltooid]
I took the programs from the Red tree under Darwin

Oh the Hello World looks very much better than is shown above. Still 
unreadable by me but I recognise different charactersets.

Do you need a print?

screenshot would be fine, or just take picture with your cell phone 
and upload it :-)

http://arnoldvanhofwegen.com/stuff/helloworldRed.jpg

Needed 6 programs to do that! Schermafbeelding (screenshot?) to make 
the picture, Spotlight to find it, Imagewell to change tiff to jpg 
Preview to check Finder to put it in the right directory Filezilla 
to transfer Safari to check (clipboard to transfer the url to here)

I need one HTC sensation, press of a shooter, one button press to 
get jpeg into my email or facebook. Just throw your workflow to the 
trashcan, you can do better nowadays :-)

Brian: thanks for the info.

Arnold, thanks for testing. Those results look good

Could you test the Red/System programs, as well?

If people check out the repository, you get all programs at once 
and you can keep them up to date very simply for new test versions:

http://rebol.esperconsultancy.nl/documentation/how-to-use-Fossil.html

Fossil is already included in Syllable 0.6.7

I've tested to confirm that all examples that apply to Syllable still 
work perfectly

Good to know, but we'll need to make a last testing pass once the 
merge of 0.3.0 done.

Sure, that's much easier now

Although I'll be away the coming week, so I probably won't be able 
to generate them then

If there's no new issue to fix, we should do the merge this weekend.

Oh, the particular quality of the R3 extension dispatch model that 
makes it well-suited to JIT compiler implementation is that a command 
function contains an indirect reference to the dispatch function, 
and an index integer. When the command is called, the runtime calls 
the dispatch function and passes the integer and a marshalled stack 
frame. For a JIT compiler dispatch function, the index of the command 
can be an index into an array of function pointers or something like 
that, and the dispatch function can just pass the stack frame to 
the appropriate function, then return the results. This means that 
the hard part of JIT compiling - getting the regular runtime to call 
the created functions - is something that you essentially get for 
free with the existing command mechanism.


You could also use the dispatch function to marshall arguments into 
another runtime with a different call model. You could, for instance, 
have a dispatch function that pushes the contents of a marshalled 
stack frame onto a Lua stack and calls Lua functions. Or you could 
do something similar for LLVM functions, or ActiveScripting languages, 
or V8, or ODBC queries, or even Red's JIT.


This all depends on having a good marshalling model in the first 
place that can handle the datatypes you need to support, and it would 
also help if there was a good task-safe callback mechanism that actually 
works (R3's needs a bit of work at the moment). Still, the principle 
is sound.

Interesting, that meshes with the idea that the extension interface 
should be able to function without an OS loader

I get a new error on GTK on Linux when it tries to load the Red-48x48.png 
logo:

(dressed-up-hello-GTK-world:4515): Gtk-CRITICAL **: gtk_container_add: 
assertion `GTK_IS_WIDGET (widget)' failed

But no time left to look into it further

Other than that, Linux x86 still works

It's helpful to make a conceptual distinction between the host interface 
and the extension interface, even though for R3 they are currently 
related to each other and share a lot of the same code.


For the host interface, the host is the OS (more or less) and provides 
an execution environment that the R3 runtime runs on like a program 
(this is all metaphorical, but I'm sure you get it). The OS in this 
case could be something like Windows, Linux, some microkernel, whatever, 
or it could be an application or application plugin like Eclipse, 
Visual Studio, Notepad++, Excel, Firefox, whatever.


For the extension interface, R3 is the OS, the extension-embedded 
module is the program that runs on the OS, and that program calls 
the extension's native code like a library. The program source is 
returned by the extension's RX_Init function, and that program then 
wraps the native library code. The module source is loaded like a 
normal script (slightly hacked after loading to make it a better 
wrapper), so the script could be embedded in binary data along with 
non-Rebol stuff just like with normal scripts. You could even have 
Red and Rebol scripts in the same file (if they use the same embedding 
method) so you the data the init function returns can be like a Red/Rebol 
fat binary, metaphorically.


Given this, Red could either be (or compile) a host for R3; or it 
could be (or compile) a runtime library that implements the same 
host interface as r3lib, making it a drop-in replacement for R3; 
or it could be (or compile) an extension that R3 is a client of, 
returning R3 code that calls calls the compiled Red code; or it could 
be an alternate extension container, for extensions that return both 
Red and R3 code from the same init function, which would call the 
Red code returned, which would in turn call the same native code. 
The two languages could be integrated at any point in the stack, 
along with other languages.

Much of this is already in the Red/R3 bridge

What bothers me about the module script returned by the RX_Init function 
is that it looks like an Interface Description Language, but it isn't, 
because arbitrary REBOL code can be intermixed. While the whole interface 
looks very strongly decoupled, this breaks it because it ties it 
strongly to REBOL if one is not very careful what not to put in the 
module script

It would be much better to have a real IDL for the command descriptions

Most extension module scripts are currently IDL-like, but that is 
only because they aren't (and don't need to be) very ambitious in 
their system integration because they just export a bunch of functions. 
Any native implementation of a port scheme, native dialect, or other 
system enhancement would need Rebol code to integrate that enhancement. 
Doing that in Rebol code is what allows the actual native interface 
to be that simple. It is also what would allow Red wrapper code (which 
could be returned from the same RX_Init function in the same string) 
to use the same native code unchanged, even though Red's runtime 
model is likely to require different integration code.

[rebol [] do something]
[red [] do something]

People who stick to IDL-style extension modules are really underestimating 
the power of the model :)

Yes, there's endless opportunity to make it incompatible with anything 
but R3 :-)

That has been a fabulously successful strategy for Microsoft

I was really careful to make extensions support mixed-language multi-modules. 
For that matter, with my recent patches it even supports same-language 
multi-modules.