World: r4wp

type: either dt/value = TYPE_DATATYPE [
    TYPE_DATATYPE
][ 
    dt/value
]

Isn't that equivalent to

type: dt/value

Yep :) I've changed those lines so many times today, that my mind 
should have blocked from reading them anymore. ;-)

:-)

It was a bit painful commit anyway, as the internal API is not yet 
fully stabilized and to do it right, I would need to fully define 
the public runtime API first, but that would delay the 0.3.0 way 
too much, so we'll deal with that later.


What I mean by "public runtime API", is the Red API exposed to Red/System 
and to other host languages loading Red as a library. I'm still uncertain 
if such public API can be just some internal exposed or will need 
a thin layer of wrappers to make it handier (and in some cases, safer) 
to use. 


The R3 extension isolation model is too strong for my taste and makes 
the extensions harder to write than they should. I'm also uncertain 
if this model was stricly motivated by providing the safest possible 
interface with the core or, if the willing to keep the core internals 
as secret as possible was also playing a big part in this model choice.


Once the `dyn-lib-emitter` branch merged, I plan to study the Lua 
(and others) public API, to see if and how we can do better for Red. 
I already have a rough idea of how it should look like, I just need 
to refine it.

Sounds good

In my view, the R3 interface is also motivated by the wish to be 
independent from any operating system loader mechanisms, to discard 
them for Wildman in some unknown future

Red/System extensions will usually be compiled together with the 
Red runtime, so they will be much more flexible. For interfacing 
with precompiled binaries, a more stable interface would be needed

Public API stability: right, that's a good point in favor of a set 
of wrappers on top of current Red runtime API.

A stack-oriented API might be a good candidate for that...

For example, I've noted the alias! issue before. As long as all code 
is compiled together, alias numbers are a good interface, like symbol 
IDs are assigned at runtime in REBOL. But when precompiled code needs 
to communicate they become useless, hence why the R3 interface makes 
efforts to map symbols to known numbers

Alias! type ID are attributed per compilation session, so they are 
not shareable with other binaries.

Anyway, we want to expose Red API to host languages/apps, not Red/System's 
one (at least not for now).

Think PARSE usable from any other language as example. ;-)

The same issue affects Red symbols

Red/System is already easy to expose to other languages, because 
it's C compatible, and you made the dynlib interface. :-) However, 
alias IDs break it

Part of the motivation of the R3 extension/host API was to isolate 
the extensions and hosts from changes in the underlying data model, 
which makes hosts and extensions really resilient to upgrades of 
R3. It also was designed to let you have a consistent internal execution 
model even in cases where the host has a completely different process/thread 
model.

I think it went a little too far at times, especially the lack of 
marshallers for immediate values that are more than 64 bits internally. 
I've frequently wanted to supplement it with marshallers for the 
other datatypes in R3, particularly the date, time and money types.

The big win is the command dispatch model though, because it basically 
lets you get dispatch to JIT-compiled functions for free. The dispatch 
function can manage changes between execution models completely without 
R3 even noticing. Lua has similar separation, though since it doesn't 
have to support anywhere near as many datatypes it can get away with 
a stack-based interface.

Another interesting side effect of the strong separation is that 
it would be possible to implement the client-side of the extension 
interface in other languages, such as Red, so that it could use R3 
extensions without changes. The REBOL portion of the extension might 
be trickier to implement though, because that code tends to be more 
tied into the actual R3 runtime model. You could write your own wrapper 
code if your system model is different, but the native code could 
be used as-is.

However, the way that the R3 script loader works, you could make 
an extension that has both Red and Rebol scripts in the module data, 
especially if Red uses a similar script-in-a-block embedding method,

That would make it so you could make one extension that can be used 
by both languages, that shares the same native code.

Hi Kaj on my macbook:

Last login: Fri Oct 19 07:21:16 on 
ttys001

 MacBook-van-Arnold-160:~ Arnold$ 
 /Users/Arnold/Downloads/Red\(/System\)\ 
 Testing-dc1b702068063b65/Darwin/Red/hello 
 ; exit;
Hello, world!
§±ÖÁµ, ºÌüµ!
`O}Y, NLu
Dobrý den sv
te
logout

[Proces voltooid]

 /Users/Arnold/Downloads/Red\(/System\)\ Testing-dc1b702068063b65/Darwin/Red/Fibonacci 
 ; exit;

MacBook-van-Arnold-160:~ Arnold$ /Users/Arnold/Downloads/Red\(/System\)\ 
Testing-dc1b702068063b65/Darwin/Red/Fibonacci ; exit;
Fibonacci 35: 9227465
logout

[Proces voltooid]


MacBook-van-Arnold-160:~ Arnold$ /Users/Arnold/Downloads/Red\(/System\)\ 
Testing-dc1b702068063b65/Darwin/Red/empty ; exit;
logout

[Proces voltooid]
I took the programs from the Red tree under Darwin

Oh the Hello World looks very much better than is shown above. Still 
unreadable by me but I recognise different charactersets.

Do you need a print?

screenshot would be fine, or just take picture with your cell phone 
and upload it :-)

http://arnoldvanhofwegen.com/stuff/helloworldRed.jpg

Needed 6 programs to do that! Schermafbeelding (screenshot?) to make 
the picture, Spotlight to find it, Imagewell to change tiff to jpg 
Preview to check Finder to put it in the right directory Filezilla 
to transfer Safari to check (clipboard to transfer the url to here)

I need one HTC sensation, press of a shooter, one button press to 
get jpeg into my email or facebook. Just throw your workflow to the 
trashcan, you can do better nowadays :-)

Brian: thanks for the info.

Arnold, thanks for testing. Those results look good

Could you test the Red/System programs, as well?

If people check out the repository, you get all programs at once 
and you can keep them up to date very simply for new test versions:

http://rebol.esperconsultancy.nl/documentation/how-to-use-Fossil.html

Fossil is already included in Syllable 0.6.7

I've tested to confirm that all examples that apply to Syllable still 
work perfectly

Good to know, but we'll need to make a last testing pass once the 
merge of 0.3.0 done.

Sure, that's much easier now

Although I'll be away the coming week, so I probably won't be able 
to generate them then

If there's no new issue to fix, we should do the merge this weekend.

Oh, the particular quality of the R3 extension dispatch model that 
makes it well-suited to JIT compiler implementation is that a command 
function contains an indirect reference to the dispatch function, 
and an index integer. When the command is called, the runtime calls 
the dispatch function and passes the integer and a marshalled stack 
frame. For a JIT compiler dispatch function, the index of the command 
can be an index into an array of function pointers or something like 
that, and the dispatch function can just pass the stack frame to 
the appropriate function, then return the results. This means that 
the hard part of JIT compiling - getting the regular runtime to call 
the created functions - is something that you essentially get for 
free with the existing command mechanism.


You could also use the dispatch function to marshall arguments into 
another runtime with a different call model. You could, for instance, 
have a dispatch function that pushes the contents of a marshalled 
stack frame onto a Lua stack and calls Lua functions. Or you could 
do something similar for LLVM functions, or ActiveScripting languages, 
or V8, or ODBC queries, or even Red's JIT.


This all depends on having a good marshalling model in the first 
place that can handle the datatypes you need to support, and it would 
also help if there was a good task-safe callback mechanism that actually 
works (R3's needs a bit of work at the moment). Still, the principle 
is sound.

Interesting, that meshes with the idea that the extension interface 
should be able to function without an OS loader

I get a new error on GTK on Linux when it tries to load the Red-48x48.png 
logo:

(dressed-up-hello-GTK-world:4515): Gtk-CRITICAL **: gtk_container_add: 
assertion `GTK_IS_WIDGET (widget)' failed

But no time left to look into it further

Other than that, Linux x86 still works

It's helpful to make a conceptual distinction between the host interface 
and the extension interface, even though for R3 they are currently 
related to each other and share a lot of the same code.


For the host interface, the host is the OS (more or less) and provides 
an execution environment that the R3 runtime runs on like a program 
(this is all metaphorical, but I'm sure you get it). The OS in this 
case could be something like Windows, Linux, some microkernel, whatever, 
or it could be an application or application plugin like Eclipse, 
Visual Studio, Notepad++, Excel, Firefox, whatever.


For the extension interface, R3 is the OS, the extension-embedded 
module is the program that runs on the OS, and that program calls 
the extension's native code like a library. The program source is 
returned by the extension's RX_Init function, and that program then 
wraps the native library code. The module source is loaded like a 
normal script (slightly hacked after loading to make it a better 
wrapper), so the script could be embedded in binary data along with 
non-Rebol stuff just like with normal scripts. You could even have 
Red and Rebol scripts in the same file (if they use the same embedding 
method) so you the data the init function returns can be like a Red/Rebol 
fat binary, metaphorically.


Given this, Red could either be (or compile) a host for R3; or it 
could be (or compile) a runtime library that implements the same 
host interface as r3lib, making it a drop-in replacement for R3; 
or it could be (or compile) an extension that R3 is a client of, 
returning R3 code that calls calls the compiled Red code; or it could 
be an alternate extension container, for extensions that return both 
Red and R3 code from the same init function, which would call the 
Red code returned, which would in turn call the same native code. 
The two languages could be integrated at any point in the stack, 
along with other languages.

Much of this is already in the Red/R3 bridge

What bothers me about the module script returned by the RX_Init function 
is that it looks like an Interface Description Language, but it isn't, 
because arbitrary REBOL code can be intermixed. While the whole interface 
looks very strongly decoupled, this breaks it because it ties it 
strongly to REBOL if one is not very careful what not to put in the 
module script