World: r3wp

Ok, so I've not yet provided everything that you'll need to do it. 
I divided the
 extensions release into a few stages:

1. simple - 
just simple access to commands and args
2. series - access to series 
values of various types
3. objects - access to objects (of all types)
4. 
codecs - support for codecs
5. host-lib - ability to bundle extensions 
with the host-lib itself.

So, I need to get you a bit more... in 
fact along the lines that you mention.

Re #6156: Pekr, we ARE NOT giving up on dialects!! There are many 
dialects in RE
BOL, and it is one of the main concepts.

What we 
are doing is removing the strong overlap in DELECT and COMMAND. If 
you l
ook at the DELECT method, it is a small subset of full dialects. 
It implements a
 form of function with optional arguments.

So, it's 
better to move that code into COMMANDS, and allow them to work that 
way
. This makes it much easier for people to learn and use. Even 
me!

Also, REBOL/Services will use this same method, because COMMANDS 
are not limited
 to just extensions... ah the secret is out: COMMANDS 
can also be attached to a
context, making them generally useful in 
REBOL code.

I will check the blog comments to make sure it's not 
misunderstood.

Please could someone translate to me, what does it mean that COMMANDS 
can be attached to a context, and that it will make them useful in 
REBOL code? :-)

The way I see it is  that the code inside a command probably can 
be late bound to a context, rather than the global context, as it 
is now.


when extensions will support objects, this can be pretty powerfull, 
since commands can become virtual and private methods for an object 
where the data is stored in a stuct in the binary (C) side... 


which is EXACTLY what I need for liquid, where I need rebol dispatchers 
but native data storage, so it can scale to hundreds of billions 
of nodes, and yes I already have the solution for the storage/memory 
engine if Carl can give me the means.   :-D

I already found a way to make callbacks extension callbacks in the 
current host distribution, even if nothing in the current rebol native 
code supports it  :-D


will be testing this out tonight and will report on this... I hope 
my idea works.  this would reactivate the OpenGL project along with 
other stuff on the backburner.

oops... one too many callbacks in previous sentence.. hehe

It's the dispatch. Right now with extensions, when you make a command! 
it makes a function that is dispatched by a function in the extension 
based on a number (which you can think of ay a key), to code that 
handles the command (the value associated with the key). In theory 
this is not that different from an object! grabbing data from one 
of its slots based on the keyword you pass it. Apparently commands 
will be able to dispatch to objects soon, and the functions assigned 
to slots of that object will handle the command code.


The DELECT dialect model was based on rebcode, mostly on its JIT 
binding. DELECT added the out-of-order, possibly optional argument 
handling to the dialect decoding phase, but the dispatch phase was 
mostly left out (I commented on this at the time). The command! type 
has the dispatch model, but uses the function call model for calling 
the commands. The overlap that Carl mentions is in the mapping of 
keys to command handlers.


If you unify the command mapping models between DELECT and command!, 
then that code can be shared. This means that the DELECT function 
could do the out-of-order dialect decoding, then dispatch the operations 
as commands. Values of the command! type would continue to be called 
like regular functions in DO code or by APPLY, and then dispatch 
using the same dispatch code as above. On the other end, commands 
would either dispatch to objects (including modules perhaps) or extensions.


By the sound of it, this might also allow the command! type to serve 
as a method pointer, but we'll have to wait to see about that :)

they would be globally bound, but still, usefull  I wonder how extension 
re-entry from a callback will react , if it even works... the stack 
can get a mighty mangled hehe    :-)

That was in reply to Pekr, btw.

btw... I wish there more host <-> r3lib hooking.   I really wish 
he'd push some of the extension handling code into the host.  right 
now there is no real extension code within the host, and there is 
no integration possible from new runtime features into the extension... 
basically, the extensions are running blind.

just a single place where we can put data which is accessible by 
extensions.  that would already make the host that much more usefull, 
especially for testing new host models or devices.  which add new 
possibilities for extensions.


the event device is also not useable for my specific task and I'm 
not sure I can really play around with it without breaking the r3lib 
<-> host integrity... testing will provide clues, I guess.

He has said that having extensions in the host is part of the planned 
model - check R3 chat.

He has also said that the extension model isn't finished, just the 
overall structure.

I know... just stating it out loud... to make it known that this 
is needed... its not just wishfull thinking  ;-)

and we are talking (non web) commercial rebol work here.

but on we go with the callback hack  I think I'll name this a   "hackback" 
   the back door, like a "hatchback"    :-D

I know you are really chomping at the bit for callbacks, and were 
really hoping that the host code would allow you to hack them in 
yourself, but you're out of luck there. Your callback proposals didn't 
take tasks into account anyways. You're still going to need device 
extensions. Fortunately, you can probably help design the model for 
those :)

I think I could have something working for non multi-threaded stuff 
in a little while... I'm working on this now... its the time I have 
to do it ... after that...
I return from a sanity preserving week of vacation.


and yes I hope I can help with the development, especially since 
I have two different devices which need to be added.  

with the current host I might make my hackback do some usefull tests 
to help shape a working model aka prototype.

Well, with the current host code you can add devices in the host, 
and don't need to wait for the device extension model :)

I've been 3 times to the dentist in 2 weeks... (and going back next 
week!)...  needles and my gums are practically on first name basis 
as of now  :-(.

but the coupling with the core run-time is practically abscent.  
there is only one function I can use to have the run-time do anything 
and thats a pretty simple... do_rebol_string() which basically runs 
a block of code in the global space... beyond that I've only got 
network/file like ports, which basically are streamed I/O.  


I can't create data directly and leave it at the port, in a block, 
like I'd do for a proper event queue.   This is currently my pet 
peeve about the host... 


but let's not be judgmental... I'm VERY happy I have the host, so 
I can at least try to rig something up with bailing wire, duct tape, 
pliers, a bit of string & epoxy glue.  


 Extensions & the core allows me to hide this under a nice fiberglass 
 body  ;-)

just don't go looking under the hood... its going to look like a 
honda vtec... hehehe

phew... I'd forgotten how much work it was to express this in C...

variable: new-def: make type? other-def none data

that is... when the other-def isnt' a trivial integer... but something 
like a definition of a callback function ...

Max - please share your needs with Carl in R3 Chat .... I mean - 
the need to move some stuff into Host ....

I will, don't worry... but I need to have some meat for him to chew 
on if I can get this callback thing working, then we can talk about 
something concrete, not just thin air.  :-)

I know what you mean, but the thing is, that Carl might be doing 
some design decisions right now. So it might be better to act before 
it is too late for the change ....

good point.  I was planning on getting his attention (by any & all 
means necessary ;-)  tomorrow

Maxim, you are taking the wrong approach. If you are having problems, 
don't work around them, fix them. Working around is the R2 way.

I could have something working for non multi-threaded stuff
 Not good enough. Tasks are going to be in R3. Prepare :)

Don't look at how the existing ports are implemented, look at the 
port model. You can create your own port types if you like.

They don't have to be network or file port types either :)

How do I put NONE into a block?

I guess, that it is like integer 0, except, that the datatype should 
be RXT_NONE

So it will be int64 in the union.

But this could than be interpreted as integer 0.

But I will try to do this and specify the datatype. We will see.

If the c-level side uses UTF-8 strings as well, can I just use the 
Rebol series as is? get_string returns a decoded string.

I got my SQLite R3 extension working. Really nice. Overall it takes 
about 10KB of C code.

very nice

:-)  and its probably really fast :-)

in my tests, extensions are quite performant.

I will do a stress-test with a triple engine I have for R2. We will 
see.

there is no reason for it being any faster than R2 version, no? Most 
of the work in DB area is done by DB engine anyway ... the rest is 
passing the arguments around ... or do you think extension interface 
is faster than R2 DLL interface?

In R2 the collection loop to build the result was done on the Rebol 
side. Now it's done on the C-side. This is a lot faster.

And, I directly build internal R3 block, this is directly used I 
think. So no copying, internal handling etc.

Performance test will show.

can I give a guess that the series building part of the request will 
be about 100 times faster now?

@ brian: I understand what you say about working around... but, with 
the host right now, I can't do anything else than hack up a solution 
in order for extensions to have callbacks.  99% of the real work 
is inside the extension.  when the host will be improved, the extension 
work won't change.  I can't wait for things to happen... this solution 
is just for my own use, and it gives me a good perspective on how 
to help with a real solution.  


I'm also brushing up on my C skills (I haven't done any in over a 
decade!) so doing this stuff is a good exercise anyways.


right now, if I can show to Carl how complicated it is to do some 
stuff, he will have an explicit example and have a better reference 
for practical solutions.  Doing anything... its all just theories. 
 look at the notes on delect and command... this is similar... lets 
start with something.  see where that leads us and then, we have 
a reference to pick on and critique.  


We can have better ideas for the api, just by having something bad 
to start with.... and believe me.. I know my hack sucks ... but I've 
got part of the solution complete, and compiling without warnings, 
and some code sharing between extensions api and the host.  :-)

Maxim, are you preparing a Glass extension ?