World: r3wp

cool maxim, I have a significant interest in connecting a powerful 
lib for 
complex financials analysis with rebol. A very dry area but

if I can support your activities in the way I have mentioned before, 
please 
let me know.

maxim the thing you are looking for is known as ABI, application 
binary interface

basically the ABI specifies (among other things) the calling conventions

and yes, that's extremely platform-dependent stuff

for windows, the calling conventions used by the OS seem to be pretty 
standardized and simple.  which is nice.  So far it seems to be a 
bit more sprawled for linux.


first release of the /library extension will be 32 bit only, but 
when we have a 64 bit REBOL it will be pretty easy to adapt... in 
fact, MS/intel did their homework for 64bit, it seems, and there 
is only ONE calling convention for that platform.

afaik microsoft has different calling conventions for x86 and x64

and iirc they have at least 3 support cconvs anyway (cdecl, stdcall, 
fastcall; maybe more)

yep, the x64 is better in fact.  x86 has three official conventions, 
some on the stack, some using registers, and different cleanup.  
 the x64 has only one and is based on __fastcall

because the x64 has much more registers.... it makes sense.

but as i suggested a few days ago, using e.g. libffi or dyncall would 
really help getting started quickly

maxim, any change the extension will be capable to handle nested 
strucs and pointers?

dyncall is BSD and should even work with MSVC

tom, yes.    that will start out simple, but will be handled in the 
rebol & extension part of the code.  


once I have the actuall subroutine jump (ASM call) code in place 
with an unlimited number of run-time functions registering, we'll 
see how the community wants to evolve the struct! dialect.


I am thinking of a rebol to struct converter and a way to keep handles 
to those structs, and even use them within sub structs, so that we 
can easily reuse allocated structures without doing the convertion 
over and over.

so you can work like draw, keeping just the rebol version and convert 
it at every call, or manually convert it once, and then reuse it 
over and over.  

this will allow us to chose between ease of use or raw speed (with 
a bit more management in our code).

internally, it won't make a difference.  depending on how you call 
the library function, it will convert on the fly or reuse the handle.

Andreas, I won't need an external lib, as we are in effect building 
a dyncall specifically for rebol.  which makes it simpler and faster... 
and it seems very simple to implement ourself, in fact.

dyncall is precisely what you will implement anyway :)

supporting a multitude of platforms quickly becomes a PITA

but i won't stop you in doing that :)

in fact i'll happily cheer you along :)

hehehe  I'll do my first tests, and then I'll see If I can wrap the 
platform-specifc parts of the process through dyncall's source code. 
 this way we will be able to compile the /library extension for any 
platform without any source code changes.


my design is to use header file macros which actually map the calling 
conventions based on your platform... THAT I can definitely borrow 
from dyncall if it BSD licensed :-)

ABIs are a bitch :)

this whole project is fun... I haven't done any ASM since my Amiga 
days... 17 years ago... I'm remembering stuff I had placed in a deep 
black hole in my mind...  :-)

yes, it sure is a lot of fun

I'm officially a geek... I actually laughed when I read a bit of 
ASM that had the   lea   instruction in it.. and instantly remembered 
what it meant hehehe  


three little characters, pop up 17 years later and my mind does a 
180 degree shift    :-)

:)

yes sire! just signed in as you first beta tester please  ;-))

created a new group for this specific extension.  we should continue 
any discussion about /library  there .

Max, I don't know what you do but the overall concept would be:
1. Loading the DLL dynamically

2. Finding all requested functions dynamically (Windows has a call 
for this where you provide a string of the function name (just can't 
remember the call name)).

3. Implementing a generic R3 extension function that will be marshalled 
on Rebol site so that the first parameter will the function name 
and than all paraemters (I would use a block here).
4. Rearrange all this to call the Win32 function.

You shouldn't deal with calling conventions etc. at all. At least 
if I had to it would be a sign for the wrong way.

Robert, three problems with that approach (even just on Windows):

- Not all DLLs export that info, some just export numbered entry 
points ane expect you to use a .def file or some such.

- Even for the DLLs that do export names, the C-compatible APIs don't 
have info about function arguments, data structures, ...

- The languages that have real reflection models and encoded info 
(Delphi, C++, .NET, Java) aren't compatible with an R2-style model.

Which is what Maxim is working on. You can't use reflection with 
C-style APIs, and API models that you can use reflection on can get 
their own extensions to access them :)

DLL by enum: Yes, than you need to know what it's about. But within 
R3 you could use a name.

Arguments must always be known up front to DLL functions. I never 
meant that it can be derived automatically.

At least not for C APIs :(

If there are only ordinals for func in the DLL, REBOL should be able 
to specify those. Just use an integer! rather than a string! (in 
R2 library parlance).

That's not the problem. But you need to know what functionn Nr-1 
is about?

Yes, and that can only come from docs of some kind.

Or from header files.

I would consider those docs "of some kind". :-)

Robert,  even if you know the function arguments, how do you call 
the function itself?


the extension will be compiled, so we can't create generic function 
calls.   The calling convention is part of a function's compilation 
process, on Windows any DLL may have three different argument passing 
methods simultaneously!!


get the problem is that you have to push arguments on the stack and 
retrieve them.  there are NO C functions which manipulate the stack 
directly.  these are always done via assembly, either inline or libraries.

Ok, I might have missed something. If you want to make it totaly 
dynamic than you are right. My ideas was more like a hybrid soltuion. 
Keeping the R2 structure how to deal with DLLs within R3(so you avoid 
porting code) but get use the R3 extensions model. Well, but you 
need to do the R3 extension. Not what most people would expect...

Here's the place to ask that question, Robert. The answer is that 
there may not be a way to do that until the next (redesigned) host 
kit comes out.

And the associated new extension model.

Don't worry, it's mostly an enhance ment of the existing model, but 
the command! type will be enhanced.

Ah, I was searching for !REBOL3 Extension. Maybe we can rename this 
group.

We can't, but a world master could. !REBOL3 Extensions sounds good.

agreed on the rename ...

How would this than work? What does the COMMAND! stuff do?

The command! type is the function type that you export from extensions 
into REBOL. There are supposed to be many enhancements to the command! 
type in the next host kit, and extensions are supposed to be embeddable 
in the host as well. And DELECT is going to be redone too. With all 
of these changes, more enhancements to the extension model are likely 
as well.

the question is ... when? :-)