World: r3wp

Can you paste me here the result of : readelf -l empty

(just the program header entries)

I don't think it would be any different than on Linux, but I will

Should be 2 lines starting with LOAD.

OK

Also, do you know if Syllable requires a "section headers" part in 
executable binaries? (it's optional in ELF specifications)

[[kaj-:-syllable]:~/Red]readelf -l empty

                                                                                                                                                                                                        
Elf file type is EXEC (Executable file)
Entry point 0x8048074
There are 2 program headers, starting at offset 52

                                                                                                                                                                                                        
Program Headers:

  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg 
  Align

  LOAD           0x000000 0x08048000 0x08048000 0x00184 0x00184 R E 
  0x1000

  LOAD           0x000184 0x08048184 0x08048184 0x00032 0x00032 RW 
   0x1000
[[kaj-:-syllable]:~/Red]

Indeed, exactly the same as on Linux, as it's the same executable, 
and the same readelf

I have to mention that Syllable does something odd by compiling all 
its own programs as shared libraries, so it could also be a problem 
in our loader that wasn't hit before

Seems good, sizes are specified, 0x32 (40) for the 2nd segment. Don't 
know what could cause the issue for the image loader, maybe the lack 
of section headers?

oops, 0x32 = 50

I don't know, I'll have to dive into our loader. It's much like Red: 
the minimum we needed over time to get things working :-)

AFAIK, shared libraries on Linux have special requirements, like 
for the code, it must be compiled in PIC mode (Postion Independent 
Code). Red/System doesn't support that mode.

Yes, I expect it not to work once loaded, but it doesn't even load 
yet

A good opportunity to learn more about Linux kernel hacking. :-)

Not really, this is the Syllable kernel :-)

We determined that the shared library trick isn't even necessary, 
so we have a backburner plan to change it. So I hope it will be possible 
to get regular executables to run

Btw, PIC mode will have to be supported at some point of Red evolution 
in order to be able to build dynamic libraries for Linux (don't know 
for OS X).

Yep, that will be another impressive addition

DLL generation support could be added for Windows without too much 
effort (just by extending the PE.r code), but for Linux, it requires 
both work at the linker and at the compiler level for PIC support.

Here's our loader:

http://syllable.cvs.sourceforge.net/viewvc/syllable/syllable/system/sys/kernel/kernel/elf.c?view=markup

Code looks somehow cleaner and easier to read than the linux loader. 
:-)

Ok it seems it relies on section headers rather than program headers: 

if ( sElfHdr.e_nSecHdrSize != sizeof( Elf32_SectionHeader_s ) )
{

    printk( "Error: load_image() Invalid section size %d, expected %d\n", 
    sElfHdr.e_nSecHdrSize, sizeof( Elf32_SectionHeader_s ) );
    nError = -EINVAL;
    goto error;
}

i think we can add a basic shdr table rather quickly

nSecHdrSize is at 0 in Red's emitter.

Andreas: yes, it's not a big deal...seems you're volunteering? :-)

i'll at least have a look or two :)

That would be cool

readelf gave me the impression that section and program headers are 
the same

Program headers describe segments. Segments are composed of one or 
several sections merged together. In our current ELF implementation, 
each segment contains only one section, so headers should be almost 
the same. That will change with the addition of dynamic linking support 
(which seem to require a lot of additional sections).

That could be the reason that our loader always expects segment headers, 
because a Syllable program is also a dynamic library

I think we ran into the multiple sections per segment problem halfway 
our development, when we needed to implement it

Wouldn't position-independent code be required even on Windows if 
you wanted to support address space layout randomization (ASLR) and 
other such tricks?

It also helps when running Windows code on fake Windows-like OSes 
like SanOS that load at a different address.

ASLR: I suppose it's required, but I can't find any information from 
an official source explaining the exact requirement for ASLR on Windows 
(except the additional flag to set in the executable header).


While searching for that, I found this interesting reading about 
PIC: http://www.gentoo.org/proj/en/hardened/pic-internals.xml#doc_chap7

I would like, if possible, to support PIC without the additional 
cost of an indirection table for global data.

Brian: loading at a different address should not be a problem, for 
that, relocatable code is sufficient. Full ASLR, however, will require 
not only relocatable but position-independent code.

(Take that with a sufficiently sized grain of salt, as I don't know 
anything about how ASLR on Windows works. But I find it hard to imagine 
how they would get away without PIC :)

But at least for PaX-based Linux ELF ASLR, full ASLR requires "position-independent 
executables" (PIE) which are  ELF binaries with a special type flag 
and PIC.

But I guess I wouldn't worry about ASLR for Red at this point :))

Interesting, it sounds like Syllable is prepared for that :-)

You have PaX on Syllable?

Some basic ASLR (stack randomisation and mmap-base randomisation) 
was added to the mainline Linux kernel in 2.6.12:
http://lwn.net/Articles/121845/

We don't have PaX, we have PIC executables, as far as I know

Beyond that, full ASLR is still quite rare in Linux, at least as 
far as I know.

New revision released: preprocessor replaced by a new one running 
at block-level (so much more accurate now). Includes and simple macros 
are now also possible.

Those are some good improvements

Includes make Red a real programming language :-)

Well, that's not my intention, but it wouldn't need much to make 
it a C replacement language.