World: r3wp

Have to examine assembler before knowing such things :)

Well I have already :)

Thats why i try to get away by just asking you :) But seems i should 
look too.

Actually, I pored over every line of code, tested most of the opcodes 
too. I've been taking this rebcode development very seriously, making 
suggestions, finding bugs. Trying to keep from making too much of 
a jerk of myself. I expect that I'll be using and generating a lot 
of rebcode in the future so I want it to be the best that it can 
be.

If anything, the conversations with Gabriele remind me of those we 
had on the list where we reverse-engineered REBOL's context handling, 
back when it wasn't documented.

I agree. For small byte-crunchers its high-level enough. Maybe  it 
should integrate with parse too, that would be great for dialects 
(replacing chars and such).

and features to write specialized loops, like remove-each .

(indirect set mainly)

It does integrate with parse - have you tried the rewrite rules? 
More advanced stuff can be done with preprocessors like your altsyn.

No, at runtime.

apply x parse [rules]

in parse
  "<" rebcode[ append out "&lt;" ]

a: rebcode [args] [stuff]
parse dat [here: some-rule (a here)]

I am talking speed ;) Your example goes thru a rebol-style call. 
a direct call may be much faster

ANd some1 (Gabriele) suggested parse can compete with rebcode in 
its area, its a vm too.

(Gabriele) -> (Gabriele?)

(yeah, that's right)

It would be interesting to have a parse opcode, but keep in mind 
that this kind of speedup would likely be implemented in the JIT, 
when we get that. And however fast parse is, its overhead dwarfs 
that of a function call. And remember, using apply would be significantly 
faster than calling a function in the do dialect because there isn't 
any evaluator overhead.

Keep in mind that parse can be jitted too :)

No, it really can't. You could in theory build a compiler for a static 
subset of parse, but parse rules can change at runtime, even while 
they are running. They are passed by reference, remember. The reason 
you can JIT rebcode is because every code block must be supplied 
as an immediate literal and so can be considered static. If you do 
runtime modifications of the code blocks, code-as-data stuff or other 
REBOL-like tricks a JIT wouldn't really work, and certainly provide 
any speedup. Keep in mind that rebcode is a compilable subset of 
REBOL - the rest of REBOL mostly isn't compilable.

JITs like hotspot can handle such changes.

Hotspot compiles a VM that doesn't allow self-modifying code, just 
replacable code.

It can switch from native code back to bytecode if the mathod is 
changed. and from bytecode to native once something is compiled.

All you need is tracking references to a parse-rule, and signal such 
references to fall back to interpreter if rule is changed.

When Java bytecode is changed, the entire function is replaced with 
a new function with the changes. It's not changed in place.

its practically changed in place. Thats magic inside vm, adapting 
stacks etc.

there was an example with a main which contained only a loop. it 
startd with bytecode, after a while it was compiled, and that loop 
was faster. without re-entering that method.

so practically changed in place.

But we would not need that, only a change-check when entering sub-blocks. 
and doing that block by interpreter. then.

The JIT replaces the bytecode function with native code. If the bytecode 
is changed, a new function is generated. The JIT can work on that 
new function too.

Technically yes. Practically the current method is changed to a faster 
one.

Thats similar to changing a sub-rule at runtime. changing the current 
block would not work with my simpler propoal, but then i consider 
that bad style.

Back to parse, you could in theory statically translate the rules 
to an internal rebcode-like form for a different VM, and then JIT 
that. You wouldn't get as much of a speedup as you think though. 
The existing parse engine spends most of its time actually doing 
the parsing with the native code in the engine - a JIT would only 
speed up the reading of the parse rules, something it doesn't spend 
as much time doing in comparison.

Bernd Paysan once wrote a regexp compiling to jitted forth. 10* faster 
than the usual c-interpreter.

Bad style? Certain parse tasks require you to at least swap between 
parse rules on occasion - for instance incremental parsing.

parse processes bytes. thats what a compiler is good at. now you 
add some interpreter steps, one for any, one for the char. In comparison 
to byte-ops interpreter-steps are heavy.

bad style:
 rule: [ "<" (append rule [ | ">") ]

Parse may be a C interpreter, but the rules are REBOL blocks, not 
text. In some ways the compiler step of a regex compiler is done 
already in parse.

sub-rule: [ ..] sub-rule-2: [ .. ]
rule: ["<" (sub-rule: sub-rule-2) sub-rule]
; is better imho.

the rexexp-rules are byte-compiled in perl to, at least optional.

Those "interpreted" aren't interpreted themselves, they are implemented 
in efficient native code in the parse engine itself.

No, it has to look up what 'any does, then there is a char. Its done 
quickly by lookup-tables, but we compare to tiny native loops here.

Yeah, that is the style I generally use too, when possible.

Right now any of those flexible parsing and rule changes are implemented 
in do dialect code in the parens. To make the rules static, you would 
have to compile that code too.

thats what 'rebcode would do.

(rebol-code) -> use rebol
rebcode [rebcode-code] -> use rebcode

But rebcode only implements a subset of the REBOL semantics. General 
REBOL isn't really compilable.

In theory.

Thats what i mean. i don't need full rebol when appending a char. 
rebcode would do.

And parse processes a lot of bytes. The parsing is fast. then the 
() are slowing down things.