World: r3wp

I've been rewriting bnf generated parse rules (and often a bit cryptically) 
into proper parse ordered rules for 3 days now... <sigh>  

C is sooo complex for what it really does.  I''ve discovered a few 
quite mind-boggling language capabilities... 
stuff like:    

char *( *(*var)() )[10];


it takes 7 steps to define what that really is and there are other 
"fun" examples which end up being interpretation nightmares, but 
look really simple.


one thing is certain at this point... although I will be able to 
build a C to rebol converter with relative precision under specific 
goals, some of the crazy stuff just will have to be finished manually 
by humans.


at least I rarely see such twisted C code in most of what I've been 
reading so far.

BNF is just a syntax form, with a *lot* of variation. The real difference 
that matters between Yacc and PARSE is the parsing model. Yacc implements 
an LR parser (or some variant thereof), and PARSE implements a variant 
of TDPL parsing (related to PEG), though more powerful and with a 
completely different syntax. How you structure the parse rules depends 
on the parsing model, not the syntax.


For instance, LR parsers tend to do recursion rather than iteration, 
and when they recurse the recrsive call tends to be on the left, 
with the distinguishing clause on the right. For PEG parsers, recursion 
goes the other way. This is not an error, this is a difference in 
parsing model.


If you are translating from Yacc to PARSE, it's not just a syntax 
change. You have to reorganize the rules to match the new model. 
And watch out: Certain patterns are easier to express in some parsing 
models than in others. Some patterns aren't supported at all in some 
models, and so no amount of translation will help you. We chose the 
TDPL model for PARSE because it is more expressive than the LR model, 
so in theory you should be able to translate LR rules to PARSE with 
some topological twists (redoing the sturcture of the rules). However, 
there are patterns that you can express in PARSE that can't be translated 
to LR, even with topological changes.

Unfortunately, the C grammar was designed with LR parsers in mind.

You might be better off translating a C grammar for a PEG or TDPL 
parser generator into PARSE - less topological shifts needed.

well, considering that I just finished the basic rule re-organisation... 
eheheh I think I'll apply the unit testing phase right now to test 
if all the rules perform as they shoudl using input text.  there 
is probably going to be about 100kb of unit test code for what is 
now about 12kb of parse rules.

Sounds about right.

Are you sure you have enough test code/data?

there is all in all only two or three rules that I'm unsure of the 
transformation, as some aspects of the C syntax are a bit obscure 
to represent.

you are being sarcastic right? :-)

No, really. The syntax of C is so complex that you would need a lot 
of data to test all of the common variations.

data
 in this case being C source.

my goal is to get the host code and OpenGL headers past the parsing 
phase.  once that is done, I'll start work on adding the production 
phase.


I still have to write the pre-processor, but that in fact is pretty 
straight forward.  there are little rules and they are much more 
static and well defined on the MS web site.

Well, good luck! :)

the funny thing is that the C language reference on the MSDN is actually 
pretty well done... there are a lot of evil C examples for some of 
the more obscure parts of  the language like pointers, structs and 
unions.


funny thing is that some of the most complex things to express where 
the litteral constants!  integers, with octal, hex notation... not 
as simple as some [digits]  ;-)

Maxim, maybe you thought I was kidding the other day... ;)

hehe

Looking at the new WHILE keyword and I was quite baffled by Carl's 
use of it in his latest blog example. Then I read the docs and it 
didn't get much better:

- WHILE is a variant of ANY
- ANY stops, if input does not change
- WHILE doesn't stop, even if input does not change

What does "input does not change" mean?

Is it about changing the parse series length during parse?

Is it actively moving the parse index back or forth using special 
commands?

Is it normal progression of parse index with each cycle of WHILE 
or ANY?

Is it alteration of the parse series content while maintaining length 
during parse?

Henrik - according to docs explanation, 'parse contains some internal 
protection for the case, when input stream does not advance its position. 
In R2, following code causes infinite loop, in R3, it returns false:

parse str [some [to "abc"]]


(I am not sure I like that it returns false - normally I expect it 
to cause infinite loop. This is imo overprotecting programmer, and 
you have to think, why your code returns false anyway, which for 
me is the same, as if it would cause an infinite loop)

Further from docs:


To avoid infinite looping, a special internal rule is triggered based 
on the fact that the rule did not change the input position.

However, this shows a problem with this rule:

parse str [some [to "a" remove thru "b"]]


Here the input did not appear to advance, but something useful happened. 
In such cases, the some word should not be used, and the while word 
is better:

parse str [while [to "a" remove thru "b"]]

I don't probably understand usefullness of 'while at all. Because 
now I have to think, if my code would cause infinite loop, or not, 
and use 'some or 'while accordingly ...

Running above examples, my opinion is, that in fact adding 'while 
was probably not a good decision. I can understand, that now we have 
more power - our code will not easily cause an infinite loops, but 
otoh you now have to think, if it can happen or not, and 'some becomes 
your enemy ...

I probably need more examples ..

The WHILE keyword is the simplest possible cycle. The rule:

    a: [while b]

is equivalent to recursive:

    a: [b a]

sorry, I meant a: [b a |]

More complicated rules can be easily simulated using the While keyword, 
the opposite isn't true. Carl's example just proves, why While is 
useful.

>> parse [1 2 3] [?? thru [integer! string!] ?? integer!]
thte: [1 2 3]
integer!: [2 3]
== false

What's that "thte" thing?

?? not initialized after first match?  And secondly, how do I match 
thru a series of things (e.g. integer! integer!, but just wondering 
about the thte.  ?? problem before the first match?)

>> parse [1 2 3][?? thru [integer! string!] ?? integer!]
thru: [1 2 3]
integer!: [2 3]
== false

what do you mean by "match thru a series of things"?

Hm.  Version:   2.100.96.2.5  I quit and restarted.

And it stopped doing that.  I'll see if I can get it to do it again.

Is a sequence of things one of the complex rules that you can't use 
in a thru?

Yes. You can express a sequence of characters in a string as a string 
literal, but not a sequence of types in a block. You are going to 
need first sets and the other LL tricks for that.

Fortunately typesets work for block parsing like bitsets do for string 
parsing, so first sets are easy.

>> parse [a b c] [(value: none) copy value to 3 skip to end (probe 
value)] 
[a b]
== true


>> parse [a b c] [(value: none) copy value thru 3 skip to end (probe 
value)]
[a b]
== true

brian - so we can use things like any-string! or other typesets to 
match?

Should the latter be [a b c] ?

I would expect that ...

>> parse [a b c][?? 3 skip ??]
3: [a b c]
end!: []
== true

to/thru were reimplemented to allow multiple options. There are cases, 
where they are not supposed to work, but in above case I would regard 
it being a bug .... unless some guru finds a theory showing us why 
it should be regarded being a correct result :-)

Fork, the fact that both of those examples work incorrectly instead 
of throwing an error is a bug in PARSE. It should be CureCoded.

FYI still seeing some erratic behavior with ?? at head of the parse 
rule

>> parse [a b c] [?? copy value thru 1 skip to end]              
            
co? : [a b c]
== true

(That question mark not visible in the terminal, showed up when I 
pasted here)

Seems like a Unicode to ANSI translation error.

Indeterminate, e.g. just ran it again and:

But no such characters should be output by ??

>> parse [a b c] [?? copy value thru 1 skip to end]              
            
coo:: [a b c]
== true

Definitely another bug. CureCode it.

Well, I should find a way to reproduce it before doing that.  Left 
a note about how getting a CureCode account didn't work the other 
day.