World: r3wp

It needs to know how many arguments the function takes.

That's non-context-free. Though this lexer can be done in Coco/R.

I guess that a context-free grammar would have required some arguments 
list delimiter, such as C and most other languages have.

Also, the "Each context-free rule" expression on page 1 of the BNF 
grammar description is not accurate, it would need a specific comment 
for <fixed-arguments-function-call> rule.

no. the delimiter is space. it's ok.

if you want to parse function calls into a call tree, you'd need 
some argument delimitations

spaces are delimiters

how do you parse `foo bar 1 2`?

depends on the definitions of  foo and other tokens. I.e. on context

exactly

so if you want a context-free grammar which models nested calls, 
you'd need funcall delimitation

(which is what doc said)

no. not delimiters are issue.

end of list of arguments is uncertain

with CFG

read again: i said "funcall delimitation"

this is ambiguous

and nested calls are implementable without any delimitation

on the other hand, i'm certain that you can model Red/System's syntactical 
structure with a context-free grammar. it's just that the CST/AST 
would look quite different from other languages (i.e. in that it 
does not explicitly model function call structures)

(pretty much the same as for REBOL)

you're wrong

for example

as i've done this several times before, i'm quite confident that 
i'm not wrong :)

funcallA arglist next_token ----- CFG grammar cannot distinguish 
args from arglist from next_token.

you probably used hacks over CFG that make it non-CFG

arglist --- args separated by spaces

no. as i've said before, you simply cannot explicitly model e.g. 
funcalls

i.e. you won't get a "funcall" node in the AST

well. I'll go rest a bit today =)

Evgeniy: thanks for bringing up that issue in the BNF doc, we'll 
need to fix that description somehow.

Dockimbel: I think there may be more issues with the grammar. I think 
I'll continue tomorrow.

I suppose there are some internal priorities for Red as well, such 
as when for example networking becomes relevant.

Henrik, networking is already available

Haha. The following is indistinguishable in space-ignoring versions 
of Coco/R, so spaces must be modeled via the grammar, too:

word1: word2

word1 :func2

the first line is assignment, the second line depends on the meaning 
of word1

though :func2 could be included into TOKENS section

<code-block> ::= [ {<statement> | <expression> | <comment>}+   ]
      A

btw are definitions allowed inside the code blocks?

Evgeniy, the : is part of the first token in the first case, and 
part of the second token in the second case, it isn't a separate 
token. The values a: and :a are related semantically, not syntactically.

The assignment is semantic, not syntactic. Other Red dialects may 
or may not associate a set-word with assignment.

Or you could think of it as a higher-level syntax, above the tokenization 
level.

You could try to make the Coco/R syntax specific to the Red/System 
dialect or you could make at least the tokenizer general for Red 
code and implement the Red/System dialect in the parse rules. The 
latter would be a more useful approach for REBOL-like languages :)

Well. I give up. I don't like languages with preprocessors since 
they are slower than languages with no preprocessor. So I send an 
original .ATG to Dockimbel, and stop my work. Looking at the RED 
spec made me sigh about the preprocessor.

And I am restarting my work on simple Oberon-like language.

I was just talking about the token portion of the Coco/R syntax rules, 
as opposed to the parse rule portion. But agreed, any Oberon-like 
language will have a simpler syntax than any REBOL-like language, 
and will have been designed with LL(1 or so) in mind.

paths seems to be tokens, too

Paths are made up of tokens, yes, just like the other block types 
except with more restrictions about what types of data may go in 
the paths. I don't (currently) know what subset of path syntax and 
semantics that Red/System supports though.

In some other REBOL-like languages there are some inherent conflicts 
between some path element types (notably dates) and the path separator 
/ itself, plus the final : on a set-path is considered part of the 
path instead of being a set-word element contained in the path, and 
the same for a leading : in a get-path not being part of a get-word 
first element (in R3). There's a fairly well-defined set of precedence 
rules, but for REBOL-like languages other than Red those rules are 
not very well documented, and they can therefore sometimes vary from 
language to language.