World: r3wp

while other operators work ok as lit-words:

>> '=
== =

Any use of those words conflicts with the tag! syntax. The trick 
is to set priorities.

No, the trick is to program the lexer correctly without bugs! ;-)

The lexer is where the priorities are set. What you are talking about 
isn't bugs, it is preferences (and good ones at that).

Non-configurable preferences, mind you. They have to be set in the 
source of the lexer.

Ok, I see it differently. If it's possible to make a lexer, that 
can cope with tags and lit-words and get-words like '< and :<, then 
I see it as bugs in the current implementation.

You could see it either way. Those "bugs" are backwards compatible 
with R2. However, this is clearly related to bug#666 :)

I prefer to think of them as justifiable improvements :)

We take Wish tickets as seriously as Bug tickets. Whether it is a 
Wish or a Bug doesn't affect prioritization of ticket handling.

Output from my work-in-progress lexer:

:>
Valid get-word: :>
'>
Valid lit-word: '>
<a>
Tag begin


Next it should get the whole tag and return that as a valid tag. 
I think, it's possible to make a lexer, that can handle all this 
correctly (as I see it).

It all sounds doable so far. Please remember #537 in your proposal.

Why would you write

a,


? Comma is not allowed in words. I wondered about this some months 
ago, like if Carl had some future idea for the use of comma!?

I don't want to allow , in words, I want to have the word be recognized 
and then have the lexer complain about the , next time.

Try not to break things like:
>> url? *::,
== true
(It's not always as easy as it looks :-)

Read the comments - this would be word-only. This is why I was so 
precise in the ticket, referring to TRANSCODE behavior.

Brian, ah ok.


As : is not allowed in words, that url example should be ok. But 
I'm wondering, why comma isn't allowed in words, as point is.

>> a.: 1
== 1
>> a,: 1
** Syntax error: invalid "word" -- "a,:"


I don't see the reason, other than if Carl has some special future 
idea for comma.

The comma is so bad syntactically that throwing an error every time 
someone tries to use it is considered a valuable feature.

Or maybe because comma is used in many languages to separate arguments, 
and by not allowing comma in words, REBOL might be easier to read 
for everyone.

Plus, having something as difficult-to-see as a comma in any syntax 
role is considered a bad thing. REBOL syntax didn't come out of nowhere 
- these choices were maade for good reasons. It's why we don't use 
periods as well.

If you get it, like to describe in #537, what should this return 
then?

transcode "a,0"

[a ",0"]

period is usable

and

reduce [a,0]

Except in binary. TRANSCODE works on UTF-8 binaries now. I need to 
adjust that ticket accordingly.

Geomol, that reduce example uses TRANSCODE/all, not TRANSCODE.

ok, I'm not too familar with transcode. Let me make another example. 
This works today:

>> load "a ,0"
== [a 0.0]

What should this return:

load "a,0"

Sounds like it should return [a 0.0].

Steeve, a period is usable, but not used (in general). And the period 
being usable is likely why the comma is an error.

Today it's an invalid word. You suggestion would turn something invalid 
into being valid.

comma can be _used_ in words, but not in words that have to be serialised 
and then reloaded

    to-word "a,"    ;; this works  
    == a,

    to-word ",a"   ;; there are some limits
    ** Syntax error: invalid character in: ",a"

Don't serialise and reload O:
    o: make object! reduce [to-set-word "a," 1]
    == make object! [
        a,: 1
    ]

lol, good ones, Sunanda!

Sunanda, TO-WORD "a," being allowed sounds like a bug. Report it.

It's the general thing, we've talked a bit about before, that TO 
some datatype should put the result through LOAD or something, so 
valid REBOL comes out of it.

But it's R2 compatible :)

There are other edge cases -- Latin-1 chars that can be _in_ a word 
not not _start_ them, and do not serialise well.....I did a script 
and found them all once

not not ==> but not

We've already put restrictions on the character set of words - Sunanda 
just found a hole in those that needs repairing.

But it's R2 compatible :)
 - See bug#666 :)

Some languages only allow 7-bit ascii in the source except for strings.

re: 666 -- I'll dig that script out and run it tomorrow :)

I could use 8-bit danish letters in my REBOL source, if I would:

>> æble: "apple"
== "apple"


But I don't do it. I'm not sure, if it's a good idea to allow this. 
I guess, 8-bit values are allowed, because it makes the lexer faster.

All standard functions and syntax in REBOL fit within 7-bit ASCII, 
which is why R3 source is UTF-8.

UTF-8 encoded binary!

Adjusted #537 to reflect the post-alpha-39 behavior of TRANSCODE 
(can't believe I forgot to do so before now).

Geomol, please take another look...

Sunanda, #1167 created for that to-word "a," error.

{Is there a problem with getting operators?

>> get to word! "="
** Script error: = word is not bound to a context}


This is not operator-specific, no variable can be handled like that:

a: 1
get to word! "a"
** Script error: a word is not bound to a context

{>> get to-word '=
== op!}

- in the above code the usage of TO-WORD is just a no-op

{Brian knows the internals.} - see the http://www.rebol.net/wiki/Comparisons
article containing the general principles

{>> get to-word quote <>}

- again, TO-WORD is just no-op above

Knowing the internals isn't hard. I haven't seen the native source 
- I've just seen the mezzanine source, followed the conversations, 
and read sites like the onee Ladislav linked above. The rest is deduction.