World: r3wp

I'm not sure whether not having a seperate tokenizer is a plus or 
a minus, though.

I guess you could think of block parsing as using load as a tokenizer.

IMHO that would add overhead for the simple things.

and you can use parse to tokenize first?

Two rounds of parsing, one for tokenizing and one to parse? Interesting. 
That would work if you don't have control over the source syntax 
- otherwise load works pretty well for simple languages.

Thats where i got the idea: tokenize first and use block-parser :)

I've been using that approach for XML processing.

sounds good. if one finds a good tokenized representation. I am not 
an xml-guru :(

My next personal project is to go through the XML/XSL/REST specs 
and create exactly that. I already have an efficient structure, I 
just need to fill out the semantics to support the complete logical 
model of XML.

I am also not an XML guru, but I will be by the time I'm done :)

After i read " go through the XML/XSL/REST specs" ithought soo. Beeing 
undecised ifiprefer to run away or participate curiously.

Well, I know enough to know where to look to figure out the rest.

Still, "run away" is a common and sensible reaction to XML.

*nod*

Later, I must run errands...

cu

I'm a bit stuck because this parse stop after the first iteration. 
 Can anyone give me a hint as to why it stops after one line.

Here is some code:

data: read to-file Readfile

print length? data
224921


d: parse/all data [thru QuoteStr copy Note to QuoteStr thru QuoteStr 
thru quotestr

    copy Category to QuoteStr thru QuoteStr thru quotestr copy Flag to 
    QuoteStr
    thru newline (print index? data)]
1
== false


Data contains hundreds of "memos" in a csv file with three fields: 

 Memo, Category and Flag ("0"|"1")  all fileds are enclosed in quotes 
 and separated by commas.
  

It would be real simple if the Memo field didn't contain double quoted 
words; then 
parse data none
would even work; but alas many memos contain other "words".
It would even be simple if the memos didn't contain commas, then
parse data "," or parse/all data ","
would work; but alas many memos contain commas in the body.

Does every field is quoted?

Gordon, can you post a copy of short lines of the data?

if QuoteStr = "\"", then this looks like it to me:
Note
, "Category", "Flag" 
Note
, "Category", "Flag"

But you don't have a loop or anything - try this:
d: parse/all data [
   some [

      thru QuoteStr copy Note to QuoteStr thru QuoteStr thru quotestr

      copy Category to QuoteStr thru QuoteStr thru quotestr copy Flag to 
      QuoteStr
      thru newline (print index? data)
   ]
]

James: Yes every field is quoted.

Izkata:  Sorry, I left that out.
QuoteStr: to-char 34
probe QuoteStr
==  #"^""

hm, I was thinking in C++.... very unusual for me lol

Do you need to loop?  I thought parse looped by itself
ie: data: parse data none

not as far as I know

This change in the parse looks like it works:

>> data: {"Note", "Category", "Flag"
{    "Note", "Category", "Flag"
{    "Note", "Category", "Flag"
{    "Note", "Category", "Flag"
{    }
== {"Note", "Category", "Flag"
Note
, "Category", "Flag"
Note
, "Category", "Flag"
Note
, "Category", "Flag"
}
>> QuoteStr: to-char 34
== #"^""
>> d: parse/all data [
[    some [

[        X: thru QuoteStr copy Note to QuoteStr thru QuoteStr thru 
quotestr

[        copy Category to QuoteStr thru QuoteStr thru quotestr copy 
Flag to QuoteStr
[        thru newline (print index? :X)
[        ]
[    ]
1
29
57
85
== true

Okay, trying it now.  I see that the phrase: "print index? data" 
stays stuck on "1".  


I see that you have posted a new example.  I'll try that.  Be right 
back.

I'm pretty sure that you are right in that I have to loop throught 
the "Data".  That was my big stumbling block and the rest is just 
logic to figure out.  Thanks a bunch.

No problem  (I'm glad I could actually help  '^^ )

In the phrase.  "Print index :x", what does putting a colon before 
a variable do again?

Oops I meant "Print index? :x"

Not sure - I remember seeing it in others' parse rules, so I just 
put it there and it worked  '^^
Take it out and see what happens lol

:)

I think it was like get-word or something

; Did you try this?
data: read/lines to-file Readfile
fields: [note category flag]
foreach x data [
    set fields parse x ","
    ; do something
]

In particular, remember not to use parse/all

>> parse {"Hello, World", "Blah"} ","
== ["Hello, World" "Blah"]

Hi BrianH;

  Yes I did try that and the problem was that even though I specified 
  the "," as the delimiter, it came across an embedded quote #"^"" 
  and split the input at the quote.  Rebol Shouldn't have split it 
  up that way, to my understanding.  I will post some simple data to 
  test.

Embedded quotes should be escaped somehow.

Please include some troublesome data if you could.

This data was exported by PalmOS.  I like the Palm desktop for keeping 
track on notes/,memos addresses but the  search engine sucks badly. 
 Therefore I wanted to export the data to allow a nice Rebol search 
on it..  Therefore, the PalmOS export function does "escape" an embedded 
quote by quoting it again.  Ex:
Press the "Home" button
becomes
Press the 
Home
 button.

truth (as far as i know) is:   word  is is a shortcut for :word  
but there are a few places such as inside parse where the shortcut 
does not work so you need to make it explicit

I will get some troubleshooting data posted in a minute.

Tomc:  Do I understand that :word would be like "get word" except 
in a parse sentence?

Wait I said that wrong

Tomc:  Do I understand that :word would be like "get word" and needed 
in a parse sentence but you can just use the shortcut 'word' most 
everywhere else?

The colon before the word prevents the interpreter from evaluating 
active values like functions and parens. It's a safety thing.

works for me

you can use :word everywhere you would use word

as far as i know

Except when you want an active value assigned to the word to be evaluated, 
like when you are calling a function.