World: r3wp

i gave the design, it's simple to ehance

what is good to return in that case ?

It depends on the rules chosen to interpret the string content. That's 
the point, there's no simple way to implement a general case load-foreign 
that would return a block! of mixed REBOL values and "foreign" values 
as string. It's not possible to define a general set of rules because 
"foreign" can be just anything. LOAD just returns an error! when 
the syntax is incorrect and that's the best thing it can do.

doc, never said it should accept anything...

just that I have seen it would work so many times as to not make 
it a useless feature.

its also a question of speed.  load is blighthingly fast.

If you can define a set of rules for your specific needs, then you 
can just use parse or Steeve's load-foreign for that (or ask someone 
to write it for you if you don't want to implement it). Everyone 
can have its own specific needs, I can't see how to define a set 
of rules for "foreign" data that would be or could be useful to most 
users. That would probably end up defining a few additionnal datatypes 
(or alternative syntax for existing datatypes).

Max, did you ever consider WHY load is so fast?

What makes you think that it would be so fast if it was able to do 
what you ask?

Because.  ;)   If REBOL had a foreign! datatype! could the lexer 
not just ... about to throw syntax error, make a foreign! and continue?


I still don't see why this is deemed a bad extension to REBOL.   
Sure it "stinks" to guru's but it opens doors to fishermen, contractors 
and linguists, imho.  And yes it could look like a load of gunk if 
a quote, paren, brace or bracket was misaligned.  So?


For code purity, I'm sure Steeve could spend a few minutes to write 
4 lines to scan a loaded series! and recursively highlight the foreign! 
data, providing a semblance of a lint procedure for those writing 
apps.  My push for foreign! has little to do with apps, and just 
about all to do with data, human real-world data.   For people using 
data for sums, control, etc... just use the lint scan to reject data 
blocks that have ANY foreign! substances??

Why not just load the bad data as a string! ?

should be simple to parse out string!s when a numeric datatype is 
expected...

Are you guys listening?

yep

I'm just a bit amazed at the lack of comprehension.  I agree with 
DocKimbel and I advise to think on his words longer.

what !?

I have never met with anything I could not handle. Gee, we are complaining 
about REBOL parser not being able to handle ANY format, while even 
lamers as me are being able to use 'parse in opposition to regexp? 
Why this possession? What is exact deal breaker? Isn't it a bit naive? 
Brian, really - what fishermen are you talking about? For anything 
more complicated than one-liner, you have to come up with script 
anyway. And if you store code into script, you can write few parse 
rules, no? Well, maybe it is me, who never uses load (I don't like 
the fact it mysteriosly does decoding of few things here or there 
- libraries, jpg, which is fixed in R3), but with REBOL string parsing, 
you can do many things. I don't even agree to Max opinion, that parse 
will let you down. Well, if you want to parse streamed mp3 binary 
content, maybe so, but for some general data format, specifically 
delimited? Come on :-)

Parse is my beloved function in Rebol, i use it all the tuime.

*time

btw - there is now new blog posted - encoders/decoders. We should 
better care to get those things done right. You can write your decode-anything 
codec to input your mysterious data :-)

eh ?

(i don't understand)

Even my compromise has been rejected as too slow. Use TRANSCODE/error 
in a loop if necessary, but otherwise come to accept that LOAD loads 
REBOL data, and whatever DECODE can load (which is *anything* in 
theory).

Anton ... it's not (I don't think, but I guess it could be) lack 
of comprehension.  It's a desire to see REBOL usefulness expand beyond 
the borders of us 200 loyal fans.
>> book: load %somefile
>> sort book
>> length? unique book

etc...   Can you not see a linguist loving this?  And perhaps adding 
to the size of the REBOL user base?
>> account: [paid $10,000.00 to bob]
A general contractor would love to type that in.

It just seems so close to being a thing that could be done without 
too much fuss or bother if it was internal.

Everyone here in REBOL3 ... we could make REBOL "more" restrictive 
and magic would still happen.  Carl and we advocates don't have to 
worry about you/us lot.  We're hooked already.


For instance.   What if    ``\\ was made a special! datatype?  Would 
you complain?  Or just accept it as a new feature of REBOL?   I don't 
see what a catch-all foreign! would harm.   The lexical complexity 
of REBOL is wonderous, I don't see what could hurt if it was more 
human friendly to boot.

Pekr ... I grew up a commercial fisherman.   ;)

As someone who writes some of the internals, I can tell you that 
internalizing the fuss and bother doesn't make it less bothersome. 
It's not magic: The overhead would be the same if we added it to 
TRANSCODE - we might as well write the REBOL syntax in PARSE.

Flexibility is slow. If you want to load another syntax, try something 
like this:
    install-codec fishing-jargon
    load %blah.fish

Theoretical functions of course, but LOAD will call DECODE, which 
will look for an installed codec that it passes the data to.

Before I read this I was going to add ...

Feeling a little bit responsible for being this argument to the rebol 
community, in a small attempt to advocate for more users ...  I'd 
also like to say, that push come to shove; if the argument gets to 
a point of driving away the likes of Nenad, or Gabriele in some sense 
of disgust at the un-understanding of some of us versus the potential 
of a few thousand new faceless strangers that may be attracted to 
a "garbage loading dialect" language.  I'd vote to keep Doc and Gab 
happy and skip the faceless masses.   ;)

Check out TRANSCODE/error.

But ... I still don't see the performance hit of a lexical scanner 
that defaulted to "make foreign!" and continue whenever a position 
in the code would trigger a syntax error?  Or as Anton states ... 
is this an amazing lack of comprehension?


I've been working with Bison and Flex ... and from what I understand, 
this would not make the parse tree any more complex.  It would simply 
change the runtime semantics to "make foreign!" at every point where 
the scanner would fail.

We want that failure in LOAD though - it's what makes it trustworthy 
and fast. Check out TRANSCODE/error, but keep in mind that code to 
cope with foreign data is much more complex than regular code. Beginners 
should stick to REBOL data.

Or to data loaded by codecs called by DECODE.

Again ... I'll side with Doc, push come to shove ... but ... it can't 
be more complex than throwing a runtime error in an action! when 
given invalid datatypes, no?   foreign! on action! or evaluation 
would trigger a throw.  The only operation I'd allow is  to string! 
 or a test such as foreign?   But I'll now try and hold my tongue 
for another 2 months.   ;)

In regards to my "Why not load as a string! ?" comment, I was thinking 
of some sort of load/safe refinement

LOAD is safe already. The fact that it throws an error on bad data 
is what makes it safe.

We changed it to be even more strict in R3 to make it even safer.

The "more strict" is making LOAD/header actually require a REBOL 
script, if not the header itself.

I haven't followed this thread

Ok, fine two minutes then.   ;)   "bad data" would be a new foreign! 
datatype.  No longer "bad".  It'd be no more a code hassle than doing 
email! + url!      Take the hit then.  But you still have a "valid 
REBOL series!" to play with.  Even with a $$10,000.00\\  floating 
around in the series ... semantically equivalent to email! or file! 
 out of context for those functions that require particular datums.

but usually you want to parse data knowing what datatypes you are 
expecting.

would it be possible to do a to-block on that data accepting only 
certain datatypes and anything else is coerced to string?

eg telephone numbers are recognized as invalid time values

>> to-block "  345-6789  "
** Syntax Error: Invalid date -- 345-6789
** Near: (line 1) 345-6789
>>

Foreign data will have codecs. No need to try to treat it like REBOL 
data. The foreign! type is the worst possible situation, since it 
would take the overhead of dealing with foreign data away from the 
actual input proces, and add that overhead to every bit of REBOL 
code that called LOAD, because LOAD would no longer be trustworthy.

or perhaps rebol by adding a telephone number data type

fix rebol

Right now you can count on LOAD returning REBOL data, or an error. 
This is what makes LOAD trustworthy enough for DO.

seems a significant omission ...

Really? Do you mean #1-773-555-1212 ?

how many times do you see phone numbers like that in real data ?