[bug?] Problem with 'load
[1/23] from: SunandaDH:aol at: 27-Nov-2003 16:49
Is this a bug, or am I just being thick?
I just want to evaluate a header, but REBOL won't let me even load it if the
needs aren't met:
load/all/header "rebol [needs: [1.2.1 view]]"
will work if you are running view.
But ask for a version you definitely don't have:
>> load/all/header "rebol [needs: [1.2.1 xxxx]]"
** Script Error: This script needs xxxx or better to function correctly
** Near: load/all/header "rebol [needs: [1.2.1 xxxx]]"
Load/all is not meant to evaluate, so I reckon it's a bug. It's certainly a
problem.
Thanks to Chris RG for spotting this while trying to contribute a script to
the Script Library....We try to load the header for validation, but we're
running Core so the problem was reported as an "unable to load" error, leading to
some mutual head-scratching.
Sunanda.
[2/23] from: maximo:meteorstudios at: 27-Nov-2003 17:24
I didn't know the needs: attribute was actualy evaluated.
actually, I barely even knew that was even significant...
I just looked at the online docs and the word is referenced 3 times, where they explain
the script header... but don't explain the dialect anywhere!
is this dialect actualy usable as a function given a block?
-MAx
---
You can either be part of the problem or part of the solution, but in the end, being
part of the problem is much more fun.
[3/23] from: antonr:iinet:au at: 28-Nov-2003 14:22
I think words in the needs block are just compared to:
extract system/components 3
In my tests it looks like the needs block isn't evaluated,
just examined.
Anton.
[4/23] from: g:santilli:tiscalinet:it at: 28-Nov-2003 10:03
Hi SunandaDH,
On Thursday, November 27, 2003, 10:49:49 PM, you wrote:
>>> load/all/header "rebol [needs: [1.2.1 xxxx]]"
Sac> ** Script Error: This script needs xxxx or better to function correctly
Sac> ** Near: load/all/header "rebol [needs: [1.2.1 xxxx]]"
Looks like a bug to me...
Regards,
Gabriele.
--
Gabriele Santilli <[g--santilli--tiscalinet--it]> -- REBOL Programmer
Amiga Group Italia sez. L'Aquila --- SOON: http://www.rebol.it/
[5/23] from: SunandaDH:aol at: 28-Nov-2003 4:07
Thanks Anton and Max
Anton:
> In my tests it looks like the needs block isn't evaluated,
> just examined.
We might be seeing different behaviour on different versions of REBOL -- I'm
looking at Win and Unix. It looks to me like the header is *always*
evaluated, including the needs check, despite the /all refinement. Try this
===save next lines as temp.r===
A preamble: there is text before header, containing "bad things
More text before rebol [print "gets printed if this is taken as the header"]
rebol this isn't a header despite starting with the magic word
rebol [print "gets printed if header is evaluated" file: %fff needs: [1.2.1
view]]
print "gets printed if script is executed"
=====end of test script=====
This is what I see:
>> load %temp.r ;; evaluates header; returns body
>> load/all %temp.r ;; crashes because preamble is not valid REBOL
words
>> load/header %temp.r ;; evaluates header; returns header
>> load/header/all %temp.r ;; evaluates header; returns header
What I want is a way to load the header of *any* REBOL script regardless of
its needs setting. Any one any ideas?
Max:
>is this dialect actualy usable as a function given a block?
Looks like you can do a lot with it. I guess part of the original idea is
that a script can load another's header and thus learn a lot about it. You could
use that to build some sort of documentation tool.
Or if I wrote something that alters a script (like a pretty printer) I could
use the header to give it directives:
rebol [
prettyprint [maxwidth: 50 comments: 'keep blanklines: 'drop]
]
Trouble is. my prettyprinter would seem to be only able to load scripts that
run under the same version that it does....Annoying or what?
Sunanda.
[6/23] from: g:santilli:tiscalinet:it at: 28-Nov-2003 10:26
Hi SunandaDH,
On Friday, November 28, 2003, 10:07:23 AM, you wrote:
Sac> We might be seeing different behaviour on different versions of REBOL -- I'm
Sac> looking at Win and Unix. It looks to me like the header is *always*
Sac> evaluated, including the needs check, despite the /all refinement. Try this
More recent versions of REBOL use CONSTRUCT on the header instead
of MAKE OBJECT!, just to avoid evaluation.
Regards,
Gabriele.
--
Gabriele Santilli <[g--santilli--tiscalinet--it]> -- REBOL Programmer
Amiga Group Italia sez. L'Aquila --- SOON: http://www.rebol.it/
[7/23] from: SunandaDH:aol at: 28-Nov-2003 4:31
Thanks Gabriele
> Looks like a bug to me...
I'll report it to feedback.
My last email should have said:
Trouble is. my prettyprinter would seem to be only able to load scripts that
run under the same version that it does if the target script has a needs
header entry....Annoying or what?
Sunanda
[8/23] from: rebolview:y:ahoo at: 28-Nov-2003 11:29
I use this on View 1.2.1 maybe could help you.
REBOL []
script: get in context
[script: func
["^/ "
File
/header
/content
/compress
/decompress
/local
script
]
[script: system/words/read File
script: load/all script? script
while [block? first script] [script: first script]
script: context
[header: second script
content: copy at script 3
]
if binary? first script/content
[script/content: load system/words/decompress first script/content]
switch/default true reduce
[header [script/header]
content [script/content]
decompress [save/header File script/content script/header]
compress
[save/header File
system/words/compress mold/only script/content script/header
]
] [script]
]
] 'script
-yos
[9/23] from: SunandaDH:aol at: 28-Nov-2003 6:25
Thanks Yos,
> I use this on View 1.2.1 maybe could help you
Unfortunately, it fails if the preamble (stuff before the REBOL header)
contains invalid words, If you try it on the test script below, you'll see what I
mean:
===save next lines as temp.r===
A preamble: there is text before header, containing "bad things
More text before rebol [print "gets printed if this is taken as the header"]
rebol this isn't a header despite starting with the magic word
rebol [print "gets printed if header is evaluated" file: %fff needs: [1.2.1
xxxx]]
print "gets printed if script is executed"
=====end of test script=====
I was trying to think of a way of safely doing this myself, so your script
has given me hope that it is possible....We just need to solve the bad preamble
problem,
Thanks again,
Sunanda.
[10/23] from: brett:codeconscious at: 28-Nov-2003 23:26
> I was trying to think of a way of safely doing this myself, so your script
> has given me hope that it is possible....We just need to solve the bad
preamble
> problem,
It would be better to have it fixed, but if you need the workaround I'm
pretty sure it could be done with Parse and using a parse-rule that uses
load/next (like the cookbook example to parse code). The trouble is knowing
accurately what the rules are well enough to mimic the logic of REBOL when
it validates a script.
For example a valid file could have a preamble as you have shown, but it
could also have an embedded script like:
blah blah
[ REBOL [Title: "Sample"] print now]
blah blahblah blah
blah blah
Strangely enough, the Script? function that Yos pointed out seems to give a
useful result in this case - the beginning of the embedded script, but not
when the script is not embedded. Odd. Anyway, at least it will tell you if
there is no point continuing :-)
I think they are the two cases.
If it is not embedded, it may have a preamble, then the rest is script. If
it has an embedded script, then the script? function tells you were it is
and you ignore the stuff coming after it. The embedded case should be easier
to validate, the non-embedded case (having seen your example) means you
probably need to be able to do a successful Construct on what you think is
the header before you can call it the header.
That's my guess - would be nice not to have to guess though.
Regards,
Brett.
[11/23] from: rebolview:y:ahoo at: 28-Nov-2003 15:28
Thanks Brett for explanations.
I think it is simpler for Sunanda to work with embedded script with
the script function i give you than parsing for header and content
information like script? do (script? is a native and we can't see the source).
So we add [ at start and ] at end of temp.r example script:
===save next lines as temp.r===
A preamble: there is text before header, containing "bad things
More text before rebol [print "gets printed if this is taken as the header"]
rebol this isn't a header despite starting with the magic word
[ rebol [print "gets printed if header is evaluated" file: %fff needs: [1.2.1
xxxx]]
print "gets printed if script is executed" ]
=====end of test script=====
>> probe script %temp.r
make object! [
header: [print "gets printed if header is evaluated" file: %fff needs: [1.2.1
xxxx]]
content: [
print "gets printed if script is executed"]
]
-yos
[12/23] from: rotenca:telvia:it at: 28-Nov-2003 15:33
Hi Brett,
> blah blah
> [ REBOL [Title: "Sample"] print now]
<<quoted lines omitted: 4>>
> when the script is not embedded. Odd. Anyway, at least it will tell you if
> there is no point continuing :-)
Can you do an example? Here it works (1.2.10.3.1).
>From my tests, the header is not seen only if
before REBOL [] there is some no space chars
AND
there is not a newline between the chars and REBOL []
The only difference i see is this:
script? "aa^/rebol[]" ;== "aa^/rebol[]"
script? "aa^/[rebol[]" ;== "[rebol[]"
This could be a bug.
Note also that in the embedded case the final ] can be omitted.
So i think that this is enough:
load-header: func [
"Load and construct a Rebol header object"
str [file! url! string!]
][
if str: script? str [
attempt [
construct/with first load/next find find str "rebol" "["
system/standard/script
]
]
]
probe load-header "# aa^/rebol[needs: [1]]"
The function return none if the header block is bad formed, like in case like
these:
rebol [
Note that script? returns a positive answer in these cases:
script? "rebol ["
so we need the attempt in the function.
---
Ciao
Romano
[13/23] from: SunandaDH:aol at: 28-Nov-2003 13:50
Thanks to everyone who contributed responses.
I've now updated REBOL.org to use Romano's load-header script rather than a
direct load/all/header in all the places (lots!) we load a header to get values
about a script.
This was a potentially damaging integrity exposure (my old-timer IBMese for
hackable security flaw) in the Library.....
.....Any one could have inserted active code in a header and contributed it
as a script. That code would have been executed on the server when we did the
load/header. The result could have been nasty.
This remains a potential danger for any REBOL-based CGI site that loads
headers of contributed code in the old manner.....If you have such a site, please
check your code and see if you need to replace load/header with Romano's script.
I'm impressed (as usual) with the usefulness of this ML and the helpfulness
of the people on it. Less than 24 hours from reporting a problem to having a
fix available. Outstanding, everyone!!
Thanks everyone again!
Sunanda.
[14/23] from: maximo:meteorstudios at: 28-Nov-2003 14:11
damn I was about to show you a way of doing it with encompass... :-)
I wish I could rebol full time.
That is its main purpose... fixing/improving/limiting things around... while still keep
the basic functionality.
-MAx
---
You can either be part of the problem or part of the solution, but in the end, being
part of the problem is much more fun.
[15/23] from: nitsch-lists:netcologne at: 28-Nov-2003 23:51
Am Freitag, 28. November 2003 19:50 schrieb [SunandaDH--aol--com]:
> Thanks to everyone who contributed responses.
>
> I've now updated REBOL.org to use Romano's load-header script rather than a
> direct load/all/header in all the places (lots!) we load a header to get
> values about a script.
>
> This was a potentially damaging integrity exposure (my old-timer IBMese for
> hackable security flaw) in the Library.....
>
If we use /core 2.5.6, then not.
this is oold /view 1.2.1:
>> load "rebol[print {hehe}]"
hehe
== [
]
and this core 2.5.6:
>> load "rebol[print {hehe}]"
== [
]
IIRC its mentioned somewhere in the /core change-log.
> .....Any one could have inserted active code in a header and contributed it
> as a script. That code would have been executed on the server when we did
<<quoted lines omitted: 8>>
> Thanks everyone again!
> Sunanda.
-Volker
[16/23] from: SunandaDH:aol at: 29-Nov-2003 6:08
Thanks Volker:
> If we use /core 2.5.6, then not.
> this is oold /view 1.2.1:
<<quoted lines omitted: 7>>
> ]
> IIRC its mentioned somewhere in the /core change-log.
Yep -- RT got there first!
http://www.rebol.com/docs/changes.html#section-4.2
That fixes the security problem.....But it doesn't fix the
evaluation/checking of the 'needs word -- so Core can't load a REBOL script that needs
View:
>> print system/version
2.5.6.3.1
>> load "rebol [print 'hi needs: [1.2.1 view]]"
** Script Error: This script needs view or better to function correctly
** Near: load "rebol [print 'hi needs: [1.2.1 view]]
Sunanda.
Sunanda.
[17/23] from: SunandaDH::aol::com at: 29-Nov-2003 6:13
Max:
> damn I was about to show you a way of doing it with encompass... :-)
> That is its main purpose... fixing/improving/limiting things around...
while
> still keep the basic functionality.
I'd appreciate seeing some more examples of what encompass can do -- it's
often hard to see all the possibilities of a new tool without being spoon-fed
some inspiration first.
If you have some examples of using encompass to add goodies, it might be
worth adding them to the documentation in the Library.
> I wish I could rebol full time.
>
Ah, me too!
Sunanda.
[18/23] from: moliad:aei:ca at: 29-Nov-2003 12:05
Sunanda et al.
thanks for asking me to post an example of encompass in action. I, myself, was looking
for meaningfull examples, and its fun to have one in the context of a real problem.
here is an example of how to patch load so that the /all refinement works as expected.
I have used the method of Romano (hope you don't mind... I take no credit for that :-)
to safely load the header (with a tiny improvement which allows me to find the 'rebol
word even if its not at the begining.
rload: :load
load: encompass/pre 'rload [
; check if header refinement was used
if header [
; header and library modes are not compatible (but could be).
if not library [
; the /all refinement is the ONLY case where the header should not be read
if all [
; source is defined in load, so it will have the same meaning here
if string! <> type? source [
source: read source
]
; at this point we ARE SURE TO HAVE TEXT!
; source: a
if source: script? find source "rebol" [
attempt [source: construct/with first load/next
find source "[" system/standard/script]
]
return source ; this will actually shortcut the load call.
]
]
]
]
script: {rebol [author: "someone" needs: [view 2.0.0]] gkjnfdkljhn dfihroih ro[h rtoi
}
; --- THIS WILL WORK !!! ---
probe load/header/all script
; --- THIS WILL FAIL (which is expected) !!! ---
probe load/header script
Hope this helps.
-MAx
[19/23] from: SunandaDH:aol at: 4-Dec-2003 3:39
More problems with load.....
This may not be a bug, but I can't find a workaround.
I want to be able to safely load *any* script to check it has acceptable
syntax before allowing it into REBOL.org. But we absolutely don't want to evaluate
the header as that may execute untrusted code in server CGI mode.
But here I have a very simple, three-line script that defeats the existing
checks.
,
REBOL [print "header evaluated"]
print "body evaluated"
We know 'load is bad as it evaluates the header:
>> load {,^/REBOL [print "header evaluated"]^/print "body evaluated"}
header evaluated
== [
print "body evaluated"
]
So we are supposed to use 'load/all. But in this case:
>> load/all {,^/REBOL [print "header evaluated"]^/print "body evaluated"}
** Syntax Error: Invalid word -- ,
** Near: (line 1) ,
I can successfully and safely load the header using Romano's load-header
function:
http://www.rebol.org/cgi-bin/cgiwrap/rebol/view-script.r?script=load-header.r
But this problem looks like it needs a different approach.
I could try saving it as a file and Do'ing it. The "proves" it is a
validly-formed script. But it evaluates the header and the body. Not what I want!
>> write %temp.r {,^/REBOL [print "header evaluated"]^/print "body evaluated"}
>> do %temp.r
header evaluated
body evaluated
Any ideas?
Thanks,
Sunanda.
[20/23] from: brett:codeconscious at: 4-Dec-2003 23:52
Hi Sunanda,
> We know 'load is bad as it evaluates the header:
>
Not true - Core 2.5.2 introduced a safer LOAD. Please see:
http://www.rebol.com/docs/changes.html#section-4.2
> But this problem looks like it needs a different approach.
...
> Any ideas?
I might have missed a message on this, but is there any reason why you
cannot use an upgraded version of REBOL?
Regards,
Brett.
[21/23] from: SunandaDH:aol at: 4-Dec-2003 9:03
Hi Brett,
> I might have missed a message on this, but is there any reason why you
> cannot use an upgraded version of REBOL?
Thanks for the reply.
My mistake.....I got too many version of REBOL around, and I did all my
pre-post testing on an older version.
Load does now perform as you say, so that eliminates that problem.
Just leaves me this problem -- I still want to load any abritrary script to
check that it is valid.
'script? alone is not enough to check that it is valid
But 'load will return an error if their is a 'needs in the header that
doesn't match the interpreter running:
troublesome-script: {
,
REBOL [needs: [9.9.9 xxx]]
}
script? troublesome-script ;; finds a valid header : correct!
load-header troublesome-script ;; Romano's script -- finds a valid header:
correct!
load troublesome-script ;; fails due to the 'need
** Script Error: This script needs xxx or better to function correctly
** Near: load troublesome-script
load/all troublesome-script ;; fails due to the preamble
** Syntax Error: Invalid word -- ,
** Near: (line 2) ,
Looks like I am going to have to do a 'load and then trap and accept the
'needs error.
A more elegant solution would be welcome,
Thanks,
Sunanda.
[22/23] from: rotenca:telvia:it at: 4-Dec-2003 16:24
Hi Sunanda,
> troublesome-script: {
> ,
<<quoted lines omitted: 12>>
> 'needs error.
> A more elegant solution would be welcome,
troublesome-script: {
,
REBOL [needs: [9.9.9 xxx]]
}
load-all-script: func [s][
if s: script? s [
load/all s
]
]
load-all-script troublesome-script
---
Ciao
Romano
[23/23] from: SunandaDH:aol at: 4-Dec-2003 17:06
Romano:
> troublesome-script: {
> ,
<<quoted lines omitted: 5>>
> ]
> ]
That's ingenious!!
Thanks a lot,
Sunanda.
Notes
- Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted