http scheme to support cookies
[1/15] from: thomas::cr::gmail::com at: 16-Apr-2005 2:14
has anyone looked into adapting the default http scheme to support
cookies (ie. be able to set them and/or remember them across reads)?
tc
[2/15] from: SunandaDH::aol::com at: 16-Apr-2005 6:15
Thomas:
> has anyone looked into adapting the default http scheme to support
> cookies (ie. be able to set them and/or remember them across reads)?
There are several scripts in the Library that do things with cookies -- do a
search on www.rebol.org for details.
Cookies are fairly easy to handle anyway -- could you say what you see the
advantages would be to extending the scheme?
Basic cookie handling:
To set a session cookie -- just print it as part of the HTTP headers:
my-cookie: "name=data"
print join "set-cookie: " my-cookie
To set a longer cookie -- just add an expiry date:
my-cookie: "name=data ; expires=" to-idate now + 31 ;; 31 days
print join "set-cookie: " my-cookie
To access a cookie:
my-cookie: select system/options/cgi/other-headers "HTTP_COOKIE"
Things are slightly more complicated if you have multiple cookies. Some code
assistance may be useful there.
Sunanda.
[3/15] from: thomas::cr::gmail::com at: 16-Apr-2005 14:17
it might not have been clear based on the end of your reply, what i'm
interested in is handling cookies on the client side, not in a cgi.
> > has anyone looked into adapting the default http scheme to support
> > cookies (ie. be able to set them and/or remember them across reads)?
>
> There are several scripts in the Library that do things with cookies -- do a
> search on www.rebol.org for details.
yes in fact, i've tried them (cookies-client.r and http-tools.r) but
couldn't get them to work for the site i tried (nytimes).
> Cookies are fairly easy to handle anyway
kind of, in the same way that the rest of http is... if you look at
the scripts you talk about, they're still 250 lines of code.
> could you say what you see the
> advantages would be to extending the scheme?
i didn't really ask myself the question in these terms...
disclaimer: i have never tried to write a scheme so i might be totally wrong!
rebol deals with network protocols using what it call schemes. and it
was designed so that people could add and extend these schemes. the
advantage of doing this is that each protocol deals with its details
within the scheme but uses a standard interface with the rest of
rebol. which is why you can use the same 'read' with pop, http, ftp...
this is a _good_ thing. it makes things more elegant. code is easier
to read and write for users, functionnalities and new protocols are
easier implement...
i don't have anything against the authors of those scripts, they
probably didn't know about schemes or were impressed by them (as you
seem to be). what they ended up doing doesn't seem to me to correspond
to how rebol was intended to be used. it is quite confusing to use for
users and i bet their implementation has been more complicated than if
they had used a scheme.
the scripts in question basically hacked functionnality on top of
rebol instead of doing it within rebol. they use weird global
variables called "cookie-data" and "cookie-data2" instead of the
common datastructure for ports and protocols, they use weird function
interfaces such as "tmp: HTTP-TOOLS http://www.rebol.com []" (to fetch
a page which sends you a cookie) instead of the familiar read...
tc
[4/15] from: gabriele::colellachiara::com at: 16-Apr-2005 16:39
Hi Thomas,
On Saturday, April 16, 2005, 2:17:14 PM, you wrote:
TC> the scripts in question basically hacked functionnality on top of
TC> rebol instead of doing it within rebol. they use weird global
TC> variables called "cookie-data" and "cookie-data2" instead of the
TC> common datastructure for ports and protocols, they use weird function
TC> interfaces such as "tmp: HTTP-TOOLS http://www.rebol.com []" (to fetch
TC> a page which sends you a cookie) instead of the familiar read...
I agree, however I'm not sure about making REBOL deal with cookies
automatically
.
Anyway, the functionality is almost already there.
>> p: open http://www.qtask.com/
connecting to: www.qtask.com
>> probe p/locals
make object! [
list: []
headers:
make object! [
Date: "Sat, 16 Apr 2005 14:28:09 GMT"
Server: "Apache/2.0.52 (Fedora)"
Last-Modified: none
Accept-Ranges: none
Content-Encoding: none
Content-Type: "text/html; charset=iso-8859-1"
Content-Length: none
Location: none
Expires: "Sat, 16 Apr 2005 14:28:09 GMT"
Referer: none
Connection: "close"
Authorization: none
Set-Cookie: "previousurl=http%3A%2F%2Fwww.qtask.com%2Findex.cgi"
Vary: "Accept-Encoding"
Cache-Control: "max-age=0, must-revalidate"
]
]
You can get the cookie from p/locals/headers/Set-Cookie, except
that PARSE-HEADER has problems with repeated headers and you only
get to see the last one in the object. A quick hack migth be to
read directly header-rules/head-list which has all of them.
You can send cookies back using the /CUSTOM refinement. Here the
problem of using repeated headers is harder to workaround, and you
probably need to poke into the handler code to make it work;
anyway, it's something like:
read/custom http://somesite/ [
header [Cookie: "my-cookie=something"]
]
I think that some of these problems have been fixed during the
View 1.3 project.
Regards,
Gabriele.
--
Gabriele Santilli <[g--santilli--tiscalinet--it]> -- REBOL Programmer
Amiga Group Italia sez. L'Aquila --- SOON: http://www.rebol.it/
[5/15] from: gchiu::compkarori::co::nz at: 18-Apr-2005 19:39
>i don't have anything against the authors of those scripts, they
>probably didn't know about schemes or were impressed by them (as you
You're correct on this. The cookies-client script was one I hacked in 1999 for a specific
reason .. to grab and send cookies to mtnsms.com, and enable a web to sms gateway ..
which worked until mtnsms.com killed the free sms. The script was not really intended
for public consumption.
The first article that was even vaguely official on schemes from rebol.com came out of
Jeff Kreis in 2001.
>variables called "cookie-data" and "cookie-data2" instead of the
Yep, I only needed to capture two cookies.
>common datastructure for ports and protocols, they use weird function
>interfaces such as "tmp: HTTP-TOOLS http://www.rebol.com []" (to fetch
>a page which sends you a cookie) instead of the familiar read...
>
Yes again. But it does indeed capture the nytimes.com cookie for me .. I just tried
it.
>has anyone looked into adapting the default http scheme to support
>cookies (ie. be able to set them and/or remember them across reads)?
And yes again. Turned out to be quite straight forward to do for both http and https.
--
Graham Chiu
http://www.compkarori.com/cerebrus
http://www.compkarori.com/rebolml
[6/15] from: SunandaDH::aol::com at: 18-Apr-2005 9:07
Thomas:
> i don't have anything against the authors of those scripts, they
> probably didn't know about schemes or were impressed by them (as you
> seem to be). what they ended up doing doesn't seem to me to correspond
> to how rebol was intended to be used. it is quite confusing to use for
> users and i bet their implementation has been more complicated than if
> they had used a scheme.
I think that's a little unfair, even if true. It's a bit like blaming
pioneers for not having used the freeways their grandchildren built.
REBOL has a deep (as opposed to steep) learning curve. You can hack things
at the surface level (I know I do) and you can delve deep into its elegance.
But delving deep takes time.
Many early examples of code show people on that learning curve. I know, just
from looking at the code I've written over the past three years, I can see
archaeological layers in the styles and approaches I've taken.
What would be good in the Library is some "state of the art" scripts to
replace some of the pioneering efforts. The best way to accomplish that is
contribute such scripts: do what the pioneers did; it's the way we can all make
progress.
Sunanda.
[7/15] from: thomas:cr:gmai:l at: 18-Apr-2005 15:59
> Yes again. But it does indeed capture the nytimes.com cookie for me ..
> I just tried it.
hmm, weird. you were able to access pages which require login? it did
store a couple cookies for me but i wasn't able to reuse them to
access pages. i didn't spend much time on it but it might have to do
with the fact that nytimes uses half a dozen different cookies. i'll
look more into it...
> >has anyone looked into adapting the default http scheme to support
> >cookies (ie. be able to set them and/or remember them across reads)?
>
> And yes again. Turned out to be quite straight forward to do for both http and
> https.
what was the functionnality you added? did you replace them or did you
add them as another scheme? would you mind posting them? (I won't
complain about it not being a nice clean and perfect library ;) please
see reply to next post btw)
tc
[8/15] from: thomas::cr::gmail::com at: 18-Apr-2005 16:24
> > i don't have anything against the authors of those scripts, they
> > probably didn't know about schemes or were impressed by them (as you
<<quoted lines omitted: 4>>
> I think that's a little unfair, even if true. It's a bit like blaming
> pioneers for not having used the freeways their grandchildren built.
i apologize for sounding for sounding like complaining. i did not mean
to complain about the contributed code and i'm definitely thankful to
graham (and others) for those examples which will be of great help for
what i need to do. i hope he/you didn't interpret my original mail
this way (didn't seem like it).
i just tried to answer to your question: "it's easy to hack, there's
code that does it, why would you want to bother doing it with
schemes?"
> What would be good in the Library is some "state of the art" scripts to
> replace some of the pioneering efforts.
exactly, which is why i was curious about people who had experience
with schemes... which seemed to me like the correct, long term way of
dealing with something which is now-a-days used extensively all over
the web (ie. cookies)
tc
[9/15] from: gchiu::compkarori::co::nz at: 19-Apr-2005 8:33
tc wrote.. apparently on 18-Apr-2005/15:59:43+2:00
>> Yes again. But it does indeed capture the nytimes.com cookie for me ..
>> I just tried it.
<<quoted lines omitted: 3>>
>with the fact that nytimes uses half a dozen different cookies. i'll
>look more into it...
Ah, you didn't state that you wanted to login .. I don't have a nytimes account so didn't
try.
And I've noticed that there are quite a few websites that are built to deliberately make
it hard to scrape the data .. using javascript etc to generate values which are needed
to be posted into forms.
>> And yes again. Turned out to be quite straight forward to do for both http and
>> https.
>
>what was the functionnality you added? did you replace them or did you
>add them as another scheme? would you mind posting them? (I won't
I added to it.
It was a few years ago now, but I recall that I created a global cookie-jar object, and
added cookies to it that were encountered using the http scheme. And when sites were
being accessed that had cookies in the jar, I sent them. Some sites used up to 6 cookies,
and it seemed to handle those okay, as well as handling redirections. It wasn't perfect,
but it did the job I needed to do.
As for posting, I'll have to discuss with Carl. Posting the schemes, modified or not,
seems to be a breach of the sdk license.
--
Graham Chiu
http://www.compkarori.com/cerebrus
http://www.compkarori.com/rebolml
[10/15] from: thomas:cr:gm:ail at: 18-Apr-2005 23:02
> As for posting, I'll have to discuss with Carl. Posting the schemes, modified
> or not, seems to be a breach of the sdk license.
really? hmm... had almost forgotten that sdk licenses still existed in
the "free" world we now live in ;)
what you did sounds a lot like what i had in mind! (except i had
forgotten about redirections)
correct me if i'm wrong but from what i saw (thx for the reference of
the Jeff Kreis article btw) i wont need to pay/sign any sdk license to
implement this correct? i can just reuse public code from the http
scheme and the net-install function, no?
tc
[11/15] from: andreas::bolka::gmx::net at: 19-Apr-2005 0:29
Monday, April 18, 2005, 10:33:37 PM, Graham wrote:
> As for posting, I'll have to discuss with Carl. Posting the schemes,
> modified or not, seems to be a breach of the sdk license.
even though they are available under system/schemes/.. in standard
REBOL/Core ?
--
Best regards,
Andreas
[12/15] from: gchiu::compkarori::co::nz at: 19-Apr-2005 13:10
Andreas Bolka wrote.. apparently on 19-Apr-2005/0:29:19+2:00
>Monday, April 18, 2005, 10:33:37 PM, Graham wrote:
>
>> As for posting, I'll have to discuss with Carl. Posting the schemes,
>> modified or not, seems to be a breach of the sdk license.
>
>even though they are available under system/schemes/.. in standard
>REBOL/Core ?
Correct. Here's the header from prot-http.r
REBOL [
Title: "REBOL Protocols: HTTP"
Author: "Sterling Newton"
Rights: "Copyright REBOL Technologies 1997-2002"
Note: "Confidential and Proprietary. All rights reserved."
; CONFIDENTIAL AND PROPRIETARY OF REBOL TECHNOLOGIES, USA.
; This source code module shall only be used according to
; the terms and conditions of the REBOL/SDK license.
; This source code is not for distribution or publication.
; All changes or modififications must be submitted to REBOL.
]
I believe Carl is working on a new license for LNS .. maybe that will percolate back
to the sdk sources.
--
Graham Chiu
http://www.compkarori.com/cerebrus
http://www.compkarori.com/rebolml
[13/15] from: Izkata::Comcast::net at: 19-Apr-2005 6:15
> Andreas Bolka wrote.. apparently on 19-Apr-2005/0:29:19+2:00
>>Monday, April 18, 2005, 10:33:37 PM, Graham wrote:
<<quoted lines omitted: 9>>
> Author: "Sterling Newton"
> Rights: "Copyright REBOL Technologies 1997-2002"
Forgive my possible misconception, but doesn't the 2002, being
in the past, mean it's no longer valid?
I'm usually not interested in legal stuff, so I don't really know..
-Izzy Boy
[14/15] from: SunandaDH:aol at: 19-Apr-2005 8:54
Izzy:
> Forgive my possible misconception, but doesn't the 2002, being
> in the past, mean it's no longer valid?
No. Copyright law varies across the world, so how long a copyright remains in
force varies.
But if the principle you suggest were true, I'd be republishing all Harry
Potter novels today; the most recent one has a copyright date of 2003.
But let's hope that RT will forget copyright and just do what's right.
Sunanda
[15/15] from: Izkata:Comcast at: 19-Apr-2005 18:13
>> Forgive my possible misconception, but doesn't the 2002, being
>> in the past, mean it's no longer valid?
<<quoted lines omitted: 3>>
> But if the principle you suggest were true, I'd be republishing all Harry
> Potter novels today; the most recent one has a copyright date of 2003.
Ah. Not quite what I meant, but nice suggestion (^.-)
I meant when there's a range like that. For example, WinXP says something
like
(c)1985-2002 (I forget the exact year) And, apparently, people (Mostly
white-hat
hackers) have been releasing dummed-down versions of Windows on the
internet -
all the capabilities, nearly no extra features, so it runs much faster and
costs less/nothing.
> But let's hope that RT will forget copyright and just do what's right.
Who isn't? =^þ
> Sunanda
-Izzy Boy
Notes
- Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted