Mailing List Archive: 49091 messages
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

http scheme to support cookies

 [1/15] from: thomas::cr::gmail::com at: 16-Apr-2005 2:14


has anyone looked into adapting the default http scheme to support cookies (ie. be able to set them and/or remember them across reads)? tc

 [2/15] from: SunandaDH::aol::com at: 16-Apr-2005 6:15


Thomas:
> has anyone looked into adapting the default http scheme to support > cookies (ie. be able to set them and/or remember them across reads)?
There are several scripts in the Library that do things with cookies -- do a search on www.rebol.org for details. Cookies are fairly easy to handle anyway -- could you say what you see the advantages would be to extending the scheme? Basic cookie handling: To set a session cookie -- just print it as part of the HTTP headers: my-cookie: "name=data" print join "set-cookie: " my-cookie To set a longer cookie -- just add an expiry date: my-cookie: "name=data ; expires=" to-idate now + 31 ;; 31 days print join "set-cookie: " my-cookie To access a cookie: my-cookie: select system/options/cgi/other-headers "HTTP_COOKIE" Things are slightly more complicated if you have multiple cookies. Some code assistance may be useful there. Sunanda.

 [3/15] from: thomas::cr::gmail::com at: 16-Apr-2005 14:17


it might not have been clear based on the end of your reply, what i'm interested in is handling cookies on the client side, not in a cgi.
> > has anyone looked into adapting the default http scheme to support > > cookies (ie. be able to set them and/or remember them across reads)? > > There are several scripts in the Library that do things with cookies -- do a > search on www.rebol.org for details.
yes in fact, i've tried them (cookies-client.r and http-tools.r) but couldn't get them to work for the site i tried (nytimes).
> Cookies are fairly easy to handle anyway
kind of, in the same way that the rest of http is... if you look at the scripts you talk about, they're still 250 lines of code.
> could you say what you see the > advantages would be to extending the scheme?
i didn't really ask myself the question in these terms... disclaimer: i have never tried to write a scheme so i might be totally wrong! rebol deals with network protocols using what it call schemes. and it was designed so that people could add and extend these schemes. the advantage of doing this is that each protocol deals with its details within the scheme but uses a standard interface with the rest of rebol. which is why you can use the same 'read' with pop, http, ftp... this is a _good_ thing. it makes things more elegant. code is easier to read and write for users, functionnalities and new protocols are easier implement... i don't have anything against the authors of those scripts, they probably didn't know about schemes or were impressed by them (as you seem to be). what they ended up doing doesn't seem to me to correspond to how rebol was intended to be used. it is quite confusing to use for users and i bet their implementation has been more complicated than if they had used a scheme. the scripts in question basically hacked functionnality on top of rebol instead of doing it within rebol. they use weird global variables called "cookie-data" and "cookie-data2" instead of the common datastructure for ports and protocols, they use weird function interfaces such as "tmp: HTTP-TOOLS http://www.rebol.com []" (to fetch a page which sends you a cookie) instead of the familiar read... tc

 [4/15] from: gabriele::colellachiara::com at: 16-Apr-2005 16:39


Hi Thomas, On Saturday, April 16, 2005, 2:17:14 PM, you wrote: TC> the scripts in question basically hacked functionnality on top of TC> rebol instead of doing it within rebol. they use weird global TC> variables called "cookie-data" and "cookie-data2" instead of the TC> common datastructure for ports and protocols, they use weird function TC> interfaces such as "tmp: HTTP-TOOLS http://www.rebol.com []" (to fetch TC> a page which sends you a cookie) instead of the familiar read... I agree, however I'm not sure about making REBOL deal with cookies automatically . Anyway, the functionality is almost already there.
>> p: open http://www.qtask.com/
connecting to: www.qtask.com
>> probe p/locals
make object! [ list: [] headers: make object! [ Date: "Sat, 16 Apr 2005 14:28:09 GMT" Server: "Apache/2.0.52 (Fedora)" Last-Modified: none Accept-Ranges: none Content-Encoding: none Content-Type: "text/html; charset=iso-8859-1" Content-Length: none Location: none Expires: "Sat, 16 Apr 2005 14:28:09 GMT" Referer: none Connection: "close" Authorization: none Set-Cookie: "previousurl=http%3A%2F%2Fwww.qtask.com%2Findex.cgi" Vary: "Accept-Encoding" Cache-Control: "max-age=0, must-revalidate" ] ] You can get the cookie from p/locals/headers/Set-Cookie, except that PARSE-HEADER has problems with repeated headers and you only get to see the last one in the object. A quick hack migth be to read directly header-rules/head-list which has all of them. You can send cookies back using the /CUSTOM refinement. Here the problem of using repeated headers is harder to workaround, and you probably need to poke into the handler code to make it work; anyway, it's something like: read/custom http://somesite/ [ header [Cookie: "my-cookie=something"] ] I think that some of these problems have been fixed during the View 1.3 project. Regards, Gabriele. -- Gabriele Santilli <[g--santilli--tiscalinet--it]> -- REBOL Programmer Amiga Group Italia sez. L'Aquila --- SOON: http://www.rebol.it/

 [5/15] from: gchiu::compkarori::co::nz at: 18-Apr-2005 19:39


>i don't have anything against the authors of those scripts, they >probably didn't know about schemes or were impressed by them (as you
You're correct on this. The cookies-client script was one I hacked in 1999 for a specific reason .. to grab and send cookies to mtnsms.com, and enable a web to sms gateway .. which worked until mtnsms.com killed the free sms. The script was not really intended for public consumption. The first article that was even vaguely official on schemes from rebol.com came out of Jeff Kreis in 2001.
>variables called "cookie-data" and "cookie-data2" instead of the
Yep, I only needed to capture two cookies.
>common datastructure for ports and protocols, they use weird function >interfaces such as "tmp: HTTP-TOOLS http://www.rebol.com []" (to fetch >a page which sends you a cookie) instead of the familiar read... >
Yes again. But it does indeed capture the nytimes.com cookie for me .. I just tried it.
>has anyone looked into adapting the default http scheme to support >cookies (ie. be able to set them and/or remember them across reads)?
And yes again. Turned out to be quite straight forward to do for both http and https. -- Graham Chiu http://www.compkarori.com/cerebrus http://www.compkarori.com/rebolml

 [6/15] from: SunandaDH::aol::com at: 18-Apr-2005 9:07


Thomas:
> i don't have anything against the authors of those scripts, they > probably didn't know about schemes or were impressed by them (as you > seem to be). what they ended up doing doesn't seem to me to correspond > to how rebol was intended to be used. it is quite confusing to use for > users and i bet their implementation has been more complicated than if > they had used a scheme.
I think that's a little unfair, even if true. It's a bit like blaming pioneers for not having used the freeways their grandchildren built. REBOL has a deep (as opposed to steep) learning curve. You can hack things at the surface level (I know I do) and you can delve deep into its elegance. But delving deep takes time. Many early examples of code show people on that learning curve. I know, just from looking at the code I've written over the past three years, I can see archaeological layers in the styles and approaches I've taken. What would be good in the Library is some "state of the art" scripts to replace some of the pioneering efforts. The best way to accomplish that is contribute such scripts: do what the pioneers did; it's the way we can all make progress. Sunanda.

 [7/15] from: thomas:cr:gmai:l at: 18-Apr-2005 15:59


> Yes again. But it does indeed capture the nytimes.com cookie for me .. > I just tried it.
hmm, weird. you were able to access pages which require login? it did store a couple cookies for me but i wasn't able to reuse them to access pages. i didn't spend much time on it but it might have to do with the fact that nytimes uses half a dozen different cookies. i'll look more into it...
> >has anyone looked into adapting the default http scheme to support > >cookies (ie. be able to set them and/or remember them across reads)? > > And yes again. Turned out to be quite straight forward to do for both http and > https.
what was the functionnality you added? did you replace them or did you add them as another scheme? would you mind posting them? (I won't complain about it not being a nice clean and perfect library ;) please see reply to next post btw) tc

 [8/15] from: thomas::cr::gmail::com at: 18-Apr-2005 16:24


> > i don't have anything against the authors of those scripts, they > > probably didn't know about schemes or were impressed by them (as you
<<quoted lines omitted: 4>>
> I think that's a little unfair, even if true. It's a bit like blaming > pioneers for not having used the freeways their grandchildren built.
i apologize for sounding for sounding like complaining. i did not mean to complain about the contributed code and i'm definitely thankful to graham (and others) for those examples which will be of great help for what i need to do. i hope he/you didn't interpret my original mail this way (didn't seem like it). i just tried to answer to your question: "it's easy to hack, there's code that does it, why would you want to bother doing it with schemes?"
> What would be good in the Library is some "state of the art" scripts to > replace some of the pioneering efforts.
exactly, which is why i was curious about people who had experience with schemes... which seemed to me like the correct, long term way of dealing with something which is now-a-days used extensively all over the web (ie. cookies) tc

 [9/15] from: gchiu::compkarori::co::nz at: 19-Apr-2005 8:33


tc wrote.. apparently on 18-Apr-2005/15:59:43+2:00
>> Yes again. But it does indeed capture the nytimes.com cookie for me .. >> I just tried it.
<<quoted lines omitted: 3>>
>with the fact that nytimes uses half a dozen different cookies. i'll >look more into it...
Ah, you didn't state that you wanted to login .. I don't have a nytimes account so didn't try. And I've noticed that there are quite a few websites that are built to deliberately make it hard to scrape the data .. using javascript etc to generate values which are needed to be posted into forms.
>> And yes again. Turned out to be quite straight forward to do for both http and >> https. > >what was the functionnality you added? did you replace them or did you >add them as another scheme? would you mind posting them? (I won't
I added to it. It was a few years ago now, but I recall that I created a global cookie-jar object, and added cookies to it that were encountered using the http scheme. And when sites were being accessed that had cookies in the jar, I sent them. Some sites used up to 6 cookies, and it seemed to handle those okay, as well as handling redirections. It wasn't perfect, but it did the job I needed to do. As for posting, I'll have to discuss with Carl. Posting the schemes, modified or not, seems to be a breach of the sdk license. -- Graham Chiu http://www.compkarori.com/cerebrus http://www.compkarori.com/rebolml

 [10/15] from: thomas:cr:g:mail at: 18-Apr-2005 23:02


> As for posting, I'll have to discuss with Carl. Posting the schemes, modified > or not, seems to be a breach of the sdk license.
really? hmm... had almost forgotten that sdk licenses still existed in the "free" world we now live in ;) what you did sounds a lot like what i had in mind! (except i had forgotten about redirections) correct me if i'm wrong but from what i saw (thx for the reference of the Jeff Kreis article btw) i wont need to pay/sign any sdk license to implement this correct? i can just reuse public code from the http scheme and the net-install function, no? tc

 [11/15] from: andreas::bolka::gmx::net at: 19-Apr-2005 0:29


Monday, April 18, 2005, 10:33:37 PM, Graham wrote:
> As for posting, I'll have to discuss with Carl. Posting the schemes, > modified or not, seems to be a breach of the sdk license.
even though they are available under system/schemes/.. in standard REBOL/Core ? -- Best regards, Andreas

 [12/15] from: gchiu::compkarori::co::nz at: 19-Apr-2005 13:10


Andreas Bolka wrote.. apparently on 19-Apr-2005/0:29:19+2:00
>Monday, April 18, 2005, 10:33:37 PM, Graham wrote: > >> As for posting, I'll have to discuss with Carl. Posting the schemes, >> modified or not, seems to be a breach of the sdk license. > >even though they are available under system/schemes/.. in standard >REBOL/Core ?
Correct. Here's the header from prot-http.r REBOL [ Title: "REBOL Protocols: HTTP" Author: "Sterling Newton" Rights: "Copyright REBOL Technologies 1997-2002" Note: "Confidential and Proprietary. All rights reserved." ; CONFIDENTIAL AND PROPRIETARY OF REBOL TECHNOLOGIES, USA. ; This source code module shall only be used according to ; the terms and conditions of the REBOL/SDK license. ; This source code is not for distribution or publication. ; All changes or modififications must be submitted to REBOL. ] I believe Carl is working on a new license for LNS .. maybe that will percolate back to the sdk sources. -- Graham Chiu http://www.compkarori.com/cerebrus http://www.compkarori.com/rebolml

 [13/15] from: Izkata::Comcast::net at: 19-Apr-2005 6:15


> Andreas Bolka wrote.. apparently on 19-Apr-2005/0:29:19+2:00 >>Monday, April 18, 2005, 10:33:37 PM, Graham wrote:
<<quoted lines omitted: 9>>
> Author: "Sterling Newton" > Rights: "Copyright REBOL Technologies 1997-2002"
Forgive my possible misconception, but doesn't the 2002, being in the past, mean it's no longer valid? I'm usually not interested in legal stuff, so I don't really know.. -Izzy Boy

 [14/15] from: SunandaDH:aol at: 19-Apr-2005 8:54


Izzy:
> Forgive my possible misconception, but doesn't the 2002, being > in the past, mean it's no longer valid?
No. Copyright law varies across the world, so how long a copyright remains in force varies. But if the principle you suggest were true, I'd be republishing all Harry Potter novels today; the most recent one has a copyright date of 2003. But let's hope that RT will forget copyright and just do what's right. Sunanda

 [15/15] from: Izkata:Comcast at: 19-Apr-2005 18:13


>> Forgive my possible misconception, but doesn't the 2002, being >> in the past, mean it's no longer valid?
<<quoted lines omitted: 3>>
> But if the principle you suggest were true, I'd be republishing all Harry > Potter novels today; the most recent one has a copyright date of 2003.
Ah. Not quite what I meant, but nice suggestion (^.-) I meant when there's a range like that. For example, WinXP says something like (c)1985-2002 (I forget the exact year) And, apparently, people (Mostly white-hat hackers) have been releasing dummed-down versions of Windows on the internet - all the capabilities, nearly no extra features, so it runs much faster and costs less/nothing.
> But let's hope that RT will forget copyright and just do what's right.
Who isn't? =^þ
> Sunanda
-Izzy Boy

Notes
  • Quoted lines have been omitted from some messages.
    View the message alone to see the lines that have been omitted