World: r3wp

SQLite has a function extension mechanism, so it seems likely that 
you will access the extensions through that. We'll see.

Hi, could it be that the COLUMNS information isn't update if I use 
an ATTACH command?

Sorry I mean TABLES information.

I have made a simple change to the sqlite.r driver, so it now "supports" 
nested transactions in that it uses reference counting to determin 
when to send a BEGIN or END to SQLite. Works good. Ashley, should 
I mail you the changed version?

Yes please.

it looks to me, that the TIME variable is unused?

... and the last line is "off topic" in my opinion, violating the 
KISS principle - it should be the caller who determines what to do 
next, not the called script

'time is set in 'sql and used by 'format

Agreed on the 2nd point, I'll remove it once I've integrated Robert's 
changes (which I'm doing now).

I know that 'time is set in SQL, but it isn't used by FORMAT AFAICT

aha, I overlooked it

1.0.1 available at: http://www.dobeash.com/download.html


Bumped version to 1.0.x as code is no longer "beta" (I actually had 
a few people email me about that!)


Incorporates new IMPORT function (twice as fast as a tight INSERT 
loop) and Robert's nested transaction handling logic (so nested BEGIN, 
COMMIT, END and ROLLBACK statements are correctly handled). Also 
changed logging behavior so all logging is off by default (i.e. use 
/log to turn on *all* logging).

By the way Ashley, your site redesign is very nice.  Good work.

I have a problem when I import a CSV file. I read the file (1.5 -2 
MB), parse it and than write it out to SQLite.


For some records I get scrambeld (invalid strings) in the database. 
Importing the CSV file via other SQLite tools work without any problem.


It looks like the Rebol memory somehow gets messed up. Or could it 
be on the way to the DLL?

Robert, are you escaping the apostrophes before inserting in DB? 
What datatype are the strings being stored in ? nvarchar, varchar 
etc ?

Thanks Brock. Robert, looks like you've hit the same error I RAMBOed 
here: http://www.rebol.net/cgi-bin/rambo.r?id=4063&


A real pain as it can be consistently reproduced (and hence avoided), 
but with no pattern [that I can see]. If Carl implements SQLite-like 
support (see latest blog entry) then hopefully this will become a 
moot point as we won't require the library call anymore.

From his blogs it appears that Carl is just extracting SQLite's btree 
and indexing engine, but leaving out the SQL stuff that duplicates 
functionality already in REBOL (think blocks and series functions). 
You may be able to access the data (a little unlikely), but it won't 
be SQLite support.

I understand it the way, that we will get RIF, so you can design 
your own small db engine upon that. Linkintg to sqlite will be still 
possible using .dll, no?

or would you welcome including sqlite directly into rebol?

there is one thing I really don't like about sqlite - it stores everything 
into one file. I want one file for table, one file for index, as 
with mysql, because for me it means simplicity - I can just look 
into file system and see how big some table is, or selectively backup 
some tables .... mySQL works that way IIRC

It also seems as if Carl is talking about a generic DB layer, such 
as is defined in python.  so any db, could just implement a derivative 
of the base class and be compatible to some extent.

This is probably a real dumb question, but what is the advantage 
of SQLite over say a REBOL object database?

Depends on the number and size of records you wish to handle, whether 
ACID is important to you, and whether you need SQL access or not.

Carl mentions paging. Does that not mean that you can have really 
big tables that are swapped to disk?

is it possible to easily add a new column to a db table already populated 
with data without losing that data?

Louis, It's not on the SQL92 features not supported http://www.sqlite.org/omitted.html
 so it must support alters.

Mike, thanks. I'll keep playing around with it then.

Yes, see http://www.sqlite.org/lang_altertable.html

Thanks, Ashley.  But that website has a flaw; it can't be downloaded 
easily. It should be either one html page or else one pdf file.

Related: has anyone written a script to download a complete website?

I would like to download the complete sqlite website.

try some of wget tools (webget). IIRC there is even some rebol based 
one on rebol.org iirc?

Can't find any. i wrote a crude one once, but can't find it on my 
harddrive.

Louis: there's batch-download by Anton, try it - http://anton.wildit.net.au/rebol/util/batch-download.r

Louis: if you are on os x, this one work pretty neat http://www.hexcat.com/deepvacuum/index.html
, it will dl the whole site and rewrite urls to make them relative, 
or you coud try with acrobat pro if you want a pdf version of the 
site

if you can use non-rebol stuff, use wget. It's very effective at 
mirroring sites to a local drive.

and DeepVacuum is a GUI for wget ;-)

batch-download is not suited for downloading a whole website. It 
doesn't do any spidering.

Pekr, Rebolek, Will, and Henrik, thanks. I'm on XP.  Too bad. DeepVacuum 
looks nice.

Well, I suppose if you do the spidering, you can supply the links 
to batch-download. Still, it's not designed for that. Better to get 
a specific tool for that.

wget is available for windows.

http://pages.interlog.com/~tcharron/wgetwin.html

Thanks, Anton. I'm going to study your script for other uses later.

Henrik, I'm going to try wget. Thanks.

as long as you use wget in the same location as where you put your 
downloaded files temporarily, you don't need to set the environment 
variables.

Then it's just in a DOS prompt:

wget -r -l <depth> <some site>

And it will download the site to <some site>/ directory

depth is how deep links should be gathered for downloading. Values 
above 3 or 4 can be dangerous. :-)

wget is also very good at resuming downloads:

wget -c <very incredibly big file>

It's probably much better than batch-download at resuming. I should 
study its source to see what it knows.

That's right. It's very rare that wget fails to resume a download.

C:\.SQLite>wget -r -l 1 http://www.sqlite.com/index.html
--23:27:13--  http://www.sqlite.com:80/index.html
           => `www.sqlite.com/index.html'
Connecting to www.sqlite.com:80... connected!
HTTP request sent, awaiting response...
23:27:14 ERROR -1: Malformed status line.