World: r3wp

ah, so this variable applies only to disable/enable logging of sql 
statements, right?

anyway - inability to work directly in cgi mode will drive ppl away 
from driver usage .... Logging should be disabled by default imo, 
and the name of the variable suggests that too ....

the log can't be used for rollbacks anyway, can it? (so not so important)

Hi, I know that some talk of encryption was held here some time ago, 
but currently I was asked to eventually protect sqlite data and I 
am not sure what is correct aproach. I would not go DB-as-a-file 
encryption, then "unpacking" into memory, or so. I prefer app level 
encryption, but I am not sure about searches, using LIKE directive. 
Would it work?

hmm, would be problematic ....

>> (encloak "3" "pass") > (encloak "2" "pass")
== false
>> "3" > "2"
== true

the proper encryption would have to come at lower db level, namely 
- storage level ....

that is nearly useless then, because there is no easy way of how 
to protect your data

I think that SQLite itself can be extended with encrypted fields.

hmm, author offers kind of extension for 2000USD .... there is no 
way how poor pekr would know C code to the extent of writing such 
an extension myself :-)

but I found - http://sqlite.phxsoftware.com/

I haven't given this ADO stuff a try to see if it fits my requirements. 
But will do so.

At the moment I tend to use the 2K$ extension. Yes, not quite cheap 
and I might do it myself but it will take some time too. So not being 
cheaper.

The strategy is to en/decrypt every block that gets written to disk. 
Even all B*-tree stuff etc. with this it's fully transparent to the 
database engine.

It appears that the SQLite version at Petr's link will work without 
ADO if you want to use the traditional APIs.

I can't install it, need to download .NET framework first, will do 
so at home :-)

how much slower the encrypted db is?

You can run it without the .NET framework.

yes, but binary dll comes in installer .msi, and it refuses to run, 
unless you have .NET installed, but that is detail :-)

Why wouldn't you have .NET installed? There are so many good apps 
that use it...

I known none - NET is crap :-)

it means - I use many fine apps, yet it is not on my PC ....

ppl should use technology where appropriate - ATI uses it for Control 
Center, it takes 96MB of RAM ... for what? To configure driver in 
nice juicy, painfully slow environment? Well, at least on 850MHz 
Duron, 256MB RAM, it took some 20 - 30 sec to appear :-)

hopefully at home, with my XP, it is already installed, so will try 
it at home ...

.NET is less crappy than Win32, believe me. I've written for both. 
I agree about ATI's control center though.

Performance will be about 50% without encryption for the SQLite extension. 
But I don't think that it's that much. Reading/writing to disk is 
a lot slower compared to execution speed of processors. So, I expect 
about 25% performance loss.

is .NET complete API replacement for Win32? I thought it is kind 
of virual machine (competition to JAVA), and hence for kind of internet 
ready apps, but not system wide apps?

It will become a replacement over time.

Anyway, I read it that .NET isn't required. So, it should be a drop-in-replacement. 
The only thing I don't know is, how to access the exteneded functions 
than.

There is a virtual machine, but the code you run is native code by 
the time you run it. All new APIs for Windows are built on .NET.

SQLite has a function extension mechanism, so it seems likely that 
you will access the extensions through that. We'll see.

Hi, could it be that the COLUMNS information isn't update if I use 
an ATTACH command?

Sorry I mean TABLES information.

I have made a simple change to the sqlite.r driver, so it now "supports" 
nested transactions in that it uses reference counting to determin 
when to send a BEGIN or END to SQLite. Works good. Ashley, should 
I mail you the changed version?

Yes please.

it looks to me, that the TIME variable is unused?

... and the last line is "off topic" in my opinion, violating the 
KISS principle - it should be the caller who determines what to do 
next, not the called script

'time is set in 'sql and used by 'format

Agreed on the 2nd point, I'll remove it once I've integrated Robert's 
changes (which I'm doing now).

I know that 'time is set in SQL, but it isn't used by FORMAT AFAICT

aha, I overlooked it

1.0.1 available at: http://www.dobeash.com/download.html


Bumped version to 1.0.x as code is no longer "beta" (I actually had 
a few people email me about that!)


Incorporates new IMPORT function (twice as fast as a tight INSERT 
loop) and Robert's nested transaction handling logic (so nested BEGIN, 
COMMIT, END and ROLLBACK statements are correctly handled). Also 
changed logging behavior so all logging is off by default (i.e. use 
/log to turn on *all* logging).

By the way Ashley, your site redesign is very nice.  Good work.

I have a problem when I import a CSV file. I read the file (1.5 -2 
MB), parse it and than write it out to SQLite.


For some records I get scrambeld (invalid strings) in the database. 
Importing the CSV file via other SQLite tools work without any problem.


It looks like the Rebol memory somehow gets messed up. Or could it 
be on the way to the DLL?

Robert, are you escaping the apostrophes before inserting in DB? 
What datatype are the strings being stored in ? nvarchar, varchar 
etc ?

Thanks Brock. Robert, looks like you've hit the same error I RAMBOed 
here: http://www.rebol.net/cgi-bin/rambo.r?id=4063&


A real pain as it can be consistently reproduced (and hence avoided), 
but with no pattern [that I can see]. If Carl implements SQLite-like 
support (see latest blog entry) then hopefully this will become a 
moot point as we won't require the library call anymore.

From his blogs it appears that Carl is just extracting SQLite's btree 
and indexing engine, but leaving out the SQL stuff that duplicates 
functionality already in REBOL (think blocks and series functions). 
You may be able to access the data (a little unlikely), but it won't 
be SQLite support.

I understand it the way, that we will get RIF, so you can design 
your own small db engine upon that. Linkintg to sqlite will be still 
possible using .dll, no?

or would you welcome including sqlite directly into rebol?

there is one thing I really don't like about sqlite - it stores everything 
into one file. I want one file for table, one file for index, as 
with mysql, because for me it means simplicity - I can just look 
into file system and see how big some table is, or selectively backup 
some tables .... mySQL works that way IIRC