World: r3wp

after a text file being read from a directory is another db operation.

after all ..

the filing system is a database

is it?

Ok, doing some timing tests.. using 10000 loops... 


Firstly, the mysql is choking around the 9800 -9850 mark by denying 
access at that point...

(i think my test code is messed up)

Actually, i was counting down, and mySQL is choking after 150 calls 
or so.

are you waiting for responses before you call it again?

Forced to reduce the number of iterations to 100 for mysql sake.. 
and even then it would choke 50% of the time.. results.. 


Reading the DB used mysql-protocol.r with join mysql://.. localhost 
{select email from table where name='name' }
Reading the file used a: load %file.txt, b: select a 'email
mysql - 1.35 seconds (using time/now/precise

file - 0 seconds (timestart - 18:35:25.531, timefinish - 18:35:25.531)


Conclusion.. no contest.. loading and selecting from file wins hands 
down.

Graham, if i need to wait for a response, then i would need to add 
that time to the total for mysql.

and if you have 1million files in the local directory?

good question... and one million files in the DB as well.

10000 iterations using the load file method.. 0:00:00.781

I doubt very much that the DB could do that.

and then later on if you need to copy the 1million files from the 
filing system, or copy the mysql database .. which is faster?

another good point.. think ill load up a directory with a few 100,000 
files and see what happens ;)

If you dont' here from me, you'll know that I mashed my drive :)

hear, that is

well, copying the messages from ios took absolutely ages ...

10,000 iterations .. folder has 100,000 27byte files  - 0:00:00.765 
seconds

Took awhile to create 100,000 27 byte files.. long after the rebol 
script had finished, the os was still handling the cache...
Size: 2.57mb
Size on Disk: 390 mb

but we needed 28 byte files to test this out!

I think the results speak volumes.

and how much space does the db require to store the same data?

don't know

Does anyone know how to setup a subject filter in gmail?  Can you 
do wild cards like *[Rebol]* for subject filters?

size of DB approx 36 mb.

Took about 6 mins to move the directory.   So it appears to be a 
trade off.
Benefits of using files... 

- much more flexible (ie: a 10mb binary file in a DB is not a good 
idea)
- easier to manipulate.
- magnitude faster access time

the magnitude faster time breaks down pretty quickly when you need 
to retrive a file based on it contents not its title

No, my experiments were using the file AND it's contents via SELECT

using indexes in your database?

primary key?

my DB only had a single row.

that does not answer my question

and no, it wasn't indexed :)

there should be an imprivement with any indexing

not only that, MySQL via the protocol couldn't handle the 'rapid 
fire' query rate.

sure .. doing a DOS attack on mysql://

exactly

so if you only have one row  what is the difference between athe 
fontents of a given file and its name?

contents

the contents of the file, and the row inthe DB were identical.. a 
word °email°, and an email address

relational databases start to be useful when they are given relations 
which typically require more than one column

it is hard to relate apples and oranges if you only have apples

ahh, but I have relations as well with °7°

in fact, my relations can be spread across the net

so the net becomes on big relational DB.. just without the archaic 
DB system.

Im happy for you but to have your test make sense the relation should 
be in the relational database

in the file system the file name is a key and the contents a value

the same in a db would nbe two colums one for the name one for the 
value