r3wp [groups: 83 posts: 189283]
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

World: r3wp

[All] except covered in other channels

Terry
22-Jan-2005
[731]
Actually, i was counting down, and mySQL is choking after 150 calls 
or so.
Graham
22-Jan-2005
[732]
are you waiting for responses before you call it again?
Terry
22-Jan-2005
[733x2]
Forced to reduce the number of iterations to 100 for mysql sake.. 
and even then it would choke 50% of the time.. results.. 


Reading the DB used mysql-protocol.r with join mysql://.. localhost 
{select email from table where name='name' }
Reading the file used a: load %file.txt, b: select a 'email
mysql - 1.35 seconds (using time/now/precise

file - 0 seconds (timestart - 18:35:25.531, timefinish - 18:35:25.531)


Conclusion.. no contest.. loading and selecting from file wins hands 
down.
Graham, if i need to wait for a response, then i would need to add 
that time to the total for mysql.
Graham
22-Jan-2005
[735]
and if you have 1million files in the local directory?
Terry
22-Jan-2005
[736x3]
good question... and one million files in the DB as well.
10000 iterations using the load file method.. 0:00:00.781
I doubt very much that the DB could do that.
Graham
22-Jan-2005
[739]
and then later on if you need to copy the 1million files from the 
filing system, or copy the mysql database .. which is faster?
Terry
22-Jan-2005
[740x3]
another good point.. think ill load up a directory with a few 100,000 
files and see what happens ;)
If you dont' here from me, you'll know that I mashed my drive :)
hear, that is
Graham
22-Jan-2005
[743]
well, copying the messages from ios took absolutely ages ...
Terry
22-Jan-2005
[744x2]
10,000 iterations .. folder has 100,000 27byte files  - 0:00:00.765 
seconds
Took awhile to create 100,000 27 byte files.. long after the rebol 
script had finished, the os was still handling the cache...
Size: 2.57mb
Size on Disk: 390 mb
Graham
22-Jan-2005
[746]
but we needed 28 byte files to test this out!
Terry
22-Jan-2005
[747]
I think the results speak volumes.
Graham
22-Jan-2005
[748]
and how much space does the db require to store the same data?
Terry
22-Jan-2005
[749]
don't know
Graham
22-Jan-2005
[750]
Does anyone know how to setup a subject filter in gmail?  Can you 
do wild cards like *[Rebol]* for subject filters?
Terry
22-Jan-2005
[751x2]
size of DB approx 36 mb.
Took about 6 mins to move the directory.   So it appears to be a 
trade off.
Benefits of using files... 

- much more flexible (ie: a 10mb binary file in a DB is not a good 
idea)
- easier to manipulate.
- magnitude faster access time
Tomc
23-Jan-2005
[753]
the magnitude faster time breaks down pretty quickly when you need 
to retrive a file based on it contents not its title
Terry
23-Jan-2005
[754]
No, my experiments were using the file AND it's contents via SELECT
Tomc
23-Jan-2005
[755x2]
using indexes in your database?
primary key?
Terry
23-Jan-2005
[757]
my DB only had a single row.
Tomc
23-Jan-2005
[758]
that does not answer my question
Terry
23-Jan-2005
[759]
and no, it wasn't indexed :)
Tomc
23-Jan-2005
[760]
there should be an imprivement with any indexing
Terry
23-Jan-2005
[761]
not only that, MySQL via the protocol couldn't handle the 'rapid 
fire' query rate.
Graham
23-Jan-2005
[762]
sure .. doing a DOS attack on mysql://
Terry
23-Jan-2005
[763]
exactly
Tomc
23-Jan-2005
[764x2]
so if you only have one row  what is the difference between athe 
fontents of a given file and its name?
contents
Terry
23-Jan-2005
[766]
the contents of the file, and the row inthe DB were identical.. a 
word °email°, and an email address
Tomc
23-Jan-2005
[767x2]
relational databases start to be useful when they are given relations 
which typically require more than one column
it is hard to relate apples and oranges if you only have apples
Terry
23-Jan-2005
[769x3]
ahh, but I have relations as well with °7°
in fact, my relations can be spread across the net
so the net becomes on big relational DB.. just without the archaic 
DB system.
Tomc
23-Jan-2005
[772x3]
Im happy for you but to have your test make sense the relation should 
be in the relational database
in the file system the file name is a key and the contents a value
the same in a db would nbe two colums one for the name one for the 
value
Terry
23-Jan-2005
[775]
well, why not run some tests yourself and let us know the results... 
talk is cheap.
Tomc
23-Jan-2005
[776]
the nales and falued are relatted
Terry
23-Jan-2005
[777]
i had two columns in the DB ..
Tomc
23-Jan-2005
[778x3]
in a database you could give a value and return the keys as easly 
as give the keys and return the values
in a filesystem thsi is much less symetric
if one cloum cintained a constant it ought not have been there