World: r3wp
[All] except covered in other channels
older newer | first last |
Terry 22-Jan-2005 [731] | Actually, i was counting down, and mySQL is choking after 150 calls or so. |
Graham 22-Jan-2005 [732] | are you waiting for responses before you call it again? |
Terry 22-Jan-2005 [733x2] | Forced to reduce the number of iterations to 100 for mysql sake.. and even then it would choke 50% of the time.. results.. Reading the DB used mysql-protocol.r with join mysql://.. localhost {select email from table where name='name' } Reading the file used a: load %file.txt, b: select a 'email mysql - 1.35 seconds (using time/now/precise file - 0 seconds (timestart - 18:35:25.531, timefinish - 18:35:25.531) Conclusion.. no contest.. loading and selecting from file wins hands down. |
Graham, if i need to wait for a response, then i would need to add that time to the total for mysql. | |
Graham 22-Jan-2005 [735] | and if you have 1million files in the local directory? |
Terry 22-Jan-2005 [736x3] | good question... and one million files in the DB as well. |
10000 iterations using the load file method.. 0:00:00.781 | |
I doubt very much that the DB could do that. | |
Graham 22-Jan-2005 [739] | and then later on if you need to copy the 1million files from the filing system, or copy the mysql database .. which is faster? |
Terry 22-Jan-2005 [740x3] | another good point.. think ill load up a directory with a few 100,000 files and see what happens ;) |
If you dont' here from me, you'll know that I mashed my drive :) | |
hear, that is | |
Graham 22-Jan-2005 [743] | well, copying the messages from ios took absolutely ages ... |
Terry 22-Jan-2005 [744x2] | 10,000 iterations .. folder has 100,000 27byte files - 0:00:00.765 seconds |
Took awhile to create 100,000 27 byte files.. long after the rebol script had finished, the os was still handling the cache... Size: 2.57mb Size on Disk: 390 mb | |
Graham 22-Jan-2005 [746] | but we needed 28 byte files to test this out! |
Terry 22-Jan-2005 [747] | I think the results speak volumes. |
Graham 22-Jan-2005 [748] | and how much space does the db require to store the same data? |
Terry 22-Jan-2005 [749] | don't know |
Graham 22-Jan-2005 [750] | Does anyone know how to setup a subject filter in gmail? Can you do wild cards like *[Rebol]* for subject filters? |
Terry 22-Jan-2005 [751x2] | size of DB approx 36 mb. |
Took about 6 mins to move the directory. So it appears to be a trade off. Benefits of using files... - much more flexible (ie: a 10mb binary file in a DB is not a good idea) - easier to manipulate. - magnitude faster access time | |
Tomc 23-Jan-2005 [753] | the magnitude faster time breaks down pretty quickly when you need to retrive a file based on it contents not its title |
Terry 23-Jan-2005 [754] | No, my experiments were using the file AND it's contents via SELECT |
Tomc 23-Jan-2005 [755x2] | using indexes in your database? |
primary key? | |
Terry 23-Jan-2005 [757] | my DB only had a single row. |
Tomc 23-Jan-2005 [758] | that does not answer my question |
Terry 23-Jan-2005 [759] | and no, it wasn't indexed :) |
Tomc 23-Jan-2005 [760] | there should be an imprivement with any indexing |
Terry 23-Jan-2005 [761] | not only that, MySQL via the protocol couldn't handle the 'rapid fire' query rate. |
Graham 23-Jan-2005 [762] | sure .. doing a DOS attack on mysql:// |
Terry 23-Jan-2005 [763] | exactly |
Tomc 23-Jan-2005 [764x2] | so if you only have one row what is the difference between athe fontents of a given file and its name? |
contents | |
Terry 23-Jan-2005 [766] | the contents of the file, and the row inthe DB were identical.. a word °email°, and an email address |
Tomc 23-Jan-2005 [767x2] | relational databases start to be useful when they are given relations which typically require more than one column |
it is hard to relate apples and oranges if you only have apples | |
Terry 23-Jan-2005 [769x3] | ahh, but I have relations as well with °7° |
in fact, my relations can be spread across the net | |
so the net becomes on big relational DB.. just without the archaic DB system. | |
Tomc 23-Jan-2005 [772x3] | Im happy for you but to have your test make sense the relation should be in the relational database |
in the file system the file name is a key and the contents a value | |
the same in a db would nbe two colums one for the name one for the value | |
Terry 23-Jan-2005 [775] | well, why not run some tests yourself and let us know the results... talk is cheap. |
Tomc 23-Jan-2005 [776] | the nales and falued are relatted |
Terry 23-Jan-2005 [777] | i had two columns in the DB .. |
Tomc 23-Jan-2005 [778x3] | in a database you could give a value and return the keys as easly as give the keys and return the values |
in a filesystem thsi is much less symetric | |
if one cloum cintained a constant it ought not have been there | |
older newer | first last |