r3wp [groups: 83 posts: 189283]
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

World: r3wp

[Core] Discuss core issues

[unknown: 5]
18-Dec-2008
[11602x3]
Yeah I just pasted it
hold on
64   0:00:10.639     0:00:00.000020633   515625
128      0:00:05.444     0:00:00.000021116   257813
256      0:00:02.761     0:00:00.000021418   128907
1024     0:00:00.733     0:00:00.000022744   32227
2048     0:00:00.374     0:00:00.000023209   16114
4096     0:00:00.218     0:00:00.000027057   8057
8192     0:00:00.124     0:00:00.000030776   4029
10240    0:00:00.109     0:00:00.000033819   3223
16384    0:00:00.078     0:00:00.000038709   2015
32768    0:00:00.078     0:00:00.00007738    1008
65536    0:00:00.047     0:00:00.000093253   504
131072   0:00:00.063     0:00:00.00025   252
>>
BrianH
18-Dec-2008
[11605]
Looks like 64k is a good value for you.
Steeve
18-Dec-2008
[11606x2]
yeah it seems
but in fact, you should execute it several times, results can be 
different
BrianH
18-Dec-2008
[11608]
Particularly on Windows.
Steeve
18-Dec-2008
[11609x4]
in fact you can see that 16ko buffer would be a better choice (regarding 
to the first column)
(the third column)
you must do more tests
brb, i have to request some cigarettes and food
[unknown: 5]
18-Dec-2008
[11613]
Steeve, I'm finding that copy/part using 'at on /seek is working 
at more speed for me.
Steeve
18-Dec-2008
[11614]
you must have a special computer, it's not quiet logical and i have 
opposite results on my computer
[unknown: 5]
18-Dec-2008
[11615x2]
s: now/precise
a: open/direct/binary %blah
idx: 0
a/state/index: 0
buff: make binary! 16

loop 100000 [read-io a buff 16 a/state/index: a/state/index + 16 
clear buff]
close a
t: now/precise
difference t s
my datafile is %blah
Steeve
18-Dec-2008
[11617]
oh really ? ;-)
[unknown: 5]
18-Dec-2008
[11618x5]
== 0:00:01.934
That is what I get with read-io.
s: now/precise
a: open/seek %blah
idx: 0
loop 100000 [copy/part at a (idx: idx + 16) 16]
close a
t: now/precise
difference t s
what I get with copy/part:
== 0:00:01.747
Steeve
18-Dec-2008
[11623x2]
using a buffer of 16 bytes is YOUR problem, didn't you understand 
the result of the pofiling script
use a 16ko buffer instead (16ko = 16 * 1024)
[unknown: 5]
18-Dec-2008
[11625]
Yes, I did Steeve, but that is my point - I'm saying for MY purposes 
read-io i s not faster.
Oldes
18-Dec-2008
[11626]
Do you mean you MUST use only 16 bytes?
Steeve
18-Dec-2008
[11627]
but copy use a larger buffer that 16 bytes, you don't compare same 
things
[unknown: 5]
18-Dec-2008
[11628x3]
I'm comparing the same for what I need.
I only need to copy 16 bytes
Yes Oldes
Steeve
18-Dec-2008
[11631]
no you are wrong Paul, copy is using an internal buffer of the port 
which are of several Kbytes
[unknown: 5]
18-Dec-2008
[11632x2]
Steeve, your missing what I'm saying.
I ONLY WANT to read 16bytes.  I don't want any more than that.
Steeve
18-Dec-2008
[11634]
not really
[unknown: 5]
18-Dec-2008
[11635]
heh,ok Steeve, what do I really want?
Steeve
18-Dec-2008
[11636]
i think it's you who don't understand, but anyway do as you want
[unknown: 5]
18-Dec-2008
[11637]
ok steeve.
BrianH
18-Dec-2008
[11638]
Paul, the data you are retrieving is in 16 byte increments and not 
contiguous?
[unknown: 5]
18-Dec-2008
[11639]
Correct Brian.
BrianH
18-Dec-2008
[11640]
Accessed randomly, not in order?
[unknown: 5]
18-Dec-2008
[11641x2]
Yes.
I'll use read-io for my contigious stuff but not for this portion
Steeve
18-Dec-2008
[11643]
each time you do a copy/part a new internal buffer is created, it's 
better to use always the same buffer. easy to understand.

more of that, the profiling script say that it''s faster to read 
16ko than 2 times  16 bytes
BrianH
18-Dec-2008
[11644]
In that case, read-io would not work well for you directly without 
implementing your own buffering. Barring that, the buffering built 
into REBOL and used by COPY would probably be good enough. If yo 
want better performance you might to make your own buffers.
[unknown: 5]
18-Dec-2008
[11645x2]
Steeve, why should I read 16k  when I only want 16 bytes?  I would 
then have to copy/part on the  portion of the buffer that I want 
if that was the case to get only the first 16 bytes of the 16k.
Correct Brian.
BrianH
18-Dec-2008
[11647]
You would have some memory overhead if you go the COPY route because 
of what Steeve said, but that may be dwarfed by the disk overhead 
savings.
[unknown: 5]
18-Dec-2008
[11648x2]
correct.
its the speed of the read which will be the most  impact.
Steeve
18-Dec-2008
[11650]
because each time you do a copy/part you create new buffers in memory 
which are not erased by the recycler so that you should consider

using always the same buffer especially if you do thousands and thousands 
access in one second
[unknown: 5]
18-Dec-2008
[11651]
I would still have to use copy/part on the buffer if I went with 
Steeves idea.