World: r3wp
[Core] Discuss core issues
older newer | first last |
Steeve 18-Dec-2008 [11654] | memory overhead is the point |
[unknown: 5] 18-Dec-2008 [11655] | performance is the point. |
BrianH 18-Dec-2008 [11656] | Not in this case, Steeve. |
[unknown: 5] 18-Dec-2008 [11657] | I don't think steeve gets that I don't need more than 16 bytes |
Steeve 18-Dec-2008 [11658] | performance you will have too with a well sized buff |
[unknown: 5] 18-Dec-2008 [11659x3] | let me put it another way Steeve, I will not be reading more than 16 bytes per request (because I don't need any more than that) and it isn't a 16 bytes segment that is next to another 16 byte segment really I'm moving back and forth all over the file to get 16 byte segments each time. |
Thats for this particular portion of what I'm doing. | |
The read-io is optimal in my other needs. | |
BrianH 18-Dec-2008 [11662x2] | Steeve, he is doing random access. Unless he figures out how to queue his accesses, it won't matter if he uses the internal buffering or his own. The only way he could improve performance is through queueing. |
Paul, there are other ways to reduce memory overhead that will get you more benefit, like using INSERT/part. | |
[unknown: 5] 18-Dec-2008 [11664x2] | in what way? |
can't read with insert/part | |
BrianH 18-Dec-2008 [11666] | Not here. I was just using an example. |
[unknown: 5] 18-Dec-2008 [11667x2] | ok |
I can't think of a better options than copy/part at this point | |
BrianH 18-Dec-2008 [11669] | Without read queueing, neither can I. |
[unknown: 5] 18-Dec-2008 [11670x4] | what about pick? |
be nice to have pick/part | |
without the overhead of copy | |
pick already works on /seek ports | |
BrianH 18-Dec-2008 [11674] | Do you need to open/direct with read-io? |
[unknown: 5] 18-Dec-2008 [11675x3] | no |
but the head moves as you read with /direct | |
must like forall does | |
BrianH 18-Dec-2008 [11678] | It is not COPY that is buffering internally, it is OPEN without /direct. |
[unknown: 5] 18-Dec-2008 [11679x5] | I might do a test to see how fast 4 successive picks would be |
and 16 picks | |
/seek is not buffered | |
it is like /direct | |
difference is that the index stays at head as you reference it. | |
BrianH 18-Dec-2008 [11684] | How often do you need to do these reads, and can they be sorted in batches? |
[unknown: 5] 18-Dec-2008 [11685] | They do get sorted but they are done often and the batch sort is random sized depending on the request |
BrianH 18-Dec-2008 [11686] | You are using variable-length records rather than fixed length? |
[unknown: 5] 18-Dec-2008 [11687x2] | yes ;-) |
Your wondering how right? | |
BrianH 18-Dec-2008 [11689] | That is a lot slower. I am not wondering how, I've followed the discussions in your group. |
[unknown: 5] 18-Dec-2008 [11690] | slower than what? |
BrianH 18-Dec-2008 [11691] | Fixed length. Databases usually work in increments of disk pages because it is faster. |
[unknown: 5] 18-Dec-2008 [11692x2] | But at a cost |
What if declared a field varchar[2000] and you only populate it with 50? | |
BrianH 18-Dec-2008 [11694] | Varchars are slower in practice. You can minimize the overhead of disk page access by packing pages if you have to. The OS reads in pages anyways. |
[unknown: 5] 18-Dec-2008 [11695] | I have minimized it to a greater extent. |
BrianH 18-Dec-2008 [11696] | A read of less than 1 disk page is just as slow as a read of 1 disk page. |
[unknown: 5] 18-Dec-2008 [11697] | I'm hitting you privately. |
BrianH 18-Dec-2008 [11698] | Ouch :) |
Steeve 18-Dec-2008 [11699x2] | say Oooooo to relax your sphincter muscles Brian |
forget that | |
Chris 19-Dec-2008 [11701x3] | A 'bind question: how do I construct an object containing blocks without changing the blocks' contexts? This bad: >> id: #foo == #foo >> blk: [id] == [id] >> reduce blk == [#foo] >> ctx: context append/only [id: #bar bk:] blk >> reduce ctx/bk == [#bar] This good: >> id: #foo == #foo >> blk: [id] == [id] >> reduce blk == [#foo] >> ctx: magic-context append/only [id: #bar bk:] blk >> reduce ctx/bk == [#foo] How to 'magic-context ? |
Ugly answer: magic-context: func [blk [block!] /local out][ out: context join extract blk 2 [none] set out extract/index blk 2 2 ] | |
I was hoping 'construct would do it, but alas not. | |
older newer | first last |