World: r3wp

I've always needed it to run on both Syllable and Linux. I finally 
got R3 to work on Syllable a year ago, but it had proved buggy when 
porting my CMS

Most things I still can't do with it, because there's no GUI

Then eventually, the open source issue will remain, because I can't 
use it for Syllable system tools

The lack of GUI means only web apps can be made, but there wasn't 
an acceptable web framework for REBOL

I used QuarterMaster in the past year, but I can't take that and 
Cheyenne with me now that R3 finallt starts working

Hence I'm forced to make my own CMS

And I'm forced to keep it working on both R2 and R3, so I can't use 
new R3 features

Not exactly a sales story, is it?

Is there anything like WAIT/ANY [3 read http://www.rebol.com]. It 
would either  return none if 3 seconds elapsed, or the content of 
www.rebol.com ?

as in a timeout?

I guess so. I think it has to be programmed by opening in async mode

yeah, I did a few tests using async, but its a bit tricky, nothing 
as elegant as what Awi proposes/asks

is there a way that I can do to pre-allocate a huge file? Pre-allocated 
file will have a continuous disk space, which makes seeking fast.

something like? :

write %file head insert "" 1000000

oops... 

write %file head insert/dup "" " " 1000000

Maxim, I am not sure if doing that will have a continuous disk space. 
Why don't we have something like this: make %file 100000000

usually it will depend on disk fragmentation.  I'm not sure all OS 
allow you to force a contiguous disk area. 

on any OSes which allow it, it could be a good suggestion for R3.

Actually, I am making a NoSQL in R3 for Chinese Social Network Analysis. 
I have more than 20,000,000 records, and every records might need 
a few KB.

you should post a CC wish ticket for it.

(and its a good idea to give some details as to why you need (in 
the CC ticket) this in order to give persepective on the ticket)

When data are not much, my NoSQL was very fast, I called it Lightning 
DB. Now with so much data, It's very slow, and I call it Snail DB. 
:-(

20,000,000 * 1kb *  n   =  a massive file!  :-)

OK, I will post this wish in CC.

are you sure its related to fragmentation?

Well... I am 95 % sure. By the way, Every column of every table has 
its own one or two huge files. I didn't put all the data in a single 
file.

As far I remember It's working with R2 if you write the %file past 
the end, it's filled with #{00].

I remember having asked the same feature in R3 one year ago at least 
(in the R3 chat)

Are you sure it must be in one file?

Thanks again, Jerry, for pushing the resource usage limits of R3. 
http://issue.cc/r3/1799is about reallocating a 256+ MB map! to an 
even larger map!, and then getting a slowdown probably because of 
virtual memory use. We really need tasks so this can go on in the 
background :)

More likely a 512+MB map.

I think the problem was a realloc from 256+MB to 512+MB, which would 
temporarily have 768+MB in memory, plus a hash table recalculation.

If a single R3 value slot is 128bit and a map needs two value slots 
for each (key, value) pair:
(128 / 8) * 2 * (2 ** 24) / (1024 ** 2) == 512.0

Weird, I did a similar calculation and got 256. I should revise my 
comment.

Oh right, I put a 16 in there when it should have been a 32.

so should maps allow to be pre-allocated like series?

They are already.

m: make map! n

Well, at least that allocates _something_ :)

hehe

:-)

So is it about the initial sufficient prediction of programmer allocating 
enough of memory, or is there some artificial limit for the map size?

its about the fact that some things have to be arrays in ram, and 
if you don't make them big enough to begin with eventually, you have 
to live with this sort of "cleanup'

REBOL didn't crash so I'd assume it did its job correctly.  
but a few tests might prove that something is not optimal.

Why do guys need so large map array? Don't you remember Bill Gates 
once said, that 640KB is enough for everyone? :-)

this seems to prove that pre-allocating maps does indeed store all 
the space required:

>> stats
== 921576
>> m: make map! 20000000
== make map! [
]
>> stats
== 909353688

Jerry, you might want to try the above and see if the wait occurs 
on the 21 millionth item.

(20 millionth item + 1)

I'm with Brian in that most likely the only thing "not optimal" is 
the amount of RAM in Jerry's system.

yep!

if the pre-allocation fixes the setup. then there's no bug in REBOL.

Preallocation won't help with insufficient memory :)