World: r3wp

Right, but you just want to get started now, don't you?

The BMW has arrived now. Don't let it pass you by :-)

Too late ... for now ... I wrote the data shuffle code over a year 
ago.  And until I can show a hassle free install (by others), I don't 
want to push for REBOL training again.   When 3.0 gels a bit, I'll 
stop say ... for now ... and ask, now Canwecanwecanwe?   :)  It will 
come.

By that time, you'll still have to go through the same path of creating 
a COBOL interface

Oh, sorry Kaj, two different topics really.   OpenCOBOL is for fun. 
   Getting REBOL (not embedded, but full on REBOL apps) in the shop 
has been a slow starter so far.  Not out, just down. ... for now 
...

Yeah, that has been a difficult proposition for a very long time

I'm only now able to move from Ruby to REBOL myself

Kaj: "I'm only now able to move from Ruby to REBOL myself" ... what 
is the reasong/condition, which allowed you to move to REBOL? Just 
curious ...

I've always needed it to run on both Syllable and Linux. I finally 
got R3 to work on Syllable a year ago, but it had proved buggy when 
porting my CMS

Most things I still can't do with it, because there's no GUI

Then eventually, the open source issue will remain, because I can't 
use it for Syllable system tools

The lack of GUI means only web apps can be made, but there wasn't 
an acceptable web framework for REBOL

I used QuarterMaster in the past year, but I can't take that and 
Cheyenne with me now that R3 finallt starts working

Hence I'm forced to make my own CMS

And I'm forced to keep it working on both R2 and R3, so I can't use 
new R3 features

Not exactly a sales story, is it?

Is there anything like WAIT/ANY [3 read http://www.rebol.com]. It 
would either  return none if 3 seconds elapsed, or the content of 
www.rebol.com ?

as in a timeout?

I guess so. I think it has to be programmed by opening in async mode

yeah, I did a few tests using async, but its a bit tricky, nothing 
as elegant as what Awi proposes/asks

is there a way that I can do to pre-allocate a huge file? Pre-allocated 
file will have a continuous disk space, which makes seeking fast.

something like? :

write %file head insert "" 1000000

oops... 

write %file head insert/dup "" " " 1000000

Maxim, I am not sure if doing that will have a continuous disk space. 
Why don't we have something like this: make %file 100000000

usually it will depend on disk fragmentation.  I'm not sure all OS 
allow you to force a contiguous disk area. 

on any OSes which allow it, it could be a good suggestion for R3.

Actually, I am making a NoSQL in R3 for Chinese Social Network Analysis. 
I have more than 20,000,000 records, and every records might need 
a few KB.

you should post a CC wish ticket for it.

(and its a good idea to give some details as to why you need (in 
the CC ticket) this in order to give persepective on the ticket)

When data are not much, my NoSQL was very fast, I called it Lightning 
DB. Now with so much data, It's very slow, and I call it Snail DB. 
:-(

20,000,000 * 1kb *  n   =  a massive file!  :-)

OK, I will post this wish in CC.

are you sure its related to fragmentation?

Well... I am 95 % sure. By the way, Every column of every table has 
its own one or two huge files. I didn't put all the data in a single 
file.

As far I remember It's working with R2 if you write the %file past 
the end, it's filled with #{00].

I remember having asked the same feature in R3 one year ago at least 
(in the R3 chat)

Are you sure it must be in one file?

Thanks again, Jerry, for pushing the resource usage limits of R3. 
http://issue.cc/r3/1799is about reallocating a 256+ MB map! to an 
even larger map!, and then getting a slowdown probably because of 
virtual memory use. We really need tasks so this can go on in the 
background :)

More likely a 512+MB map.

I think the problem was a realloc from 256+MB to 512+MB, which would 
temporarily have 768+MB in memory, plus a hash table recalculation.

If a single R3 value slot is 128bit and a map needs two value slots 
for each (key, value) pair:
(128 / 8) * 2 * (2 ** 24) / (1024 ** 2) == 512.0

Weird, I did a similar calculation and got 256. I should revise my 
comment.

Oh right, I put a 16 in there when it should have been a 32.

so should maps allow to be pre-allocated like series?

They are already.

m: make map! n

Well, at least that allocates _something_ :)

hehe

:-)

So is it about the initial sufficient prediction of programmer allocating 
enough of memory, or is there some artificial limit for the map size?

its about the fact that some things have to be arrays in ram, and 
if you don't make them big enough to begin with eventually, you have 
to live with this sort of "cleanup'

REBOL didn't crash so I'd assume it did its job correctly.  
but a few tests might prove that something is not optimal.