World: r3wp
[!REBOL3]
older newer | first last |
Jerry 13-Dec-2010 [6599] | Andreas, would you post a ticket in CC on this? You probably can describe the issue better than me. |
Andreas 13-Dec-2010 [6600] | If I can pin down an issue, I will. |
BrianH 13-Dec-2010 [6601] | On my system it had to allocate virtual memory for the process from the OS, and swap memory in RAM to the VM so it would have room to allocate the map in the working RAM. It took as long as I would have expected it to take given that circumstance. |
Andreas 13-Dec-2010 [6602] | As soon as you start swapping, all bets are off. |
BrianH 13-Dec-2010 [6603x2] | An empty map! of 22,000,000 entries took nearly 1GB of RAM on its own, and that doesn't include memory for any strings, blocks or structures that you might add to the map after it is allocated. |
I used DP instead of DT, and it gave me all the details. | |
Andreas 13-Dec-2010 [6605x2] | Can't tell you about the nature of your virtual memory, though. |
Well, it technically could, but it doesn't :) | |
BrianH 13-Dec-2010 [6607] | I used process explorer. |
Andreas 13-Dec-2010 [6608] | allocating 22M entries leads to 929MB resident, for me |
BrianH 13-Dec-2010 [6609] | Yup, that was the nearly part :) |
Andreas 13-Dec-2010 [6610x3] | Well, at around 2^24 entries things start to deteriorate noticeably. |
Long before we even reach 2GB resident. | |
So probably Jerry was right from the beginning after all :) | |
BrianH 13-Dec-2010 [6613] | The OS and other apps are also resident. Anything over 512MB would push things on a 2GB system running OSX.. I wish my main system was running - it has 4GB, which is plenty even on Win7. |
Andreas 13-Dec-2010 [6614x2] | Believe me when I say I have sufficient amounts of free RAM :) |
Only the R3 process looks like it doesn't even want to go beyond 1.9GB resident :) | |
Kaj 13-Dec-2010 [6616] | Are you aware that on a 32 bits system, not the full 4 GB is available, because that's the size of the total address space? |
Andreas 13-Dec-2010 [6617] | Yes. |
Kaj 13-Dec-2010 [6618] | A considerable part of that is needed by the OS for its kernel space. 2 GB is a bit low for that limit on the main OSes, but it makes you think that there's a sort of limit like that in R3 |
Andreas 13-Dec-2010 [6619] | 64-bit host system here. |
Kaj 13-Dec-2010 [6620] | Yes, but R3 is 32 bits, so its address space is at most 4 GB and could be 2 GB if for example one bit of the address is used as a flag |
Andreas 13-Dec-2010 [6621x2] | Of course. |
But it's easy to confirm that this is not the case for R3. | |
Kaj 13-Dec-2010 [6623] | You said it doesn't seem to want to go beyond 1.9 GB resident |
Andreas 13-Dec-2010 [6624] | For a single map, yes. |
Kaj 13-Dec-2010 [6625] | So maybe a 31 bits limit for a map |
Andreas 13-Dec-2010 [6626x3] | >> m5: make map! 22'000'000 ** Internal error: not enough memory ** Where: make ** Near: make map! 22000000 |
>> stats == 3890639112 | |
So obviously no PAE hacks, but we get 4GB usable address space for the R3 process, at least. | |
BrianH 13-Dec-2010 [6629] | On pre-Vista server versions of Windows you used to be able to change the balance to 1GB OS, 3GB apps, as an install opton. This is likely done dynamically now. |
Andreas 13-Dec-2010 [6630] | The famous /3GB switch :) |
BrianH 13-Dec-2010 [6631] | Recently Carl increased the limit on maps - it used to be less than 500,000 pairs - but if there is a hard limit now it is unlikely to be increased for the 32bit builds. |
Andreas 13-Dec-2010 [6632] | Do you have a CureCode reference for that handy? |
BrianH 13-Dec-2010 [6633] | He even wrote a blog about it. |
Andreas 13-Dec-2010 [6634x3] | Here are my current findings: https://gist.github.com/1ece58c8302b2d1f734b |
Ah, of course! | |
A hard-coded allocation progression would have been my current guess :) | |
BrianH 13-Dec-2010 [6637] | http://issue.cc/r3/1344 |
Andreas 13-Dec-2010 [6638x2] | http://www.rebol.net/r3blogs/0295.html |
We clearly have a different concept of "recently" :) | |
BrianH 13-Dec-2010 [6640] | On this project, anything in the last year is recently. |
Andreas 13-Dec-2010 [6641] | Disqualifying that ticket and blog :) |
BrianH 13-Dec-2010 [6642x2] | Which this wasn't |
I'm having AltME freezes again. | |
Andreas 13-Dec-2010 [6644] | A single map! is limited to 2 ** 26 - 1 entries. Or ~3GB memory. |
BrianH 13-Dec-2010 [6645] | You should put that in a comment to #1344. |
Andreas 13-Dec-2010 [6646x2] | Already did. Also added a (rather unlovingly formatted) comment to the map! docs. |
(http://www.rebol.com/r3/docs/datatypes/map.html) | |
Andreas 14-Dec-2010 [6648] | Here's a plot of the DT for each "put" operation (m/(i): i) for 19M puts: http://bolka.at/2010/rebol3/tmp/raw19.png |
older newer | first last |