World: r3wp

But it is not necessary week point if the design of data would be 
good

Anyway it leads to multiple lookup when data is difficult

I agree, Doc, that sounds like a good plan. Map! storage and implementation 
is in theory more efficient than hash!, so having two of them shouldn't 
be a problem, as long as your wrapper functions keep track of the 
mirroring.

The only trick is that you remember that none means nothing, so don't 
bother with a none key. The theory of map! is either:

- Keys that are associated with none don't exist in the map!, so 
assigning none to a key makes it go away.

- All possible keys are in the map! but only the ones associated 
with something other than none are displayed.

There is no difference between these two theories in practice, and 
whether there is memory allocated for keys that you can't see is 
an implementation detail that is irrelevant to the use of map! (though 
there is usually not).

something related, in the past i made some tests to simulate hashs 
with integer keys in R2. I used a bitset as an index, mixed with 
block of blocks to store data.

my tests show that for 10000 records,  finding data is near as fast 
as with hashs. 
actually it's incomplete but you have the idea with this:

REBOL []
f: fast-dic: context [
	size: 100000

 hash: 128 - 1	;** hash size speed up the search, must be a power 
 of 2 - 1 (ie. 15, 31, 63, 127, 257 ...)
	master: copy/deep head insert/dup/only [] [] hash + 1
	index: make bitset! size
	flag: func [idx [integer!]][
		unless find index idx [
			insert index idx

   insert/only insert tail pick master idx and hash + 1 idx copy []
		]
	]
	flag?: func [idx [integer!]][find index idx]
	deflag: func [idx [integer!]][
		remove/part index idx
		remove/part find pick master idx and hash + 1 idx 2
	]
] 

t: now/time/precise
loop 10000 bind [flag random 99999] f
print now/time/precise - t
t: now/time/precise
loop 10000 bind [flag? random 99999] f
print now/time/precise - t

Just a question. Is there a way to rappresent  Unicode Characters 
inside a string with an escape sequence ?

^(hex characters)

. The console may not render the character properly if the font doesn't 
support it though - it may look like a space.

Same as R2, but you can provide more hex characters.

Yes Pavel, that is the way it looks to me also regarding the vector.

Yeah, vector! doesn't work yet. There are a lot of tickets for vector!, 
all deferred. That to-block bug is not there, but not unexpected.

Brian, I think we need to address the lack of list in R3.  Maybe 
a list like handling of block data that still returns a block.

at the performance level of list.

I think that would be an excellent thing to address with a user-defined 
datatype.

People in general didn't use list!, because of the bugs and because 
block! was good enough for most uses. We haven't felt its lack for 
a year now. Most of the advantage of list! in regular code was handled 
by allowing insert and remove at the head of a block! to expand or 
contract the block, just like it does at the end, without a block 
copy. Just by making block! more efficient for inserts and removes, 
we have made list! even less necessary.

For that matter, you can make blocks even faster by getting rid of 
list!, just by enabling more optimizations.

The real advantage of list! wasn't the speed though, it was the alias 
persistancy. That was the source of the bugs in list! too.

I tested vector! extensively about one and half year ago. Not much 
changed, I think. The BIGGEST problem with vector! is that it's zero 
based.

Trust me, vector! has bigger problems than that right now. I expect 
that it will not end up 0-based though, even with all of the advantages 
that 0-based vectors give us. It is likely that we will instead have 
0-based versions of PICK and POKE that work on all series.

Brian, all the vector! tickets are mine, so I know very well about 
vector! problems :) I'm not sure about zero based PICK and POKE, 
I would probably prefer PICKZ and POKEZ, but I can get used to it.

Yes, those would be good names for the aformentioned 0-based versions 
of PICK and POKE. PICK0 and POKE0 have also been suggested as names, 
though I prefer *Z. They could be natives (or mezzanines?) that call 
PICK and POKE internally. I think the only reason we don't have them 
already is that PICK is an action!, and the naming disagreement.

Are blocks stored in memory as linked fragments?

no

How can inserts at the head and tail be done without moving the block, 
then?

you asked for blocks not for lists

Yes, Brian talks about blocks above

where ?, i don't think so

I´m not talking about links for each item, but for fragments determined 
during use

uh ? he just said that the performance have been improved with block 
operations, so that lists are no more requested. He didn't say that 
blocks are working like lists now

I didn´t say that, either. But there must be something going on if 
you can insert without moving the memory

where did he said that ?

One screen up

this ? "Just by making block! more efficient for inserts and removes, 
we have made list! even less necessary."

The only way I can think of to keep the memory contiguous is advanced 
use of the hardware MMU, which would lead to partially used memory 
pages at the start and end of each block

>> a: []
== []

>> dp [append a 1]
== make object! [
    timer: 32
    evals: 11
    eval-natives: 4
    eval-functions: 1
    series-made: 2
    series-freed: 1
    series-expanded: 1
    series-bytes: 464
    series-recycled: 0
    made-blocks: 1
    made-objects: 0
    recycles: 0
]


as you can see the block has been expanded, which means copied in 
another place

That would contradict Brian´s statement even about R2

Don't see when he stated that, anyway we just have to wait his return

¨People in general didn't use list!, because of the bugs and because 
block! was good enough for most uses. We haven't felt its lack for 
a year now. Most of the advantage of list! in regular code was handled 
by allowing insert and remove at the head of a block! to expand or 
contract the block, just like it does at the end, without a block 
copy. Just by making block! more efficient for inserts and removes, 
we have made list! even less necessary.¨

So it already worked that way for appends in R2

Is that profiling accurate Steeve?

>> dp [a: [] b: []]
== make object! [
    timer: 15
    evals: 12
    eval-natives: 3
    eval-functions: 1
    series-made: 1
    series-freed: 0
    series-expanded: 0
    series-bytes: 432
    series-recycled: 0
    made-blocks: 1
    made-objects: 0
    recycles: 0
]

>>

Shouldn't made-blocks indicate 2 blocks were created?

no to Kaj, no to Paul :-)

To kaj: Brian is talking about lists, not blocks. Your test do not 
append or insert data in the input blocks, it's not relevant.

To Paul: No, the 2 blocks are created before entering in the dp function, 
dp doesn't create them.

>> dp []
== make object! [
    timer: 14
    evals: 8
    eval-natives: 3
    eval-functions: 1
    series-made: 1
    series-freed: 0
    series-expanded: 0
    series-bytes: 432
    series-recycled: 0
    made-blocks: 1
    made-objects: 0
    recycles: 0
]

see, a block is always created by dp

If the block needs to be expanded because there isn't enough allocated 
the system does a block copy. If there *is* enough allocated, as 
when you preallocate using make block!, then the system doesn't have 
to do a block copy. That is R2 and R3.


What is new in R3 is that the "head" pointer of a block doesn't have 
to point to the beginning of  the allocated memory, just like the 
"tail" pointer in R2. This means a remove from the head of the block 
just shifts the pointer over one in R3, while in R2 you had to copy 
over the rest of the block contents to shift it towards the head 
of the allocated memory. Preallocated memory can also exist before 
the head of the block contents in R3. This means that there is no 
difference in overhead between inserts at the head or the tail of 
a block in R3.


In theory, inserts inside the block in the first half could be more 
efficient because you would only have to shift from the nearest end, 
not the tail. I don't know whether this optimization has been implemented.


Block operations in general could be faster because with no list! 
type we wouldn't have to special-case as much code, so we could make 
our code much faster through more aggressive optimization.


Btw, I submitted a tweak to DP to make it more accurate by subtracting 
its own overhead. It still has some variance though - have to tweak 
the native to fix that. Plus there is the extreme variance caused 
by Windows.

I hope this answers your question, Kaj.

Back to map Brian's note words-of returns all keys is easy to overlook, 
but this is only way how to traverse thru map (when no next is at 
hand) THX Brian

There is a proposal (looking likely to be implemented) to have FOREACH 
work on object! and map! types. The word list syntax would be restricted, 
but you could do your traversal that way. In the meanwhile you have 
WORDS-OF to get the keys in a block, VALUES-OF to get the values 
in a block, BODY-OF object! to get both in a block (map! proposed 
too) and TO-BLOCK of map! to get both in a block. It works, but the 
FOREACH proposal would create fewer intermediate blocks.

THX again, is it possible something like "select" ie <,>,between 
etc?

Of course FOREACH of map! would operate in the order that TO-BLOCK 
map! would return the keys and values at that moment. In the long 
run you would have to consider the order of FOREACH map! to be non-deterministic 
between calls. The map! type has no inherent ordering, so position 
and sorting are meaningless for it.

hashed doesn't mean ordered?