World: r3wp

uh ? he just said that the performance have been improved with block 
operations, so that lists are no more requested. He didn't say that 
blocks are working like lists now

I didn´t say that, either. But there must be something going on if 
you can insert without moving the memory

where did he said that ?

One screen up

this ? "Just by making block! more efficient for inserts and removes, 
we have made list! even less necessary."

The only way I can think of to keep the memory contiguous is advanced 
use of the hardware MMU, which would lead to partially used memory 
pages at the start and end of each block

>> a: []
== []

>> dp [append a 1]
== make object! [
    timer: 32
    evals: 11
    eval-natives: 4
    eval-functions: 1
    series-made: 2
    series-freed: 1
    series-expanded: 1
    series-bytes: 464
    series-recycled: 0
    made-blocks: 1
    made-objects: 0
    recycles: 0
]


as you can see the block has been expanded, which means copied in 
another place

That would contradict Brian´s statement even about R2

Don't see when he stated that, anyway we just have to wait his return

¨People in general didn't use list!, because of the bugs and because 
block! was good enough for most uses. We haven't felt its lack for 
a year now. Most of the advantage of list! in regular code was handled 
by allowing insert and remove at the head of a block! to expand or 
contract the block, just like it does at the end, without a block 
copy. Just by making block! more efficient for inserts and removes, 
we have made list! even less necessary.¨

So it already worked that way for appends in R2

Is that profiling accurate Steeve?

>> dp [a: [] b: []]
== make object! [
    timer: 15
    evals: 12
    eval-natives: 3
    eval-functions: 1
    series-made: 1
    series-freed: 0
    series-expanded: 0
    series-bytes: 432
    series-recycled: 0
    made-blocks: 1
    made-objects: 0
    recycles: 0
]

>>

Shouldn't made-blocks indicate 2 blocks were created?

no to Kaj, no to Paul :-)

To kaj: Brian is talking about lists, not blocks. Your test do not 
append or insert data in the input blocks, it's not relevant.

To Paul: No, the 2 blocks are created before entering in the dp function, 
dp doesn't create them.

>> dp []
== make object! [
    timer: 14
    evals: 8
    eval-natives: 3
    eval-functions: 1
    series-made: 1
    series-freed: 0
    series-expanded: 0
    series-bytes: 432
    series-recycled: 0
    made-blocks: 1
    made-objects: 0
    recycles: 0
]

see, a block is always created by dp

If the block needs to be expanded because there isn't enough allocated 
the system does a block copy. If there *is* enough allocated, as 
when you preallocate using make block!, then the system doesn't have 
to do a block copy. That is R2 and R3.


What is new in R3 is that the "head" pointer of a block doesn't have 
to point to the beginning of  the allocated memory, just like the 
"tail" pointer in R2. This means a remove from the head of the block 
just shifts the pointer over one in R3, while in R2 you had to copy 
over the rest of the block contents to shift it towards the head 
of the allocated memory. Preallocated memory can also exist before 
the head of the block contents in R3. This means that there is no 
difference in overhead between inserts at the head or the tail of 
a block in R3.


In theory, inserts inside the block in the first half could be more 
efficient because you would only have to shift from the nearest end, 
not the tail. I don't know whether this optimization has been implemented.


Block operations in general could be faster because with no list! 
type we wouldn't have to special-case as much code, so we could make 
our code much faster through more aggressive optimization.


Btw, I submitted a tweak to DP to make it more accurate by subtracting 
its own overhead. It still has some variance though - have to tweak 
the native to fix that. Plus there is the extreme variance caused 
by Windows.

I hope this answers your question, Kaj.

Back to map Brian's note words-of returns all keys is easy to overlook, 
but this is only way how to traverse thru map (when no next is at 
hand) THX Brian

There is a proposal (looking likely to be implemented) to have FOREACH 
work on object! and map! types. The word list syntax would be restricted, 
but you could do your traversal that way. In the meanwhile you have 
WORDS-OF to get the keys in a block, VALUES-OF to get the values 
in a block, BODY-OF object! to get both in a block (map! proposed 
too) and TO-BLOCK of map! to get both in a block. It works, but the 
FOREACH proposal would create fewer intermediate blocks.

THX again, is it possible something like "select" ie <,>,between 
etc?

Of course FOREACH of map! would operate in the order that TO-BLOCK 
map! would return the keys and values at that moment. In the long 
run you would have to consider the order of FOREACH map! to be non-deterministic 
between calls. The map! type has no inherent ordering, so position 
and sorting are meaningless for it.

hashed doesn't mean ordered?

It doesn't. SELECT works on object! and map!, but there is no comparison 
except equality for those types.

Hashed means indexed (in the database sense), not ordered.

my fault I missmatch rebol select with SQL like select

IE if you expect quite large number of keys would you have any chance 
to "narrow" like "sql between"

Map! is not indexed in the REBOL sense, ironically enough: no position.

OK now I have more clue, all this is toward key/value datastore preferably 
diskbased something I believe Paul is trying in R3

REBOL is a lower-level language than SQL. No set operations (for 
map!, ar least). Those kinds of SQL operations need to be done procedurally 
in REBOL, using block copies and such.

We will have REBOL Indexed Files (RIF) for the disk-based datastore.

I'm really currious believe me, like small boy

Well, we don't havee RIF yet so you'll have to stay curious for a 
while :(

I'm keeping finger crossed not to be too old to be able enjoy truly

But seriously thanks for all the job is done until now

hmmmm.... interesting behaviour of blocks....

Brian, does that mean that Carl had in main a possibly new feature 
asked for a while: 

RANGE :A reference which is a sub part of an existing serie. (supress 
the need of overheaded copy/part in our sripts)

I think it's not at all tricky to implement if the head of a serie 
is a logical offset now.

(had in mind)

Must be RIF native or is it possible to mimic in rebol as is now

is it question of mold load on each read?

No, there are no plans to implement range. The head and tail of the 
series are attributes of the series, not the reference to the series. 
It's like the difference between the position attribute of a port 
and a series: For a series, the position is an attribute of the reference 
to the series, while for port, the position is an internal attribute. 
Still, a subseries reference could be implemented as a user-defined 
datatype as long as it is careful to make up for changes in the underlying 
series.

I think that the idea was to be able store internal REBOL values. 
I mean that you for example close console with some defined values 
and after restart of the console you could load into the state where 
you ended.

At least I think Carl was somewhere talking about that. And that 
it could for example speed the boot time, which could be useful for 
cgi aps.

Correct me, if I'm wrong:)

I am really curious to see how binding issues are resolved with RIF. 
You could just have the data come out of the RIF unbound...

Oldes, you are not wrong here.

just to say the virtual-block scheme is an exemple of how RIF could 
be implemented (see http://sites.google.com/site/rebolish/for the 
source)

Although Carl was actually talking about rebin to speed up the boot 
times, not RIF :)

RIF files were going to be binary (rebin), not source like Steeve's 
virtual blocks.

Oldes - was it RIF? Wasn't it Rebin?

Maybe it was rebin... all these far future projects:)