World: r3wp

(doing my test with recycle/ballast activated, so that the memory 
used stay low, arround 3500 Mb.
The test file containing 500000 records)

Oups !!! not 3500 Mb, but 3,5 Mb

3.5MB :-)

yes bytes not bits :)

Steeve - what is the speed compared to R2?

don't know

cause the scheme VBS has not been backported

..

sh*****t, i lost my post

I Don't write it back.
Summary:
We can't serialize vectors to save/load them into files.
We need decisions.

Designing vectors not serializable from the start was an error to 
my mind.

We need easy way (fast and not memory consuming) to convert and convert 
back vectors from binaries.

yes, this world needs restart. I advice, before you post, press CTRL+A, 
then CTRL+C to get your post to clipboard first, to save your sanity 
:-)

Maybe it would be get to post your idea to R3 Chat, R3/Datytypes 
section, or as a wish to CureCode, as Carl will not be able to read 
it here ....

yep, it's what i do usually, but sometimes i forgot

but we can discuss here about the proposal at first

Yes ... things should be serialisable, chainable, and streamable 
- I still wait for Codecs and Parse to handle streamed data input 
.... :-)

REBOL values only need to be serializable in REBOL format from the 
start - that is all the datatype! spec allows. That means MOLD and 
MOLD/all formats, and vectors support those peoperly as of alpha 
55. There are only so many operations that a datatype! can support 
internally. These operations are known as action! functions.


All other operations on a datatype! can be implemented as REBOL or 
native functions, and these functions can be added later if need 
be. They don't need to be there from the start. Codecs seem like 
a likely choice in this case, once the codec model changes to allow 
streaming. The current model is a just-for-now placeholder.


Welcome to the wonderful world of alpha software that is still being 
designed :)

I just wanted to underline that it's not good to postpoint the design 
of vectors.

Currently their design is too limited to be usefull (not serialisable, 
no scalars operations).

I hope there will be not huge drawbacks when the time will come to 
complete them like they should behave.

I'm a little disapointed about how Carl deals with the design of 
vectors.

Vectors are serializable in the only way that REBOL datatypes can 
be inherently serializable: through MOLD. This is a basic limitation 
of how datatypes are implemented in REBOL. All other forms of serialization 
*have to* be implemented with other functions.

The actions are the only functions that can be implemented *by* a 
datatype. All other functions are addons.

R3 is being incrementally designed. Incremental design is by far 
the best design approach for programming langages.

Serialization is a general concept here.

I'm talking about convertions of vectors into another formats (like 
blocks or binaries).

molding a vector is of no use in real applications.

Well, that has to be done by functions that are not inherent to the 
datatype, addon functions. Those addon functions can be built in 
or put into plugins, or both.

Except for the conversion to blocks, which is supported as of alpha 
55.

are you serious ?
to block! and to binary! should be provided

especially to binary!

aswell as-binary

Yes they should. And they will, along with better binary conversion 
methods than to-binary.

AS-BINARY and AS-STRING will not exist in R3 - no type aliasing allowed.

Wtf, i want the reference of the hardstored binary data of the vector.
What the prob ? it's stored as a binary stream

It's a safety issue. That kind of access will be done through accessor 
functions.

you mean the data will be copied ?

I mean the access will be through functions which will hide the internal 
implementation. That doesn't mean the data will be copied.

there is nothing to hide, a vectors is a binary serie

That's the plugin model, in theory (we'll see in practice). R2-style 
structs and routines are insecure.

so to secure it, there will be a copy. If not it can be secured

*can't be

It can be secured if access is through bounds-checked accessors.

By "we'll see in practice", I mean that the current model is not 
yet documented, and has gone through design changes this month.

i don't see your point, if i want a binary serie of the internal 
data of a vector, it's to do all sort of operations we can do on 
binaries

accessor will not protect anything

Except inserting, or changing the length, or changing the data to 
something the vector type doesn't support, or...

Oh Geez, you want protect the vector from modifications ?
So forget my request, vectors will be of no use

i will continue to manage my indexes with plain binaries

No, I want to protect the vector from *bad* modifications. That's 
easy to do, as long as you *don't* do type aliasing.

what kind of bad modification could exist with a binary serie ? there 
is noone.

To my mind a vector should be just a wrapper on a binary serie.

just a new handy way, to modify or get values from a binary