World: r3wp

Unit tests will have to be rebol version dependant. Eg. A set of 
unit tests developed on Core 2.6 for the PRINT function may all pass 
on Core 2.6, but not on Core 2.5.  Recording the rebol version also 
captures the date and platform where the tests were developed.

Uhh yes, testing a vocabulary huge and multi-platform language as 
REBOL is a big task. But interesting aspects (like the history), 
as you point out.

I'm thinking more like keeping track of a few things:
- Proper behavior, and version when such behavior was achieved
- Changes in expected behavior

So there would be two sets of versions, the versions of REBOL and 
the versions of the tests. Over time, both REBOL will be fixed and 
the tests will be fixed, refined or altered. This could get pretty 
big pretty quickly I suppose - it could use a database to store the 
tests or some such.

Encyclopedia REBOLae
:-)

Don't tell me this is another thing that would have to wait for the 
REBOL/Services infrastructure...

I think we all have to wait for Rebol 3.0 :-)

It would be good to be able to answer a question like this: "Do the 
functions: [print parse encloak] exist and behave the same on Rebol/Core 
2.5 and 2.6, and Rebol/View 1.2.1 and 1.3.1?"

Brian, I think the results can be stored hierarchically, so a filesystem 
storage can be enough, although I'm not sure.

I'm wondering if everything can be automated so that there are no 
stored results (because results are in fact derived), so when the 
user asks a question, that's when the tests are run and the answer 
obtained.

(The results could be stored after this, mayb, so that only popular 
questions have web pages generated with the results in them.)

I've got a text file going with these notes in it, but I think it 
might be better to make a new project in Qtask. Anyone disagree ?

For the resulst of the use of a word, you may need to save the wanted 
output, and the actual output from different platforms (if they differ).

*results*

Absolutely, but if we have a rebol unit test server constantly running, 
able to answer arbitrary questions, why would we need to store results 
?

right, only the wanted result need to be tested, if you have access 
to all platforms to test. Maybe result from one reference platform 
should be saved, to test other platforms up against?

We could set up a server with as many versions of rebol as possible, 
so it can help as many people as possible, but each of us could also 
run the same system with a more limited set of rebol versions (ie. 
just the ones we bother keeping).

*only the wanted result need to be SAVED, if ...*

(I'm not too sharp today.) :-)

Geomol, every one has a different idea, as time progresses, as to 
what is the "reference" platform. Since it will be a collaborative 
effort everyone will be adding input from various sources. Some functions 
are only available on Rebol/Link etc..

Well for one thing, this would preclude platform-specific tests, 
as your server would only be able to be one platform (probably).

Anton, to make it perfect (total), you need acces to all the OSs, 
REBOL is supported on.

That's right, the server will get as many rebol versions from everyone 
as possible; Core, View, SDK whatever..

So the request to a certain server would be something like: run this 
little test and tell me the result (also if it lead to an error!). 
Could be done easily with REBOL.

More like: the request to the rebol unit test server would be: "Tell 
me all about PARSE on these Rebol versions..."

I see difficulty in testing something like a block!, because you 
would also have to test blocks in objects and look for possible side-effects, 
or what?

I'm not sure a hierarchy would work here - there are too many dimensions. 
Platform (Core, View, ...), platform (Windows, Linux, ...), version, 
test version, etc. Plus a test version would have applicable platforms, 
expiry both for bugs in the test and for changes in expectations, 
and cached results. I'm thinking of more of a formal test suite here 
than an arbitrary test server farm.

Test categories too, perhaps.

Blocks are probably too fundamental to rebol to worry about at an 
early stage, but unit tests could be made for any kind of behaviour 
or to test for any bug.

Brian, you're probably right.

Maybe RebDB could be used in this project? I've no experience with 
RebDB though, so I can't say, if it's suited.

I've done a relational database "NicomDB" as an education project 
2 years ago. It would be suited for this, and I've wanted to push 
it forward for some time. Maybe this is the opportunity? NicomDB 
is used on a webserver in a real application.

So it needs a database, and a web interface to be able to make queries 
to that.

The expiry and applicability info would help us distinguish between 
changes in intended behavior, buggy implementations, beta-vs-release 
and such. REBOL changes a lot even if many of those changes are fixes. 
This could act as a compatibility test suite for alternate implementations.

NicomDB could be used, possibly. We'd have to evaluate the options.

Geomol, what' s the application NicomDB is serving ?

I'll write you privately, as we don't want it to be exposed too much. 
(Users are depending on it, and this goups is web-public.)

Ok.  But this is becoming a very large and complex project. Are we 
really prepared to go through with it ?  I think even if we go a 
short way, the ideas and findings may help someone else come along 
to complete it later. So I think I will make that Qtask project.

Yes, do that as a start. We can move it later if needed.

I'll do some research and think about this some more. It may not 
be as difficult as it sounds.

rebdb is very easy too ... we now have even sqlite drivers ...

If anyone else wanna see the website powered by REBOL and running 
my NicomDB, write me privately.

Looks ok, nothing broke while I was fiddling with the website. :-) 
That's encouraging but not conclusive of how well it can really perform.

It's running on a FreeBSD server, so it's pretty robust. It hasn't 
been rebooted since July last summer.

is it on-disk db or memory based as rebdb is?

We had some problems with Netcraft sending strange HTTP headers, 
that sometimes made our REBOL application hiccup, but that should 
be solved.

Pekr, data is on-disk, multi-user locking is in memory.

interesting ... binary or plain text based? sql interface? :-)

Pekr, REBOL datatype based (so text, but you can store binary data).

Pekr, no SQL interface yet. I started to do a SQL interface doing 
my education, but didn't finish it. SQL is crap. ;)

why we did not know about this interesting project earlier? :-) was 
it private?

hehe, well. My problem is, I have so many things to do and so little 
time.