World: r3wp

Maxim -- thanks....the diff function code is an embarrassing collection 
of random hacks. I'm hoping someone else will write a better version 
that could be made public :-)

well what I am seeing impressed me ... I wasn't expecting that on 
rebol.org... really you should allow us to compare right in the update 
page.  I know I'd use it every single time  :-)

might even use rebol.org more :-)

in fact... might = will

I attempted to rename button.r to vid-button.r so the new script 
should be deleted and the old one renamed if possible.  As for the 
rest of them...

wizard.r  TO vid-wizard.r
group.r TO vid-group.r    
drop-down.r TO vid-drop-down.r   
dragbar.r TO vid-drag-bar.r       
date.r TO vid-date.r


Since we won't be calling the GUI Dialect in R3 VID the new names 
should make what the scripts do clear even after R3 GUI scripts start 
popping up...

Thanks Max -- I'll take a look at making that an option on upload.

sunanda COOL  :-)  and promote it on rebol week, on the ML and in 
"announce"  here too, really I think its a big deal/feature.


 I woundn't be surprised that most rebol.org users don't even know 
 the diff was there  ;-)

Maxim -- not sure it'll be that *cool* -- most people are probably 
not aware of it as you can only see it in action for the scripts 
you own.

Ammon -- thanks again for the license link bug. It's fixed now.....Shows 
how seldom anyone checks license T&Cs for scripts :-)

=D

sundanda:  its cool for script owners.   It adds value in suddenly 
being more than just a storage space... I can even check my code 
as I submit it.

for me anyways, it adds a feature that I can really use ... an online, 
simple VCS.


I can make sure that a new release not only is shared, but also free 
of a set of bugs.  I can suddenly say.. what did I change... and 
more easily comment on it, for example.

Ammon -- those scripts are now renamed as reqeusted.

Sweet!  Thanks.

I just added ascii-math.r to the library. It looks fine in the  view 
, but the downloaded file contains gibberish characters before each 
non-standard ascii char. This problem may be related to the problem 
that Oldes was having on 14-Mar-08.

Thanks for the script -- and for the problem report.

Looks to me like the script, as uploaded, contained non-UTF8 characters, 
and they are being treated as multi-byte characters.

REBOL, and REBOL.org can really only handle ASCII....Unicode and 
such like is for R3.


If you email (preferably in a ZIP to prevent email software from 
chewing up the code) the original script, I'll take a look.

so you support utf8?good to know.. next time I will upload it as 
utf8

The library doesn't support utf-8 yet. We  have found that many people's 
browser are set so that the browser renders the output from rebol.org 
as utf-8.

In this way, the library accidentally supports utf-8 in the sense 
that if you upload utf-8 and display it in a browser set to display 
utf-8 everything will be displayed properly.

The core of the library system is old enough that it was written 
without considering character encoding at all.

Supporting utf-8 will require a lot of changes ..... though probably 
not quite as many as moving to R3.

If most of it is currently ascii, would it not just be a case of 
adding a few filters?

'it', being current content.

Preventing any new content from posting invalid sequences, for example...

Peter beat me to it, thanks.

Sorry Oldes, the Library does not support utf-8, despite my confused 
suggestion that it did.

Because we use a charset of UTF-8 in the browser header, it is _possible_ 
that we can more-or-less handle scripts with 2+ byte UTF-8 codings 
in REBOL strings! But that's not been tested.


Good point, Chris -- we already have such a filter, but it is not 
used to turn back conributions.

Sunanda, I have emailed the zipped script to you.

Chris, one issue that we face with the mailing list archive is not 
knowing how the imput is encoded. I think this is also true of scripts.

Thanks, Scott.
--The email script looks fine
-- it's identical to what is in the Library
-- Viewing the scriot works fine
-- Downloading it doesn't

.....Which is exactly what you reported. We are now both on the same 
page :-)

We've had a similar problem before (I've just checked the source 
code, and it's prompted my memory).


To solve it, we analyse the script for various extended ascii chars 
and then perform some messing around on HTTP content-type headers.


It's messy, and it's worked up til now......But obviously, we need 
some more analysing and messing around for this script.

Results of a tiny bit of debugging on the ascii chars problem:
-- problem seems to be at the input stage:

     -- if you have exended ascii characters (top bit set, like the 1/4 
     used in the script) what we get from the webserver is bad (extra, 
     unexpected extended ascii chars)

    -- only download is (visibly) affected, although the extra extended 
    ascii chars are present in the text streams

     -- though there is some REBOL mezz code (decode-cgi) that may be 
     doing something I do not understand

    -- I can replicate the problem with both Apache and Xitami which 
    suggests the problem may be in REBOL rather than a given server.


-- the quick fix would be to add accept-charset="ISO-8859-1" to the 
<form ....> or <textarea ....>

    -- but that stops all extended ascii, including the ones we want. 
    So we won't do that.

-- the slower fix has yet to emerge from the available options.

why not standardize everything on UTF-8?

As far as I know, Core 2.5.6 (what the Library CGIs runs on) does 
not support UTF-8.

does not support UTF-8

 - what do you mean by "support"? if you mean having native encoders/decoders, 
 no, it does not. but, utf-8 is just 8 bit characters, and it is backwards 
 compatible with ascii. if you can handle ascii, and leave alone any 
 char > 127, you already support utf-8.

Sadly, REBOL running as CGI under the two servers I've tested (Apache 
and Xitami) does not seem to support the whole range of ASCII -- 
ASCII chars with the 8th bit set seem to cause problems.

I don't know where the problem really is, but right now, we do not 
even support 8-bit ASCII, let alone anything more modern :-)

At the moment, I'd be worried about standarising the Library on utf-8 
as the effect of multibyte characters would have during script and 
mail processing is not understood. It could well be that the system 
handles multibyte characters without a hitch but nobody knows yet.


I have started to write some scripts to try to help move to a consistent 
character encoding of the Library data but, due to time constraints, 
I have been very slow.

Why worry? Just do it. :-P

What version of rebol is being used by rebol.org ?

Sunanda, can you publish some files with the 8-bit ascii and note 
what the problems are ?

sunanda, you can force the character encoding in the html page header... 
I've used that before and it worked for me.

note, I don't mean the http header, but the actual <HEAD> tag.

I had the same kind of issues on another system.  nowadays, the default 
encoding has become UTF-8 for many/most html handlers, so if its 
not specified, many new browers and tools will incorrectly break 
up the character data.

Anton, REBOL.org uses 2.5.6.4.1

The obvious bad file is the one Scott added recently:
http://www.rebol.org/view-script.r?script=ascii-math.r

If you view it with that URL, all looks good.

If you click the [Download script] link you'll see many spurious 
high-ascii chars in the source.

Those high ascii _are_ actually in the source. But where they came 
from is a mystery.

Maxim, REBOL.org emits a header

   <meta http-equiv="Content-Type" content="text/html;charset=utf-8">

Yeah, I know we aren't utf-8 -- but experiment has shown that's the 
moste acceptable charset.

Not sure what you are saying we could put in <head> -- can you be 
more specific.

there is a specific charset for western -iso, which ensure the extra 
127 bytes are correct.


<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">

Thanks......We used to have that, but it created some other problems. 
I'll have to try to remember what and why :-)

And it does not solve the download problem (I know, I tried yesterday).

I think the root of the problem is that when the Library system was 
first written, no account was taken of character encoding. As a result, 
not only is the data encoded as it was when originally submitted 
but the method of encoding is not even known.


Whatever charset is specified in the http header is not going to 
be correct for all scripts and messages. Using charset=utf8 seems 
to cause the least problems. Though for example, it will cause many 
ISO-8859-1 "high bit" characters to be incorrectly displayed.

Do you have any stats on how many 'high bit' characters are now contained 
in Library content?

Or scope? - minimal; limited; too many to be trivial...

Re. ISO-8859-1 - the most obvious problem is the limitation - 256 
chars vs. UCS-1+

No actual stats. Just from feel:
* Scripts -- very few
* Posts on the ML -- a few dozen
* AltME archive -- no idea

Sunanda, I can tell you where does chars come from. if your page 
is set as utf-8, then the script as been uploaded by the browser 
as utf-8. when you view it in the brower, it shows correctly as utf-8. 
when you download it, it is still utf-8, but if you view it with 
something that believes it's latin1 (eg. the rebol 2 console on windows 
set as latin1), it won't show up correctly.

Sunanda, you're right about that ascii-math.r file. When I clicked 
the [Download script] link, the browser (konqueror) downloaded and 
directly opened it with the editor (SciTE). SciTE thought it was 
8-bit ascii, and showed the characters incorrectly. All I had to 
do was change the file encoding from 8-bit to utf-8 and the characters 
appeared correctly. I guess the editor had no way of determining 
the encoding, and incorrectly guessed 8-bit ascii.