World: r3wp

finally I got it to work!!!

IE8 required the DTD spec in order make the div center.

looks good on Chrome also.

Yeah, IE has different ideas on when to go to quirks mode.

Yeah I couuldn't figure out why I couldn't get the main container 
div to center on IE8.  Kept messing with it and then read where you 
had to declare the DTD for it to center.

I was going crazy because Chrome was centering it just fine.

Code snippet to "inline" an image:

	string: trim/lines form read/binary file
	remove back tail string

 ajoin [{<img src="data:image/} next suffix? file ";base64," skip 
 string 5 {">}]


Adds about 30-50% to image size but reduces server fetches (important 
for wireless access).

COOL!  I didn't know we could do this.

use ENBASE instead of trim/lines form etc.

Ah, even simpler then:


 ajoin [{<img src="data:image/} next suffix? file ";base64," enbase 
 read/binary file {">}]

Thanx.

remember to take care of client caching issues 8-)

and obviously you can have all the image data in a javascript file 
as well (for which you probably already have a longer cache expire 
set up)

I think the only downside of a data: url is it's not supported in 
IE, iirc.

Anyone  else use Concrete5 CMS?  Pretty good stuff.

Is there anyway to delete wiki spam off rebol.net?  If you just revert 
the page, the spam is still accessible to search engines and still 
gets indexed so the spammer still gets what they want.

ie. it remains in the revision history

the history pages are not supposed to be indexed

and they have `<meta name="robots" content="noindex,nofollow" />` 
in their head to that effect, so simply reverting the change should 
be fine

Except they are being read by my crawler ..

the history pages being _read_ is fine, their content being _indexed_ 
is not :)

Perhaps it's just indexing the user names which are also drug names!

http://129.33.196.33/search/?index=Default&query=albendazole&queryTimeout=3000&ref=http%3A%2F%2F129.33.196.33%3A80%2Fsearch%2F%3Fquery%3Dalbendazole%26queryTimeout%3D3000%26index%3DDefault

Should I exclude these http://www.rebol.net/w/index.php*??

excluding http://www.rebol.net/w/*

Finished now at 27k pages ...

you should also index other sites, like www.codeconscious.com/rebol 
  which has the best complementary view information out there, .

it has helped me on sooo many occasions.

Guess I could do ... but this is just to see if the engine is good 
enough.

I've started to crawl Brett's site .. hope he doesn't mind!

that was an example, but there are others... reboltutorial, Nick's 
learn programing site, Olde's flash site, rebol.org, rebol weekly 
news links, etc etc... I think that having a unique source for all 
of that rich rebol content is very usefull for everyone.

Well as I said someone has to decide if the quality of the search 
engine is good enough or not.

And if so, we need a permanent host for it

Anyone want to do some comparison searchs between google, and this?

Probably has to be Carl as he is the one with the issues!

Looks like it might be using Oracle as the DB ...

well, one of carl's original issues looks just as bad with this search 
engine: http://129.33.196.33/search/?query=construct

I can't find Carl's desired http://www.rebol.com/r3/docs/functions/construct.html
at all in above results :)

Seems parts of the R3 docs are not (yet?) indexed: http://129.33.196.33/search/?query=url%3Aconstruct

I looked at the html source and it should clearly float to the top. 
 strange... 

its got everything needed to be scored high (title, H1, and many 
counts of construct in the page).

Andreas, you're right... same with using a title search with construct. 
 it returns nothing.

Last crawled  	31 December 1969 16:00:00.000 PST
Crawler status 	760 - Excluded by crawl space definition

Parser and index status 	0 - The document has not been added to the 
index.

Looks like my rules were too tight

Hmm.. I had not exclusion rules for rebol.com ... you sure that there 
isn't a no robots directive higher in this path?

Adding http://www.rebol.com/r3/docs/....

if this construct page can't be found by any of the search engines 
... is there a no robots directive ?

Looking for construct now brings up http://www.rebol.com/r3/docs/functions/construct.html
as the top item

33.36k pages indexed

Collection is now 834mb ( 5, 138 documents )

5, 183

cool, we get both functions near the top (R2 & R3) so looks like 
the search engine is stepping up its results  :-)

Try searching for beer on this engine as opposed to google!

There's a lot to be said for a custom site specific search engine.