r3wp [groups: 83 posts: 189283]
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

World: r3wp

[Core] Discuss core issues

Joe
14-Jan-2007
[6633]
Gabriele, any news about when the new async core will be released 
(it was almost ready TWO years ago!)
Pekr
14-Jan-2007
[6634]
Joe - I wonder if it gets ever released. RT is now dedicated to R3, 
so I would not away some major architecture changes in 2.x family. 
And as R3 should support tasking probably via threads, my suspicion 
is, that we are going to be playing cheap tricks - suggesting ppl 
to use new thread for each new connection, instead of implementing/fixing 
async mode. IIRC even Gabriele expressed himself in the past, that 
coding async is not all that easy, and that it might be better to 
use threading ...
Joe
14-Jan-2007
[6635]
iirc there was a blog post mentioning that it would get added in 
1.3.x . I hope so !  Async networking is a must for decent performance 
apps !
Gabriele
15-Jan-2007
[6636]
petr, it's not a cheap trick. r3 is going to be better than the async 
core.
Pekr
15-Jan-2007
[6637x2]
Gabriele - I hope so - otoh - what is the expense of threads? I looked 
into my friends code programming server in Delphi - they used exactly 
that - simply new thread for each connection. I would like to know 
what is the overhead - speed, memory etc. I called it "a cheap trick", 
as my friend could not even imagine, how he would do a multiplexed 
scenario ...
With current aproach I at least learned difference between blocking, 
non-blocking io and correspondive techniques. My opinion is, that 
underlying engine (REBOL) should not use task/thread unless I ask 
it to do it. That is why I think that kernel should be anync in its 
roots too ....
Gabriele
15-Jan-2007
[6639x5]
well, i guess i should say again what i said it devcon 2004 with 
my async core presentation.
the first thing i said was: "why async?"
for two reasons: 1) it gives you more control over things; which 
means it is harder than threading, but threading is just "looking" 
simpler, while it actually isn't, so with threading you get many 
hard to debug problems later on. with async you get the hard part 
in the beginning, so you don't get fooled into believing it works 
while it doesn't. 2) rebol does not have threading.
so... with r3, the second reason just goes away.
let's get on the first reason: why did I say that threading is worse? 
for the reason holger explained a lot of time ago on the mailing 
list. shared memory.
Pekr
15-Jan-2007
[6644]
ok, maybe I mix things incorrectly. I will ask other way - async 
and non-blocking are two separate things, right? I don't need async 
console etc., that I might solve via threading. But - will it be 
possible to have fully non-blocking behavior? Even for DNS reads 
(it is now) and opening connection?
Gabriele
15-Jan-2007
[6645x4]
that is, handling concurrency in a threaded app is hard, so they 
just fool you into thinking that threading is easier.
but - now this is the difference: as far as i know, our goal with 
r3 is to have no sharing between threads (from the point of view 
of the user, of course) unless really necessary.
that is, the user does not have to handle concurrency. you just talk 
with other "threads" (tasks would probably be a better term) with 
ports
which must mean, that the ports in question must be async.
Pekr
15-Jan-2007
[6649]
Gabriele - so what is "tasking" in R3 going to be under the hood? 
There were various things said - 1) we will have something like make 
task! syntax, which will use OS threads in the background, just hiding 
the complexity of threading model from user (= simplicity in rebol 
level) 2) In one of recent blogs and their comments Carl said he 
might have its own way of how to do tasking in Rebol. So, how will 
it look? Is it already decided?
Gabriele
15-Jan-2007
[6650]
what you describe above, can be easily obtained by doing the dns 
resolution in one task. what you see is a port, that to you looks 
exactly like an async dns port. except that it is implemented in 
mezz code rather than being native.
Pekr
15-Jan-2007
[6651]
Gabriele - so you already partially answered my question - OS threads 
in the low level, but from the user pov it will be abstracted by 
eg. make task! syntax plus port model?
Gabriele
15-Jan-2007
[6652x2]
that is, from your point of view it could look exactly like the async 
core.
exactly.
Pekr
15-Jan-2007
[6654x2]
Gabriele - we have async dns already, don't we? dns:/// .... but 
what is currently blocking without ability to set timeout is trying 
to establish a connection (SYN) ... or can we make it also a non-blocking?
If there is a way of how to do fully non-blocking communication, 
as for me, I don't need async ... if I need "async" behavior, I might 
use rebol tasks ....
Gabriele
15-Jan-2007
[6656x3]
hmm, you can make connections asynchronously
eg. when you make a connection with async:// or atcp:// there is 
no wait involved
the point rather is, than normally, instead of delving into the complexity 
of an asynchronous event based interface, you just make a task and 
use read.
Pekr
15-Jan-2007
[6659]
ah. But was it possible with old cores? IIRC Holger explained that 
we can't influence connection ... if it was corrected, it should 
be satisfactory ...
Gabriele
15-Jan-2007
[6660]
i haven't done measurements accurate enough to tell you if there's 
some little blocking somewhere. but in practice i don't see any blocking 
with current core and async tcp
Pekr
15-Jan-2007
[6661]
Gabriele - correct - after all - what we want from REBOL is - provide 
simplicity in user level. Not all users can think in complex async 
way ....
Gabriele
15-Jan-2007
[6662]
in principle, you could implement this, in native code, by using 
the async core and implementing tasks over that.
Pekr
15-Jan-2007
[6663]
Gabriele - what was the reason Chord was not successfull in Rebol? 
IIRC you said it is difficult to achieve with current kernel. Why? 
If we have fully non blocking kernel, should there be any problem?
Gabriele
15-Jan-2007
[6664x2]
however, that would have the big disadvantage, of not making use 
of multiple processors if they are present.
also, it's probably harder to do, when the os already provides threading, 
and the os libs are good with threading but not with your system.
Pekr
15-Jan-2007
[6666]
hmm, that is interesting situation indeed. First we thought of parralelism 
as multiple CPUs in the system. Now the trend is to place multiple 
CPU cores in one CPU. I wonder if we can influence, what task/thread 
in OS uses what Core? Is there any API, or CPU decides on its own?
Gabriele
15-Jan-2007
[6667x2]
Chord - problem number one, udp:// has bugs that make it inusable 
in async manner. problem number two, it just stopped working after 
a couple minutes, and figuring out why was incredibly difficult. 
(maybe it'd work on 1.3... but i haven't had time to try it)
i think the os decides how threads/processes are scheduled.
Pekr
15-Jan-2007
[6669]
udp has bugs? Why not fix them then? :-)
Gabriele
15-Jan-2007
[6670x2]
they are on rambo.
they will probably be fixed someday :)
Pekr
15-Jan-2007
[6672x2]
as for R3 - networking layer is the one ported from 2.x, just plugged 
into tasking/threading model, or is it new implementation? (IIRC 
Carl said port model is going to be a bit reimplemented?)
re "fixed someday" - I thought that it is essential in regards to 
networking communication language to adhere to communication standards 
and protocol behaviour :-)
Gabriele
15-Jan-2007
[6674x2]
i would guess it is a new implementation. there are two reasons it 
must be: the os code has to be separate, and the internal model is 
probably very different.
well, udp works, as long as you don't try to use it in a 100% async 
manner.
Pekr
15-Jan-2007
[6676]
ok, thanks for your answers - looking forward to R3 Core alpha, hopefully 
released at least for DevCon time :-)
Gabriele
15-Jan-2007
[6677]
probably noone except me used it that way, so noone except me complained 
about the bugs in udp (i think there was another person at least... 
but we're just two)
Pekr
15-Jan-2007
[6678]
my first tests will be amiga one - generating 10K processes and looking 
how system is stable :-) Amiga was very OK with such tests once you 
had enough memory :-)
Gabriele
15-Jan-2007
[6679]
that might not kill rebol, but it might kill windows. ;)
Pekr
15-Jan-2007
[6680x3]
hehe, exactly :-)
but it will be interesting comparison too - for apache, there is 
some "standard" testing tool done in perl, measuring its performance. 
We should try two models and compare - having rebol http server, 
first using non-blocking io multiplexing, one task, second generating 
new thread for each connection - measure response, amount of memory 
consumed etc ....
going to lunch, thanks for the chat ...