r3wp [groups: 83 posts: 189283]
  • Home
  • Script library
  • AltME Archive
  • Mailing list
  • Articles Index
  • Site search
 

World: r3wp

[!Liquid] any questions about liquid dataflow core.

Maxim
7-May-2009
[1083x3]
amacleod: I missed your post... just noticed it... don't know why... 
the color makes it stand out... ' :-/
rebol is a pretty closed language in the sense where there isn't 
much room to change rebol itself.  you can obviously replace functions, 
but not the real heart of the engine, the datatypes.  Other languages 
like python let you have access to the complete internals of the 
language.   This is often related to class usage, for which it is 
easier to provide hooks and callbacks. 


rebol is a language which doesn't promote objects as the core paradigm, 
its much closer to imperative programming than most "recent" languages. 
  R3 was/is? supposed to let us build our own datatypes, and has 
been reported as eventually providing for some level of accessors 
for objects.   This will make it easier to integrate tools like liquid 
seamlessly.
I often feel like there is a bridge to cross when using liquid.  
on one side, you have object and imperative code which needs a management 
layer over it, and on the other side, you have code that is more 
"aware", it is self-managed.  It knows what is going on, you tell 
it in advance what it should and then let it work on its own. 


 the hard part is to cross the bridge, both in understanding HOW to 
 use dataflow effectively (and that's not related to the name of the 
 functions ;-),   and then there is the actual coding where you'll 
 pretty much always have to bridge the gap between parts of an app 
 which are dataflow and those that aren't.
Robert
7-May-2009
[1086x3]
Max, I get all this. But how about using the normal graph notation: 
nodes and edges, direct, undirected, cycles etc.
Then extend it by Petri-Net notation using Tokens, Colors etc.
I'm pretty sure your concept maps to this.
Maxim
9-May-2009
[1089]
one reason is that it doesn't map 1:1  plugs are both edges and nodes, 
for example.
RobertS
22-May-2009
[1090]
.
Maxim
26-May-2009
[1091]
a small liquid news tidbit.


liquid-paint is now being used as the core engine for two new commercial 
apps, one is going to be based on paint-lab, my up and comming vector 
paint app, which is already working pretty well.
Janko
27-May-2009
[1092]
can we try or see liquid-paint anywhere :) ?
Maxim
27-May-2009
[1093]
it will be online this week-end... as part of the roll-out of my 
web site.
RobertS
27-May-2009
[1094]
congrats !
Janko
27-May-2009
[1095]
cool
amacleod
27-May-2009
[1096]
I'm eager to se what you got, Maxim.
Graham
27-May-2009
[1097]
I think he means he might be putting up screen shots ... is that 
right max?
Maxim
27-May-2009
[1098]
I intend to put the source on-line.  

the version won't be the latest and sure is not intented to be pretty 
to look at, its just going to show the liquid-paint module in action 
and draw-based event processing.

this being said, this tool is critical in many things I need, like 
building graphs for docs, building my web graphics, server-side graphics 
module for mod_remark, so it will be updated often.
Gerard
1-Jun-2009
[1099]
Will your web site be the actual Steel-toolkit web site or another 
you will start-up ?
Maxim
2-Jun-2009
[1100]
I'm working on a completely new site, with more than just programming.
Graham
6-Jun-2009
[1101]
Did I miss the roll out?
Maxim
6-Jun-2009
[1102]
posted on remark... release post-poned due to client work. ' :-/
Maxim
16-Sep-2009
[1103]
I just want to share this cause I'm all excited... 


I've been working a lot with liquid recently and I think I've just 
tought of an algorithm which would allow "atomic" multithreading 
using liquid.


it would mean building a kernel which manages the nodes, but basically 
several threads would creep up a liquid processing network in an 
async manner and resolve branches out of order, based on a thread 
semaphore and a stack to realign dependencies...  


If this works... It means Elixir OS could possibly have a multithreaded 
kernel, resulting in ALL applications becoming multihreaded without 
any specific coding needed by the developpers.  :-)


adding a few networked services... you could leverge an entire cloud 
(render farm) without even having to code a single specific line 
of cloud and thread stuff, and this would be directly embeded at 
all levels of the OS down to the button properties... cause even 
the most basic gui properties are built up of tiny liquid nodes...
BrianH
16-Sep-2009
[1104]
Sounds cool :)
Maxim
16-Sep-2009
[1105x4]
its going to be a really brain numbing kernel to build, but I really 
see the whole flow in my head.  It doesn't feel impossible like random 
compression so far. ;-)
imagine it like if the kernel, knowing what functions are eventually 
going to be called, pre-emptively starts a concurrent stack and executes 
some stuff in advance, and then inserts the results just as the original 
thread encounters it.  now imagine this using 8 threads in parallel...

brainfuck anyone  ;-)
whenever two parts of a piece of code aren't interdependent, ex:

;-------------------------------
ctx: context [b: "111" prt: func [ data ][ print data return 77]]
result: call-some-func 66
ctx/prt result 
;-------------------------------


they can be forked independently... here both lines 1 and 2 can run 
concurrently, so that the third line, when encountered will already 
have its results, all it needs to do is make sure both ctx and result 
are done.  if it has to wait, then that line is put on the stack 
and the thread jumps to another part of the code in need of processing 
(possibly a part of  call-some-func() can be run concurrently.


when all upper dependencies are met, threads start popping back to 
parts of the code waiting down the stack. until all the code has 
been flushed and the last line of the initial function is all that 
is left to execute.
so instead of having a kernel using a stack its using a tree.  Am 
I right in saying that  this is a shift from the turing machine? 
 :-)
Robert
17-Sep-2009
[1109]
Take a look at petri-nets.
Maxim
17-Sep-2009
[1110]
I have, but I think I should find a good book about them, cause I 
admit what I read about them never fully makes them obvious to understand.
jocko
17-Sep-2009
[1111]
sounds interesting ...do you intend to do some experiments ? I would 
also be interested to know if you will release some new examples 
of liquid.
Maxim
17-Sep-2009
[1112]
some commercial stuff using liquid will be released publicly in a 
short while (within a few weeks maybe sooner).
amacleod
17-Sep-2009
[1113]
what do you mean commercial...stuff you have doen for clients or 
apps you are selling?
Maxim
17-Sep-2009
[1114]
client work which will be released as open source.
amacleod
17-Sep-2009
[1115]
sounds cool...eager to check out...
Maxim
17-Sep-2009
[1116]
I'm really excited... this is probably the furthest I've pushed liquid 
outside of elixir and so far, its the most stable part of my development 
cycle.  View being at the other end of that spectrum.
jocko
18-Sep-2009
[1117]
coming back to your reflexions on a liquid based processing network, 
it recalls me a distributed code programming language : "wave system", 
developed in the 90's, and allowing to spread a code in a large and 
unstructured network for neighbour to neighbour, wher each node executes 
a part of the code, and provides a part of the information requested, 
and transmits. It is unfortunately difficult to find info on this 
system, which was developed by Peter Sapaty. It seems that it was 
recently used in a light version for network problems studies : "wiseman" 
http://www.ece.ubc.ca/~vleung/IWCMC2008Keynote VLeung.pdf , 
and http://www.ece.ubc.ca/~sergiog/wiseman/WisemanManual.pdf
Maxim
18-Sep-2009
[1118]
very interesting talk  I'll read it to the end  its close to my idea 
for sure  :-)
Maxim
24-Jan-2010
[1119]
just thought I'd share that I've successfully optimized liquid by 
a huge margin, just by removing all the verbose print statements 
in the code.


10 minutes of deleting code... equals to an order of magnitude more 
responsiveness in some items of a graphic application I'm working 
on !

if only all of life where this simple  ;-D
Robert
24-Jan-2010
[1120]
Max, I will bite the bullet and will see if I get a graph engine 
done in C and attched to R3 via an extension.
Maxim
24-Jan-2010
[1121x2]
I'm sure you can  :-)
note that I already have access to a fully functional and highly 
optimised C-based graph engine which allows me to create billions 
of nodes, and much more.  


Integrating that system into REBOL is part of my current contract 
work.
Robert
24-Jan-2010
[1123]
Will you tell us what engine it is?
Pekr
24-Jan-2010
[1124]
I expect it being Max own implementation ... Liquid, globs, and all 
his stuff ...
Robert
24-Jan-2010
[1125]
In C? I doubt that...
BrianH
24-Jan-2010
[1126]
Nope, it's C (or maybe C++, I forget). He prototyped it in REBOL 
then went native for speed/scale.
Maxim
24-Jan-2010
[1127]
its not my engine, and am not at liberty for now... it will become 
public at some point though, its not my code, so I'm under NDA.
BrianH
24-Jan-2010
[1128]
Ah, too bad. Still, sounds fun :)
Maxim
24-Jan-2010
[1129:last]
The engine will use liquid's flexible interpreted messaging overlayed 
on the other graph engine which I will use for scalability in sheer 
volume of connections and nodes I can allocate.  just a portion of 
the tree usually needs to be in RAM at any given time, and in fact, 
parts of processing tree can now reside on different computers since 
the graph engine is refered to... it should be quite fun to use.


this will be tied in to the OpenGL and scream core scene-description 
engine as one cohesive toolset.


in this system, the binary nodes will actually be optional and should 
be invisible when used from the rebol application's point of view.