World: r3wp
[!Liquid] any questions about liquid dataflow core.
older | first |
Maxim 7-May-2009 [1083x3] | amacleod: I missed your post... just noticed it... don't know why... the color makes it stand out... ' :-/ |
rebol is a pretty closed language in the sense where there isn't much room to change rebol itself. you can obviously replace functions, but not the real heart of the engine, the datatypes. Other languages like python let you have access to the complete internals of the language. This is often related to class usage, for which it is easier to provide hooks and callbacks. rebol is a language which doesn't promote objects as the core paradigm, its much closer to imperative programming than most "recent" languages. R3 was/is? supposed to let us build our own datatypes, and has been reported as eventually providing for some level of accessors for objects. This will make it easier to integrate tools like liquid seamlessly. | |
I often feel like there is a bridge to cross when using liquid. on one side, you have object and imperative code which needs a management layer over it, and on the other side, you have code that is more "aware", it is self-managed. It knows what is going on, you tell it in advance what it should and then let it work on its own. the hard part is to cross the bridge, both in understanding HOW to use dataflow effectively (and that's not related to the name of the functions ;-), and then there is the actual coding where you'll pretty much always have to bridge the gap between parts of an app which are dataflow and those that aren't. | |
Robert 7-May-2009 [1086x3] | Max, I get all this. But how about using the normal graph notation: nodes and edges, direct, undirected, cycles etc. |
Then extend it by Petri-Net notation using Tokens, Colors etc. | |
I'm pretty sure your concept maps to this. | |
Maxim 9-May-2009 [1089] | one reason is that it doesn't map 1:1 plugs are both edges and nodes, for example. |
RobertS 22-May-2009 [1090] | . |
Maxim 26-May-2009 [1091] | a small liquid news tidbit. liquid-paint is now being used as the core engine for two new commercial apps, one is going to be based on paint-lab, my up and comming vector paint app, which is already working pretty well. |
Janko 27-May-2009 [1092] | can we try or see liquid-paint anywhere :) ? |
Maxim 27-May-2009 [1093] | it will be online this week-end... as part of the roll-out of my web site. |
RobertS 27-May-2009 [1094] | congrats ! |
Janko 27-May-2009 [1095] | cool |
amacleod 27-May-2009 [1096] | I'm eager to se what you got, Maxim. |
Graham 27-May-2009 [1097] | I think he means he might be putting up screen shots ... is that right max? |
Maxim 27-May-2009 [1098] | I intend to put the source on-line. the version won't be the latest and sure is not intented to be pretty to look at, its just going to show the liquid-paint module in action and draw-based event processing. this being said, this tool is critical in many things I need, like building graphs for docs, building my web graphics, server-side graphics module for mod_remark, so it will be updated often. |
Gerard 1-Jun-2009 [1099] | Will your web site be the actual Steel-toolkit web site or another you will start-up ? |
Maxim 2-Jun-2009 [1100] | I'm working on a completely new site, with more than just programming. |
Graham 6-Jun-2009 [1101] | Did I miss the roll out? |
Maxim 6-Jun-2009 [1102] | posted on remark... release post-poned due to client work. ' :-/ |
Maxim 16-Sep-2009 [1103] | I just want to share this cause I'm all excited... I've been working a lot with liquid recently and I think I've just tought of an algorithm which would allow "atomic" multithreading using liquid. it would mean building a kernel which manages the nodes, but basically several threads would creep up a liquid processing network in an async manner and resolve branches out of order, based on a thread semaphore and a stack to realign dependencies... If this works... It means Elixir OS could possibly have a multithreaded kernel, resulting in ALL applications becoming multihreaded without any specific coding needed by the developpers. :-) adding a few networked services... you could leverge an entire cloud (render farm) without even having to code a single specific line of cloud and thread stuff, and this would be directly embeded at all levels of the OS down to the button properties... cause even the most basic gui properties are built up of tiny liquid nodes... |
BrianH 16-Sep-2009 [1104] | Sounds cool :) |
Maxim 16-Sep-2009 [1105x4] | its going to be a really brain numbing kernel to build, but I really see the whole flow in my head. It doesn't feel impossible like random compression so far. ;-) |
imagine it like if the kernel, knowing what functions are eventually going to be called, pre-emptively starts a concurrent stack and executes some stuff in advance, and then inserts the results just as the original thread encounters it. now imagine this using 8 threads in parallel... brainfuck anyone ;-) | |
whenever two parts of a piece of code aren't interdependent, ex: ;------------------------------- ctx: context [b: "111" prt: func [ data ][ print data return 77]] result: call-some-func 66 ctx/prt result ;------------------------------- they can be forked independently... here both lines 1 and 2 can run concurrently, so that the third line, when encountered will already have its results, all it needs to do is make sure both ctx and result are done. if it has to wait, then that line is put on the stack and the thread jumps to another part of the code in need of processing (possibly a part of call-some-func() can be run concurrently. when all upper dependencies are met, threads start popping back to parts of the code waiting down the stack. until all the code has been flushed and the last line of the initial function is all that is left to execute. | |
so instead of having a kernel using a stack its using a tree. Am I right in saying that this is a shift from the turing machine? :-) | |
Robert 17-Sep-2009 [1109] | Take a look at petri-nets. |
Maxim 17-Sep-2009 [1110] | I have, but I think I should find a good book about them, cause I admit what I read about them never fully makes them obvious to understand. |
jocko 17-Sep-2009 [1111] | sounds interesting ...do you intend to do some experiments ? I would also be interested to know if you will release some new examples of liquid. |
Maxim 17-Sep-2009 [1112] | some commercial stuff using liquid will be released publicly in a short while (within a few weeks maybe sooner). |
amacleod 17-Sep-2009 [1113] | what do you mean commercial...stuff you have doen for clients or apps you are selling? |
Maxim 17-Sep-2009 [1114] | client work which will be released as open source. |
amacleod 17-Sep-2009 [1115] | sounds cool...eager to check out... |
Maxim 17-Sep-2009 [1116] | I'm really excited... this is probably the furthest I've pushed liquid outside of elixir and so far, its the most stable part of my development cycle. View being at the other end of that spectrum. |
jocko 18-Sep-2009 [1117] | coming back to your reflexions on a liquid based processing network, it recalls me a distributed code programming language : "wave system", developed in the 90's, and allowing to spread a code in a large and unstructured network for neighbour to neighbour, wher each node executes a part of the code, and provides a part of the information requested, and transmits. It is unfortunately difficult to find info on this system, which was developed by Peter Sapaty. It seems that it was recently used in a light version for network problems studies : "wiseman" http://www.ece.ubc.ca/~vleung/IWCMC2008Keynote VLeung.pdf , and http://www.ece.ubc.ca/~sergiog/wiseman/WisemanManual.pdf |
Maxim 18-Sep-2009 [1118] | very interesting talk I'll read it to the end its close to my idea for sure :-) |
Maxim 24-Jan-2010 [1119] | just thought I'd share that I've successfully optimized liquid by a huge margin, just by removing all the verbose print statements in the code. 10 minutes of deleting code... equals to an order of magnitude more responsiveness in some items of a graphic application I'm working on ! if only all of life where this simple ;-D |
Robert 24-Jan-2010 [1120] | Max, I will bite the bullet and will see if I get a graph engine done in C and attched to R3 via an extension. |
Maxim 24-Jan-2010 [1121x2] | I'm sure you can :-) |
note that I already have access to a fully functional and highly optimised C-based graph engine which allows me to create billions of nodes, and much more. Integrating that system into REBOL is part of my current contract work. | |
Robert 24-Jan-2010 [1123] | Will you tell us what engine it is? |
Pekr 24-Jan-2010 [1124] | I expect it being Max own implementation ... Liquid, globs, and all his stuff ... |
Robert 24-Jan-2010 [1125] | In C? I doubt that... |
BrianH 24-Jan-2010 [1126] | Nope, it's C (or maybe C++, I forget). He prototyped it in REBOL then went native for speed/scale. |
Maxim 24-Jan-2010 [1127] | its not my engine, and am not at liberty for now... it will become public at some point though, its not my code, so I'm under NDA. |
BrianH 24-Jan-2010 [1128] | Ah, too bad. Still, sounds fun :) |
Maxim 24-Jan-2010 [1129:last] | The engine will use liquid's flexible interpreted messaging overlayed on the other graph engine which I will use for scalability in sheer volume of connections and nodes I can allocate. just a portion of the tree usually needs to be in RAM at any given time, and in fact, parts of processing tree can now reside on different computers since the graph engine is refered to... it should be quite fun to use. this will be tied in to the OpenGL and scream core scene-description engine as one cohesive toolset. in this system, the binary nodes will actually be optional and should be invisible when used from the rebol application's point of view. |
older | first |