[From nobody Fri Feb 10 19:19:19 2006 X-Mozilla-Status2: 00000000 Message-ID: <38CB3C67.C6A71C32@geoserve.net> Date: Sun, 12 Mar 2000 06:42:47 +0000 From: "J.W. Bizzaro" <bizzaro@geoserve.net> X-Mailer: Mozilla 4.6 [en] (X11; I; Linux 2.2.5-15 i586) X-Accept-Language: en MIME-Version: 1.0 To: Jean-Marc Valin <jean-marc.valin@hermes.usherb.ca>, doumdi@yahoo.com, Brad Chapman <chapmanb@arches.uga.edu> Subject: Re: Overflow and Loci References: <38C99E07.F04785BB@geoserve.net> <38C9C595.31EB2B76@hermes.usherb.ca> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hi Jean-Marc! Jean-Marc Valin wrote: > > I'm having a look at you project... FYI, our server is down today (Saturday) but will be back up on Sunday. > have you taken a look at Overflow. I downloaded the binaries, and I had to do some updating to my libraries. BTW, setting VFLOW_PATH to /usr/lib made vflow abort with a core dump. I was only able to get it to work by setting VFLOW_PATH to nothing. > The main > difference I see so far is that your project seems more aimed at distributed > processing using "complex" blocks, while ours is meant to work with "smaller" > blocks. For instance, I use overflow to calculate FFTs on every frame in an > audio file and the overhead has to be small. I noticed that Overflow does use a lot of logic statements for its dataflow. From this, it appears that Overflow is indeed using smaller blocks or nodes. You're right that Loci is working with rather complex nodes. But these nodes (what we call 'loci' or 'locuses') are not considered part of Loci. They are, in fact, independent programs controlled by (or wrapped to run under) Loci. For example, we have wrapped the simple UNIX 'ls' program to be executed and then to pipe its output to a 'viewer'. We are starting with several basic node/locus types: * Processor * Document * Container * Viewer * Flow control The actual data is considered ephemeral/temporary, so nodes/loci actually 'do something to data': * Processor: Modifies data * Document: Stores data * Container: Stores loci * Viewer: Displays data * Flow control: Redirects data And in standard UNIX terminology, you could say that... * Processor: Is an executable * Document: Is a file * Container: Is a directory * Viewer: Is a GUI widget * Flow control: Is a pipe, etc. You see, one of our main goals is to 'bring the UNIX way to the GUI'. Okay, so Loci will use existing, rather large blocks. But we are not (or have not until now) considered restricting Loci to the larger blocks. Maybe that's where Overflow and Loci can distinguish themselves. A question I have about Overflow using 'small blocks': Are these blocks custom-made for Overflow? > On the other hand, our system isn't "network aware". Are you saying that Overflow's nodes cannot communicate over a physical network? Another goal for Loci is to allow the nodes/loci to be placed anywhere on a physical network and to keep that fact hidden from the user. I mentioned before that Loci only 'controls' its nodes. That is extended to the communication protocols used and prefered by these programs. We're calling Loci a 'connectivity' broker, because we want to allow the programs and the users to decide what communication protocol is used. For example, 2 nodes/loci can be placed across the Interner and use HTTP to communicate, or CORBA, or whatever. Oh, and BTW, the name 'Loci' comes from the nodes being placed in different 'locations'. > To be honest, I don't think python would be fast enough for what we do (we've > tried Smalltalk before switching to C++). But what is requiring the speed? Is it the dataflow control or the nodes themselves? Python is fast enough for Loci, because as I mentioned, Loci only brokers connections. Below is a flowchart depicting how Loci works. 'Front' is a shell or user interface (the first of which is being made for Gnome - using Python/Gnome in fact). 'Middle' is the middleware or server/daemon that does the real work. And 'back' is the actual locus/node/block/program/file or whatever, including a wrapper to get it to work with Loci: +-------+ | Front | +-------+ | /|\ | | LOCAL | | LOCAL | | \|/ | +--------+/_________________ | Middle |\ | +--------+ | | | | | INTERNET INTERNET | | | | \|/ | +--------+ +--------+ LOCAL | Middle | | Middle |/_____________ +--------+ +--------+\ | / | LOCAL / | / | +-----+/_/ +-----+___| | |\ | | | Back|____________________________________\|Back | | | ANY PROTOCOL /| | +-----+ +-----+ You can see from the flowchart that the nodes/loci/backs can be across the Internet and use a protocol of their (or the user's) choosing. > However, if you're interrested in > porting some of your stuff to Overflow, I could give you a hand. Also, it might > be feasable to use your GUI with our "core system", since our GUI is in its own > module. This is interesting. How does your GUI communicate with your core? As you can tell from my flowchart, we also separate our GUI. If there is some duplication of effort here (we use the Gnome canvas with pretty little boxes and icons too :-)), this may be a point of collaboration. Perhaps we can both use the same GUI. (BTW, we too are working on a console-based interface, and we're going to have a natural language interface that will let the user 'talk' to Loci. I noticed that another project of yours is speech recognition. I think there could be some exciting points of collaboration in these areas!) Our GUI is not very complete, but we have put A LOT of work into the yet-to-be-implemented design. We have actually been hammering out this design for more than a year and have nearly 2,000 messages on our mailing list. But our GUI is written in Python. You may not think it is fast enough for some of the calculations done by your nodes, but Python/Gnome is great for GUI construction. Also, I think we can make use of Overflow for doing low-level iconic programming. As I mentioned, Loci can embed other programs as nodes (even Loci can be embedded in itself!). This means we could have Overflow work as a Loci node, and have Overflow's workspace appear in a Loci 'windowlet'! I would like us to consider us having a single, shared Gnome GUI in Python. Let me know your thoughts on this. Let me know if you have any questions on the Loci GUI or other parts of Loci. There a lot in our mailing list archives that has not worked its way into the code yet. Feel free to explore the archives when the server is back up (Sunday). (BTW, do you have a mailing list?) Cheers. Jeff -- +----------------------------------+ | J.W. Bizzaro | | | | http://bioinformatics.org/~jeff/ | | | | BIOINFORMATICS.ORG | | The Open Lab | | | | http://bioinformatics.org/ | +----------------------------------+ ]