[Pipet Devel] Propoal for integrating Overflow

[Brad Chapman]

Hello all;
    I've been thinking a lot about how to start getting Overflow 
integrated into Piper so that we can start getting all three programs 
working together to actually start doing something. Towards this goal, 
I went back to the simple example of using the unix command "ls" as 
our starting point. What I'd like to try and do is use the Overflow 
ExecStream class (in data-flow/src/ExecStream.cc), which runs a 
command line argument, returning the output as a string. So the simple 
goal is to re-do what I had going a long time ago (but now in a real 
format that will actually do something more interesting later) and 
pipe the output of an ls statement into a text viewer.
    So here is the way I'd like to propose to do that:

1. Create a simple XML document to describe 'ls'. I've done this 
already (yay! step 1 done!) and it is at the bottom of this mail, 
along with a dtd which is just a start towards generating a format for 
writing these xml files). The idea is that this XML file can be used 
to read all of the information about how to deal with a program. Right 
now, we can just write these by hand (for simplicity) and later can 
explore Jeff's plans to have the user be able to develop these in the 
user interface, and Jean-Marc's ideas to be able to generate these 
files automatically by dlopening a library. 

2. Use this xml file to be able to build a command line in the user 
interface. Right now I propose that we use a really simple widget, 
with a text box to type in the command line, and a button which pops 
up the description. This will free Jeff and Deanne from having to feel 
pressured to produce something quickly, but will still allow everyone 
else to be able to use the UIs for testing (since it is hard to test 
things when you have nothing to drive them :-). Of course, this UI can 
expand to be a lot more user friendly as Jeff and Deanne see fit. I 
need to work on the dl as well to be able to give this description to 
the uil in a nice format.

3. Once the work-flow diagram (of a 'ls' command piped into a viewer) 
is completed, the ui will need to submit this diagram to the dl. From 
there, the dl will need to convert this diagram into a Overflow style 
XML file. This is coming along as I've wrapped the Overflow classes 
that do this...

4. Have the dl submit the created xml description to gms (the bl). 
Jarl and I worked out this interface and are working on connecting us, 
so hopefully this will be ready soon.

5. For right now, just have the bl submit the xml document directly to 
Overflow, and use Overflow's 'pull technology' to run the document and 
return the output to the bl. Looking at Overflow, I'm not exactly 
positive about the way to do this in Overflow, but it seems like using 
the UIDocument class (in vflow/src and include) with a 
UIDocument::load and a UIDocument::run might be the way. I think there 
might need to be some modifications here becaue of the next step...

6. Overflow (the pl) returns the results to the bl. Then the bl passes 
them back to the dl, and voila, they are sent back to the user 
interface where they are displayed.

    I know this isn't perfect and that 'ls' isn't the most exciting 
program to get working through this system but I think this might 
provide us with some initial connectivity, and a place to build on for 
more exciting things, and will help us get the problems connecting 
things ironed out right away. What do people think of this idea? 
Opposition? Suggestions? Mistakes I made? Things I didn't think of? 
Random comments about unbelievably bad the Detroit Tigers are doing 
yet again this year? Anything?
    Anyways, thanks for reading all the way though this. Below, for 
your further reading enjoyment, are the ls.xml class and the _very 
preliminary_ dtd for representing a program "plugged in" to Piper. 
These will be in cvs whenever piper gets to go to its new home.

Brad

<!-- Example document for plugging 'ls' in to work with Piper -->

<!DOCTYPE piperplugin SYSTEM "piperplugin.dtd">

<piperplugin>

<module name = "unix">
<module name = "utilities">
  <node name = "ls" type="ExecStream">
  
    <parameter name = "COMMAND"/>
	
	<output name = "STRING_OUT"/>
	<output name = "STATUS_OUT"/>
  
    <description>
	 ls [-ACFLRTWacdfiloqrstu1] [file ...]

     For each operand that names a file of a type other than 
directory, ls
     displays its name as well as any requested, associated 
information.  For
     each operand that names a file of type directory, ls displays the 
names
     of files contained within that directory, as well as any 
requested, asso-
     ciated information.

     If no operands are given, the contents of the current directory 
are dis-
     played.  If more than one operand is given, non-directory 
operands are
     displayed first; directory and non-directory operands are sorted 
sepa-
     rately and in lexicographical order.
	</description>
	
  </node>
</module>
</module>
</piperplugin>

<?xml version="1.0" encoding="US-ASCII"?>

<!--Preliminary dtd for pluging programs in to work with piper-->

<!ELEMENT piperplugin (module)>

<!ELEMENT module (module | node)>
<!ATTLIST module 
   name CDATA #REQUIRED>

<!ELEMENT node (parameter*, input*, output*, description?)>
<!ATTLIST node
   name CDATA #REQUIRED
   type CDATA #REQUIRED>

<!ELEMENT parameter EMPTY>
<!ATTLIST parameter 
   name CDATA #REQUIRED>

<!ELEMENT input EMPTY>
<!ATTLIST input
   name CDATA #REQUIRED>

<!ELEMENT output EMPTY>
<!ATTLIST output
   name CDATA #REQUIRED>

<!ELEMENT description (#PCDATA)>