[Pipet Devel] Code updates

[Brad Chapman]

Hey everyone;
    Just wanted to let everyone know that I just made a huge cvs 
commit, and now the front and middle are officially separated! Yay! 
All communication between them is now going through sockets, and I 
should have restored all of the functionality that we had previously, 
with a few exceptions:

1. My "wrapping"/connection code isn't currently compatible with the 
structure of everything, so it isn't used right now. It probably won't 
be very useful in the future either, with GMS and Overflow and all so 
it'll probably be out for a while, if not permanently.

2. The database connectivity stuff only half works because of some 
issues I'm having with sockets (more on this below). You can only 
connect to a gadfly db, and can't view the contents of tables. 

If anything else generates errors, please post on it. I've fixed the 
error Jeff noticed with containers (thanks for this!), and so 
everything except for what I've mentioned _should_ work right.
    I also added a doc directory which has the communication protocols 
I'm using kind of outlined.
    Maybe if Jeff is feeling like he has free time, and the code seems 
okay, he might be able to develop a new snapshot from this, so people 
can check it out:) If you are taking a look at the code, the parts to 
look at are basically divided as following:

front/gnome/library and subdirectories: The code for the gnome/gtk 
based front end. 

middle/library/modules/xml: This is all of the code for the current 
middle/core. Basically it contains code to create a server to 
communicate with the front via XML through sockets, code to create 
XML descriptions of the workspace (which can be seen in middle/temp 
when the Loci program is running), and code for dealing with the 
filesystem and databases.

middle/library/modules/wrap: The old barely started "wrapping" 
and "connectivity" code. This isn't currently used, so it can be 
ignored.

There is tons o' commenting, but if stuff isn't clear, please feel 
free to post. Any comments/criticism are highly encouraged. I 
apologize, I'm having trouble getting any of the docstring processing 
programs to work without complaining, so there isn't any nice api 
documentation like there should be.

    Okay, the rest of this has to do with problems I'm having with the 
socket connection between the front and back. Basically, I'm having 3 
issues:

1. Speed: Both the database and filesystem container views come up 
*really* slowly. This is due (I think) to all the extra dom/xml 
generation, and also the lag time waiting for the front.

2. Password passing: I've been passing around passwords for databases 
in an encrypted format (using the rotor module in the standard 
distribution). When encrypted, a password is a jumble of random 
characters. The problem I'm having is in passing these through the 
socket. When the server receieves a message, it receieves it in small 
pieces of strings, and then adds them up (until it gets a newline, the 
end of message signal). The problem is that python doesn't like to add 
the weird characters generated in the encryption, and so complains 
when it tries to add these.

3. Data passing: One thing which we haven't talked about, although I 
should have thought of it before, is how to pass a large amount of 
"unstructured info" between the front and the middle. For instance, 
how should I pass back the columns and row information for a database? 
    If I format this as XML, this will take impossibly forever for 
large tables (which I have quite a few of in my databases), but I 
can't think of another way way to pass info cleanly and reusably. 

Does anyone have any thoughts on fixes for these problems? 
Suggestions? Am I misssing something simple? 

Thanks for reading! 

Brad