[Pipet Devel] hierarchy (was: icons)

[Brad Chapman]

Jeff wrote:
> So, we can nix the whole concept of any real "BL network".

Okay dokee. It seems like this was kind of confusing terminology 
anyways. Maybe we should refer to remote stuff with the same 
terminology as local stuff, just with the word "remote" prepended to 
it? So we would have a "remote network" or a "remote node."

> The BL network that I mentioned before is/was a "collection of nodes 
where
> each node is an instance of the BL (or Piper) located on a 
different, remote
> computer".  This means the nodes communicate via BL -> BL 
connection.  It
> seems though, looking again at your whitepaper, that a BL -> BL 
connection is
> supposed to be a BL -> DL -> DL -> BL connection.
> 
> Brad, can you  explain why the BL's can't/shouldn't communicate 
directly, but
> instead should communicate back through the Build-Time Subsystem and 
the DL?

I guess the idea to start with was that users wouldn't be creating 
workspaces like:

Local Node -> Remote Node -> Local Node -> Remote Node

ie. workspaces where there is a lot of switching between local and 
remote nodes. But rather, they would create networks all of which ran 
at a remote location. Thinking about it this way, the DL to DL 
communication was supposed to make more sense because it kept all of 
remote authentication and communication in a single layer.
    I think Jarl's point on this was that it was "good enough" to 
start with. Right now we are supposed to be focusing on a local 
implementation of Piper, though, so I think as long as we agree that 
remote and local nodes should be dealt with the same in the user 
interface (which I agree with) we are probably still good enough to 
move forward. Then we can come back to the whole remote communication 
thing when we can actually implement different ideas and try them out.

> Actually, I never thought the BL would tell the PL how to execute 
its own
> workflow.  It seems Jarl agrees that Overflow's "pull" method is 
very good. 
> I
> thought that the BL would manage PL network -> remote PL network
> communication, directly, with a remote BL.

Okay, when I was talking about this I was only talking about the fact 
that thinking about things as a "pull" was nicer than thinking about a 
"push" mentality. However, I don't think the current pull system the 
PL has will be sufficient for dealing with a network that is 
remotely distributed. The reason is that all it does is keep calling 
getOutput() on the input nodes for a given node. If you strictly stick 
with this way of doing things, then you'll get stuck on a long running 
node (while it is calculating it's getOutput()) and not be able to 
distribute two long running processes on different machines. This is 
why I thought the BL would be primarily responsible for breaking up 
the wfd into pieces that the PL would calculate via the getOutput() in 
some kind of fashion, and then would assemble these pieces back 
together.

> I'm not sure how genetic algorithms in the BL will improve the 
execution of
> networks, when most of that responsibility lies with the PL.  Remote 
(BL)
> connections will be less common.  In any case, I think the PL and 
the BL need
> to use the same approach for executing networks.  And, it seems like 
Jarl
> likes the pull approach.

The idea behind the genetic algorithms stuff is that it is just a 
smarter way to execute a work flow diagram so that remote processes 
will run simultaneously and the whole processing thing will be done 
faster. Right now, the PL is responsible for having a way to wrap 
programs and a simple "pull" methodology for piping information 
between nodes. Since Jarl has a lot of good ideas about making a 
smarter system, this is kind of "in addition" to what already exists 
in the PL. Having a smart system that can manage simultaneously 
running programs on different systems is very important to me, and I 
think that these are the kind of things Jarl is interested in working 
on. This is *a lot* of functionality to be implemented. Hopefully Jarl 
will step in and smack me upside the head if I am misrepresenting what 
he wants to do.

> Perhaps we should take a look at direct BL -> BL communication (keep
> execution
> in the RTS).  From what Jarl has said, it seems the BL is very 
capable of
> doing that.

Sure, if Jarl thinks this is better, then maybe we should look at 
this. If we want to have all of the remote authentication occur at the 
level of the definition layer, then the definition layer can be 
responsible for fetching object references for remote nodes, and then 
the BL can do all of the calling directly. At any rate, I am for 
leaving this thread for the time being and focus on local stuff in our 
coding and discussions right now. 

> You know we already trimmed a lot of functionality out of the BL 
[..snip...]
> what's the purpose of the BL???

I don't think it is very productive to be looking at things as "we 
should give the BL something to do because it doesn't do anything":

1. The BL is *very important*, IMO, because of all the things 
mentioned above. I really want the kind of functionality that Jarl was 
talking about for GMS, and I think this can be worked in with how 
Overflow works now. How we handle remote requests should be determined 
by where it makes the most sense (and we'll figure that out later).

2. Questioning the purpose of other people's code in Piper is a very 
good way to get people disillusioned about coding on Piper. When we 
were going through that endlessly long thread about "what is the 
puprose of the DL?," I know that I was very close to wanting to quit 
from sheer frustration. Now we are going through the exact same thing 
with the BL--suggesting Jarl code in a different language, trying to 
micromanage what libraries he uses, saying that the BL isn't 
important, blah, blah, blah. Since I think this sucks really hard, can 
I suggest that we cut it out? This is a very unproductive way to talk 
about things. The BL and Jarl's contributions to Piper are extremely 
important, and we should never have to question this. Please, let's 
keep things positively focused. Positive energy.

    I think in general we are probably not ready for this whole thread 
yet. When we were talking a while back about security stuff, I 
remember Jarl suggesting that we stick with a simple local 
communication model to start with, and then add onto this. I like this 
idea and think we should work towards it and not worry about 
remote communication until we are done with this. Another 2 bits for 
you all.

Brad