Piper Ends and Goals

Abstract

This document tries to explain what the ends of Piper are. In other words, it tries to answer the following questions: What is the intended usage? What will piper be good for? What do we agree it can be bad at? What piper capabilities are needed to get piper to be as useful and easy to use as we expect it to become? What will the user see in terms of features? How will the user(s) use it?

Status of this document

This document is work in progress. Please post your feedback to vsh-general@bioinformatics.org for futher discussion.

TODO: read through the ML archives to find relevant things that will help in fleshing out this doc.

Overview

Piper is meant to be a visual shell, providing the user with a GUI and a pipe mechanism to glue programs together. Piper exhibits several key advantages over the old unix shell: distribution (as in distributed computing), P2P capabilities (for resources discovery), non-linearity (scripts are networks of pipes and not a sequences of pipes), artificial intelligence aware, etc.

Let's examine each of these features in detail...

Piper features

Visual shell

System that let one graphically build programs by linking nodes.

Piper will let one glue and reuse existing programs.

For now refer to Jeff's presentation of Aug 2000 and Oct 1999, to the Piper's website introduction section and to the command-compilation description.

Distribution

Programs can naturally be executed on different hosts.

Explain how this will work for the user.

compare to the use of rsh

Peer-to-Peer

What is P2P used for?

Resource discovery, instances/host coordination.

Read the [piper] Universal Description, Discovery, and Integration thread in the mailing list archives.

Non-linearity

XXX: find better term?

In a shell, pipes let one build sequences of commands, but not network of commands without any difficulty (think of tee)

XXX: what about flow control that seemed to exist in Loci?

Artifical Intelligence Aware

Offers an AI toolbox of algorithms or other approach? Possible algorithms: genetic programming, rule-based system, etc.

What would they be used for? Find examples where AI algorithm/techniques are useful in Piper. Do they need to be part of Piper? Can't these algorithm be nodes that would be linked in Piper to process Piper internal information? Export internal information or implement reflexivity/introspection as in Narval?

Working material

Here are pieces of information for those who would like to participate in solving the big puzzle "what should piper do?".

Website abstract

Piper is a peer-to-peer (P2P) distributed workflow system. It is an independent, GNU-based project which brings the power and flexibility of the GNU/UNIX command-line interface (CLI) to the graphical user interface (GUI) and Internet-distributed computing.

Networks, programs, files, widgets, and so on, can be Internet-distributed components represented in a GUI as the nodes of a flow chart. The user can join nodes via lines that depict links for data flow, procedural steps, relationships, and so forth.

Overflow command-compilation

More info about "command compilation" and "what will using existing commands and programs be like?" http://www.bioinformatics.org/piper/documentation/command-compilation.html.

Jeff's Aug 2000 presentation

Jeff put on-line the slides of a presentation he gave in august 2000. The most interesting ones, relatively to Piper's use and features are:

Jeff's Oct 1999 presentation

Describes the older Loci Project.

Website FOCUS & GOAL section

Connect...

Website INTRODUCTION section

Piper has two key components: an XML language for dynamically assembling components across networks to construct new components and a graphical interface that makes this trivial for end users. Piper plays an even larger role in the GUI and networks than pipe plays in the command line.

...

Piper is transparent component glue

Enter Piper. Piper, like pipe, moves the processes of gluing components from C and and C++ code to the user level. Piper uses XML rather than C or C++ to describe how components are related to each other. For example, if an email component talked to a spell checker component, this would be represented in an XML document as a relationship between two components. When this document was loaded at runtime, Piper would dynamically create the relationship.

Piper provides a simple graphical representation of component relationships that users can easily manipulate. Rather than use the "|" symbol, Piper uses simple lines and arrows to connect different components together and then allows users to change and add to these connections at will.

Piper is significantly richer than the traditional world of pipes. The goal of Piper is to be a complete XML component language that allows users to construct any program they need from a set of components located anywhere on the Internet.This gives tremendous power to the user.

Piper is multidimensional in that one component can have multiple connections to other components, though this is not displayed above. When a set of Piper components is finished, users can create a BlueBox user interface to the components. The resulting program will then be accessible from any platform that BlueBox has been ported to.

Piper Paradigm

  1. Everything is a component.
  2. Every component can accept (input) or produce (output) data, or do both.
  3. Components can be connected or "piped" according to their input and output of data.
  4. All components have a network "location." Components can therefore be refered to as "loci."
  5. Nodes are only represented locally, if possible.

Benefits of Piper

Narval

From the list archives:


Nicolas Chauvat
Last modified: Sat Feb 17 15:04:04 CET 2001