[Pipet Users] Re: making more CLI programs Piper-compliant

[J.W. Bizzaro]

(late reply)

Roland Walker wrote:
> 
> There is absolutely a better way.  Don't parse the help output.  This
> is harder than doing things the nominally hard way.
> 
>    1 First, you must wrap some CLI elements by hand.  This is needed
>      to get started, yes? and you are doing it already, yes?  Wrapping
>      basically means standardizing the handling of STDIN, STDOUT,
>      and STDERR, trying to tame buffering, and providing some
>      formalization of the options.  This is not too hard; it's my own
>      stock in trade.

Actually, we've come up with a way to do the wrapping using connect-the-dots:

  http://www.bioinformatics.org/piper/documentation/command-compilation.html

Sure, programmers will find it simpler to write one by hand, but perhaps
non-programmers will find it usable.  In any case, it's been an academic
excercise to see how many things can be done using connect-the-dots.

>    2 Provide an _easy_ way for CLI authors to export options, making
>      makes their programs drop-in elements for you.  This means not
>      writing yet another standard, which authors will just ignore, but
>      giving authors the needed code.  This is not as hard as it seems.

Is there ANY standard out there?  Can you provide us with an example?

>      You won't have to conquer SEALS, as I'll just give it to you.

Great.

> I assume you have written an XML spec to define the options?  That's
> what I would have done if XML was around when I started my thing.

Sort-of.  It's a long story.  Basically, we already *CAN* define options using
XML, but that is due to the dataflow nature of Piper.  We should probably come
up with clear specs though.

> Give me your spec and a give me a couple of hours, and I can give you
> a Getopt::Piper that is drop-in compatible with Getopt::Long.

Super.

> I'm not claiming this is perfect, just useful as heck.  Cram new
> programs in to your system as hard as you can.  If you reach a certain
> critical mass, all kinds of stuff will start to come your way.

That's our motto :-)

> There are dozens of scripts here at NCBI that use the same library for
> argument processing.  For some reason argument processing and CLI
> usability is an obsessive interest of mine.  Anyway, I build a data
> structure that holds more than anyone could want to know about
> processing the options for a CLI script.

You're the man to talk to about this then!

> Then I like to do more with
> that than merely process the options.  For instance, each script can
> generate programmable completions for tcsh based on its expected
> arguments:

For Piper, grepping what the program needs for parameters is more important
than grepping what the parameters do.  Piper is "dumb" that way: it doesn't
care what data passes through it; it's the responsibility of the node/program
to deal with it (although we'd like to help the user make the right
connections).

> So to you Piperians, I say, show me your spec, I'll throw it into
> the main SEALS library so that all scripts can be queried, say,
> like this
> 
>    gi2fasta -piper_options
> 
> to return your spec, giving you a sizable wad of new CLI elements
> for your system.

Again, I'd be interested in seeing what others have come up with and if there
may be some sort of a standard approach.  We'd be interested in promoting a
standard (say for GNU programs).

> There is certainly some duplication, but duplication
> can even be good.  I am not into the idea of a single perfect
> system, or a single best way, but rather in opening up new ways
> which are new paths for users to wander.

The old standards vs. freedom to innovate argument :-)

> Furthermore, you may have our module if you find it useful.  I'd say
> the module is wonderful, though idiosyncratic.  Fixing it for general
> use means mostly indenting and deleting.

Thanks.

> In addition, the research side at the NCBI is a powerfully active
> testbed.  If you could get folks using Piper here, you would get
> a _lot_ of feedback.
> 
> Think critical mass.

>From the bioinformatics side of Piper development, I can say that it would be
a Great Thing to have NCBI involved :-)

> So I think that the only hitch between us and a world of excitement
> is the lamentable lack of a packaging system that can tame the SEALS
> monster.  We have multiple revisions we need to reconcile, and
> tangles of code that we rather urgently need to clean up and push
> out the door in some usable form.

Are you thinking along the line of RPM, Deb, or CPAN?

Thanks for the message.  It was very helpful.

Cheers.
Jeff
-- 
J.W. Bizzaro                                jeff at bioinformatics.org
Director, Bioinformatics.org        http://bioinformatics.org/~jeff
"As we enjoy great advantages from the inventions of others, we
should be glad of an opportunity to serve others by any invention
of ours; and this we should do freely and generously."
                   -- Benjamin Franklin
--