[Bioclusters] [long-ish] Advice on getting started with clustering, LSF, Xserve?

[Elia Stupka]

> machines to solve some problems. There are solutions which, for
> example, use java as a layer to distribute work across a variety of
> architectures.
[...]
> There are also pipe lines available which might use things like 
> SOAP, XML, etc to abstract analysis from the researcher 

My two cents: SOAP, XML, CORBA, etc. all fun, but hard hard work... but
really if it is only the hardware that is heterogenous LSF is your
friend. If it is the software that is heteregoneous asses which bits are
"necessary" and which are "replaceable" and try to concentrate on one
platform (be it a database, or a language or a pairing of those) to
converge onto, will save you lots of headaches. Not to mention the HR
headaches of getting people who are religious about certain languages or
platforms to talk to each other. Unless it is a very big operation I think
keeping one backbone is always good, even if hardware is very
heteregoneoue (usually heterogenous hardware will actually tickle the
heart of a good sysadmin who can play with different CPUs...)

> In the cycle stealing/borrowing sceanrio it may never be clear just
> how many systems will be working on a given job making it difficult to
> present good estimates on completion times. This is a problem if you
> are attempting to accomodate the interests of several researchers all
> of whom need data by a grant deadline.

The other scenario is that the machines that are used as dektops which
usually you would be "stealing" CPU from be setup to allow end-users to
easily run things via LSF/etc. thus freeing their own machine and allowing
for easier estimates, i.e. if all blasts are always run via LSF, even the
"personal" ones, then the machines stay more free (of course most people
wouldn't go to the extreme of running Microsoft Office remotely...)

> resellers. LSF is good , flexible software no doubt about it.
> But we are always under pressure to do more with less money
> hence my interest in SGE.

Another option to explore is that some vendors (such as IBM and Apple) are
partners of Platform, so if in the initial purchase you specify you want
LSF bundled they can go off and negotiate a bundled price for you, often
it's a very good deal...

> I tend to toss data out onto the nodes which means I might use
> software RAID to improve performance. I don't do much over NFS except
> to serve the LSF tree but even thats not a requirement. I like to
> cache data locally which means quicker Blasting in my case. I split up
> my databases so when I add new nodes I repslit and redistribute
> (programmatically of course).

Couldn't agree more, you will often notice it is better to spend money on
serious local storage (like 2*120GB on the Xserve, or even 4) and do RAID,
striping,etc. to have as fast and reliable access to local data as
possible, rather than spend lots of money on the latest processor, and
then have data fetched by 200 processors via LSF from the same poor disk,
often a big misconception....


> As far as Xserves I've yet to get my hands on an eval unit so can't
> say much about them. If their advertised price is the actual street
> price then I can buy two of what I'm already using (dual atlons 1800+
> ,2X40 GB HD,and 2GB RAM) for the same money. The Xserves are worth
> checking out though.

Advertised price is not the price you get after some solid
negotiations... of course depends the country you are in, your
skills,etc.... but you can definitely go well below.... can't quote real
numbers but trust me you can *slash* the prices right now because they
want to get into bioinformatics big time....

Elia

********************************
* http://www.fugu-sg.org/~elia *
* tel:    +65 6874 1467        *
* mobile: +65 9030 7613        *
* fax:    +65 6779 1117        *
********************************