[Bioclusters] Cluster ideas/suggestions

[Micha Bayer]

On Thu, 2005-01-13 at 15:54, Nick D'Angelo wrote:
> Thank you Micha.
> 
> Just so I am clear on this, do I need to migrate to Biobrew or can I run
> Condor on my current three machines (FC2) to achieve the same thing - a
> cluster of machines running blast queries and Clusa1W?

No, you should be okay just using Fedora - we do. You just have to get
your bio apps separately (e.g. from NCBI or EBI) and then you can just
preinstall them on all your machines - that saves even more time.

Condor installation takes minutes and it works out of the box usually.
It's a great example of how good open software can be if it has matured
over a number of years (since 1989 in this case).

If you want I can give you our config files - that would save you time.
With our config files we force our jobs to run always (regardless of
user activity) because our cluster is pretty much dedicated to running
jobs at the moment, with only the occasional unsuspecting student
wandering in.... :-)

cheers
Micha


> 
> Sorry for the repeated questions, I guess I venturing into unknown (to me)
> territory.
> 
> Nick
> 
> 
> 
> -----Original Message-----
> From: Micha Bayer [mailto:michab at dcs.gla.ac.uk]
> Sent: Thursday, January 13, 2005 10:44 AM
> To: Clustering, compute " farming & distributed computing in life
> science informatics
> Subject: RE: [Bioclusters] Cluster ideas/suggestions
> 
> 
> Nick,
> 
> if you have dedicated machines then Condor and BLAST can work reasonably
> well together because you can keep target databases preinstalled on all
> your machines. This gets you around the i/o issues.
> 
> We run BLAST on Condor that way and it seems pretty reasonable. There is
> also a very large Condor Pool at Wisconsin Uni which is used for BLAST
> and it does a hell of a lot of useful work. 
> 
> Micha
> 
> 
> On Thu, 2005-01-13 at 15:34, Nick D'Angelo wrote:
> > All great ideas, thank you for them so far.
> > 
> > I should have mentioned, that the cluster is required to run Blast,
> CLusta1W
> > and RNAfold.
> > 
> > Currently I have three Fedora machines, kind of all doing independent work
> > in isolation of each other and we manually setup the jobs to run until
> > completed.
> > 
> > Nick 
> > 
> > -----Original Message-----
> > From: Micha Bayer [mailto:michab at dcs.gla.ac.uk]
> > Sent: Thursday, January 13, 2005 10:01 AM
> > To: Clustering, compute " farming & distributed computing in life
> > science informatics
> > Subject: Re: [Bioclusters] Cluster ideas/suggestions
> > 
> > 
> > Hi,
> > 
> > this sounds like you might want to look at Condor. Condor is an
> > excellent tool for doing just that sort of thing (desktop cycle
> > harvesting). 
> > 
> > It can be installed on Linux or Windows machines, and it is highly
> > configurable. One can set machines up so that jobs never run when the
> > user is working, so that users don't even know a job is lurking in the
> > background on their machine.  
> > 
> > This would save you the trouble of having to reboot the machines, and
> > you would get extra cycles during the day when users are in their coffee
> > break etc. 
> > 
> > Very nice package, free , very mature and great documentation. We run a
> > Condor pool here and we think it is excellent. There is also a large
> > user community already with mailing lists etc and good support
> > otherwise.
> > 
> > http://www.cs.wisc.edu/condor/
> > 
> > If you have a pool of users who trust you to run jobs on their machines
> > then this is just the ticket.
> > 
> > cheers
> > Micha
> > 
> > 
> > On Thu, 2005-01-13 at 14:34, Nick D'Angelo wrote:
> > > In speaking to our R and D managers, they posed a good question.
> > > 
> > > What about their desktop machines (relatively New Dell PCs), have them
> > > automatically shutdown and reboot, say at 8:00 pm and then join the
> > cluster
> > > until 05:00 am.
> > > 
> > > Then at 5:00 am, initiate another reboot that will present the Windows
> > Login
> > > for the 'normal business day'.
> > > 
> > > This would make great use of the hundreds of computers that are mostly
> > > sitting idle during the off hours 'normal business day'.
> > > 
> > > Thoughts, suggestions always appreciated, thanks.
> > > 
> > > Nick
> > > 
> > > In case you need to know,  I am likely going to install the BioBrew v3.x
> > > which has just been pre-leased in Beta I believe.
> > > _______________________________________________
> > > Bioclusters maillist  -  Bioclusters at bioinformatics.org
> > > https://bioinformatics.org/mailman/listinfo/bioclusters