[Bioclusters] Cluster ideas/suggestions

[Chris Dwan]

I've done this in the past with workstation labs in an academic 
setting, and it's *much* easier than the alternative (integrating your 
code into a cycle stealing system, and hooking that system into an 
existing workstation environment).  My experience was that both side 
(the workstation users and the partial duty cycle cluster environment) 
suffer.

What worked for us was almost exactly what you describe:  We dedicated 
a decent sized partition to the cluster image.  The first step after 
the reboot was an rsync (or similar) of the popular shared datasets.  
This was much cheaper / simpler than trying to upgrade an office 
building network to handle intra-cluster style loads.

Obviously, you will also want to avoid running truly parallel jobs with 
interprocess communication across the larger network gaps.

Joe Landman is correct about scheduling and potential lost time because 
of nodes rebooting in the middle of a job.  I believe that all of the 
major DRMs provide a facility to "drain" nodes for scheduled downtime.  
If you can put runtime estimates on your jobs, the scheduler should be 
able to handle this for you.

Good luck!

-Chris Dwan
  The BioTeam


On Jan 13, 2005, at 9:34 AM, Nick D'Angelo wrote:

> In speaking to our R and D managers, they posed a good question.
>
> What about their desktop machines (relatively New Dell PCs), have them
> automatically shutdown and reboot, say at 8:00 pm and then join the 
> cluster
> until 05:00 am.
>
> Then at 5:00 am, initiate another reboot that will present the Windows 
> Login
> for the 'normal business day'.
>
> This would make great use of the hundreds of computers that are mostly
> sitting idle during the off hours 'normal business day'.
>
> Thoughts, suggestions always appreciated, thanks.
>
> Nick
>
> In case you need to know,  I am likely going to install the BioBrew 
> v3.x
> which has just been pre-leased in Beta I believe.
> _______________________________________________
> Bioclusters maillist  -  Bioclusters at bioinformatics.org
> https://bioinformatics.org/mailman/listinfo/bioclusters