[Bioclusters] OS X and NFS

[William Harman]

If there are interested parties that would like to try Yellow Dog Linux, and
the corresponding cluster distro, Y-HPC, let me know off line, and I will
arrange for a 30-45 days, no cost evaluation, will also provide technical
support if required.  Supported are ext3, XFS, PVFS2.
http://www.terrasoftsolutions.com/products/y-hpc/applications.shtml

I would also suggest that you might want to check our web site in early
August for an announcement concerning a Bio package which Terra Soft will
release for a PowerPC - Linux platform, as well as some new PowerPC
products.  www.terrasoftsolutions.com


Bill Harman,
Director of Sales
Terra Soft Solutions
Salt Lake City office
P - (801) 572-9252  F - (801) 571-4927
wharman at terrasoftsolutions.com
wharman at prism.net
billharman at comcast.net
skype: harman8015729252
 

-----Original Message-----
From: bioclusters-bounces+wharman=prism.net at bioinformatics.org
[mailto:bioclusters-bounces+wharman=prism.net at bioinformatics.org] On Behalf
Of John H. Lee
Sent: Thursday, July 14, 2005 4:38 PM
To: Clustering, compute farming & distributed computing in life science
informatics
Subject: Re: [Bioclusters] OS X and NFS


Since Juan's system is OS X, can anyone offer a suggestion for OS X?   
Cluster filesystems like Lustre and PVFS sound promising for Linux, but what
options do we have on Macs?  Any experience with Xsan?  With a fibre channel
limit of 64 endpoints, can Xsan even be considered for systems as large as
Tim's?

We have a 16-node Xserve cluster with one Xserve RAID.  The RAID is
connected via FC to one server, which exports the volumes via AFP and NFS
over gigabit ethernet.  As expected, the AFP/NFS server is a bottleneck.

-John

On Jul 14, 2005, at 1:32 AM, Tim Cutts wrote:

> On 13 Jul 2005, at 7:01 pm, M. Michael Barmada wrote:
>
>
>> Hi Carlos,
>>
>> If its any help, we also had similar problems with our cluster.  
>> Our solution
>> was to train the users to include code in their scripts that would  
>> create
>> local directories (on the compute node - in /tmp) and copy the  
>> files they
>> needed to those directories, then do their computing locally and  
>> copy back
>> the results.
>>
>
> Absolutely.  And preferably do the copying with something other  
> than NFS too - rcp or rsync work well, or the scheduler's built-in  
> mechanism.
> Most batch schedulers have built in abilities to this - LSF  
> certainly does, in the form of lsrcp and various options to bsub.   
> I don't know about SGE - I'm not familiar with it, but I imagine  
> the same sort of features are available.
>
> It really is quite amazing how badly NFS scales.  I remember having  
> serious problems with it on the first Linux cluster I built at  
> Incyte's UK office about 6 years ago, and that was just 7 dual-CPU  
> nodes talking to a Sun E3000 NFS server.  It didn't crash, but it  
> got *really* slow - and that was deliberately caching the data  
> locally (I wrote wrapper scripts around blastall and other  
> applications to cache the databases locally, blowing them away by a  
> least-recently-used method if there wasn't room).
>
> Sanger's current 1100 node cluster still has NFS in places, and it  
> regularly causes us grief.  Our medium-term aim is to remove pretty  
> much all NFS from the cluster altogether, with the possible  
> exception of automounted home directories, and use cluster  
> filesystems like Lustre for shared data.
>
> Tim
>
> -- 
> Dr Tim Cutts
> Informatics Systems Group, Wellcome Trust Sanger Institute
> GPG: 1024D/E3134233 FE3D 6C73 BBD6 726A A3F5  860B 3CDD 3F56 E313 4233
>
> _______________________________________________
> Bioclusters maillist  -  Bioclusters at bioinformatics.org
> https://bioinformatics.org/mailman/listinfo/bioclusters
>

--
John H. Lee
Berkeley Phylogenomics Group
http://phylogenomics.berkeley.edu

_______________________________________________
Bioclusters maillist  -  Bioclusters at bioinformatics.org
https://bioinformatics.org/mailman/listinfo/bioclusters