[Bioclusters] NFS performance with multiple clients.

[Tim Cutts]

> Any comments, tips or suggestions?

Avoid NFS whenever possible.  Buy nodes with enough local storage that 
they can have local copies of the data files that need to be searched.

More complex solutions involving cluster filesystems like GPFS and 
Lustre are possible, but they're still far from perfect, especially in 
the multiple-writer scenario.

Plus user education to write temporary files and stage their output to 
local storage (i.e. /tmp) on the execution host.

We use all these solutions in our 1,200 CPU cluster.  NFS contention is 
still a  problem on our machines, but not as bad as it used to be.

There are also things you can do to reduce the NFS traffic - specifying 
noatime on the NFS mounts helps quite a bit, for example.

Tim

-- 
Dr Tim Cutts
Informatics Systems Group, Wellcome Trust Sanger Institute
GPG: 1024D/E3134233 FE3D 6C73 BBD6 726A A3F5  860B 3CDD 3F56 E313 4233
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://bioinformatics.org/pipermail/bioclusters/attachments/20041124/5cf09048/PGP.bin