[Bioclusters] Login & home directory strategies for PVM?

[Tony Travis]

Tim Cutts wrote:
> 
> [...]
> Most very large clusters in the HPC world don't allow NFS at all, or 
> minimise it.
> 
> Our 1000-node cluster does allow some NFS, but this is to scratch 
> directories, and *not* to all users' home directories, in general.

Hello, Tim.

We only have a 'small' 64-node cluster here :-)

However, I've opted to use BOBCAT architecture:

	http://www.epcc.ed.ac.uk/bobcat/

Although the original EPCC BOBCAT no longer exists, it's spirit lives on 
in our RRI/BioSS cluster:

	http://bobcat.rri.sari.ac.uk

The important thing is to have TWO completely separate private network 
fabrics: One for DHCP/NFS, the other for IPC. The main problem we have 
is that IPC (i.e. Inter Process Communication) can swamp the bandwidth 
of a single network fabric and you rapidly lose control of the cluster.

I think there are some MYTHS about NFS and clusters around because of 
the bandwidth contention on a single network fabric. The NFS network 
traffic on our cluster is completely segregated from the IPC traffic 
which is throttled by the bandwidth of its own network fabric. The 
switches on the two network fabrics are NOT connected in any way...

	Tony.
-- 
Dr. A.J.Travis,                     |  mailto:ajt at rri.sari.ac.uk
Rowett Research Institute,          |    http://www.rri.sari.ac.uk/~ajt
Greenburn Road, Bucksburn,          |   phone:+44 (0)1224 712751
Aberdeen AB21 9SB, Scotland, UK.    |     fax:+44 (0)1224 716687