[bioclusters] self-built clusters and red hat

[Donald Becker]

On Wed, 18 Dec 2002, Patrick Audley wrote:

>     Andrew> always useful.  If there are people out there who are
>     Andrew> running production services on beowulf-like clusters with
>     Andrew> a non-redhat based distribution (meaning a customized
>     Andrew> red-hat image like rocks or the old VALinux distro, not
>     Andrew> mandrake), I would really appreciate a note about it.  If
>     Andrew> you migrated to something else from redhat, and care to
>     Andrew> tell me why you did it and how it went, that would even be
> 
> Hi Andrew,
>    We have a 100+ processor cluster that we are happily using Debian
> on right now.  I runs very happily with Grid Engine and MPI though I
> haven't personally tried it with Beowulf yet (there is a port of
> Debian for Beowulf[1]).  If you have any specific questions, feel free
> to ask.

I think that you misunderstand what Beowulf was and is, based on
shorthand statements on web pages like the one above.  Beowulf started
out as a software system to make building dedicated clusters easier.
The system was both specific tools, which have evolved pretty
dramatically over time, and system changes, most of which were folded
back in standard Linux packages.

Among the Beowulf features were tools to configure, run and monitor PVM
and later MPI jobs.  But Beowulf is not PVM or MPI -- those are libraries
interfaces to message passing routines, and just one component of a
cluster.  Beowulf system do automatically configure the MPI and PVM
environment and correctly-designed systems offer features such as direct
execution (no need for "mpirun") and single-binary deployment (no
recompile needed for different cluster configurations or different
network types).

The web page you pointed out talks about a "classic" Beowulf system
circa 1998, where there was a full, static OS installation on each node.
Current generation Beowulf system have progressed well beyond that.
They have a designated master machine or a reference install, automatic
node addition and removal, scalable cluster nameservices, single point
updates that are consistent even when nodes are down, and defined
plug-in interfaces for job monitoring and control.

-- 
Donald Becker				becker@scyld.com
Scyld Computing Corporation		http://www.scyld.com
410 Severn Ave. Suite 210		Scyld Beowulf cluster system
Annapolis MD 21403			410-990-9993