[Bioclusters] Operating system choice.
Joseph Landman
bioclusters@bioinformatics.org
Wed, 12 Nov 2003 14:00:44 -0500
Hi David:
Many commercial applications are guaranteed to work only upon
particular distributions. Most of these tend to be variations of
RedHat, SUSE, etc. In a number of cases, you will get no support for
running your application on a non-supported platform.
I might suggest looking carefully at yum, apt-rpm, and other systems.
Dependency issues are small compared to support issues if you use
commercial code. Moreover, the easiest to install and manage
distributions for clusters are based upon RedHat or variants. You can
slog through your own cluster system, but I normally (strongly)
recommend using one of the existing packaged systems (ROCKS, OSCAR,
WareWulf, Callident/BioBrew). It sounds like WareWulf or
Callident/BioBrew may be reasonable choices for you. I am using yum on
both without problems.
If commercial software is not important now or in the future, and you
have complete freedom here, and a strong linux support ability, you
might be able to roll your own. However, and this is a big however,
building a cluster is not a trivial process, and it really helps to
leverage knowledge that others before you have found. Pure
roll-your-own clusters are great if you have the staff to support it,
and they are stable, and you have no real commercial needs. Management
traditionally wants someone to lean on outside the company to help with
the support. The sales pitch to them, from a well supported, well
integrated, well understood set of systems to something not as well
supported, understood, and externally supportable would be ... hard ...
at best.
That said, there are BSD clusters, though not many. There are debian
clusters, also though not terribly many. The vast majority I have seen
are RPM based on RH, SUSE, and variants. Yum and apt-rpm handle
dependency issues quite nicely.
Also, depending upon the size of the cluster, NFS file systems may be
ok, though you quickly run out of bandwidth on your server pipe. This
is highly problematic with larger clusters if you plan to do even
moderate I/O. Local I/O is almost always faster (in aggregate, even if
you build a single local server file system capable of 1 GB/s, your
network pipe and the TCP stack will be your bottleneck). Numbers I have
seen from others put the crossover point for gigabit at around 16 nodes
with moderate I/O.
Good design is *critical* to getting the most value out of the
cluster. A cluster is not simply a pile of PCs with a network. If you
expect performance, you will need to design to your specific problems.
The distribution choice generally comes last, after the end users needs
are addressed, OS issues decided upon by their needs.
Veritas is supported on SUSE and RedHat
http://eval.veritas.com/downloads/pro/netbackup/nbu_50_ds.pdf . This
will likely be the managements' counter-argument. Using yum and apt-rpm
pretty much removes the rest of the argument against an RPM based
distribution.
Arguments against BSD involve a lack of an SGE port (though MacOS X
has one), lack of real Platform support for LSF. SGE should work on
Debian, though I do not know if LSF is supported there.
I might suggest reversing the analysis to see managements point of
view on it, and then figuring out how you can work with the distribution
that you need to, based upon the type/level of support that they desire.
Also, it is highly recommended that you ask your users what they
want, what they need, and specifically avoid indicating your preferences
to them. I have one customer who suddenly needs a RH9.0 cluster due to
one application (Gaussian) being supported only on RH8 or RH9. I have
other customers with differing needs, including one which requires
RH7.2. All of this is completely driven by applications. The
distribution follows the application, as does the design/implementation
of the nodes.
Just a suggestion, email me offline if you want to talk about these
issues.
Joe
David Robillard wrote:
> Good evening everyone,
>
> I'm in the process of building a compute farm and my main concern is with the Operating System choice.
>
> As a unix systems admin, I'm not a big fan of Red Hat linux products (or anything RPM based for that matter), because of dependency problems inherent to RPM. I would prefer to go with Debian GNU/Linux or with FreeBSD. Unfortunately, my Upper Management team thinks Linux and Red Hat are identical.
>
> I thus face two barriers:
>
> a) The OS needs to be compatible with Platform LSF and Veritas NetBackup client.
>
> b) I need proper arguments to demonstrate to Upper Management why we should NOT use Red Hat.
>
> Therefore,
>
> Does anyone have experience with LSF/NetBackup on Debian or FreeBSD?
>
> Could someone help me put together a "sales pitch" for either Debian or FreeBSD? Otherwise, the OS choice will be a business choice and not a technical/systems administration choice. Hence I'll be "stuck" with it...
>
> Here are some details or the environment:
>
> -I have 5+ years experience running Solaris, Debian, FreeBSD and Red Hat machines.
> -This compute farm will be running on x86 architecture, the Intel Xeon CPU.
> -Gigabit ethernet will be used for interconnect, nothing fancy here.
> -Master node will be connected to a SAN and will be backed-up by Veritas NetBackup.
> -Compute nodes will boot via PXE/Etherboot (undecided yet) and mount their kernel via NFS. A local drive will be there for swap.
> -The applications running on this compute farm are in-house algorithms which don't use MPI nor PVM.
>
> Thanks for your help,
>
> David
>
> --
> David Robillard
> UNIX systems administrator
> david.robillard@galileogenomics.com
> +1 514 270 3991 x285
> Galileo Genomics
> _______________________________________________
> Bioclusters maillist - Bioclusters@bioinformatics.org
> https://bioinformatics.org/mailman/listinfo/bioclusters
--
Joseph Landman, Ph.D
Scalable Informatics LLC,
email: landman@scalableinformatics.com
web : http://scalableinformatics.com
phone: +1 734 612 4615