[Bioclusters] Fwd: pre-configured clusters

[Jeffrey B. Layton]

Robert Myers wrote:

> Rayson Ho wrote:
>
>> --- "Sean C. Garrick" <garrick@me.umn.edu> wrote:
>>  
>>
>>> To: scitech@lists.apple.com
>>> From: "Sean C. Garrick" <garrick@me.umn.edu>
>>> Subject: pre-configured clusters
>>> Date: Thu, 22 Jan 2004 09:30:54 -0600
>>>
>>> This is good news. Hopefully there will other communications options 
>>> soon. I spent several hours yesterday learning about Infiniband.
>>>
> Infiniband is starting to show some life again, but it is still very 
> expensive as opposed to the other components of most commodity 
> clusters. Look for prices to come down. Wait if you can. Take note of 
> the price of Infiniband switches now and see what they look like at 
> the end of 2004. Also expect more cost-effective solutions on the HCA 
> (host channel adapter) side, also very expensive at the moment.
>
>>>
>>> http://www.apple.com/xserve/cluster/wgcluster.html
>>>
> SGI Altix 350 probably a better buy
>
> http://www.serverwatch.com/news/article.php/3298041
>
> "The SGI Altix 350 is priced starting at $12,199. A 4-processor 
> configuration carries at a list price of $21,599, or $5,400 per 
> processor. " 


I will agree/disagree. It depends upon your application(s). Some will
run faster on the Altix, some won't. Test your application on both
and then do the price/performance.

Personally, if you need a single-system image machine and you know
you are not going to grow the machine, then the 350 is a great deal.
Otherwise, look at clusters. I think the Xserve deserves a second look
as does the IBM 970 (same thing really). Also look at the Xeon
nodes and the Opteron nodes. I think the Opterons are especially
promising.

> The Apple cluster comes standard with gigabit ethernet: an acceptable 
> interconnect only for a limited number of problems. The SGI uses 
> proprietary Numa-Link, which is at least competitive with Infiniband, 
> a much faster and much more expensive solution than gigabit ethernet, 
> but the Altix 350 looks like a really good buy, especially if you're 
> looking for a pre-packaged solution. 


I don't see anything wrong with GigE at all. Again, it depends upon
your application. If you need a highbandwidth low latency interconnect
then something like IB, Myrinet, Quadrics, or Dolphin are the way to go.
If you need single system image, then NUMAflex is the way to go
(just a small correction - the SGI website and literatures refers to it as
NUMAflex, not NUMA-Link). Believe it or not, my main application
scales very well with Fast Ethernet (over 90% scaling for at least 300
processors). It's not a Bio application, but some bio apps aren't even
really parallel apps at all - just HUGE amounts of serial processing with
varying data sets.

Oh, another thing about NUMAflex, it's not the same across their
product line. While I can't confirm this, I was told that the Altix
350 had a ring type connection for NUMAflex. The larger and more
expensive machines had a fat-tree type connection for NUMAflex.
If you're going this route, be sure to ask (and be sure to test!).

If your apps are MPI based, then there are some pretty neat MPI
packages out there. Scali's MPI Connect is pretty cool for GigE
clusters (OS by-pass technology). They also claim that they can
channel-bond GigE with pretty decent results (I seem to remember
that it was something like 50-75% of the bandwidth of the second
GigE line). If your apps only use a small amount of MPI, then try
something like MP_Lite. It can channel-bond GigE pretty damn
well.

There's also a new interconnect that I just heard about called ATOLL
(www.atoll-net.de). I've seen some test numbers for the MP_Lite
folks and it looks VERY promising. I don't know about the pricing
though (it still may be beta). Something to watch.

If you want a small cluster, I can price out a simple 8 node cluster with
GigE (jumbo packet capable) where each node is just a lowly 1.4 Ghz
Duron and 128 Megs of memory (you can upgrade to 1 Gig each). The
NICs are only 32-bit/33 Mhz, but they're not bad at all. The last time I
priced this out, it was about $2,600. Not bad for a starter cluster!

Just remember the old adage - test your own apps!

Enjoy!

Jeff

>
>
> RM
>
> _______________________________________________
> Bioclusters maillist  -  Bioclusters@bioinformatics.org
> https://bioinformatics.org/mailman/listinfo/bioclusters
>