[Bioclusters] Re: (no subject)
Michael Will
bioclusters@bioinformatics.org
Tue, 30 Mar 2004 18:33:28 +0000 (UTC)
Dan Bolser wrote:
>> Hmm strange that you are getting 2GB per process, my large job normally
>> died around the 3GB on a standard linux kernel.
> The job is my dumb approach to memory allocation ...
>
> #!/usr/bin/perl -w
> ...
You probably found a limitation in the perl interpreter rather than in your
operating system. As pointed out earlier, you have about a 3GB limit when
using ia32 architecture.
For opteron systems when using 64bit linux like Suse SLES8, you could have
4G per 32bit-process and a much larger (limited only by your physical RAM
for at least all of 2004 :-) ) amount per 64bit-process.
> Is there some way I can find out the make / model of the machines without
> going down to the server room and looking at the label?
Not really. What you can find out is some specs for your CPUs.
Try cat /proc/cpuinfo
> They are dual cpu (hyperthreaded) xeon ... My colegue told me that often
> on dual machines the memory is split, one block per cpu?
Thats not true for Xeons, there both CPUS access the same block of memory
via the front-side-bus.
On the opterons however, you do have one block of memory per CPU, but again
it does not affect the maximum amount of memory in one process because each
CPU can access all of the RAM via Hypertransport.
To you it just looks like one piece of RAM, except for that you might see
different access speed when not using 'node interleave memory access' which
averages out the effect of part of the RAM being closer to the CPU.
Michael Will
>> >> The limitation actually lies in the 32 bit architecture of Xeon.
>> >> Physical 32 bit limitation is 4GB, PAE gives us up to 64 GB. For
>> >> normal
>> >> Linux, you should have about 3GB per process and maybe can tune the
>> >> kernel to use 3.5GB per process.
>> >
>> > Hmmm..... Looks like we are only getting 2GB per process (2GB per
>> > CPU?).
>> >
>> > We are using standard linux, but with a swap space below the size of
>> > the
>> > ram (2GB). Could this be a problem? (i.e. true64 style).
>> >
>> > ulimit -a
>> > core file size (blocks, -c) 0
>> > data seg size (kbytes, -d) unlimited
>> > file size (blocks, -f) unlimited
>> > max locked memory (kbytes, -l) unlimited
>> > max memory size (kbytes, -m) unlimited
>> > open files (-n) 1024
>> > pipe size (512 bytes, -p) 8
>> > stack size (kbytes, -s) 8192
>> > cpu time (seconds, -t) unlimited
>> > max user processes (-u) 7168
>> > virtual memory (kbytes, -v) unlimited
>> >
>> > Mem: 4124724k av,
>> > Swap: 2040244k av
>> >
>> > Thanks very much for your help,
>> >
>> > Dan.
>> >
>> >
>> >>
>> >> LAI Loong-Fong
>> >>
>> >> On Mar 30, 2004, at 2:46 PM, Dan Bolser wrote:
>> >>
>> >>>
>> >>> I have a question about Xeon and memory... It looks like I have one
>> >>> Gb
>> >>> per
>> >>> CPU, and not 4 Gb for the 4 CPU without restriction. Is this problem
>> >>> at
>> >>> the hardware level?
>> >>>
>> >>> What is the maximum amount of memory a CPU can use?
>> >>>
>> >>> I heard talk of a Tb memory machine, but it was part of a 1000 node
>> >>> cluster, so I am thinking OK 1 Gb per node.
>> >>>
>> >>> Can anyone clarify this for me?
>> >>>
>> >>> Cheers,
>> >>> Dan.
>> >>>
>> >>> On Mon, 29 Mar 2004, Philip MacMenamin wrote:
>> >>>
>> >>>> On Monday 29 March 2004 05:38 am, you wrote:
>> >>>>> hello all,
>> >>>>>
>> >>>>> i'm interested to the constructiong of a linux cluster of computer
>> >>>>> for
>> >>>>> bioinformatics purpose. but i dont have a cluu about the performans
>> >>>>> it
>> >>>>> should have. is there any one who can give any suggestion?
>> >>>>>
>> >>>>> in the hope of an answer
>> >>>>>
>> >>>> Its kind of a nebulous question.
>> >>>>
>> >>>> Basically... buy Xeon || AMD two way boxes. Be boring, look at Dell
>> >>>> or
>> >>>> penguin computing or something. Dont bother with 64 bit. Think about
>> >>>> that in
>> >>>> about 2 years time. It will irritate you in 2 months after you buy
>> >>>> it, *i
>> >>>> promise*.
>> >>>>
>> >>>> Dont buy from some indy very cheap manufacturer, cause, if the nodes
>> >>>> will
>> >>>> fail, you will be left waiting... or make them give you a decent
>> >>>> guarantee.
>> >>>> Or a big box of bits.
>> >>>>
>> >>>> Look at the price of the chip : speed ratio. This graph will have an
>> >>>> inflection point, at which more money input gives diminishing
>> >>>> returns
>> >>>> speed-wise. Buy at this inflection point. This changes all the
>> >>>> time... of
>> >>>> course. There is no point in buying the *best* out there right this
>> >>>> second.
>> >>>> Just buy a little behind it, and buy an extra box or two.
>> >>>>
>> >>>> Find out what you want to run.
>> >>>> Buy as enough memory so it doesn't thrash your disks. (Or just fill
>> >>>> it with
>> >>>> memory).
>> >>>>
>> >>>> Spending about 2 grand (USD) on them a piece (I dont know what that
>> >>>> is in
>> >>>> Lira) should buy you something decent operating at 3 gigs, with a
>> >>>> couple of
>> >>>> gigs ram.
>> >>>>
>> >>>> Then have a look at
>> >>>> http://bioinformatics.org/biobrew/
>> >>>> www.rocksclusters.org/
>> >>>>
>> >>>> Sorry if this is very uninspiring/boring advice... its just, you
>> >>>> want
>> >>>> to keep
>> >>>> everything as simple as you can. There will be tricky bits no matter
>> >>>> how
>> >>>> simple you make everything.
>> >>>> Good luck.
>> >>>> Philip.
--
Michael Will, Linux Sales Engineer
NEWS: We have moved to a larger iceberg :-)
NEWS: 300 California St., San Francisco, CA.
Tel: 415-954-2822 Toll Free: 888-PENGUIN
Fax: 415-954-2899
www.penguincomputing.com