[Bioclusters] Re: (no subject)

Wed, 31 Mar 2004 21:40:05 +0100 (BST)

On Tue, 30 Mar 2004, Michael Will wrote:

> 
> Dan Bolser wrote:
> >> Hmm strange that you are getting 2GB per process, my large job normally 
> >> died around the 3GB on a standard linux kernel. 
>  
> > The job is my dumb approach to memory allocation ...
> > 
> > #!/usr/bin/perl -w
> > ...
> 
> You probably found a limitation in the perl interpreter rather than in your
> operating system. As pointed out earlier, you have about a 3GB limit when 
> using ia32 architecture. 

Ahhh.... that answers a question!

> For opteron systems when using 64bit linux like Suse SLES8, you could have 
> 4G per 32bit-process and a much larger (limited only by your physical RAM
> for at least all of 2004 :-) ) amount per 64bit-process.
> 
> > Is there some way I can find out the make / model of the machines without
> > going down to the server room and looking at the label?
> Not really. What you can find out is some specs for your CPUs. 
> 
> Try cat /proc/cpuinfo

Cool...

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 2
model name      : Intel(R) Xeon(TM) CPU 2.80GHz
stepping        : 7
cpu MHz         : 2791.842
cache size      : 512 KB
physical id     : 0
siblings        : 2
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
bogomips        : 5570.56

processor       : 1
...
physical id     : 0
...

processor       : 2
...
physical id     : 3
...

processor       : 3
...
physical id     : 3
...

> 
> > They are dual cpu (hyperthreaded) xeon ... My colegue told me that often
> > on dual machines the memory is split, one block per cpu?
> Thats not true for Xeons, there both CPUS access the same block of memory 
> via the front-side-bus. 
> 
> On the opterons however, you do have one block of memory per CPU, but again 
> it does not affect the maximum amount of memory in one process because each 
> CPU can access all of the RAM via Hypertransport. 
> 
> To you it just looks like one piece of RAM, except for that you might see 
> different access speed when not using 'node interleave memory access' which
> averages out the effect of part of the RAM being closer to the CPU.
> 

Thanks for the info,

Cheers,
Dan.

> Michael Will
> 
> >> >> The limitation actually lies in the 32 bit architecture of Xeon.
> >> >> Physical 32 bit limitation is 4GB, PAE gives us up to 64 GB. For 
> >> >> normal
> >> >> Linux, you should have about 3GB per process and maybe can tune the
> >> >> kernel to use 3.5GB per process.
> >> >
> >> > Hmmm..... Looks like we are only getting 2GB per process (2GB per 
> >> > CPU?).
> >> >
> >> > We are using standard linux, but with a swap space below the size of 
> >> > the
> >> > ram (2GB). Could this be a problem? (i.e. true64 style).
> >> >
> >> > ulimit -a
> >> > core file size        (blocks, -c) 0
> >> > data seg size         (kbytes, -d) unlimited
> >> > file size             (blocks, -f) unlimited
> >> > max locked memory     (kbytes, -l) unlimited
> >> > max memory size       (kbytes, -m) unlimited
> >> > open files                    (-n) 1024
> >> > pipe size          (512 bytes, -p) 8
> >> > stack size            (kbytes, -s) 8192
> >> > cpu time             (seconds, -t) unlimited
> >> > max user processes            (-u) 7168
> >> > virtual memory        (kbytes, -v) unlimited
> >> >
> >> > Mem:  4124724k av,
> >> > Swap: 2040244k av
> >> >
> >> > Thanks very much for your help,
> >> >
> >> > Dan.
> >> >
> >> >
> >> >>
> >> >> LAI Loong-Fong
> >> >>
> >> >> On Mar 30, 2004, at 2:46 PM, Dan Bolser wrote:
> >> >>
> >> >>>
> >> >>> I have a question about Xeon and memory... It looks like I have one 
> >> >>> Gb
> >> >>> per
> >> >>> CPU, and not 4 Gb for the 4 CPU without restriction. Is this problem 
> >> >>> at
> >> >>> the hardware level?
> >> >>>
> >> >>> What is the maximum amount of memory a CPU can use?
> >> >>>
> >> >>> I heard talk of a Tb memory machine, but it was part of a 1000 node
> >> >>> cluster, so I am thinking OK 1 Gb per node.
> >> >>>
> >> >>> Can anyone clarify this for me?
> >> >>>
> >> >>> Cheers,
> >> >>> Dan.
> >> >>>
> >> >>> On Mon, 29 Mar 2004, Philip MacMenamin wrote:
> >> >>>
> >> >>>> On Monday 29 March 2004 05:38 am, you wrote:
> >> >>>>> hello all,
> >> >>>>>
> >> >>>>> i'm interested to the constructiong of a linux cluster of computer
> >> >>>>> for
> >> >>>>> bioinformatics purpose. but i dont have a cluu about the performans
> >> >>>>> it
> >> >>>>> should have. is there any one who can give any suggestion?
> >> >>>>>
> >> >>>>> in  the hope of an answer
> >> >>>>>
> >> >>>> Its kind of a nebulous question.
> >> >>>>
> >> >>>> Basically... buy Xeon || AMD two way boxes. Be boring, look at Dell 
> >> >>>> or
> >> >>>> penguin computing or something. Dont bother with 64 bit. Think about
> >> >>>> that in
> >> >>>> about 2 years time. It will irritate you in 2 months after you buy
> >> >>>> it, *i
> >> >>>> promise*.
> >> >>>>
> >> >>>> Dont buy from some indy very cheap manufacturer, cause, if the nodes
> >> >>>> will
> >> >>>> fail, you will be left waiting... or make them give you a decent
> >> >>>> guarantee.
> >> >>>> Or a big box of bits.
> >> >>>>
> >> >>>> Look at the price of the chip : speed ratio. This graph will have an
> >> >>>> inflection point, at which more money input gives diminishing 
> >> >>>> returns
> >> >>>> speed-wise. Buy at this inflection point. This changes all the
> >> >>>> time... of
> >> >>>> course. There is no point in buying the *best* out there right this
> >> >>>> second.
> >> >>>> Just buy a little behind it, and buy an extra box or two.
> >> >>>>
> >> >>>> Find out what you want to run.
> >> >>>> Buy as enough memory so it doesn't thrash your disks. (Or just fill
> >> >>>> it with
> >> >>>> memory).
> >> >>>>
> >> >>>> Spending about 2 grand (USD) on them a piece (I dont know what that
> >> >>>> is in
> >> >>>> Lira) should buy you something decent operating at 3 gigs, with a
> >> >>>> couple of
> >> >>>> gigs ram.
> >> >>>>
> >> >>>> Then have a look at
> >> >>>> http://bioinformatics.org/biobrew/
> >> >>>> www.rocksclusters.org/
> >> >>>>
> >> >>>> Sorry if this is very uninspiring/boring advice... its just, you 
> >> >>>> want
> >> >>>> to keep
> >> >>>> everything as simple as you can. There will be tricky bits no matter
> >> >>>> how
> >> >>>> simple you make everything.
> >> >>>> Good luck.
> >> >>>> Philip.
> 