[BioBrew Users] Help with sge, scheduling dual cpu nodes?

Glen Otero gotero at linuxprophet.com
Fri Dec 12 18:14:24 EST 2003


Sorry about the delay in response--been too busy for my own good : )

Just to be sure, what's the output of the allocation rule for the
default mpi pe configuration? Run this:

# qconf -sp mpi

If it's $fill_up, it means that when you use the mpi pe, it will use up 
the slots on each node before moving on to the next available node. The 
default found in BioBrew is $round_robin. If it's $round_robin, then 
one job will be scheduled on that node, and the next job will be 
scheduled on another node. Sounds like you need to change the attribute 
allocation_rule to $fill_up or $pe_slots.

If what you want is to run only one processor per machine (which it 
sounds like you don't) you would want to
create another similar PE like mpi, call it mpi-uni maybe, and change
the allocation rule to $round_robin. Here's a quick way to achieve that:

# qconf -sp mpi > mpi-uni

Edit the mpi-uni file and change the pe_name to mpi_uni and
allocation_rule to $round_robin. Add this new PE back to the SGE pool:

# qconf -Ap mpich-uni

You may also need to increase the number of slots for each queue. The 
slots, by default, correspond to the number of processors for each 
node. So if the node has dual processors, the number of slots will be 
two. But you can change it. To change it for a queue (e.g 
compute-0.0.q):

# qconf -mq compute-0-0.q

This opens up a queue configuration session with your current $EDITOR.
Look for an attribute called slots and change that.

Hope that helps!

Glen


On Dec 11, 2003, at 5:30 AM, Humberto Ortiz-Zuazaga wrote:

> I'm testing biobrew v0.9 on a 5 node cluster, each with dual 2.4 GHz 
> Xeons.
> I've never used sge or rocks before, so I'm a little confused.
>
> I want to run a 2 cpu mpi job, but sge schedules 2 nodes with one mpi 
> thread
> each instead of both processors on a single node.
>
> I'm running everything with the defaults chosen by biobrew.
>
> Here's my qsub script, in PBS, I can specify
>
> #PBS -l nodes=1:ppn=2
>
> to get both processors on a single node, what's the equivalent sge 
> magic?
>
> #!/bin/sh
> #$ -pe mpi 2
>
> /opt/mpich/ethernet/gcc/bin/mpirun -np $NSLOTS -machinefile 
> $TMPDIR/machines
> /opt/hpl-eth/bin/gcc/xhpl
>
>
>
Glen Otero, Ph.D.
Linux Prophet
619.917.1772

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/enriched
Size: 2246 bytes
Desc: not available
Url : http://bioinformatics.org/pipermail/biobrew-users/attachments/20031212/883644f9/attachment.bin


More information about the BioBrew-Users mailing list