[Bioclusters] Which Sun Grid Engine?

[Bernard Li]

I haven't tried the ARCo (accounting) stuff, but it sounds great and I'm sure it's great in practise.  I don't have the exact link, but perhaps Rayson can point you at the correct direction :)

It's free (not open source) but there is no problem integrating with the open source version of Grid Engine.

Have you looked at mpiBLAST?  I think Joe Landman has some integration scripts for SGE to make it even easier to use.

Regarding SGE 5 & 6's definition of what a "queue" is - actually the way it is defined in 6 is the more "popular" defintion as you allocate resources (nodes) to it, whereas previously in 5, each node has its own queue, which is somewhat "strange" to traditional clustering.  For instance our cluster is still 5.x based and our users submit jobs to Projects, however when we migrate to 6, they will be submitting to queues.

Cheers,

Bernard


-----Original Message-----
From: Michael James [mailto:Michael.James at csiro.au]
Sent: Wed 16/08/2006 23:21
To: HPC in Bioinformatics; Bernard Li
Subject: Re: [Bioclusters] Which Sun Grid Engine?
 
On Thu, 17 Aug 2006 03:04 pm, Bernard Li wrote:

> You can get official support from Sun
> if you purchase Sun N1 Grid Engine.
> "Sun Grid Engine" is the open source version.
> As far as I know they are working off the same tree,
> and even some previously proprietary things
> like Windows execution hosts,
> accounting are now freely downloadable (not open source though).      

"Official Support" phooey.
"Windows execution hosts" even more phooey.

However  "Accounting"  Oh yes please!
I lost hair and sleep over that confounded "accounting" file.
Eventually restored sanity with a perl script
 that put the entries in order by month
 and weeded out the half that were dead jobs
 with no resource usage attached ...

> The users at gridengine.sunsource.net mailing
> is extremely high traffic, so you usually get
> pretty good support for questions.  

Thanks, I'll have a look.

> In terms of upgrade -
> I suppose they have some upgrade scripts,
> but 5.x is fundamentally different from 6...  

Our cluster is simple enough (66 identical nodes)
 that we can re-build that from scratch.

More difficult is any adjustment to the calling scripts,
 the batchblast script that cuts a concatenated fasta file
 into single sequences, and issues the qsub -t  command

And then there's grocking the underlying paradigm;
 in sge5.3 the queue is something that describes a computing resource
 in sge6 (I believe) a queue characterises a job.

So in 6 you submit a job to a queue ("urgent", "batch", etc)
 whereupon it takes on that queue's characteristics.
In 5 it was the job of the scheduling system to decide
 which computing resource (also called a queue)
 should process a given job
 based solely on the flags in the qsub command.

Thanks again,
michaelj

-- 
Michael James                         michael.james at csiro.au
System Administrator                    voice:  02 6246 5040
CSIRO Bioinformatics Facility             fax:  02 6246 5166

No matter how much you pay for software,
 you always get less than you hoped.
Unless you pay nothing, then you get more.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://bioinformatics.org/pipermail/bioclusters/attachments/20060816/878beec0/attachment.html