[Bioclusters] parallel blast???

David Thomas bioclusters@bioinformatics.org
Mon, 30 Sep 2002 15:30:26 -0700

I just came from IEEE Cluster 2002 where there was a talk from the PVFS -- 
parallel virtual file system.

I understand that the performance of this software is outstanding.
It's not stable enough and fault-tolerant enough for the enterprise but it 
might be interesting to test it out for ourselves on BLAST.  Has anyone
played with it?


On Sat, Sep 21, 2002 at 07:01:34PM -0400, Chris Dagdigian (dag@sonsorol.org) wrote:
Subject: Re: [Bioclusters] parallel blast???
Message ID: <13526FA8-CDB6-11D6-8DCD-00039397F4C8@sonsorol.org>

> Last time I looked at them solid state disks were amazingly expensive. 
> I was thinking about trying them out as swap devices on a big 
> alphaserver but ended  up deciding to spend the $$ on more physical 
> memory for the system.
> In a blast or blast-farm context I'd probably just skip the solid state 
> disks and instead put the databases into a ramdisk. That would be a 
> cheaper approach since you don't really need the 
> data-is-kept-when-power-goes-away or the backup hard disk that solid 
> state systems give you. You also are limited by whatever pipe connects 
> the SSD to the system (SCSI?).
> Even ramdisks are of limited utility given the size and growth rate of 
> some of the more common sequence databases -- you'd fall behind 
> eventually.
> Although -- if you put 1 or 2 GB ramdisks in each of your cluster nodes 
> and then set up a system for chunking blast databases into 
> ramdisk-friendly sizes you could build a really fast blast farm. In 
> that context the performance bottleneck would then become the time and 
> resources needed to merge the XML output from N queries against split 
> databases into a single result file. I've seen such systems in the past 
> and merging the results could in some cases take longer than the actual 
> search did.
> -Chris
> On Tuesday, September 17, 2002, at 08:38  AM, Steve Gaudet wrote:
> > Hello Chris,
> >
> >> <snip>
> >>
> >> You can have the fastest server on earth but if you searching with
> >> blast against an NFS mounted database and your network or
> >> fileserver is
> >> slow then your blast searching speeds will be horrible. Give
> >> me a small
> >> number of speedy linux boxes and I can bring a $300,000
> >> NFS/NAS system
> >> to its knees. Storage does matter.
> >
> > Anyone ever look or try solid state disks?
> >
> >
> >> <snip>
> _______________________________________________
> Bioclusters maillist  -  Bioclusters@bioinformatics.org
> https://bioinformatics.org/mailman/listinfo/bioclusters