[Bioclusters] openmosix cluster again

[Martín Sarachu]

I think Joe is right, we are not saying contradictory things. The only patched 
tool we use is BLAST.
EMBOSS, for example, is the exact same version you would use in a non-cluster 
environment. EMBOSS jobs are quite variable in its run-times and resources they 
need. Jobs that requires multiples accesses to a database perform very well, and 
jobs that do very little (check available databases) don't even migrate.
EMBOSS apps are not paralell, so you don't get a reduction in the run-time of a 
single job (in fact there is some little overhead for the migration), what you 
get is a reduction on the time it would take to run 20 jobs.
There's a nearly-official patch (MASK patch) for openMosix that allows to 
migrate shared-memory applications (Apache, PostgreSQL).


Regards,

martin

Joseph Landman wrote:
> Hi Hong:
> 
> I don't think these are contradictory.  Martin has been able to use 
> OpenMosix with a variety of tools.  If you note, he used the patched 
> version of BLAST (which specifically disables pthreads).  The other 
> tools are likely also either patched, or compiled without pthreading. 
> Postgres is a database, and you really don't want it migrating from its 
> data store.  Apache is a web server, and likewise, you don't want it 
> migrating (extra latency etc.).
> 
> Joe
> 
> hong.zhang@research.dfci.harvard.edu wrote:
> 
>> Dear Joe and Martin,
>>
>> Thanks for your instant reply. But it seems you two have contrary 
>> opinion.
>> I am kinda confused. Would you please give me more explanation?
>> I attached both of your answers below.
>>
>> --------------------------------------------------------------------
>> Martin said:
>>
>> openMosix works great for our site (AR.EMBnet). We have many bio web 
>> tools
>> (EMBOSS, SMS, WebPhylip) here and all of them (except SRS) work fine with
>> openMosix. They migrate very well.
>> On top of openMosix we use SGE for some jobs users submit by 
>> command-line.
>> We use the patched version of BLAST to be able to run it (migrate) under
>> openMosix.
>> -------------------------------------------------------------------------- 
>>
>> Joe said
>>
>>>   Openmosix will not migrate pthreaded jobs.  Postgres and the web
>>> interfaces (apache based) are all pthreaded.
>>>
>>>   What open mosix is as far as bio-clusters are concerned, is a tool to
>>>
>>> do a "semi-automatic" load balance.  Jobs which meet specific criteria
>>> can be migrated to less busy nodes.  For this to make sense, you would
>>> need a) the jobs to require long running times (as compared to potential
>>> migration time), b) sufficiently low IO and other resource utilization
>>> such that the migration will not generate hot-spots (resource
>>> contention, such as network bandwidth for accessing a file over NFS).
>>>
>>>   It is good for some types of jobs, though in a shared nothing
>>> environment (cluster) with limited network bandwidth and relatively high
>>> latencies, its isn't necessarily better than the queuing systems.
>>>
>>> Joe
>>>
>>
>>
>>
>>
>> _______________________________________________
>> Bioclusters maillist  -  Bioclusters@bioinformatics.org
>> https://bioinformatics.org/mailman/listinfo/bioclusters
> 
> 

-- 
Martín Sarachu
mad@biol.unlp.edu.ar
EMBNet Argentina
http://www.ar.embnet.org