[Bioclusters] notes on Platform LSF Webgui / Apache 2.x integration within a MacOSX/Linux
hybrid cluster
Chris Dagdigian
bioclusters@bioinformatics.org
Fri, 09 May 2003 16:09:32 -0400
BioTeam gets asked all the time to explain concrete differences between
the features and functionality one gets with Sun Grid Engine vs.
Platform LSF.
The answer is alway different as people have many reasons for asking
that question. Some are deeply interested in the architectural
differences, others are freaked out by how different 'queues' can be
between LSF/PBS/GridEngine etc. and many just want to know "will the
freely available SGE suit my needs or do I have to pay $$ to Platform
for some LSF licenses?"
I'm agnostic myself on the SGE vs LSF front and try to make sure that
people are able use the best tool for the job. Of the 4 bioclusters that
I'm currently working on or have just completed 50% are running Grid
Engine and the other 50% are running Platform LSF. Different tools for
different environments and production needs...
On the whole "differences between SGE and LSF" front...
One thing I noticed recently when working with LSF 5.1 is the new
Java/Tomcat web GUI package that is distributed for 'free' along with
LSF Standard. The tomcat appserver inside the webgui apparently talks
via SOAP to LSF which in itself is potentially very interesting for
people doing webservices stuff.
I'm always way more productive at the commandline so this
"lsf_5.1_webgui" package was something that I never really bothered to
look at until I realized that this was one of the concrete examples of
something one gets along with LSF that is certainly not included with
Grid Engine. The existing SGE web tools one can find on the net are
basic scripts that do little more than wrap the 'qstat' binary around a
perl or php CGI. Sun may have wrapped SGE into their "SunOnePortal"
appserver efforts but I've never seen it in action.
With that in mind I decided to install the LSF webgui package onto a
hybrid Apple/Linux cluster so I could take it for a spin. (pictures of
the Apple/Dell hybrid cluster are now in the biocluster gallery at
http://bioteam.net/gallery/album06)
<break>
Anyone who wants to see screen shots of the LSF Web GUI in action can
stop reading and just point a browser to http://bioteam.net/gallery/album04
Read on for notes on how to put Apache 2.0.x in front of the LSF web gui
</break>
The LSF webgui installs just like any other LSF add-on. You put the
zipped up tarball into an installation directory and call the regular
old LSF installer shell script. The process was straightforward and the
archive was unpacked and automatically configured for use within the
existing LSF cluster environment.
The web GUI is started from the commandline:
> $LSF_HOME/bin/gaadmin start|stop
The first thing I realized is that the app was starting up on port 8080.
The port can be changed by altering the port value found in:
$LSF_HOME/tomcat/conf/server.xml but...
This was not optimal as I did not want to deal with opening up another
firewall port for HTTP traffic _and_ I already had a perfectly good
SSL-enabled Apache 2.0.40 set up with all sorts of static and dynamic
content at the usual port 80 location.
What I wanted to do was 'embedd' the LSF GUI into the regular old
cluster webserver which had already been set up with monitoring tools,
documenation and all sorts of other things.
A bit of google searching for "tomcat apache integration" shows that the
standard way that people do this is to use the "webapp" Apache module
which acts as a "connector" to tomcat application servers.
After a bit of trial and error I was able to download and build
mod_webapp from the Apache.org CVS server and get it cleanly loaded as a
DSO into the Apache 2.0.40 server running on the cluster.
Once that was done; it looked as the config would be trivial. All I
should have had to do was put something like this into my httpd.conf file:
<IfModule blah blah blah...>
WebAppConnection warpConnection warp localhost:8008
WebAppDeploy examples warpConnection /examples/
WebAppDeploy Platform warpConnection /Platform/
WebAppInfo /webapp-info
</IfModule>
Problem was, I could not do that
The Apache 2.0.4 server kept thinking that the webapp configuration
directives were virtual host directives and kept complaining about
"syntax error in virtual host name" etc. etc.
Google searches seemed to indicate that others have had problems with
apache thinking that webapp commands were vhost commands so at this
point I gave up on mod_webapp and moved on to working with mod_proxy
which I have had better experiences with in the past.
All the other tomcat/apache integration websites seemed to be based on
the use of the Apache-1.x series so no luck there.
I blew away the mod_webapp stuff and made sure that mod_proxy was
available to me. Then I put these lines into httpd.conf:
## Test proxy for LSF web gui
ProxyPass /tomcat http://localhost:8080
ProxyPassReverse /tomcat http://localhost:8080
A restart of the webserver and things started working!
I was able to get to the tomcat start page by pointing a browser at
/tomcat/index.html
Next step was to try getting the LSF web interface loaded by pointing my
browser at /tomcat/Platform/
Aargh!
I can obviously talk to the LSF interface but the HTML is all screwed
up. The appserver spits back HTML with relative inline links that point
to /Platform/ which result in a screen full of 404 not found errors.
My first thought was to use mod_rewrite within Apache to fix this but I
ended up going for the quick and dirty fix. My httpd.conf file now has
these lines in it:
## Test proxy for LSF web gui
ProxyPass /tomcat http://localhost:8080
ProxyPassReverse /tomcat http://localhost:8080
ProxyPass /Platform http://localhost:8080/Platform
ProxyPassReverse /Platform http://localhost:8080/Platform
It works!
All the screenshots from the LSF web GUI that are online at
http://bioteam.net/gallery/album04 were taken while SSL-connected to the
Apache 2.0.40 server with the above mod_proxy configuration.
I wanted to post these notes because google was not all that helpful
this time around and perhaps this will save some other poor soul a few
hours of effort...
-chris
--
Chris Dagdigian, <dag@sonsorol.org>
BioTeam Inc. - Independent Bio-IT & Informatics consulting
Office: 617-666-6454, Mobile: 617-877-5498, Fax: 425-699-0193
PGP KeyID: 83D4310E Yahoo IM: craffi Web: http://bioteam.net