BIRCH/Meeting/Jan092012

From Bioinformatics.Org Wiki

Jump to: navigation, search

Contents

Overview

Key points:


Phylip and premake

Since the makefiles for phylip have been reverse engineered into premake scripts, we can now generate makefiles for each platform automatically by following the following steps:

  1. git clone git@github.com:daleha/birch-native-sources.git # the sources used to generate the build artifacts
  2. git clone git@github.com:daleha/birch_build.git # premake binaries for every platform, as well as the premake source code
  3. cd birch-native-sources/phylip-3.69
  4. ../../birch_build/bins/[PLATFORM]/premake4 gmake
  5. ../../birch_build/bins/[PLATFORM]/premake4 build

That's it! Nothing more needs to be done, the steps to build phylip are now the same on all platforms!


Notes:

BIG QUESTION: When a new release of Phylip or FASTA or whatever comes out, and they change the makefiles, doesn't that break the entire premake process?

The FASTA situation

clean-up:
       rm -f *.o $(PROGS) $(APROGS);   rm -rf $(BIN)/*

(there is no assurance that BIN is defined, and hence it deleted every file that I had permission to delete on my system). Luckily, I was using git-kit, and didn't lose much work related to BIRCH, but it clobbered my entire home directory. With automatic make file generation, cleanup actions are much safer



Gitkit

Gitkit is ready for deployment. (demonstrate gitkit now).

Winbirch

Winbirch now has phylip, and will have FASTA very soon. After that, we will only need to write premake scripts for any remaining sources, and create a repository for the binary-only distributed programs, then we should be ready for testing and release.

Premake

The steps to making a premake script are pretty simple. You can use an existing premake script as a template, and you basically just have to tell it which files need to be compiled, where they are, and what options they need.

Premake really simplifies cross-platform build quirks. For instance, i ran into some linker errors on OSX for some of the phylip programs, and i was simply able to specify "OSX " and "not OSX" configurations as a quirk work-around.

When a new version of phylip, FASTA, or another source program that we are using premake to build comes out, all that we will have to do is update any rules that have change for the build. If no new files have been added, then there is nothing to do, and the same script can continue to build multiple versions of the program.

In fact, I suspect that the FASTA developers will love our premake framework.


Possible Feature(s) Discovered

Static libraries

I don't think that we will be needing a "lib-xxx-xxx" folder anymore! Rather than ship system libraries with birch, we can simply keep a repository of static libraries (.a files) that each platform needs to link to. This way, the symbols that would normally be looked up at runtime using LD_LIBRARY_PATH will be pulled out of the static library at compile time, and placed into the executable. This will likely improve performance, definitely reduce the size of BIRCH, and reduce the file bloat in BIRCH.

Cross compilation

If we use mingw for all of our windows builds, then we should theoretically be able to build windows binaries on a linux server equipped with the mingw development toolset.

Furthermore, using premake, we can cross-compile linux32 binaries from linux64

This would allow us to build all of our binaries using only 3 machine configurations:


Connecting the Dots

Premake + git-kit + integration APIs (github, cvs/svn) = an open-source, distributed build-automation system designed to maintain open-source submodules.

This applies perfectly to the requirements of the BIRCH framework. BIRCH is naturally made of sub-modules, as it ties 3rd party code together into a framework. By defining these submodules semantically, along with using a modern build framework, will be able to maintain our own branches of open-source 3rd party developer's software. For those programs that are no longer maintained, we would be able to take over as maintainers, and breath new life into dead programs. For programs that are in active development, we will be able to keep current rapidly and with elegant simplicity. If we can link directly in to 3rd party developer's VCSs, we will be able to establish a "birch nightly build", that serves up fresh copies of 3rd party software for end-users and developers alike to benefit from.

But that's not all, git-kit integration make's this a distributed build framework. Literally, with a single API call, a series of linked repositories will be able to produce code in the time it takes to compile it.

I would like to make this aspect the focus of the paper that you requested I outline at our last meeting. BIRCH/Developer_resources/git-kit


To Do



Things that I still need:

Things that would help me:


  1. Compile the binaries listed above
  2. Get copies of netblast, sequin
  3. Windows virtualization
    1. Reproduce getbirch virtualization error, determine solutions
    2. Find a way to get getbirch running faster (vmware?)
Personal tools
Namespaces
Variants
Actions
wiki navigation
Toolbox