The BIRCH Development Process
The Structure of BIRCH
BIRCH consists of three main components: The Installers, the Framework, and the Binaries. The installers include GetBirch, a fully-automated install wizard for use by end-users, and birchconfig, a GUI installer that breaks the install process into several components, primarily as a mechanism for testing install scripts outside of the context of the install wizard.
Two components are installed in all birch sites: the Framework and the Binaries. The Framework is the set of directories, scripts, and non-binary applications, as well as documentation and ancilary data files. All of the functionality of BIRCH resides in the Framework, and in principle it would be possible to have a functional
BIRCH system with no binaries. When choosing 3rd party applications to include in BIRCH, we tend to favour platform-independent ones (eg. Java, Python) over programs needing to be compiled. The Binaries include both compiled binaries and libraries for 3rd party applications, such as BLAST, FASTA, or Phylip.
How We Work on BIRCH
Development of BIRCH components could occur in the context of any BIRCH system. However, most development work is currently done by the BIRCH team at the Bio Information Technologies lab at the University of Manitoba.
There are several servers on which BIRCH is developed. When a new release is ready, the framework and binary files are uploaded to the FTP site at www.umanitoba.ca. The figure below summarizes the roles of these servers.
Note: The figure above has not yet been revised to correctly show the BIRCHBINDEV directory for linux-intel.
CCL - framework and binaries - Most work on the BIRCH framework occurs on the Univ. of Manitoba CCL Linux system. Work on the BIRCH system is done in /home/psgendb/BIRCHDEV, where the current Development version of BIRCH is found. This includes all files in the BIRCH framework, as well as binaries and libraries for solaris-sparc, solaris-amd64, and linux-x86_64. You either need to be a member of the 'birchdev' Unix group, or be logged in as 'psgendb'.
In all BIRCH development sites, there is are directories called 'install' and 'build'. The install directory is the place where 3rd party programs are downloaded, untarred, and in the case of C code, compiled. The build directory contains a script that builds a .tar.gz file containing the binaries and libraries for that platform.
There are three additional servers on which the development of BIRCH binaries and libraries are done. On each server, there is an account with userid 'BIRCHBINDEV' or 'birchbindev', which includes a complete BIRCH install within a directory called $HOME/BIRCHBINDEV. As well, there are also install and build directories for compiling code and building .tar.gz files. the bin-xxx-xxx.tar.gz files must be uploaded to www.umanitoba.ca by sftp or scp.
BIRCHBINDEV@flamingo.plants.umanitoba.ca This server, an Lubuntu 64-bit Linux system for production work and testing.
PeacockThis is a MacOSX laptop for development of Mac OSX binaries and libraries.
Why don't we use a version management system? - There are two reasons why we don't use a system like Git. First, there is a steep learning curve with most version management systems, and a lot of potential for developers who don't have a strong grasp of the subtleties of Git to do a lot of damage. Secondly, Git doesn't scale well, for systems such as BIRCH which have over 15,000 files. We are working to better automate Git access to make it more foolproff, and to break BIRCH into smaller components that will scale better with Git.
The BIRCH Installers
The GetBirch installer is intended for end users, and is designed to provide a foolproof install and update mechanism. However, it does not allow for fine control of the install and update process. For this reason, there is also a semi-automated installer called birchconfig, meant to be used by developers.
Both installers are linked from the BIRCH Administrator's Guide.
Why use the semi-automated install? There are several possible reasons. First, GetBirch is an evolving tool, and as it evolves there may be cases where you want to test BIRCH without worrying about whether GetBirch is doing the right thing. Secondly, for the special cases of BIRCHDEV and BIRCHBINDEV, birchconfig is, at this point, probably safer. Finally, there are cases in which a developer wants to try out new things, especially with regard to the install process. If one had to go through GetBirch, it would be necessary to make each change in BIRCHDEV, rebuild the framework, and then run GetBirch to download the new build. Instead, one can just keep a copy of framework_D.tar around, and run through install/uninstall cycles with that copy. This saves the extra steps of downloading the framework each time.
The BIRCH Framework
To work on files that are part of the BIRCH framework, install a copy of the current Development version of BIRCH on your own account.
For most scripts and BioLegato blmenu files, you should be able to make copies of files from the BIRCH framework and move them to $BIRCH/local, where they will supersede the default files in the BIRCH framework. Links to documentation can be added to your lbirchdb database. See the BIRCH Administrator's Guide for full documentation on the procedures for creating content in $BIRCH/local.
When files are ready to transfer to BIRCHDEV, make a tar archive of just the files you have changed, and send them to Brian, who will integrate them into BIRCHDEV.
BIRCH binaries
To work on files that are part of the BIRCH binaries, install a copy of the current Development version of BIRCH on your own account.
For most binaries, you should be able to make copies of files from the BIRCH framework and move them to $BIRCH/local, where they will supersede the default files in the BIRCH bin-xxx-xxx and lib-xxx-xxx directories. Links to documentation can be added to your lbirchdb database. See the BIRCH Administrator's Guide for full documentation on the procedures for creating content in $BIRCH/local.
When files are ready to transfer to BIRCHDEV, make a tar archive of just the files you have changed, and send them to Brian, who will integrate them into BIRCHDEV.
Generating a Release
Development Release
At any time, a Development version of BIRCH can be created from the build directories. The framework is built in BIRCHDEV/build. The binaries are built in either BIRCHDEV/build or the various BIRCHBINDEV/build directories, as described above.
cd BIRCHDEV/build
./makeframework.csh
Excluded and included files
The build directory contains two files that tell makeframework which files and directories to include in the release or to exclude. makeframework.includefile is a list of all directories or files to include in the .tar.gz file. Specific files or subdirectories can be excluded by adding names to makeframework.excludefile. In particular, .git directories should always be excluded, because these tend to be large.
Stable Release
When the decision is made to freeze a release, the current development version is rebuilt as a stable release using
makeframework.csh -v number
where number is a decimal number for the new stable release eg. 2.5.
makeframework.csh doesn't directly build bin-xxx-xxx_version.tar.gz files. However, when run with the -v option, it automatically copies the binary files from the FTP/BIRCH/Development directory and renames the file with the stable release number. Thus, the Development directory will contain a new copy of the framework, along with binary files for all platforms.
At present, only psgendb as the permissions to generate a BIRCH release.
BIRCH Stable Release Checklist
Projects, Ideas and ToDo Lists
Birch Bugzilla at Bioinformatics.Org
BIRCH/Release To Do list
BIRCH/Meetings
Discussions
Current Projects
Ideas for the Future
Criteria for inclusion of 3rd party software in BIRCH
- does it add a new function, or a better way to do something?
- are the results accurate over a boad range of cases?
- is it freely-distributable by others?
- does it run on Linux/Unix and Mac?
- how straightforward is it to use?
- does it lend itself to running by BioLegato?
- how hard will it be to adapt to BIRCH and to maintain?
BIRCH/New Applications under consideration
Keeping track of competing software packages
BIRCH as a software layer
Outreach
BIRCH is now listed with Omicstools.com at https://omictools.com/biological-research-computing-hierarchy-tool
Old/Completed Projects
