NBIC Galaxy Server: Maintenance Guideline

From BioAssist
Revision as of 11:38, 13 January 2013 by Freek de Bruijn (Talk | contribs)

Jump to: navigation, search

Schedule

For the Galaxy server, a monthly maintenance is scheduled on first Monday afternoon of each month. Prior to each maintenance, the collected update requests from previous month are discussed within the maintenance team in the form of email or skype call. If a decision is made for updating the Galaxy server and pipelines, or adding tools and libraries, the actual update will be executed by a dedicate team member.

Note that the changes will be stored at a new disk image, so in case system is broken, we could roll back to the previous Galaxy image.

Responsibility

How to update system packages

The update of Galaxy OS packages is done by the NBIC BET on a weekly base.

To make the changes persistent, we should follow the instruction to shutdown and restart the VM.

How to install/update Galaxy

General Policy

Galaxy at Penn State is updated very frequently. However we will not push ourselves to follow their pace. We will only update the server if there are:

  • New features or bug fixes we want to have on the NBIC Galaxy server

or

  • Security patches

Instructions

NBIC Galaxy Server: How to update NBIC Galaxy

How to install tools and libraries

On each distribution of the NBIC Galaxy server (e.g. online version or downloadable VMs), only one version of a tool/library exists. To guarantee a certain life span of each tool (which could be useful for users if they want to replicate their data analysis), we will support each version of every tool and library for at least 1 year. Since we plan updates the tools twice a year, we will clone the current online server into a downloadable VM before the update and make it available as the previous galaxy. The main VM can then be updated. When we additionally archive all the older VMs, an analysis can always be repeated.

A list of tools installed at the NBIC Galaxy server can be found at NBIC Galaxy Server: Current Tools

Linux tools/libs from a distro

If tools or libs are available from a distro repository these should be installed as root using a suitable package maneger depending on the repo. Currently we use Debian Squeeze flavored Linux OS. You can use apt-get to install any tool

  • sudo apt-get install <package> (highly recommended way to install tool/package)
  • sudo dpkg -i package.deb


To check if package/tool is available in distro repository

  • dpkg-query -W | grep -i mytool


To update repository and packages

  • sudo apt-get update
  • sudo apt-get upgrade

When you need to add or modify config files for the package you've just installed, please make sure to add a copy or commit your changes to the conerice repo.

Linux tools/libs not available from a distro repo and 3rd Party software

Software should never be installed in default paths, but in /usr/local/ instead. If you need a newer version of some package, don't overwrite the version installed as part of a distro!

Perl modules available from CPAN

Perl specific: additional Perl modules go into /usr/local/perl_plus/. If a module is available from CPAN you can install it as follows (using your own account; hence never as root):

  1. Start the CPAN shell. If this is the first time you use it you'll have to answer a bunch of questions to configure CPAN. The defaults should be mostly fine.
    • perl -MCPAN -eshell
  2. Change the location where new Perl modules will be installed:
    • o conf makepl_arg "PREFIX=/usr/local/perl_plus/"
    • o conf mbuildpl_arg "--install_base /usr/local/perl_plus/"
  3. To store the settings above, so you won't have to type them again next time:
    • o conf commit
  4. Try to install your module of interest:
    • install MY::Perl::Module

Python eggs available as tar.gz

You can try to install these with easy_install:

  1. Save the downloaded archive to /mnt/galaxyTools/sources/python/
  2. run easy_install
easy_install --prefix /usr/local/python_plus/ /mnt/galaxyTools/sources/python/downloaded-python-egg.tar.gz

Everything else

When you need to install something from source, please keep a copy of the source in /mnt/galaxyTools/sources/ for future reference. Perl modules go into /usr/local/perl_plus/, Python eggs into /usr/local/python_plus/ and everything else either in /usr/local/galaxytools/. In case of complicated installs you prefer to keep in a separate sub dir and version number /usr/local/galaxytools/<tool>/<version>/ i.e. /usr/local/galaxytools/R/2.11/.

Install on a different root directory.

  • sudo dpkg -i package.deb --root=<directory> --instdir=<directory>
  • i.e. sudo dpkg -i package.deb --root=/usr/local/

NBIC Tools

NBIC Galaxy server presents a number of tools developed tools by NBIC members. These tools should be of interest for a large number of users. We cannot support fancy tools that only address one's individual problem unless the developer of that tool joins the maintenance crew.

Installation

At the moment, we request developers to upload their tools to https://trac.nbic.nl/galaxytools/ and provide a README file to explain its installation.

For admins, here are the steps to add tool in the NBIC Galaxy server:

  1. Go to the directory of /mnt/galaxyTools/sources/galaxytools and check out the latest version from the repo: http://trac.nbic.nl/galaxytools/
  2. Update "tool_conf.xml" and optionally "datatypes_conf.xml" accordingly.
  3. Always validate the XML files you touched (i.e. tool_conf.xml, datatypes_conf.xml etc.) use xmllint command-line utility e.g. xmllint --valid tool_conf.xml
  4. Commit the modifications to these *_conf.xml files to the conerice repo.
  5. Check the README at the root of the galaxytools repo and optionally for a README in the subdir for a tool to check if additional changes are required.