NBIC Galaxy Server: Maintenance Guideline

From BioAssist
Revision as of 15:40, 10 September 2010 by David.van.enckevort (Talk | contribs)

Jump to: navigation, search

Schedule

A monthly maintenance is scheduled on first Monday afternoon of each month. Prior to each maintenance, the collected update requests from previous month are discussed within the maintenance team in the form of email or skype call. If a decision is made to update the Galaxy server, pipelines, tools and libraries, the actual update will be executed by a dedicate team member.

(Tentative) Responsibility

  • Pieter will monitor the release notes and bug reports of Proteomics tools and Galaxy
  • Leon will monitor the release notes and bug reports of NGS tools
  • Jeroen will coordinate the maintenance task and perform the update
  • David will maintain the virtual machine package and assist Jeroen

Galaxy

General Policy

Galaxy at Penn State is updated very frequently. However we will not push ourselves to follow their pace. We will only update the server if:

  • there is a new feature we want to have on the NBIC Galaxy server

or

  • there is known bugs or security breach in our current version

Steps

To install a new version of Galaxy from Penn State, here are the steps we should follow as a maintenance team member. Jeroen is planning to automate the entire procedure.

  1. log into galaxy.nbic.nl with your account. You only need a "devs" account to do all the following. Never use root!
  2. go to the directory of /nbic/prog
  3. Check out the latest code from Penn State: hg clone http://www.bx.psu.edu/hg/galaxy galaxy-<year>-<month>-<date>
  4. Install the new galaxy
    • cd /nbic/prog/galaxy-<year>-<month>-<date>
    • sh setup.sh
  5. Copy the following customized interface files from the corresponding location of the previous galaxy installation.
    • static/welcome.html
  6. Link NBIC Galaxy module repository
    • cd /nbic/prog/galaxy-<year>-<month>-<date>/tools
    • ln -s /nbic/prog/nbic_gmr nbic_gmr
  7. Configure tool location files and other tool-data files
    • cd /nbic/prog/galaxy-<year>-<month>-<date>/tool-data
    • you need to copy the following files/directories from the corresponding location of the previous galaxy installation. This list will grow when more tools are added and customized
      1. bowtie_indices.loc
      2. lastz_seqs.loc
      3. sam_fa_indices.loc
      4. msCompare
  8. Configure universe_wsgi.ini, tool_conf.xml, datatype_conf.xml to include useful NBIC customization. You can do this by running a diff program to compare with the same file of the previous galaxy installation
  9. Update the Galaxy symbolic link
    • rm /nbic/prog/galaxy
    • ln -s /nbic/prog/galaxy-<year>-<month>-<date> /nbic/prog/galaxy
  10. Restart Galaxy server
    • /etc/init.d/galaxy restart (to run this, you need to be root but "runuser" command makes sure that Galaxy instance will be run with the user account of "nbic")
    • Sometimes, you are required to upgrade the database schema. Do backup the galaxy DB before you run the upgrade script.

Pipelines

General Policy

The pipelines provided by NBIC Galaxy server should be of interest for a large number of users. Thus:

  • We will take into simple but useful pipelines
  • We will NOT take into fancy pipeline that only address one's individual problem.

Steps

At the moment, we request the developer to upload their pipeline to https://trac.nbic.nl/galaxytools/ and provide a README file to explain its installation.

Here are the steps:

  1. Log into galaxy.nbic.nl with your account. You only need a "devs" account to do all the following. Never use root!
  2. Go to the directory of /nbic/prog/nbic_gmr and check out the requested pipeline code.
  3. Update "tool_conf.xml" accordingly.

Tools/libraries

General Policy

To guarantee a certain life span of each pipeline (which could be useful for users if they want to replicate their data analysis), we will support each version of every tool and library for at least 1 year. This implies all the tools supported at NBIC Galaxy should be labeled with their version explicitly and be called with the explicit version number by pipelines.

A list of tools installed at the NBIC Galaxy server can be found at NBIC Galaxy Server: Current Tools

Installation Location

  • General Linux tools/libraries should be installed in standard Linux directories, e.g. /usr/, /usr/local/, etc.
  • Specific data analysis tools should be installed in NBIC tool directory, e.g. /nbic/prog/lib, /nbic/prog/share.

Data Files

Installation Location

  • Reference genomes data files should be installed at: /nbic/data/Genomes/

Creating new user accounts

To keep the server safe it is important to have a policy on the user accounts.

  • Create a new user with the following command (replace group1,group2 and user with the correct values of course, if no special group memberships are necessary, you can omit the -G ...)
 useradd -c 'Comment describing the user' -G group1,group2 user
  • Always use a strong password on user accounts (8 or more characters and use letters, numbers and specials). Set the initial password with
passwd user
  • When you create an account and have to e-mail the password to the user enforce a password change on first use with the following command:
chage -d 0 user
  • When you are giving temporary access to someone set an expiration date for the account with the following command:
chage -E YYYY-MM-DD user
  • make sure the user is member of the right groups. Users who need to have access to galaxy need to be in the devs group, but only give group membership if it is really necessary. Anyone who is member of the devs group can potentially break stuff...
  • To change the group memberships later on you can use the following command:
usermod -a -G devs user