NBIC Galaxy Server: How to update NBIC Galaxy

From BioAssist
Revision as of 12:07, 24 October 2012 by Leon (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Cloud template

  • change BUCKET_CLUSTER, BUCKET_DEFAULT, CLUSTER_NAME if a different bucket name is used at the cloudman script download site. For example http://downloads.nbiceng.net/brs-galaxy-cm/
  • PASSWORD is set to have authentication on cloudman UI login. The default username is "admin"
  • NETWORK=nbicgalaxy is set to obtain a static IP for NBIC Galaxy:
 TESTFLAG=false ]
 IMAGE_UNAME=hailiang.mei@nbic.nl ]
 TYPE=vnc ]
NAME="NBIC Galaxy"
 NETWORK_UNAME=oneadmin ]
 NETWORK_UNAME=hailiang.mei@nbic.nl ]
 BOOT=hd ]
 TYPE=kvm ]

VM Shutdown and Start Instructions

In order to stop the running VM at the SARA cloud and save it into a image file, we need to shutdown the VM and restart a new instance. Below are the steps:

  1. Log into cloud management console: https://ui.cloud.sara.nl/
  2. Verify the existence of a "save as" image of the current running VM, e.g. "GalaxyMaster_2012_10_18"
  3. Shutdown the VM
  4. Update "IMAGE=GalaxyMaster_2012_10_18" in the template "NBIC Galaxy"
  5. Remove bet1a:/var/www/vhosts/downloads.nbiceng.net/cloudman-on/cm.tar.gz
  6. Update "IMAGE=GalaxyMaster_2012_10_18" in bet1a:/var/www/vhosts/downloads.nbiceng.net/cloudman-on/cm/clouds/opennebula.py and recreate cm.tar.gz using the cm directory.
  7. In the cloud management console, initiate a new instance of the template "NBIC Galaxy".
  8. save the VM into a new image using "save as" and name it as "GalaxyMaster_yyyy_mm_dd".

Cloudman Startup Instructions

You need to restart cloudman in case the VM is restarted.

  • stop the running cloudman process
sudo sh /mnt/cm/run.sh --stop-daemon
  • kill the hanging Galaxy process that still occupies the Galaxy port if necessary
sudo kill -9 `sudo fuser -n tcp 42284`
  • kill all running sge process (let cloudman to handle the start of SGE)
ps aux|grep sge
sudo kill xxx
  • update /etc/hosts with master node's internal IP address and restart /etc/init.d/networking
sudo mount -t nfs4 /mnt/galaxyData
sudo mount -t nfs4 /mnt/galaxyTools
sudo mount -t nfs4 /mnt/galaxyIndices
sudo mount -t nfs4 /mnt/cloudman_configuration
  • restart rabbitmq server if it is not yet running
sudo /etc/init.d/rabbitmq-server start

  • start cloudman script
sudo sh /mnt/cm/run.sh --daemon
  • Check the log file /mnt/cm/paster.log to see if cloudman services (including postgres, SGE, Galaxy) start successfully.
  • To start/stop the Galaxy service only
sudo /mnt/galaxyTools/galaxy-central/run.sh --stop-daemon
sudo /bin/su - galaxy -c "export SGE_ROOT=/opt/sge; sh /mnt/galaxyTools/galaxy-central/run.sh --daemon"

Galaxy Update Instructions

When PennState Galaxy team releases new versions, we need to update our Galaxy codebase.

Stop the service

  1. lock the job submission at the NBIC Galaxy server via the admin UI.
  2. Stop the Galaxy daemon.
sudo /mnt/galaxyTools/galaxy-central/run.sh --stop-daemon

Download new release of Galaxy

Galaxy codebase is located at /mnt/galaxyTools (which is a NFSv4 mount of the cloud VM. All mount points are owned by user "postgres" as a workaround to support cloudman and postgres service)

  • go to the directory of /mnt/galaxyTools/sources/ and check out the latest code (central version) from Penn State:
hg clone https://bitbucket.org/galaxy/galaxy-central galaxy-<year>-<month>-<date>

Note: this is just a copy for reference. Do not modify this copy!

  • Copy this folder to /mnt/galaxyTools/. This will become the active installation with local mods.
cp /mnt/galaxyTools/sources/galaxy-<year>-<month>-<date> /mnt/galaxyTools/ -r
  • Update the Galaxy symbolic link
cd /mnt/galaxyTools
rm galaxy-central
ln -s galaxy-<year>-<month>-<date>/ galaxy-central

Update database

  • Configure the new galaxy: we keep the all the files (datasets) associated with histories stored in the Postgres database in a separate database folder outside the galaxy root folder.
cd /mnt/galaxyTools/galaxy-central
rmdir database 
# Note: make sure this is the EMPTY database folder of the new install and not 
# a symlink pointing to our data in /opt/galaxy/data/database/
ln -s /opt/galaxy/data/database/ database
  • Test starting up the Galaxy server (and fetch/check eggs if necessary)
sudo /bin/su - galaxy -c "export SGE_ROOT=/opt/sge; sh /mnt/galaxyTools/galaxy-central/run.sh --daemon"
# If you are getting conflicting eggs warnings, 
# try to fetch and check the eggs in separate steps:
sudo -u nbic python ./scripts/fetch_eggs.py
sudo -u nbic python ./scripts/check_eggs.py

Copy NBIC tools and data

  • Customize the new Galaxy: We keep copies of modified (config) files in the conerice repository. You should diff our customized versions in the conerice repo with the new versions in Galaxy. When necessary merge updates from the Galaxy developers with our local customizations and commit a new version to the conerice repo. If there are no changes you can simply copy the files below:
cd /mnt/galaxyTools/sources/
svn update conerice
cp conerice/masterfiles/galaxy/sombrero/static/welcome.html              /mnt/galaxyTools/galaxy-central/static/
cp conerice/masterfiles/galaxy/sombrero/config/universe_wsgi.ini          /mnt/galaxyTools/galaxy-central/
cp conerice/masterfiles/galaxy/sombrero/config/tool_data_table_conf.xml          /mnt/galaxyTools/galaxy-central/
cp conerice/masterfiles/galaxy/sombrero/config/tool_conf.xml              /mnt/galaxyTools/galaxy-central/
cp conerice/masterfiles/galaxy/sombrero/config/datatypes_conf.xml          /mnt/galaxyTools/galaxy-central/
cp conerice/masterfiles/galaxy/sombrero/lib/galaxy/datatypes/registry.py  /mnt/galaxyTools/galaxy-central/lib/galaxy/datatypes/
cp conerice/masterfiles/galaxy/sombrero/tools/data_source/upload.py       /mnt/galaxyTools/galaxy-central/tools/data_source/
  • Create symlinks for our tools and datatypes from the NBIC Galaxy Module Repository
cd /mnt/galaxyTools/galaxytools
svn update
svn export trunk /mnt/galaxyTools/nbic_gmr
cd /mnt/galaxyTools/galaxy-central/tools
ln -s /mnt/galaxyTools/nbic_gmr/ nbic_gmr
cd /mnt/galaxyTools/galaxy-central/static/images
ln -s /mnt/galaxyTools/nbic_gmr/data/image-links/ nbic_gmr
cd /mnt/galaxyTools/galaxy-central/lib/galaxy/datatypes/
for item in $(ls /mnt/galaxyTools/nbic_gmr/data/datatypes/); do ln -s /mnt/galaxyTools/nbic_gmr/data/datatypes/$item; done
  • Create symlinks for custom tool-data files in /mnt/galaxyTools/galaxy-central/tool-data/
cd /mnt/galaxyTools/galaxy-central/tool-data/
for item in $(ls /mnt/galaxyData/tool-data/); do ln -s /mnt/galaxyData/tool-data/$item; done

Restart the service

sudo /bin/su - galaxy -c "export SGE_ROOT=/opt/sge; sh /mnt/galaxyTools/galaxy-central/run.sh --daemon"
  • Check the log file /mnt/galaxyTools/galaxy-central/paster.log to see if Galaxy started (without errors). Sometimes, you are required to upgrade the database schema. If this is the case, Galaxy won't start and you receive a message to run manage_db.sh. Backup the galaxy DB before you run the upgrade script.