Java Bioinformatics Analyses Web Services (JABAWS) manual

JABAWS MANUAL

Quick Start Guide

JABAWS Manual

JABAWS Installation

JABA web services require Java 6 to function. We recommend using Sun Java Virtual Machine. JABAWS can be deployed on any Servlet 2.4 compatible web application server. We have tested JABA on the Apache-Tomcat version 6 web application container, however, version 5.5 should do as well. To deploy JABAWS follow your web application server specific deployment procedures. For Apache-Tomcat just drop the war file into webapps directory. Tomcat will do the rest. At this point you should have JABA web services up and running on windows machines. There are more steps to do if you are deploying JABAWS on other operation systems.

JABA web services depend on the alignment programs to do the actual alignment, so it must have an access to the alignment programs which can be executed on your platform. JABAWS is bundled with native windows executables, native Linux i386 executables, and the source code for all alignment programs JABAWS wraps. If you are installing it on Linux should not worry about installing the alignment programmes, the ones that comes with JABAWS should work. For any other operation systems you need to make the alignment programs available to JABAWS.

If you are installing on Linux it may be just a matter of setting executable flag for the binaries bundled with JABAWS. There is a script which would help you with that. To use it cd to <webapplicationpath>/binaries/src and run sh setexecflag.sh script. Restart the Tomcat. That is it JABAWS should work at this point. If not read on... or have a look at deploying on Tomcat tips. You may want to enable logging, see below how to do that.

Making Alignment Programs Available to JABAWS

There are three possible solutions for this problem

Reuse the binaries that are already have in your system
Use bundled with JABAWS alignment programs
Obtain the alignment programs for your OS elsewhere

They are described in greater details below

If you would like to use the binaries you already have

Than you just need to let JABAWS know there they are. For this edit conf/Executable.properties file. Please provide an absolute path to each executable in your system. For example default path for clustalw executable is defined as local.clustalw.bin=binaries/src/clustalw/src/clustalw2 replace it with absolute path to executable in your system. Instead of changing Executable.properties you could replace executables bundled with JABAWS with the once you have. Then the default configuration will work for you. If you like to find out more about Executable.properties file see below.

If you would like to use bundled with JABAWS alignment programs

Choose JABAWS distribution

There are three main packages you could use

Set executable flag for binaries. For this you could use the script provided. cd to binaries/src
- run setexecflag.sh script.
Make sure binaries supplied work under your OS. For this run each binary. You do not need any input or flags. If you see a error message complaining about missing libraries or other problems with an executable itself rather than with an input, you need to (a) recompile the binary. -
- To recompile all the binaries for your system you could use a script supplied with the JABAWS.
  - For this cd to folder binaries/src execute script compilebin.sh - (either chmod +x it or sh compilebin.sh)
- Makes sure everything compiled and work
  - correctly performing first part of a step 2 again.
- If not everything compiled then, it may be that your system does not have all the tools required for recompilation. At a very least check that you have gcc, g++ and make installed in your system. If not install these packages and repeat step (a).
- If compilation fails with unclear reason, try obtaining the pre compiled binaries for your OS
-
Consider recompiling all binaries for your platform, this is likely to give you a performance increase.
A JABAWS Server Virtual Appliance (or Virtual Machine) - for anyone who wants to run JABAWS locally for themselves
A JABAWS Server Web Application Archive (WAR) package - for anyone who wants to run JABAWS for their group, lab or organization. Wants to use the cluster or perform very large tasks
A client package - for anyone who wants to use JABAWS from their own code, without Jalview, scripting against local or public version of JABAWS Server.

Obtain alignment program for your operation system elsewhere

You could search for pre-packaged compiled executable in your system package repository or try downloading compiled version from alignment program's home page. Then, executables supplied with the system can be replaced with downloaded or new path to the executable can be defined in executable.properties as described above.

Default JABA Web Services Configuration

By default JABAWS configured with local engine enabled and an output directory called "jobsout" within web application itself. This will give you a working copy of JABAWS straight after the deployment. You may be interested to know what is the the war file.

JABAWS Configuration Details

-There are three part of the system you can configure. The local and cluster engines and individual executables. -

Local Engine Configuration

-Local engine configuration is achieved by changing settings in the properties file conf/Engine.local.properties. -Supported configuration settings are: -
-engine.local.enable=true - # enable or disable local engine, valid values true | false -
-local.tmp.directory=D:\\clusterengine\\testoutput - a directory to use for temporary files storage, optional, defaults to java temporary directory -
-engine.local.thread.number=4 - Number of threads for tasks execution (valid values between 1 and 2x cpu. Where x is a number of cores available in the system). Optional defaults to the number of cores for core number <=4 and number of cores-1 for greater core numbers.

If you are planning to use local engine (which you have to if you do not have a cluster) it is a good idea to increase the amount of memory available for the web application server. Apache-Tomcat will be happy to take its memory settings from JAVA_OPTS environmental variable. We would recommend using Sun Java Virtual Machine (JVM) in preference to Open JDK. To specify which JVM to use for Apache-Tomcat just define a JAVA_HOME environmental variable. Below is an example of code which can be added to <tomcat_dir>/bin/setenv.sh script to define which JVM to use and a memory settings for Tomcat server. Tomcat server startup script (catalina.sh) will execute setenv.sh on each server start automatically.
- export JAVA_HOME=/homes/ws-dev2/jdk1.6.0_17/
- export JAVA_OPTS="-server -Xincgc -Xms512m -Xmx1024m"

Cluster Engine Configuration

Supported configuration settings:
-engine.cluster.enable=true - # enable or disable local engine true | false, defaults to false -
-cluster.tmp.directory=/homes/clustengine/testoutput- a directory to use for temporary files storage. The value must be an absolute path to the temporary directory. Required. The value must be different from what is defined for local engine. This directory must be accessible from all cluster nodes.
-For the cluster engine to work SGE_ROOT and LD_LIBRARY_PATH environmental variable have to be defined. They tell the cluster engine where to find DRMAA libraries. These environmental variables should be defined on the web application server e.g.

SGE_ROOT=/gridware/sge
- LD_LIBRARY_PATH=/gridware/sge/lib/lx24-amd64

Finally, do not forget to configure executables for the cluster execution, they may be the same as for the local execution but may be different. Please refer to the executable configuration section for further details.

Executable Configuration

All executables, the binary multiple sequence alignment programs are configured in conf/Executable.properties file. Each executable configured with a number of options they are: - - local.X.bin.windows=<path to executable under windows system, optional> -
local.X.bin=<path to the executable under non-windows system, optional> -
cluster.X.bin=<path to the executable on the cluster, all cluster nodes must see it, optional> -
X.bin.env=<semicolon separated list of environmental variables for executable, use hash symbol as name value separator, optional> -
X.--aamatrix.path=<path to the directory containing substitution matrices, optional> -
X.presets.file=<path to the preset configuration file, optional > -
X.parameters.file=<path to the parameters configuration file, optional> -
X.limits.file=<path to the limits configuration file, optional> -
X.cluster.settings=<list of the cluster specific options, optional>

- Where X is a short executable wrapper class name.

Default JABAWS configuration includes path to local executables to be run by the local engine only, all cluster related settings are commened out, but they are there for you as example. Cluster engine is disabled by default. To configure executable for cluster execution uncomment the X.cluster settings and change them appropriately.
- For example a complete Mafft configuration may look like this: local.mafft.bin.windows= -
- local.mafft.bin=binaries/mafft -
cluster.mafft.bin=/homes/cengine/mafft -
mafft.bin.env=MAFFT_BINARIES#/homes/cengine/mafft;FASTA_4_MAFFT#/bin/fasta34; -
mafft.--aamatrix.path=binaries/matrices -
mafft.presets.file=conf/settings/MafftPresets.xml -
mafft.parameters.file=conf/settings/MafftParameters.xml -
mafft.limits.file=conf/settings/MafftLimits.xml -
mafft.cluster.settings=-q bigmem.q -l h_cpu=24:00:00 -l h_vmem=6000M -l ram=6000M -

Please not that relative paths must only be specified for the files that reside inside web application directory, all other paths must be supplied as absolute!

Unfortunately, it is unsafe to modify environmental variable values, thus you need to specify the absolute path to anything defined as environment variable.

If you are using JABAWS to submit jobs to the cluster (with cluster engine enabled), executables must be available from all cluster nodes the task can be sent to, also paths to the executables on the cluster e.g. cluster.<exec_name>.bin must be absolute.

Executables can be located anywhere in your system, they do not have to reside on the server as long as the web application server can access and execute them.

Cluster settings are treated as a black box, the system will just pass whatever is specified in this line directly to the cluster submission library. This is how DRMAA itself treats this settings. More exactly DRMAA JobTemplate.setNativeSpecification() function will be called.

Defining Environmental Variables for Executable

Environmental variables can be defined in property x.bin.env Where x is one of five executables supported by JABAWS. Several environmental variables can be specified in the same line. For example.
- mafft.bin.env=MAFFT_BINARIES#/homes/cengine/mafft;FASTA_4_MAFFT#/bin/fasta34;

The example above define two environmental variables with names MAFFT-BINARIES and FASTA_4_MAFFT and values /homes/cengine/mafft and /bin/fasta34 respectively. Semicolon is used as a separator between different environment variables whereas hash is used as a separator for name and value of the variable.

Configure JABAWS to Work with Mafft

If you use default configuration you do not need to read any further. The default configuration will work for you without any changes, however, if you want to install Mafft yourself then there is a couple of more steps to do.

Mafft executable needs to know the location of other files supplied with Mafft. In addition some Mafft functions depends on the fasta executable, which is not supplied with Mafft, but is a separate package. Mafft needs to know the location of fasta34 executable.

To let Mafft know where the other files from its package are change the value of MAFFT-BINARIES environmental variable. To let Mafft know where is the fasta34 executable set the value of FASTA_4_MAFFT environment variable to point to a location of fasta34 program. The latter can be added to the PATH variable instead. If you are using executables supplied with JABAWS, the path to Mafft binaries would be like <relative path to web application directory>/binaries/src/mafft/binaries and the path to fasta34 binary would be <relative path to web application directory>/binaries/src/fasta34/fasta34. You can specify the location of Mafft binaries as well as fasta34 program elsewhere by providing an absolute path to them. All these settings are defined in conf/Executable.properties file.

Testing JABA Web Services

You can use a command line client (part of the client only package) to test you JABAWS installation as described here. If you downloaded a JABAWS server package, you can use <your_jaba_context_name>/WEB-INF/lib/jaba-client.jar to test JABAWS installation as described in how-to. If you downloaded the source code, then you could run a number of test suits defined in the build.xml ant build file.

- -

JABAWS Log Files

JABAWS can be configured to log what it is doing. This comes handy if you would like to see who is using your web services or need to chase some problems. JABAWS uses log4j to do the logging, the example of log4j configuration is bundled with JABAWS war file. You will find it in the /WEB-INF/classes/log4j.properties file. All the lines in this file are commented out. The reason why the logging is disabled by default it simple, log4j have to know the exact location where the log files should be stored. This is not known up until the deployment time. To enable the logging you need to define logDir property in the log4j.properties and uncomment section of the file which corresponds to your need. More information is given in the log4j.properties file itself. Restart the tomcat or the JABAWS web application to apply the settings.

After you have done this, assuming that you did not change the log4j.properties file yourself, you should see the application log file called activity.log. The amount of information logged can be adjusted using different logging levels, it is reduced in the following order of log levels TRACE, DEBUG, INFO, WARN, ERROR, FATAL.

If you would like to know who is using your services, you might want to enable tomcat access logging.

JABAWS War File Content

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Directory	Content description
conf/	contains configuration files such as Executable.properties, Engine.local.properties, Engine.cluster.properties
conf/settings	Contains individual executable description files. In particular XXXParameters.xml, XXXPresets.xml, XXXLimits.xml where XXX is the name of the executable
jobsout/	Individual executables input, output and some other task related files are stored. (optional)
binaries/	Directory contains native executables - alignment programs, windows binaries (optional)
binaries/src	Contains source of native executables and Linux i386 binaries.
binaries/matrices	Substitution matrices
WEB-INF	Web application descriptor
WEB-INF/lib	Web application libraries
WEB-INF/classes	log4j.properties - log configuration file (optional)
Help Pages
/	help pages, index.html is the starting page
dm_javadoc	javadoc for JABAWS client (the link is available from How To pages)
prog_docs	documentation for programmes that JABAWS uses
images	images referenced by html pages

Virtual Appliance (VA) package (1) contains TurnKey Linux with JABAWS installed. JABAWS VA contains JABAWS WAR deployed on the Apache Tomcat 6 web application server. If you use MS Windows read no further - the VA is the way to go. You would need to install VMWare Player or Oracle VirtualBox (both are free) on your computer to use this package. More details about JABAWS virtual appliance is available from the Server VA section of the manual.

Option 2, Web Application Archive package contains JABAWS ready to be deployed on the Servlet 2.4 compatible web application server such as Apache Tomcat version 6. To make this version work you would need

Install Apache-Tomcat or similar web application server.
If you are not on Linux, you would have to make binaries for your system available to JABAWS.