1 <?xml version="1.0" encoding="UTF-8"?>
\r
2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
\r
3 <html xmlns="http://www.w3.org/1999/xhtml">
\r
6 <meta name="Last-modified" content="Fri, 28 Jun 2013 12:00:00 GMT"/>
\r
7 <title>Java Bioinformatics Analyses Web Services (JABAWS): Manual</title>
\r
8 <link href="ws.css" rel="stylesheet" type="text/css" media="screen, projection, handheld, tv" />
\r
9 <link href="print.css" rel="stylesheet" type="text/css" media="print" />
\r
10 <link href='http://fonts.googleapis.com/css?family=Yanone+Kaffeesatz:700' rel='stylesheet' type='text/css' />
\r
11 <script type="text/javascript" src="prototype-1.6.0.3.js"></script>
\r
20 <td style="width:158px;"><a href="http://www.dundee.ac.uk"><img src="images/uod_lt_long.gif" width="158" height="90" alt="University of Dundee" class="logo" title="University of Dundee" longdesc="http://www.dundee.ac.uk"/></a></td>
\r
21 <td class="bg"><img src="images/jabaws21.png" width="256" height="67" alt="JABAWS-2.1" title="Java Bioinformatics Analysis Web Services version 2.1"/></td>
\r
22 <td class="bg"><img src="images/banner_right.png" alt="Disorder" width="200" height="80"/></td>
\r
31 <div id="supermenu">
\r
32 <a class="newa" href="index.html">Home</a>
\r
33 <a class="newa" href="quick_start.html">Getting Started</a>
\r
34 <a class="newpressed" href="man_about.html">Manual</a>
\r
37 <a class="newa" href="man_about.html">About</a>
\r
38 <a class="newa" href="man_servervm.html" title="JABAWS Server as Virtual Appliance">Server VA</a>
\r
39 <a class="newa" href="man_awscloud.html" title="JABAWS Server in the Amazon EC2 Cloud">Server in the Cloud</a>
\r
40 <a class="newa" href="man_serverwar.html" title="JABAWS Server as Web Application aRchive">Server WAR</a>
\r
41 <a class="newpressed" href="man_configuration.html" >Configure JABAWS</a>
\r
42 <a class="newa" href="man_client.html" title="JABAWS Command Line Client">Command Client</a>
\r
43 <a class="newa" href="man_stats.html" title="JABAWS Usage Statistics">Usage Statistics</a>
\r
44 <a class="newa" href="man_dev.html" title="Accessing JABAWS from your program">Accessing JABAWS</a>
\r
45 <a class="newa" href="man_server_dev.html" >Develop JABAWS</a>
\r
52 <h2 id="headtitle">JABAWS MANUAL</h2>
\r
54 <h2>JABAWS Server Configuration </h2>
\r
56 <li><a href="#defjabaconf">JABAWS Configuration </a></li>
\r
57 <li><a href="#locEngConf">Local Engine Configuration</a></li>
\r
58 <li><a href="#clustEngConf">Cluster Engine Configuration</a></li>
\r
59 <li><a href="#exec">Executable Configuration</a></li>
\r
60 <li><a href="#setexecenv">Defining Environment Variables for Executables</a></li>
\r
61 <li><a href="#mafftconf">Configure JABAWS to Work with Mafft</a></li>
\r
62 <li><a href="#settinglimit">Limiting the size of the job accepted by JABAWS Server </a></li>
\r
63 <li><a href="#diffbin">Using a different version of the alignment program with JABAWS</a></li>
\r
64 <li><a href="#mixuse">Load balancing </a></li>
\r
65 <li><a href="#testingJaba">Testing JABA Web Services</a></li>
\r
66 <li><a href="#logs">JABAWS requests logging </a></li>
\r
67 <li><a href="#logfiles">JABAWS internal logging </a></li>
\r
68 <li><a href="#warfile">JABAWS War File Content</a></li>
\r
69 <li><a href="#ga">JABAWS and Google Analytics</a></li>
\r
71 <h3><a name="defjabaconf" id="defjabaconf"></a>JABAWS Configuration </h3>
\r
72 <p>There are three parts of the system you can configure. The local
\r
73 and the cluster engines, and the paths to the individual executables for
\r
74 each engine. These settings are stored in configuration files
\r
75 within the web application directory (for an overview, then take a
\r
76 look at the <a href="#warfile">war file content table</a>). </p>
\r
77 <p>Initially, JABAWS is configured with only the local engine
\r
78 enabled, with job output written to directory called "jobsout"
\r
79 within the web application itself. This means that JABAWS will work
\r
80 out of the box, but may not be suitable for serving a whole lab or
\r
82 <h3><a name="locEngConf" id="locEngConf"></a>Local Engine Configuration</h3>
\r
84 <p>The Local execution engine configuration is defined in the
\r
85 properties file <span class="hightlight">conf/Engine.local.properties. </span>The supported
\r
86 configuration settings are:<br />
\r
87 <span class="hightlight">engine.local.enable=true</span> - #
\r
88 enable or disable local engine, valid values true | false<br />
\r
90 "hightlight">local.tmp.directory=D:\\clusterengine\\testoutput</span>
\r
91 - a directory to use for temporary files storage, optional,
\r
92 defaults to java temporary directory<br />
\r
93 <span class="hightlight">engine.local.thread.number=4</span> -
\r
94 Number of threads for tasks execution (valid values between 1 and
\r
95 2x cpu. Where x is a number of cores available in the system).
\r
96 Optional defaults to the number of cores for core number <=4 and
\r
97 number of cores-1 for greater core numbers.</p>
\r
99 <p>If the local engine going to be heavily loaded (which is often the case if you do not have a cluster) it is a good idea to increase
\r
100 the amount of memory available for the web application server. If
\r
101 you are using Apache-Tomcat, then you can define its memory
\r
102 settings in the JAVA_OPTS environment variable. To specify which
\r
103 JVM to use for Apache-Tomcat, put the full path to the JRE
\r
104 installation in the JAVA_HOME environment variable. (We would
\r
105 recommend using Sun Java Virtual Machine (JVM) in preference to
\r
106 Open JDK). Below is an example of code which can be added to <span
\r
107 class="hightlight"><tomcat_dir>/bin/setenv.sh</span> script
\r
108 to define which JVM to use and a memory settings for Tomcat server.
\r
109 Tomcat server startup script (<span class=
\r
110 "hightlight">catalina.sh</span>) will execute <span class=
\r
111 "hightlight">setenv.sh</span> on each server start
\r
112 automatically.<br />
\r
113 <span class="code">export
\r
114 JAVA_HOME=/homes/ws-dev2/jdk1.6.0_17/<br />
\r
115 export JAVA_OPTS="-server -Xincgc -Xms512m -Xmx1024m"</span></p>
\r
117 <h3><a name="clustEngConf" id="clustEngConf"></a>Cluster Engine Configuration</h3>
\r
119 <p>Supported configuration settings:<br />
\r
120 <span class="hightlight">engine.cluster.enable=true</span> - #
\r
121 enable or disable local engine true | false, defaults to
\r
124 "hightlight">cluster.tmp.directory=/homes/clustengine/testoutput-</span>
\r
125 a directory to use for temporary files storage. The value must be
\r
126 an absolute path to the temporary directory. This is required. The value
\r
127 must be different from what is defined for local engine. This
\r
128 directory must be accessible from all cluster nodes.<br />
\r
129 For the cluster engine to work, the SGE_ROOT and LD_LIBRARY_PATH
\r
130 environment variables have to be defined. They tell the cluster
\r
131 engine where to find DRMAA libraries. These variables
\r
132 should be defined when the web application server starts up, e.g.</p>
\r
134 <p class="code">SGE_ROOT=/gridware/sge<br />
\r
135 LD_LIBRARY_PATH=/gridware/sge/lib/lx24-amd64</p>
\r
137 <p>Finally, do not forget to configure executables for the cluster
\r
138 execution, they may be the same as for the local execution but may
\r
139 be different. Please refer to the executable configuration section
\r
140 for further details.</p>
\r
142 <h3><a name="exec" id="exec"></a>Executable Configuration</h3>
\r
144 <p>All the executable programs
\r
145 are configured in <span class="hightlight">conf/Executable.properties</span> file. Each executable
\r
146 is configured with a number of options. They are: <span class=
\r
147 "code">local.X.bin.windows=<path to executable under windows
\r
148 system, optional><br />
\r
149 local.X.bin=<path to the executable under non-windows system,
\r
151 cluster.X.bin=<path to the executable on the cluster, all
\r
152 cluster nodes must see it, optional><br />
\r
153 X.bin.env=<semicolon separated list of environment variables
\r
154 for executable, use hash symbol as name value separator,
\r
156 X.--aamatrix.path=<path to the directory containing
\r
157 substitution matrices, optional><br />
\r
158 X.presets.file=<path to the preset configuration file, optional
\r
160 X.parameters.file=<path to the parameters configuration file,
\r
162 X.limits.file=<path to the limits configuration file,
\r
164 X.cluster.settings=<list of the cluster specific options,
\r
165 optional></span></p>
\r
167 <p>Where X is either clustal, muscle, mafft, probcons or tcoffee. </p>
\r
169 <p>Default JABAWS configuration includes path to local executables
\r
170 to be run by the local engine only, all cluster related settings
\r
171 are commented out, but they are there for you as examples. Cluster
\r
172 engine is disabled by default. To configure executable for cluster
\r
173 execution uncomment the X.cluster settings and change them
\r
174 appropriately. </p>
\r
176 By default limits are set well in excess of what you may want to offer to the users outside your lab,
\r
177 to make sure that the tasks are never rejected. The default limit is 100000 sequences of 100000
\r
178 letters on average for all of the JABA web services. You can adjust the limits according to your
\r
179 needs by editing <span class="hightlight">conf/settings/<X>Limit.xml</span> files.<br />
\r
180 After you have completed the editing your configuration may look like
\r
181 this:<span class="code">local.mafft.bin.windows=<br />
\r
182 local.mafft.bin=binaries/mafft<br />
\r
183 cluster.mafft.bin=/homes/cengine/mafft<br />
\r
184 mafft.bin.env=MAFFT_BINARIES#/homes/cengine/mafft;FASTA_4_MAFFT#/bin/fasta34;<br />
\r
185 mafft.--aamatrix.path=binaries/matrices<br />
\r
186 mafft.presets.file=conf/settings/MafftPresets.xml<br />
\r
187 mafft.parameters.file=conf/settings/MafftParameters.xml<br />
\r
188 mafft.limits.file=conf/settings/MafftLimits.xml<br />
\r
189 mafft.cluster.settings=-q bigmem.q -l h_cpu=24:00:00 -l
\r
190 h_vmem=6000M -l ram=6000M</span></p>
\r
191 <p>Please not that relative paths must only be specified for the
\r
192 files that reside inside web application directory, all other paths
\r
193 must be supplied as absolute!</p>
\r
196 Furthermore, you should avoid using environment variables within the paths or options - since
\r
197 these will not be evaluated correctly. Instead, please explicitly specify the absolute path
\r
198 to anything normally evaluated from an environment variable at execution time.
\r
201 <p>If you are using JABAWS to submit jobs to the cluster (with
\r
202 cluster engine enabled), executables must be available from all
\r
203 cluster nodes the task can be sent to, also paths to the
\r
204 executables on the cluster e.g. <span class=
\r
205 "hightlight">cluster.<exec_name>.bin</span> must be
\r
208 <p>Executables can be located anywhere in your system, they do not
\r
209 have to reside on the server as long as the web application server
\r
210 can access and execute them.</p>
\r
212 <p>Cluster settings are treated as a black box, the system will
\r
213 just pass whatever is specified in this line directly to the
\r
214 cluster submission library. This is how DRMAA itself treats this
\r
215 settings. More exactly DRMAA <span class="hightlight">JobTemplate.setNativeSpecification()</span> function will be called.</p>
\r
216 <p>For further details and examples of configuration please refer to the <span class="hightlight">Executable.properties</span> file supplied with JABAWS.</p>
\r
217 <h3><a name="setexecenv" />Defining Environment Variables for
\r
220 <p>Environment variables can be defined in property <span class=
\r
221 "code">x.bin.env</span> Where <span class="hightlight">x</span> is
\r
222 one of five executables supported by JABAWS. Several environment
\r
223 variables can be specified in the same line. For example.<br />
\r
225 "code">mafft.bin.env=MAFFT_BINARIES#/homes/cengine/mafft;FASTA_4_MAFFT#/bin/fasta34;</span></p>
\r
227 <p>The example above defines two environment variables with names
\r
228 MAFFT-BINARIES and FASTA_4_MAFFT and values /homes/cengine/mafft
\r
229 and /bin/fasta34 respectively. Semicolon is used as a separator
\r
230 between different environment variables whereas hash is used as a
\r
231 separator for name and value of the variable.</p>
\r
233 <h3><a name="mafftconf" id="mafftconf"></a>Configure JABAWS to Work
\r
236 <p>If you use default configuration you do not need to read any
\r
237 further. The default configuration will work for you without any
\r
238 changes, however, if you want to install Mafft yourself then there
\r
239 is a couple of more steps to do.</p>
\r
241 <p>Mafft executable needs to know the location of other files
\r
242 supplied with Mafft. In addition some Mafft functions depends on
\r
243 the fasta executable, which is not supplied with Mafft, but is a
\r
244 separate package. Mafft needs to know the location of fasta34
\r
247 <p>To let Mafft know where the other files from its package are,
\r
248 change the value of MAFFT-BINARIES environment variables. To let
\r
249 Mafft know where is the fasta34 executable set the value of
\r
250 FASTA_4_MAFFT environment variable to point to a location of
\r
251 fasta34 program. The latter can be added to the PATH variable
\r
252 instead. If you are using executables supplied with JABAWS, the
\r
253 path to Mafft binaries would be like <span class=
\r
254 "hightlight"><relative path to web application
\r
255 directory>/binaries/src/mafft/binaries</span> and the path to
\r
256 fasta34 binary would be <span class="hightlight"><relative path
\r
258 directory>/binaries/src/fasta34/fasta34</span>. You can specify
\r
259 the location of Mafft binaries as well as fasta34 program elsewhere
\r
260 by providing an absolute path to them. All these settings are
\r
261 defined in <span class=
\r
262 "hightlight">conf/Executable.properties</span> file.</p>
\r
263 <h3><a name="settinglimit" id="settinglimit"></a>Limiting the size of the job accepted by JABAWS </h3>
\r
265 JABAWS can be configured to reject excessively large tasks. This is useful if you operate JABAWS service
\r
266 for many users. By defining a maximum allowed task size you can provide an even service for all users and
\r
267 prevents waste of resources on the tasks too large to complete successfully. You can define the maximum
\r
268 number of sequences and the maximum average sequence length that JABAWS accepts for each JABA Web Service
\r
269 independently. Furthermore, you can define different limits for different presets of the same web service.
\r
271 By default limits are disabled. You can enable them by editing <span class="hightlight">conf/Executable.properties</span>
\r
272 file. You can adjust the limits according to your needs by editing <span class="hightlight">conf/settings/<X>Limit.xml</span>
\r
275 <h3><a name="diffbin" id="diffbin"></a>Using a different version of the alignment program with JABAWS</h3>
\r
277 JABAWS is supplied with binaries and source code of the executables related to the version it supports. So normally you
\r
278 would not need to install your own executables. However, if you have a different version of an executable
\r
279 (e.g. an alignment program) which you prefer, you could use it as long as it supports all the functions
\r
280 JABAWS executable require. This could be the case with more recent executable. If the options supported
\r
281 by your chosen executable is different from the standard JABAWS executable, then you need to edit
\r
282 <em>ExecutableName</em>Paramaters.xml configuration file.
\r
285 <h3><a name="mixuse" id="mixuse"></a>Load balancing </h3>
\r
287 If your cluster is busy and has significant waiting times, you can achieve a faster response by allowing
\r
288 the server machine to calculate small tasks and then reserve the cluster for bigger jobs. This works
\r
289 especially well if your server is a powerful machine with many CPUs. To do this you need to enable and
\r
290 configure both the cluster and the local engines. Once this is done decide on the maximum size of a task
\r
291 to be run on the server locally. Then, edit
\r
292 <span class="hightlight">"# LocalEngineExecutionLimit #" </span>preset
\r
293 in <span class="hightlight"> <ServiceName>Limits.xml</span> file accordingly. JABAWS server then
\r
294 will balance the load according to the following rule: If the task size is smaller than the maximum task
\r
295 size for local engine, and the local engine has idle threads, then it calculates task locally otherwise it submit
\r
296 the task to the cluster.
\r
299 <h3><a name="testingJaba" id="testingJaba"></a>Testing JABA Web Services</h3>
\r
301 Access <span class="code"><your_JABAWS_server_URL>/ServiceStatus</span> to test all web services.
\r
302 Each time you access this URL, all services are tested.
\r
303 <!-- You can test a particular web service by adding
\r
304 its name at the end of this URL. For example <span class="code">http://localhost:8080/jabaws/ServiceStatus/MuscleWS </span>
\r
305 will test MuscleWS webservice only. -->
\r
306 For production configuration we recommend prohibiting requests to this URL for non
\r
307 authenticated users to prevent excessive load on the server.
\r
309 <p>Alternatively, you can use a command line client (part of the client only
\r
310 package) to test your JABAWS installation as described <a href="man_client.html">here</a>. If you downloaded a JABAWS
\r
311 server package, you can use <span class=
\r
312 "hightlight"><your_jaba_context_name>/WEB-INF/lib/jaba-client.jar</span> to test JABAWS installation as described <a href=
\r
313 "man_serverwar.html#usingWsTester">here</a>. If you downloaded the source
\r
314 code, then you could run a number of test suites defined in the
\r
315 build.xml Apache Ant file.</p>
\r
316 <h3><a name="logs" id="logs"></a>JABAWS requests logging </h3>
\r
317 <p>Enable Tomcat log valve. To do this uncomment the following section of <span class="hightlight"><tomcat_root>/conf/server.xml</span> configuration file. </p>
\r
318 <p class="code"> <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" <br />
\r
319 prefix="localhost_access_log." suffix=".txt" pattern="common" resolveHosts="false"/></p>
\r
320 <p> The following information will be logged:</p>
\r
321 <table width="100%" border="0" style="margin:0">
\r
325 <th>Method server_URL protocol </th>
\r
326 <th>HTTP status </th>
\r
327 <th>Response size in bytes </th>
\r
330 <td>10.31.11.159</td>
\r
331 <td>[10/Feb/2010:16:51:32 +0000]</td>
\r
332 <td>"POST /jws2/MafftWS HTTP/1.1"</td>
\r
337 <p>Which can be processed in various programs for log analysis , such as <a href="http://www.webalizer.org/">WebAlizer</a>, <a href="http://www.analog.cx/">Analog</a>, <a href="http://awstats.sourceforge.net/">AWStats</a>. </p>
\r
338 <h3><a name="logfiles" id="logfiles"></a>JABAWS internal logging </h3>
\r
339 <p>JABAWS can be configured to log what it is doing. This comes in
\r
340 handy if you would like to see who is using your web services or
\r
341 need to chase some problems. JABAWS uses <a href=
\r
342 "http://logging.apache.org/log4j/1.2/">log4j</a> to do the logging,
\r
343 the example of log4j configuration is bundled with JABAWS war file.
\r
344 You will find it in the <span class=
\r
345 "hightlight">/WEB-INF/classes/log4j.properties</span> file. All the
\r
346 lines in this file are commented out. The reason why the logging is
\r
347 disabled by default it simple, log4j has to know the exact
\r
348 location of where the log files are stored. This is not known up
\r
349 until the deployment time. To enable the logging you need to
\r
350 define<span class="hightlight"> logDir</span> property in the <span
\r
351 class="hightlight">log4j.properties</span> and uncomment section of
\r
352 the file which corresponds to your need. More information is given
\r
353 in the <span class="hightlight">log4j.properties</span> file
\r
354 itself. Restart the Tomcat or the JABAWS web application to apply
\r
356 <p>After you have done this, assuming that you did not change the
\r
357 log4j.properties file yourself, you should see the application log
\r
358 file called <span class="hightlight">activity.log</span>. The
\r
359 amount of information logged can be adjusted using different
\r
360 logging levels, it is reduced in the following order of log levels
\r
361 TRACE, DEBUG, INFO, WARN, ERROR, FATAL.</p>
\r
362 <p>If you would like to know who is using your services, you might
\r
363 want to <a href="#logs">enable Tomcat request
\r
365 <h3><a name="warfile" id="warfile"></a>JABAWS War File Content</h3>
\r
366 <table width="100%">
\r
368 <th style="width:19%">Directory</th>
\r
369 <th style="width:81%">Content description</th>
\r
373 <td>contains configuration files such as Executable.properties,
\r
374 Engine.local.properties, Engine.cluster.properties</td>
\r
377 <td>conf/settings</td>
\r
378 <td>Contains individual executable description files. In particular
\r
379 XXXParameters.xml, XXXPresets.xml, XXXLimits.xml where XXX is the
\r
380 name of the executable</td>
\r
383 <td>ExecutionStatistics</td>
\r
384 <td>The database for storing the execution statistics </td>
\r
388 <td>Web pages for usage statistics visialization and webservices status queries </td>
\r
392 <td>Contains directories generated when running an individual executable. E.g. input and output files and some other task
\r
393 related data (optional)</td>
\r
397 <td>Directory contains native executables - programs,
\r
398 windows binaries (optional)</td>
\r
401 <td>binaries/src</td>
\r
402 <td>Contains source of native executables and Linux i386
\r
406 <td>binaries/windows</td>
\r
407 <td>Contains binaries for MS Windows operating system </td>
\r
410 <td>binaries/matrices</td>
\r
411 <td>Substitution matrices
\r
412 <!-- what format ? --></td>
\r
416 <td>Web application descriptor</td>
\r
419 <td>WEB-INF/lib</td>
\r
420 <td>Web application libraries</td>
\r
423 <td>WEB-INF/classes</td>
\r
424 <td>log4j.properties - log configuration file (optional)</td>
\r
427 <td colspan="2"><strong>Help Pages</strong> </td>
\r
431 <td>help pages, index.html is the starting page</td>
\r
434 <td>dm_javadoc</td>
\r
435 <td>javadoc for JABAWS client (the link is available from How To
\r
440 <td>documentation for programs that JABAWS uses</td>
\r
444 <td>images referenced by html pages</td>
\r
447 <h3><a name="ga" id="ga"></a>JABAWS and Google Analytics </h3>
\r
448 <p>JABAWS reports web services usage to our group Google Analytics (GA) account. JABAWS usage statistics are collected for funding and reporting purposes, and no private information is collected. The data sent by JABAWS is as follows: </p>
\r
450 <li>The IP address of the JABAWS server machine (the server IP can anonymized see <span class="hightlight">conf/GA.properties</span> config file)</li>
\r
451 <li>The name of the web service that was called. </li>
\r
452 <li>A few details of the system such as JABAWS version, java version, user language, color depth, screen
\r
453 resolution and character encoding.</li>
\r
455 <p> Google Analytics can be disabled or adjusted by removing/editing <span class="hightlight">conf/GA.properties</span> Google Analytics (GA) settings file. We would appreciate it greatly if you could leave it on!</p>
\r
456 <p>All calls to GA are very lightweight, completed asynchronously, create very little overhead and do not influence the server response time or performance.</p>
\r
457 </div><!-- content end-->
\r
458 <div id="copyright">Last update: 14 October 2013<br/>Sasha Sherstnev, Peter Troshin, Jim Procter and Geoff Barton, The Barton Group, University of Dundee, UK</div>
\r
459 </div><!-- wrapper end-->
\r
460 </div><!-- page end-->
\r
463 <!-- Google analitics -->
\r
464 <script type="text/javascript">
\r
465 var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
\r
466 document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
\r
468 <script type="text/javascript">
\r
470 var pageTracker = _gat._getTracker("UA-5356328-1");
\r
471 pageTracker._trackPageview();
\r
git://source.jalview.org / jabaws.git / blob