1 <%--<?xml version="1.0" encoding="ISO-8859-1" ?>--%>
2 <%--<%@ page language="java" contentType="text/html; charset=ISO-8859-1" pageEncoding="ISO-8859-1"%>--%>
4 <%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
5 <%@ taglib uri="http://java.sun.com/jsp/jstl/functions" prefix="fn" %>
6 <%@ taglib uri="http://java.sun.com/jsp/jstl/fmt" prefix="fmt" %>
7 <%@ taglib uri="http://displaytag.sf.net" prefix="dt" %>
10 <c:import url="template_header.jsp" >
11 <c:param name="title">Documentation</c:param>
14 <ol class="breadcrumb">
15 <li><a href="${pageContext.request.contextPath}/index.jsp">Home</a></li>
16 <li><a href="man_docs.jsp">Documentation</a></li>
17 <li class="active"><a href="man_server_dev.jsp">Configuring JABAWS</a></li>
20 <div class="col-md-12">
21 <div class="panel panel-default">
22 <div class="panel panel-heading">
23 <h1 class="panel-title">JABAWS Server Configuration</h1>
25 <div class="panel-body">
26 <!--<h4>JABAWS Server Virtual Appliance</h4>-->
28 <li><a href="#defjabaconf">JABAWS Configuration </a></li>
29 <li><a href="#locEngConf">Local Engine Configuration</a></li>
30 <li><a href="#clustEngConf">Cluster Engine Configuration</a></li>
31 <li><a href="#exec">Executable Configuration</a></li>
32 <li><a href="#setexecenv">Defining Environment Variables for Executables</a></li>
33 <li><a href="#mafftconf">Configure JABAWS to Work with Mafft</a></li>
34 <li><a href="#settinglimit">Limiting the size of the job accepted by JABAWS Server </a></li>
35 <li><a href="#diffbin">Using a different version of the alignment program with JABAWS</a></li>
36 <li><a href="#mixuse">Load balancing </a></li>
37 <li><a href="#testingJaba">Testing JABAWS Web-Services</a></li>
38 <li><a href="#logs">JABAWS requests logging </a></li>
39 <li><a href="#logfiles">JABAWS internal logging </a></li>
40 <li><a href="#warfile">JABAWS War File Content</a></li>
41 <li><a href="#ga">JABAWS and Google Analytics</a></li>
43 <!--<p class="justify">-->
49 <div class="row" id="defjabaconf">
50 <div class="col-md-12">
51 <div class="panel panel-default">
52 <div class="panel panel-heading">
53 <h1 class="panel-title">JABAWS Configuration</h1>
55 <div class="panel-body">
57 There are three parts of the system you can configure. The local
58 and the cluster engines, and the paths to the individual executables for
59 each engine. These settings are stored in configuration files
60 within the web application directory (for an overview, then take a
61 look at the <a href="#warfile">war file content table</a>).
64 Initially, JABAWS is configured with only the local engine
65 enabled, with job output written to directory called "jobsout"
66 within the web application itself. This means that JABAWS will work
67 out of the box, but may not be suitable for serving a whole lab or
70 <p class="text-right">
71 <a href="#">Back to top <i class="fa fa-arrow-up" aria-hidden="true"></i></a>
77 <div class="row" id="locEngConf">
78 <div class="col-md-12">
79 <div class="panel panel-default">
80 <div class="panel panel-heading">
81 <h1 class="panel-title">Local Engine Configuration</h1>
83 <div class="panel-body">
85 The Local execution engine configuration is defined in the
86 properties file <em2>conf/Engine.local.properties.</em2> The supported
87 configuration settings are:</p>
89 <em2>engine.local.enable=true</em2> - #
90 enable or disable local engine, valid values true | false</p>
92 <em2>local.tmp.directory=D:\\clusterengine\\testoutput</em2>
93 - a directory to use for temporary files storage, optional,
94 defaults to java temporary directory</p>
96 <em2>engine.local.thread.number=4</em2> -
97 Number of threads for tasks execution (valid values between 1 and
98 2x cpu. Where x is a number of cores available in the system).
99 Optional defaults to the number of cores for core number <=4 and
100 number of cores-1 for greater core numbers.
104 If the local engine going to be heavily loaded (which is often the case if
105 you do not have a cluster) it is a good idea to increase
106 the amount of memory available for the web application server. If
107 you are using Apache-Tomcat, then you can define its memory
108 settings in the JAVA_OPTS environment variable. To specify which
109 JVM to use for Apache-Tomcat, put the full path to the JRE
110 installation in the JAVA_HOME environment variable. (We would
111 recommend using Sun Java Virtual Machine (JVM) in preference to
112 Open JDK). Below is an example of code which can be added to
113 <em2><tomcat_dir>/bin/setenv.sh</em2> script
114 to define which JVM to use and a memory settings for Tomcat server.
115 Tomcat server startup script (<em2>catalina.sh</em2>)
116 will execute <em2>setenv.sh</em2> on each server start automatically.
118 <pre><code class="bash">export JAVA_HOME=/homes/ws-dev2/jdk1.6.0_17/
119 export JAVA_OPTS="-server -Xincgc -Xms512m -Xmx1024m"</code></pre>
120 <p class="text-right">
121 <a href="#">Back to top <i class="fa fa-arrow-up" aria-hidden="true"></i></a>
127 <div class="row" id="clustEngConf">
128 <div class="col-md-12">
129 <div class="panel panel-default">
130 <div class="panel panel-heading">
131 <h1 class="panel-title">Cluster Engine Configuration</h1>
133 <div class="panel-body">
134 <p>Supported configuration settings:
135 <em2>engine.cluster.enable=true</em2> - #
136 enable or disable local engine true | false, defaults to
138 <p><em2>cluster.tmp.directory=/homes/clustengine/testoutput-</em2>
139 a directory to use for temporary files storage. The value must be
140 an absolute path to the temporary directory. This is required. The value
141 must be different from what is defined for local engine. This
142 directory must be accessible from all cluster nodes.</p>
144 For the cluster engine to work, the SGE_ROOT and LD_LIBRARY_PATH
145 environment variables have to be defined. They tell the cluster
146 engine where to find DRMAA libraries. These variables
147 should be defined when the web application server starts up, e.g.
150 <pre><code class="bash">SGE_ROOT=/gridware/sge
151 LD_LIBRARY_PATH=/gridware/sge/lib/lx24-amd64</code></pre>
154 Finally, do not forget to configure executables for the cluster
155 execution, they may be the same as for the local execution but may
156 be different. Please refer to the executable configuration section
161 <p class="text-right">
162 <a href="#">Back to top <i class="fa fa-arrow-up" aria-hidden="true"></i></a>
168 <div class="row" id="exec">
169 <div class="col-md-12">
170 <div class="panel panel-default">
171 <div class="panel panel-heading">
172 <h1 class="panel-title">Executable Configuration</h1>
174 <div class="panel-body">
176 All the executable programs
177 are configured in <em2>conf/Executable.properties</em2> file. Each executable
178 is configured with a number of options. They are:
180 <pre><code class="bash">local.X.bin.windows=<path to executable under windows system, optional>
181 local.X.bin=<path to the executable under non-windows system, optional>
182 cluster.X.bin=<path to the executable on the cluster, all cluster nodes must see it, optional>
183 X.bin.env=<semicolon separated list of environment variables for executable, use hash symbol as name value separator, optional>
184 X.--aamatrix.path=<path to the directory containing substitution matrices, optional>
185 X.presets.file=<path to the preset configuration file, optional>
186 X.parameters.file=<path to the parameters configuration file, optional>
187 X.limits.file=<path to the limits configuration file, optional>
188 X.cluster.settings=<list of the cluster specific options, optional></code></pre>
190 <p>Where X any of the bioinformatics tools available (e.g. clustalw, muscle, mafft, probcons, t-coffee, etc.).</p>
193 Default JABAWS configuration includes path to local executables
194 to be run by the local engine only, all cluster related settings
195 are commented out, but they are there for you as examples. Cluster
196 engine is disabled by default. To configure executable for cluster
197 execution uncomment the X.cluster settings and change them
201 By default limits are set well in excess of what you may want to offer to the users outside your lab,
202 to make sure that the tasks are never rejected. The default limit is 100000 sequences of 100000
203 letters on average for all of the JABA web services. You can adjust the limits according to your
204 needs by editing <em2>conf/settings/<X>Limit.xml</em2> files.<br />
205 After you have completed the editing your configuration may look like
208 <pre><code class="bash">local.mafft.bin=binaries/mafft
209 cluster.mafft.bin=/homes/cengine/mafft
210 mafft.bin.env=MAFFT_BINARIES#/homes/cengine/mafft;FASTA_4_MAFFT#/bin/fasta34;
211 mafft.--aamatrix.path=binaries/matrices
212 mafft.presets.file=conf/settings/MafftPresets.xml
213 mafft.parameters.file=conf/settings/MafftParameters.xml
214 mafft.limits.file=conf/settings/MafftLimits.xml
215 mafft.cluster.settings=-q bigmem.q -l h_cpu=24:00:00 -l h_vmem=6000M -l ram=6000M</code></pre>
218 Please not that relative paths must only be specified for the
219 files that reside inside web application directory, all other paths
220 must be supplied as absolute!</p>
223 Furthermore, you should avoid using environment variables within the paths or options - since
224 these will not be evaluated correctly. Instead, please explicitly specify the absolute path
225 to anything normally evaluated from an environment variable at execution time.
229 If you are using JABAWS to submit jobs to the cluster (with
230 cluster engine enabled), executables must be available from all
231 cluster nodes the task can be sent to, also paths to the
232 executables on the cluster e.g. <em2>cluster.<exec_name>.bin</em2> must be
236 <p class="justify">Executables can be located anywhere in your system, they do not
237 have to reside on the server as long as the web application server
238 can access and execute them.
242 Cluster settings are treated as a black box, the system will
243 just pass whatever is specified in this line directly to the
244 cluster submission library. This is how DRMAA itself treats this
245 settings. More exactly DRMAA <em2>JobTemplate.setNativeSpecification()</em2>
246 function will be called.
249 For further details and examples of configuration please refer to the
250 <em2>Executable.properties</em2> file supplied with JABAWS.
253 <p class="text-right">
254 <a href="#">Back to top <i class="fa fa-arrow-up" aria-hidden="true"></i></a>
260 <div class="row" id="setexecenv">
261 <div class="col-md-12">
262 <div class="panel panel-default">
263 <div class="panel panel-heading">
264 <h1 class="panel-title">Defining Environment Variables for
267 <div class="panel-body">
268 <p>Environment variables can be defined in property</p>
269 <pre><code class="bash">x.bin.env</code></pre>
271 Where <em2>x</em2> is one of thw executables supported by JABAWS. Several environment
272 variables can be specified in the same line. For example.
274 <pre><code class="bash">mafft.bin.env=MAFFT_BINARIES#/homes/cengine/mafft;FASTA_4_MAFFT#/bin/fasta34;</code></pre>
277 The example above defines two environment variables with names
278 MAFFT-BINARIES and FASTA_4_MAFFT and values /homes/cengine/mafft
279 and /bin/fasta34 respectively. Semicolon is used as a separator
280 between different environment variables whereas hash is used as a
281 separator for name and value of the variable.
284 <p class="text-right">
285 <a href="#">Back to top <i class="fa fa-arrow-up" aria-hidden="true"></i></a>
291 <div class="row" id="mafftconf">
292 <div class="col-md-12">
293 <div class="panel panel-default">
294 <div class="panel panel-heading">
295 <h1 class="panel-title">Configure JABAWS to Work
298 <div class="panel-body">
300 If you use default configuration you do not need to read any
301 further. The default configuration will work for you without any
302 changes, however, if you want to install Mafft yourself then there
303 is a couple of more steps to do.
307 Mafft executable needs to know the location of other files
308 supplied with Mafft. In addition some Mafft functions depends on
309 the fasta executable, which is not supplied with Mafft, but is a
310 separate package. Mafft needs to know the location of fasta34
315 To let Mafft know where the other files from its package are,
316 change the value of MAFFT-BINARIES environment variables. To let
317 Mafft know where is the fasta34 executable set the value of
318 FASTA_4_MAFFT environment variable to point to a location of
319 fasta34 program. The latter can be added to the PATH variable
320 instead. If you are using executables supplied with JABAWS, the
321 path to Mafft binaries would be like
322 <em2><relative path to web application directory>/binaries/src/mafft/binaries</em2>
323 and the path to fasta34 binary would be
324 <em2><relative path to web application directory>/binaries/src/fasta34/fasta34</em2>.
325 You can specify the location of Mafft binaries as well as fasta34 program elsewhere
326 by providing an absolute path to them. All these settings are
327 defined in <em2>conf/Executable.properties</em2> file.
329 <p class="text-right">
330 <a href="#">Back to top <i class="fa fa-arrow-up" aria-hidden="true"></i></a>
336 <div class="row" id="settinglimit">
337 <div class="col-md-12">
338 <div class="panel panel-default">
339 <div class="panel panel-heading">
340 <h1 class="panel-title">Limiting the size of the job accepted by JABAWS</h1>
342 <div class="panel-body">
344 JABAWS can be configured to reject excessively large tasks. This is useful if you operate JABAWS service
345 for many users. By defining a maximum allowed task size you can provide an even service for all users and
346 prevents waste of resources on the tasks too large to complete successfully. You can define the maximum
347 number of sequences and the maximum average sequence length that JABAWS accepts for each JABA Web Service
348 independently. Furthermore, you can define different limits for different presets of the same web service.
351 By default limits are disabled. You can enable them by editing
352 <em2>conf/Executable.properties</em2>
353 file. You can adjust the limits according to your needs by editing
354 <em2>conf/settings/<X>Limit.xml</em2>
357 <p class="text-right">
358 <a href="#">Back to top <i class="fa fa-arrow-up" aria-hidden="true"></i></a>
364 <div class="row" id="diffbin">
365 <div class="col-md-12">
366 <div class="panel panel-default">
367 <div class="panel panel-heading">
368 <h1 class="panel-title">Using a different version of the alignment program with JABAWS</h1>
370 <div class="panel-body">
372 JABAWS is supplied with binaries and source code of the executables related to the version it supports. So normally you
373 would not need to install your own executables. However, if you have a different version of an executable
374 (e.g. an alignment program) which you prefer, you could use it as long as it supports all the functions
375 JABAWS executable require. This could be the case with more recent executable. If the options supported
376 by your chosen executable is different from the standard JABAWS executable, then you need to edit
377 <em2>ExecutableName</em2>Paramaters.xml configuration file.
379 <p class="text-right">
380 <a href="#">Back to top <i class="fa fa-arrow-up" aria-hidden="true"></i></a>
386 <div class="row" id="mixuse">
387 <div class="col-md-12">
388 <div class="panel panel-default">
389 <div class="panel panel-heading">
390 <h1 class="panel-title">Load balancing</h1>
392 <div class="panel-body">
394 If your cluster is busy and has significant waiting times, you can achieve a faster response by allowing
395 the server machine to calculate small tasks and then reserve the cluster for bigger jobs. This works
396 especially well if your server is a powerful machine with many CPUs. To do this you need to enable and
397 configure both the cluster and the local engines. Once this is done decide on the maximum size of a task
398 to be run on the server locally. Then, edit
399 <em2>"# LocalEngineExecutionLimit #" </em2>preset
400 in <em2> <ServiceName>Limits.xml</em2> file accordingly. JABAWS server then
401 will balance the load according to the following rule: If the task size is smaller than the maximum task
402 size for local engine, and the local engine has idle threads, then it calculates task locally otherwise it submit
403 the task to the cluster.
405 <p class="text-right">
406 <a href="#">Back to top <i class="fa fa-arrow-up" aria-hidden="true"></i></a>
412 <div class="row" id="testingJaba">
413 <div class="col-md-12">
414 <div class="panel panel-default">
415 <div class="panel panel-heading">
416 <h1 class="panel-title">Testing JABAWS Web-Services</h1>
418 <div class="panel-body">
420 Access <em2><your_JABAWS_server_URL>/ServiceStatus</em2> to test all web services.
421 Each time you access this URL, all services are tested.
422 For production configuration we recommend prohibiting requests to this URL for non
423 authenticated users to prevent excessive load on the server.
426 Alternatively, you can use a command line client (part of the client only
427 package) to test your JABAWS installation as described <a href="man_client.jsp">here</a>.
428 If you downloaded a JABAWS server package, you can use
429 <em2><your_jaba_context_name>/WEB-INF/lib/jaba-client.jar</em2> to test JABAWS installation as
430 described <a href="man_serverwar.jsp#usingWsTester">here</a>. If you downloaded the source
431 code, then you could run a number of test suites defined in the
432 build.xml Apache Ant file.
434 <p class="text-right">
435 <a href="#">Back to top <i class="fa fa-arrow-up" aria-hidden="true"></i></a>
441 <div class="row" id="logs">
442 <div class="col-md-12">
443 <div class="panel panel-default">
444 <div class="panel panel-heading">
445 <h1 class="panel-title">JABAWS requests logging</h1>
447 <div class="panel-body">
449 Enable Tomcat log valve. To do this uncomment the following section of
450 <em2><tomcat_root>/conf/server.xml</em2> configuration file.
452 <pre><code class="bash"><Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
453 prefix="localhost_access_log." suffix=".txt" pattern="common" resolveHosts="false"/></code></pre>
455 <p> The following information will be logged:</p>
456 <table class="table">
460 <th>Method server_URL protocol </th>
461 <th>HTTP status </th>
462 <th>Response size in bytes </th>
465 <td>10.31.11.159</td>
466 <td>[10/Feb/2010:16:51:32 +0000]</td>
467 <td>"POST /jws2/MafftWS HTTP/1.1"</td>
472 <p>Which can be processed in various programs for log analysis,
473 such as <a href="http://www.webalizer.org/">WebAlizer</a>,
474 <a href="http://www.analog.cx/">Analog</a>,
475 <a href="http://awstats.sourceforge.net/">AWStats</a>.
477 <p class="text-right">
478 <a href="#">Back to top <i class="fa fa-arrow-up" aria-hidden="true"></i></a>
484 <div class="row" id="logfiles">
485 <div class="col-md-12">
486 <div class="panel panel-default">
487 <div class="panel panel-heading">
488 <h1 class="panel-title">JABAWS internal logging</h1>
490 <div class="panel-body">
493 JABAWS can be configured to log what it is doing. This comes in
494 handy if you would like to see who is using your web services or
495 need to chase some problems. JABAWS uses
496 <a href="http://logging.apache.org/log4j/1.2/">log4j</a> to do the logging,
497 the example of log4j configuration is bundled with JABAWS war file.
498 You will find it in the <em2>/WEB-INF/classes/log4j.properties</em2> file. All the
499 lines in this file are commented out. The reason why the logging is
500 disabled by default it simple, log4j has to know the exact
501 location of where the log files are stored. This is not known up
502 until the deployment time. To enable the logging you need to
503 define<em2> logDir</em2> property in the <em2>log4j.properties</em2> and uncomment section of
504 the file which corresponds to your need. More information is given
505 in the <em2>log4j.properties</em2> file
506 itself. Restart the Tomcat or the JABAWS web application to apply
510 After you have done this, assuming that you did not change the
511 log4j.properties file yourself, you should see the application log
512 file called <em2>activity.log</em2>. The
513 file called <em2>activity.log</em2>. The
514 amount of information logged can be adjusted using different
515 logging levels, it is reduced in the following order of log levels
516 TRACE, DEBUG, INFO, WARN, ERROR, FATAL.
519 If you would like to know who is using your services, you might
520 want to <a href="#logs">enable Tomcat request
523 <p class="text-right">
524 <a href="#">Back to top <i class="fa fa-arrow-up" aria-hidden="true"></i></a>
530 <div class="row" id="warfile">
531 <div class="col-md-12">
532 <div class="panel panel-default">
533 <div class="panel panel-heading">
534 <h1 class="panel-title">JABAWS War File Content</h1>
536 <div class="panel-body">
537 <table class="table">
539 <th style="width:19%">Directory</th>
540 <th style="width:81%">Content description</th>
544 <td>contains configuration files such as Executable.properties,
545 Engine.local.properties, Engine.cluster.properties</td>
548 <td>conf/settings</td>
549 <td>Contains individual executable description files. In particular
550 XXXParameters.xml, XXXPresets.xml, XXXLimits.xml where XXX is the
551 name of the executable</td>
554 <td>ExecutionStatistics</td>
555 <td>The database for storing the execution statistics </td>
559 <td>Web pages for usage statistics visialization and webservices status queries </td>
563 <td>Contains directories generated when running an individual executable. E.g. input and output files and some other task
564 related data (optional)</td>
568 <td>Directory contains native executables - programs,
569 windows binaries (optional)</td>
572 <td>binaries/src</td>
573 <td>Contains source of native executables and Linux i386
577 <td>binaries/windows</td>
578 <td>Contains binaries for MS Windows operating system </td>
581 <td>binaries/matrices</td>
582 <td>Substitution matrices
583 <!-- what format ? --></td>
587 <td>Web application descriptor</td>
591 <td>Web application libraries</td>
594 <td>WEB-INF/classes</td>
595 <td>log4j.properties - log configuration file (optional)</td>
599 <td>Static content such as CSS, JavaScript and Image files</td>
602 <td colspan="2"><strong>Help Pages</strong> </td>
606 <td>help pages, index.html is the starting page</td>
610 <td><a href="${pageContext.request.contextPath}/dm_javadoc/">JavaDoc for the JABAWS Data Model</a></td>
613 <td>full_javadoc</td>
614 <td><a href="${pageContext.request.contextPath}/full_javadoc/">JavaDoc for the complete JABAWS</a></td>
618 <td><a href="${pageContext.request.contextPath}/prog_docs/">Documentation for programs that are included in JABAWS</a></td>
622 <p class="text-right">
623 <a href="#">Back to top <i class="fa fa-arrow-up" aria-hidden="true"></i></a>
629 <div class="row" id="ga">
630 <div class="col-md-12">
631 <div class="panel panel-default">
632 <div class="panel panel-heading">
633 <h1 class="panel-title">JABAWS and Google Analytics</h1>
635 <div class="panel-body">
637 JABAWS reports web services usage to our group Google Analytics (GA) account.
638 JABAWS usage statistics are collected for funding and reporting purposes, and
639 no private information is collected. The data sent by JABAWS is as follows:
642 <li>The IP address of the JABAWS server machine (the server IP can anonymized see
643 <em2>conf/GA.properties</em2> config file)</li>
644 <li>The name of the web service that was called. </li>
645 <li>A few details of the system such as JABAWS version, java version, user language,
647 resolution and character encoding.</li>
649 <p> Google Analytics can be disabled or adjusted by removing/editing
650 <em2>conf/GA.properties</em2> Google Analytics (GA) settings file. We would
651 appreciate it greatly if you could leave it on!</p>
652 <p>All calls to GA are very lightweight, completed asynchronously, create very
653 little overhead and do not influence the server response time or performance.
660 <jsp:include page="template_footer.jsp" />