1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
\r
2 <html xmlns="http://www.w3.org/1999/xhtml">
\r
4 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
\r
5 <title>Java Bioinformatics Analyses Web Services (JABAWS) developers documentation</title>
\r
6 <link href="ws.css" rel="stylesheet" type="text/css" media="screen, projection, handheld, tv" />
\r
7 <link rel="stylesheet" type="text/css" media="print" href="print.css"/>
\r
9 <script type="text/javascript" src="prototype-1.6.0.3.js"></script>
\r
16 <tr><td style="width:158px;"><a href="http://www.dundee.ac.uk"><img src="images/uod_lt_long.gif" alt="University of Dundee" width="158" height="90" class="logo" title="University of Dundee" longdesc="http://www.dundee.ac.uk"/></a></td>
\r
17 <td class="bg"><img src="images/jabaws2.png" alt="JABAWS-2.0" title="Java Bioinformatics Analysis Web Services version 2.0"/></td>
\r
18 <td class="bg"><img src="images/banner_right.png" alt="Disorder" width="200" height="80"/></td>
\r
21 </div><!-- banner end-->
\r
25 <a href="index.html">Home</a>
\r
26 <a href="quick_start.html">Getting Started</a>
\r
27 <a class="selected" href="man_about.html">Manual</a>
\r
29 <a href="man_about.html">About</a>
\r
30 <a href="man_servervm.html" title="JABAWS Server as Virtual Appliance">Server VA</a>
\r
31 <a href="man_awscloud.html" title="JABAWS Server in the Amazon EC2 Cloud">Server in the Cloud</a>
\r
32 <a href="man_serverwar.html" title="JABAWS Server as Web Application aRchive">Server WAR</a>
\r
33 <a href="man_configuration.html" >Server<br/>
\r
35 <a href="man_client.html" title="JABAWS Command Line Client">CMD Client</a>
\r
36 <a href="man_stats.html" title="JABAWS Usage Statistics">Usage Statistics</a>
\r
37 <a href="man_dev.html" title="Accessing JABAWS from your program">Accessing<br/>
\r
39 <a class="selected" href="man_server_dev.html" >JABAWS Development</a>
\r
41 <a href="http://www.compbio.dundee.ac.uk/download">Download</a>
\r
42 <a href="PublicAnnualStat" title="JABAWS usage statistics">Usage Statistics</a>
\r
43 <a href="ServiceStatus" title="JABAWS webservices status. Click to test all web services. Please be patient while the services are being checked">Services Status</a>
\r
44 <a href="contacts.html">Contact Us</a>
\r
45 <a href="http://www.compbio.dundee.ac.uk" title="University of Dundee, The Barton Group" >Barton Group</a>
\r
50 <h2 id="headtitle">For JABAWS Developers</h2>
\r
51 <h2>The JABAWS project </h2>
\r
52 <p>This page is for anyone who wants to add a new webservice into the JABAWS framework. </p>
\r
54 <li><a href="#api">The API</a></li>
\r
55 <li><a href="#thesrc">The source code</a></li>
\r
56 <li><a href="#projstruct">Structure of the project</a></li>
\r
57 <li><a href="#thecode">The code structure</a></li>
\r
58 <li><a href="#runtests">Running tests</a></li>
\r
59 <li><a href="#prepexec">Preparing distributive's</a></li>
\r
60 <li><a href="#buildstubs">Building web services artifacts</a></li>
\r
61 <li><a href="#addnewws">Rough guide - adding a new web service</a></li>
\r
64 <h4><a name="api" id="api"></a>The API </h4>
\r
65 <p><a href="dm_javadoc/index.html">Data model</a> javadoc- read this if your are coding against JABA Web Services</p>
\r
66 <p><a href="full_javadoc/index.html">Complete</a> javadoc - for developers who want to use JABAWS framework and use Engines and Executables directly</p>
\r
67 <h4><a name="thesrc" id="thesrc"></a>The source code</h4>
\r
68 <p><a name="svn">Private (most up-to-date) SVN source repository:</a><a href="link to svn.lifesci.dundee.ac.uk/svn/barton/ptroshin/JABA_r1">https://svn.lifesci.dundee.ac.uk/svn/barton/ptroshin/JABA2_r1</a></p>
\r
69 <p>The JABAWS public code repository containing the code for each JABAWS public release is located at https://jabaws.googlecode.com/ </p>
\r
70 <p> The repository contains a complete JABAWS <a href="http://www.eclipse.org">Eclipse</a> project. </p>
\r
71 <p>Of cause If you want to make a modification to the source code you would need to generate distributives yourself. To do that first generate JAX-WS artifacts using <span class="hightlight">build-server</span> task from <span class="hightlight">wsbuild.xml</span> ant script, than you could use build.xml tasks to generate any of the distributives you need. </p>
\r
72 <h4><a name="projstruct" id="projstruct"></a>Structure of the project</h4>
\r
75 <span class="directory">binaries</span> contains native executables e.g. clustalw</li>
\r
77 <li><span class="directory">src</span> contains sources of native executables</li>
\r
78 <li><span class="directory">windows</span> contains pre-compiled Windows binaries</li>
\r
79 <li>compilebin.sh - the script to complile binaries</li>
\r
80 <li>setexecflag.sh - the script to set executable flag for the binaries</li>
\r
82 <li><span class="directory">conf</span> contains JABAWS configuration files </li>
\r
83 <li><span class="directory">ExecutionStatistics</span> the database for storing collected execution statistics </li>
\r
84 <li><span class="directory">jobsout</span> a default folder for temporary job directories </li>
\r
85 <li><span class="directory">statpages</span> the web pages for execution statistics display</li>
\r
86 <li><span class="directory">WEB-INF</span> default </li>
\r
87 <li><span class="directory">website</span> contains the JABAWS web pages </li>
\r
89 <li><span class="directory">archive</span> contains JABAWS packages, the WAR and JAR files</li>
\r
92 <h4><a name="thecode" id="thecode"></a>The code structure</h4>
\r
94 <li><span class="directory">datamodel</span> contains the JABAWS datamodel </li>
\r
95 <li><span class="directory">engine</span> contains the JABAWS engine - the code that abstract the execution environment and executes native binaries</li>
\r
96 <li><span class="directory">runner</span> contains the JABAWS runners - thin wrappers for native binaries </li>
\r
97 <li><span class="directory">webservices</span> contains the JABAWS SOAP web services </li>
\r
98 <li><span class="directory">testsrc</span> contains the JABAWS unit tests</li>
\r
100 <p>Each source folder depends on the upper folders for compilation. For example, the datamodel is the top level folder so it has no other dependencies on other JABAWS code. The Engine level depends on the datamodel to compile etc. The web services folder is the bottom layer and depends on all the other source code. </p>
\r
101 <p>So the JABAWS project is split into 4 layers. From bottom-up the first layer consists from the value classes used by all other layers of the hierarchy, in particular web services. So, to be able to use JABAWS one needs to have these classes. At the same time classes on this layer does not have any dependencies on the layers above.</p>
\r
102 <p>The second layer contains code for execution of the wrappers, which are the abstraction describing native executables. JABAWS can execute tasks locally that is on the same machine as JVM and on the cluster. Thus currently code on this layer contain two engines. This layer depends on the layer underneath, the data model layer, but is completely independent from the code above.</p>
\r
103 <p>The third layer consists of the wrappers for the native executables and classes to handle their configuration. It depends on the engines and the data model, but know nothing about the web services.</p>
\r
104 <p>Finally, the upper layer contains the web services, that depend on all the layers below.</p>
\r
105 <p>The layer isolation is archived though specially designed compilation task which is executed sequentially in several stages so that the first layer compiles before any other layers, second layer compiles after that and process continies before all the code is compiled. Any violation of the layer boundaries results in the compilation failure. Use Ant "Compile" or "Complile_with_debug" tasks to perform the staged compilation.</p>
\r
106 <p>A client package contains only classes from data model layer and a simple web services client. Framework package is for anyone who want to use JABAWS framework for controlling native executables in local or cluster environments. Framework exclude the web services layer. Server package contains all the code.</p>
\r
108 <h4><a name="runtests" id="runtests"></a>Running tests</h4>
\r
109 <p>JABAWS uses TestNG framework for testing. The test results for the JABAWS package offered for download can be found here: <a href="http://www.compbio.dundee.ac.uk/user/www-jws2/tests/index.html">Test Results</a><br/>
\r
110 JABAWS uses <a href="http://testng.org/doc/index.html">TestNG</a> for testing. There is a TestNG plugin available for Eclipse which has functionality similar to JUnit. However, no plugins are necessary to run the test cases, as testng jar is supplied with JABAWS together with an ant tasks to run the test cases. </p>
\r
111 <p>The best way to ensure that JABAWS framework is completely functional on your system is to run all test cases.
\r
112 Test cases tests all aspects of JABAWS functionality. Consequently, one need to have non windows operation system and support of the cluster to be able to run all tests. If your system does not support cluster, then you could run all test excluding those that depends on the cluster.
\r
113 Several testing groups are supported:
\r
114 <p>All tests (Test)</p>
\r
115 <p>Cluster tests (Run_cluster_dependent_test)</p>
\r
116 <p>Cluster independent tests () </p>
\r
117 <p>Windows only tests (All_cluster_independent_windows_only_tests) </p>
\r
118 <p>Performance and stability tests (Long_tests) </p>
\r
119 <p>Re-run failed tests (Rerun_failed_tests) </p>
\r
120 <p>Run custom test (CustomTest) </p>
\r
121 <p>To run the tests you need to download all sources from <a href="#svn">repository</a>. Once you have done that, enter into the command line mode, change directory to the project directory and type:
\r
122 <span class="code">ant -f build.xml <test group name></span> </p>
\r
123 <p>. Make sure you have <a href="http://ant.apache.org/">Apache Ant</a>
\r
124 installed and path to ant executable is defined in your path environmental variable.
\r
125 Replace test group name with the one of the names given in the list above to run required group of tests e.g for running cluster only tests
\r
127 command: <span class="code">ant -f build.xml Run_cluster_dependent_test</span>
\r
128 If you work under Linux you could use a simple script from the root folder of repository called <span class="hightlight">runtests.sh</span> This script simply contains a collection of the test commands described above and paths to java home directory and an ant executable, which you can define once for your system and then reuse. </p>
\r
130 <p>A handy feature of TestNG is its ability to re-run failed tests. Failed test ant file is stored in <span class="hightlight">test-output/testng-failed.xml</span>. and is used in the ant task called <span class="hightlight">Rerun_failed_tests</span>. So re-running failed tests requires no more work than running any other test group and could be accomplished with the command: <span class="code">ant -f build.xml Rerun_failed_tests</span> CustomTest runs the test defined in the project root directory file called <span class="hightlight">temp-testng-customsuite.xml</span>. This file is generated by TestNG plugin every time you run the test from Eclipse. Thus an easy way to run a test in a different environment is to run it from Eclipse first and then from ant using a custom test procedure. </p>
\r
131 <p class="hightlight">For cluster execution make sure that the property LD_LIBRARY_PATH defined in build.xml points to cluster engine LD libraries directory in your local system.</p>
\r
133 <h4><a name="prepexec" id="prepexec"></a>Preparing distributive's</h4>
\r
134 <p>There are a number of ant tasks aimed for preparing distributives for download.
\r
135 Currently a few types of JABAWS packages are offered
\r
136 <p>Client only (contains classes required to access JABA Web Services)</p>
\r
137 <p>Platform specific JABAWS (windows and other)</p>
\r
138 <p>JABAWS without binaries</p>
\r
139 <p>JABAWS framework </p>
\r
141 Corresponding build task names are:
\r
142 <p>min-jaba-client</p>
\r
143 <p>jaba-windows, jaba-complete</p>
\r
144 <p>jaba-no-binaries</p>
\r
145 <p>full-jaba-client</p>
\r
146 <p>The easiest way to build all distributives is to call <span class="hightlight">build-all </span>ant task. There are more tasks defined in build.xml than described here. They are mostly self explanatory. </p>
\r
147 <p class="hightlight">If you made any changes to the data model and would like to generate a complete JABAWS distro make sure you have rebuilt jaxws artifact as described below. </p>
\r
148 <h4><a name="buildstubs" id="buildstubs"></a>Building web services artifacts</h4>
\r
149 <p>Server side artifacts should be rebuild whenever the data model, meta model or MSA interface were changed. To do that run build-server task from wsbuild.xml ant build file. WSDL files will be generated in <span class="hightlight">webservices/compbio/ws/server/resource</span> directory. It is not necessary to edit them if any of the JABAWS clients are used. JABAWS are the standard JAX-WS web services, which are WS-I basic profile compatible. </p>
\r
150 <h4><a name="addnewws" id="addnewws"></a>Rough guide - adding a new web service</h4>
\r
152 <li> Add a new executable which you'd like to wrap as a JABAWS web service to the binaries folder. If it has the source code and can be
\r
153 recompiled for different platforms include it under binaries/src.
\r
154 Edit binaries/src setexecutableflag.sh and compilebin.sh scripts accordingly. </li>
\r
155 <li> Make sure that all the dependencies of the software being installed are satisfied.
\r
156 If there are other binaries they should be included as well. Keep the dependent
\r
157 binaries in subfolder for the main executable. Update compilebin and setexecflag </li>
\r
158 scripts accordingly.<li> Make sure that the new executable </li>
\r
159 does not have any hard links to its dependencies, e.g. is able to run from
\r
160 any installation folder and does not contain any hard coded paths.
\r
161 <li> Describe executable in conf/Exectuable.properties file. The lowercase name of the
\r
162 wrapper should be included in the name of the property for example Clustal
\r
163 properties all include clustal as a part of the name e.g. local.clustalw.bin.
\r
164 The same property for Mafft will be called local.mafft.bin. For more help please refer to the Executable.properties file. </li>
\r
165 <li> Describe the executable supported parameters in the <ExecutableName>Parameters.xml, presets in the <ExecutableName>Presets.xml and the execution limits in the <ExecutableName>Limit.xml. By convention these files are stored in conf/settings. All of these are optional. If the
\r
166 executable does not support parameters you do not have to mention the
\r
167 XXXParameter.xml file in the Executable.properties file at all. The same is true for
\r
168 Presets and Limits. </li>
\r
169 <li> Create a Java wrapper class for your executable. Create it within runner
\r
170 source directory. Examples of other wrappers can be found in compbio.runner.msa
\r
171 or compbio.runner.disorder packages. Wrapper should extend SkeletalExecutable<T>
\r
172 implements PipedExecutable<T> if you need to pass the input or collect the
\r
173 results from the standard in/out. Please see Mafft code as example. Wrapper
\r
174 should expend SkeletalExecutable<T> if input/output can be set as a parameter
\r
175 for an executable. Please see ClustalW code as example. </li>
\r
176 <li> Create a testcase suit for your wrapper and run the test cases. </li>
\r
177 <li> Create parser for the output files of your executable. Suggested location
\r
178 compbio.data.sequence.SequenceUtil </li>
\r
179 <li> Test the parser</li>
\r
180 <li> Decide which web services interfaces your executable is going to match.
\r
181 For example if the executable output can be represented as <br />
\r
182 SequenceAnnotation then SequenceAnnotation interface might be appropriate.
\r
183 For multiple sequence alignment an Msa interface should be used. </li>
\r
184 <li> If you find a web interface that matches your returning data type, then
\r
185 implement a web service which confirms to it within a webservices source folder. </li>
\r
186 <li> Register web service in WEB-INF/ web.xml and sun-jaxws.xml</li>
\r
187 <li> Add generated wsdl to wsbuild.xml ant script to generate the stubs.</li>
\r
188 <li> Run build-server task in wsbuild file. Watch for errors. If the task fails <br />
\r
189 that means that JAXB cannot serialize some of your new data structures. Add <br />
\r
190 appropriate annotations to your data types.
\r
191 Also check that <br />
\r
192 - you do not have interfaces to serialize. JAXB cannot serialize them.<br />
\r
193 - you have a default no args constructor (can be private if you do not need it)<br />
\r
194 - JAXB cannot serialize a Map, use custom data structure instead!<br />
\r
195 - Enum cannot be serialized as its abstract class (do not confuse with enum <br />
\r
196 which is fine)<br />
\r
197 - Fields serialization leave a little more space for manoeuvre, so use it. If <br />
\r
198 you do then you can accept and return interfaces, e.g. List, Map; abstract <br />
\r
199 classes etc, from your methods. <br />
\r
201 If you have the data on the server side, but nothing is coming through to the <br />
\r
202 client, this is a JAXB serialization problem. They tend to be very silent and <br />
\r
203 thus hard to debug. Check your data structure can be serialized! </li>
\r
204 <li>Modify the client to work with your new web service. Update Services
\r
205 enumeration to include new service and ensure that all the methods of this
\r
206 enumeration take into account the new service. Update the client help text
\r
207 (client_help.txt) and insert it into the Constraints class. </li>
\r
208 <li> Test the web service with the client </li>
\r
209 <li> Test on the cluster.</li>
\r
213 <!-- content end-->
\r
214 <div id="copyright">Last update: 16 September 2011<br/>
\r
215 Peter Troshin, The Barton Group, University of Dundee, UK</div>
\r
216 </div><!-- wrapper end-->
\r
217 </div> <!-- page end-->
\r
218 <!-- Google analitics -->
\r
219 <script type="text/javascript">
\r
220 var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
\r
221 document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
\r
223 <script type="text/javascript">
\r
225 var pageTracker = _gat._getTracker("UA-5356328-1");
\r
226 pageTracker._trackPageview();
\r
git://source.jalview.org / jabaws.git / blob