Further simplification of build tasks

[jabaws.git] / TODO.txt

Add more detailed tests for webservices? \r
\r
? ideally should introduce native Mac support (and maybe abandon native windows support in \r
favour of VM)   \r
\r
TODO write some help on the executable.properties within this file! \r
check that after binaries relocation build tasks point to correct locations!\r
\r
Cluster stats: \r
 -Remove hyperlinks from tasks which workdirs were removed \r
 -graph generation \r
 -user documentation (do not forget how to enable follow symlinks!)\r
\r
add to help text: To disable a web service remove it from WEB-INF/sun-jaxws.xml descriptor\r
\r
rename jaba.war to jabaws.war and make sure jabaws context path is used throughout. \r
\r
Current stat collector has to rely on file dates! \r
To change this you need \r
1) Collect all the stats in old format into the DB\r
2) Update the code to get the date out of the file.  \r
Can be done once JABAWS 1 is replaced with 2. \r
\r
\r
? Replace conservation.Method with server.ws.Method and try building WS. If this does not work - get rid of Method\r
\r
integrate the above to tweak the size of the local job\r
\r
Good toString method for Limits (test with command line client -limits)\r
Document the Limits for presets\r
JABA DOCS\r
\r
(later) Add documentation on Presets, Parameters and Limits\r
(later) Explain how to define a limit\r
\r
Add test for repeated result collection from cluster engine\r
\r
(low) Pack the test cases and build file to run them in one of the distributives \r
\r
\r
DONE\r
\r
version 2\r
\r
DONE:  GET rid of binaries/help directory!\r
DONE:  update binaries\r
 DONE:  muscle binary version 3.8.31 for win/lin32/lin64 and sources, docs in website/prog_docs\r
 DONE:  test Muscle on the cluster\r
 DONE:  test Muscle compilation\r
 DONE:  probcons up-to-date\r
 DONE:  clustal up-to-date \r
 DONE:  update tcoffee (8.99 latest) source code updated  docs updated \r
 DONE:  tcoffee compilation for IA32 \r
 DONE:  update Mafft (6.857 latest) for win/lin32/lin64 and sources, no docs updates\r
DONE:  make sure conf files are optional!\r
\r
Cluster stats: \r
 DONE: cancelled should not be reported as incomplete!\r
 DONE: ant task to clean up clean stat db task before packing web app\r
 DONE: pack web apps with stat db (NOT TESTED!)\r
 DONE: change input size bites\r
 DONE: add help info for each column in the detailed job list table\r
 DONE: replace default -1 with ? \r
 DONE: delete and re-import the database to fix clustal input staff    \r
\r
DONE:  Test all WS as names from Executables were removed\r
DONE:  Add default names for input and output in every executable and use them consistently\r
  throughts (e.g. in all WS). Best of all use the same name from SceletalExectuable \r
  For statistics. \r
\r
DONE:  FIXED Conecting to JABAWS version 2 service\r
  09-Feb-2011 15:27:53 compbio.ws.client.Jws2Client connect\r
  INFO: Connected successfully!\r
  Exception in thread "main" java.lang.NullPointerException\r
        at compbio.ws.client.MetadataHelper.getParametersList(MetadataHelper.java:30)\r
        at compbio.ws.client.Jws2Client.<init>(Jws2Client.java:179)\r
        at compbio.ws.client.Jws2Client.main(Jws2Client.java:483)\r
\r
DONE:  ScoreManager should output scores properly \r
DONE:  Allow empty Parameters and Preset files! \r
DONE:  Check the WS input and reject it on submission rather then of access with error message\r
\r
DONE: Add registry service to query services status\r
DONE: Refactor web services checker to enable a programmatic access to its methods.\r
DONE: Finish the client\r
DONE: Add AACon ws\r
DONE: Add iupred ws http://iupred.enzim.hu/\r
DONE: Add globprot ws - does not report raw scores, just regions\r
DONE: Add ronn ws\r
DONE: Define limits for presets - if required\r
DONE: Add logging facility to WSTester so more details of the problem can be reported\r
DONE: Statistics collector for engines (number of operations/timing)\r
\r
\r
version 1\r
\r
DONE: LocalExecutor service must be shutdown on web application undeploy event not on JVM shutdown. \r
      This is because JVM is still running even after web application is shut down!\r
DONE: Use absolute path for web site download links as archives are not included into distro!\r
DONE: Rearrange web site docs - add links to the top of the page to the topics discussed below\r
DONE: Put the documentation for various executables online\r
DONE: Make left gradient image one-two characters larger + change "For developer ->For Developers"\r
DONE: test local/cluster execution - test Load balancer\r
DONE: Improve SimpleWS client so it can be scripted against: give user alignment and accept parameters\r
DONE: Make a configuration tester class (check that configuration files point to the executables, and they can be executed)\r
DONE: PUT CLIENT JAR INTO WEB-INF/LIB - most likely need to get rid of dist prefix in the build file\r
DONE: Different packages for download and dundee. generic vs specific settings. \r
DONE: JAVADOC\r
DONE: Refactor EngineResourcesLeak tester to hide a map implementation!\r
DONE: Compile executables for linux in the most generic way\r
DONE: Make different log4j configuration files / web services logs.\r
DONE: System manual\r
DONE: Developer doc\r
DONE: How to\r
DONE: Prepare different packages\r
DONE: extract commonly used names into variable in the build script\r
DONE: Test for high load conditions (TestNG?)\r
DONE: (low) Fix: Started jobs with unknown completion status reported as FINISHED by  msaws.getJobStatus(taskId) (see SimpleWSClient)\r
DONE: fix compile binaries script\r
DONE: Make script to change executable flag for linux systems\r
DONE: jobsout must be created within war\r
DONE: enable relative paths for local engine conf temp dir\r
COUND NOT REPRODUCE: fix cluster job status - RUNNING -> UNDEFINED\r
DONE: throw JobSubmission exception with appropriate method instead - (low) code discoverer/tester for ws - e.g. after war deployment to find out what actually works\r
DONE: fix build scripts after reorganising project\r
DONE: package as war file for deployment\r
DONE: package for jalview distro for local execution\r
DONE: reorganise project into independent chunks\r
DONE: test Parameter & Option toCommand method\r
DONE: Replace WS methods with new CommandBuilder.toCommand()\r
DONE: Setting default values is validated as external method \r
DONE: Refactoring to extract data model, and engines is performed\r
DONE: Default value is adopted for passing new Option values to server side\r
DONE: Converting Options to commands delegated to CommandBuilder  \r
DONE: Make Arguments mutable to facilitate default value setting for WS\r
DONE: Matrix files for alignment programme\r
DONE: Clean up testcases\r
DONE: Replace List<String> as command line parameters abstraction to dedicated CommandBuilder class \r
DONE: Parallel execution for tcoffee jobs\r
DONE: Copyright added\r
DONE: WRAP ProbCons\r
DONE: think about time limits and cluster parameters for parallel execution - NativeSpecs for cluster is implemented\r
DONE: (low) Use for WS: (@WebParam(name = "name") String name) {\r
DONE: Implement rough policy for runtimes (e.g. do not run excessively long tasks on the local machine if cluster is available)\r
DONE: Write tomcat conf to expose WS\r
DONE: access control to web.xml\r
DONE: Statistics collector for WS\r
DONE: implements limits for web services\r
DONE: WS clients must be able to connect to a web service given the service url. \r
e.i. ws URL should not be compiled in using wsimport, but determined from \r
the configuration file.  \r
\r
DONE: Could not find configuration files if run from within servlet container\r
DONE: Add ways to get engine based on the key for everything but submission of the new tasks \r
DONE: Add cache to Cluster JOBID - so that at least jobs for which results have not been collected is cached.\r
DONE: FINISHED overrides FINISHED with time stamp!\r
DONE: Consider think about JobStatus and StatFileStatus\r
DONE: a method to read 3 chars at the beginning of the file to indicate a progress is added\r
DONE: ConfiguredExecutable<?> al = sexec.getResults(jobId); || Alignment ?\r
DONE: ClustalW - OUTput must be set !\r
DONE: Add accessor methods to get a DUMP of the process output \r
DONE: FilePuller should be cached to avoid excessive fs exercise\r
DONE: Test List of the processes (ensure no memory leaks)\r
PART DONE - On the client on shutdown remember the task list - server tasks could be loaded and executed from saved run configurations, client should decide for themselves what they want\r
DONE: Persist the task list to enable later result collection on the server side\r
DONE Wrap TCoffee\r
DONE: improve parameters handling - new schema + update for all parameters\r
DONE: Cluster jobs report submitting and result collecting/finishing time (for sync jobs only(!)) and cluster jobid in the corresponding files  \r
DONE: Local jobs report starting and finising times in files with corresponding names in the task directory  \r
DONE: Generalize getResult method for MSA runners\r
DONE: Create test suits\r
DONE: Convert junit tests to TestNG\r
DONE: Redirect output & error streams to the named files as some processed need this  \r
DONE: Extract generilyzed class with most of the method for individual executable to use\r
DONE: Refit wrappers to use the generalised class to avoid duplication  \r
DONE: Pass environment variable settings to the executable\r
DONE: Complete AsyncExecutors implementation\r
DONE: Look to unify methods for engines\r
DONE: Provide way around of generated directory setup for test cases(?)\r
DONE: Fix the test cases for local engine (need to set directory on executable explicitly)\r
DONE: Make log files for processes err and out for local execution\r
DONE: Trace where AsyncExcel called with null Executable and attempts to create a new directory repeatedly\r
DONE: directory generator\r
DONE: tests for cancel \r
DONE tests for getStatus\r
DONE  ws async\r
DONE: parametrized clustal operations\r
DONE: test large datasets handling \r
DONE: write load balancer \r
DONE: (low) task persistance\r
DONE: Look at CancellableTask if with Sync and AsyncExecutor \r