Commit Graph

114 Commits (1e81d521c01754b6a4c8a147d91de0c4ec6ca5b2)

Author SHA1 Message Date
David Roazen 68e19edf59 Merged bug fix from Stable into Unstable, and resolved merge conflicts.
Conflicts:
	build.xml
	settings/ivysettings.xml
2011-07-08 15:50:31 -04:00
David Roazen a3c9d9c3ff Fixing Contracts for Java, and enabling contracts by default for unit/integration tests.
The NullPointerException we were seeing when trying to run with contracts enabled was being caused
by an outdated version of the asm library.

To run tests without contracts and disable their compilation, pass in "-Duse.contracts=false" to ant.

Also did some minor unrelated cleanup in build.xml
2011-07-08 15:34:39 -04:00
Khalid Shakir 7b699f8b17 Switched GridEngine from looking from environment variable to using embedded jar. 2011-07-05 21:59:00 -04:00
David Roazen 643458d7db Updated the tribble jar -- this should fix most of the integration test
failures we've been seeing.

Note that with tribble's new svn repository the revision numbers have reset,
hence "revision 3"
2011-06-29 01:11:03 -04:00
droazen 171e20a111 Updated the tribble jar to revision 351
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@6068 348d0f76-0448-11de-a6fe-93d51630548a
2011-06-22 22:55:57 +00:00
droazen ab1de3bfda Updated the tribble jar to revision 350
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@6065 348d0f76-0448-11de-a6fe-93d51630548a
2011-06-22 22:55:46 +00:00
droazen 95614ce3d6 Updated the tribble jar to revision 345
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@6033 348d0f76-0448-11de-a6fe-93d51630548a
2011-06-22 22:53:25 +00:00
droazen 32a991c4d3 Updated the tribble jar to revision 343
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@6031 348d0f76-0448-11de-a6fe-93d51630548a
2011-06-22 22:53:17 +00:00
hanna 14d7ee073b Rev Picard to get new PF_INDEL_RATE metric. Rev preQC generator script
to incorporate PF_INDEL_RATE.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5998 348d0f76-0448-11de-a6fe-93d51630548a
2011-06-14 17:43:43 +00:00
hanna cde2b409a7 Oops. Failed to add DbSnpMatchMetrics to Picard private jar.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5994 348d0f76-0448-11de-a6fe-93d51630548a
2011-06-14 15:14:19 +00:00
hanna 1b1aefc385 Move fingerprinting metrics reader into our Picard private extract.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5990 348d0f76-0448-11de-a6fe-93d51630548a
2011-06-14 00:13:10 +00:00
droazen 480598842c Updated the tribble jar
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5982 348d0f76-0448-11de-a6fe-93d51630548a
2011-06-13 18:00:09 +00:00
depristo e87c40d89c Fix for CoFoJa exception by upgrading to latest version
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5971 348d0f76-0448-11de-a6fe-93d51630548a
2011-06-09 17:49:15 +00:00
hanna 44b98bed8c Killed sonatype repository; it's failed me too many times at this point.
Temporarily disabled contracts in integrationtests until we can find the cause
of the new error that's cropping up for Ryan.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5967 348d0f76-0448-11de-a6fe-93d51630548a
2011-06-09 16:07:02 +00:00
depristo 218354e338 Contracts for Java (http://code.google.com/p/cofoja/) infrastructure enabled. No piece of code actually uses this, so it's possible to remove easily. Does not build by default (you must modify build.xml). Really an intermediate commit so I can play around with the system in my java classes and revert safely. Very much looking forward to DVCS
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5815 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-18 18:05:59 +00:00
hanna c2e8c460cb Factor out all testing dependencies into a separate test configuration and
only download that test configuration when running unit/integration tests.
This means that the build will (hopefully) never break because it can't
fetch a file that isn't required for the GATK to run.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5775 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-05 22:42:11 +00:00
hanna 45d8634522 Intermediate commit: bring Google Caliper into our private repository (even
though sonatype is back up).  This will tide us over until I figure out how
to add caliper to test configuration, so that it's only swapped in when we
actually run our unit / performance tests.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5770 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-05 14:33:14 +00:00
hanna 6d5e45b5c6 Revbump Picard dependencies at Tim/Kathleen's request. Exclude anonymous
classes from PluginManager.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5699 348d0f76-0448-11de-a6fe-93d51630548a
2011-04-27 20:38:05 +00:00
hanna 57a4700299 Ported small BAM performance test suite to the Google Caliper microbenchmarking suite. Looks promising,
but I'm still not sure that GC is a good long-term solution.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5683 348d0f76-0448-11de-a6fe-93d51630548a
2011-04-22 22:09:17 +00:00
kshakir 4f8411f4b5 Revved Picard to access new flag to disable mmap for bam indices. Only added a 3% speed boost but the mmap was added to the heap count, making it harder to specify/restrict the total resident memory size in LSF. Specifying -Xmx4g will now stay much closer to 4g resident memory usage versus bumping up to 9g when accessing 900 x ~8Mb bai's.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5549 348d0f76-0448-11de-a6fe-93d51630548a
2011-04-01 01:40:41 +00:00
hanna 8d2c14b29c Update Picard / sam-jdk at Tim's request.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4925 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-03 02:17:25 +00:00
kshakir 56433ebf6b Switched from LSF command line wrappers to JNA wrappers around the C API. Side effects:
- bsub command line is no longer fully printed out.
- extraBsubArgs hack is now a callback function updateJobRun.
Updated FullCallingPipelineTest to reflect latest changes to fullCallingPipeline.q.
Added a pipeline that tests the UGv2 runtimes at different bam counts and memory limits.
Updated VE packages that live in oneoffs to compile to oneoffs.
Added a hack to replace the deprecated symbol environ in Mac OS X 10.5+ which is needed by LSF7 on Mac.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4816 348d0f76-0448-11de-a6fe-93d51630548a
2010-12-10 04:36:06 +00:00
kshakir 63e3848187 Added status email support with -statusTo. Will send emails on failure of an individual function or success/failure of the whole pipeline.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4496 348d0f76-0448-11de-a6fe-93d51630548a
2010-10-14 15:58:52 +00:00
kshakir db47230dd9 Wrapping ScatterGatherableFunctions with a facade instead of using slower clone library. Will require keeping Clone's facade code in sync with CommandLineFunction but runs *much* faster.
Shell invoking scripts so that even really long shell scripts make it through LSF.
Using the truncated (up to 1000 characters) of the command line for the job name for use with bjobs.
Switched the default from re-running everything to re-running only files that need to be regenerated.  --skip_up_to_date replaced with --start_clean for those who want to regenerate everything.
Updated logging to let users know when the scatter gather generator is running, which still takes a while but is orders of magnatudes faster for large lists of functions.  (40s for a 100 function graph exploding to a 2500 function graph)


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4448 348d0f76-0448-11de-a6fe-93d51630548a
2010-10-07 01:19:18 +00:00
hanna 70bb480939 The battle is over. Picard is revved.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4200 348d0f76-0448-11de-a6fe-93d51630548a
2010-09-03 05:28:01 +00:00
aaron 3dc4d3c3a9 removing the custom reflections library from the libs, and adding a release version. Hopefully this will fix the problem Menachem has been seeing with random JVM crashes. Also
removed the auto-deletion of the reflections jar, and removed the very old OmniPlan document we had checked-in.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4056 348d0f76-0448-11de-a6fe-93d51630548a
2010-08-19 00:42:37 +00:00
kshakir 4f51a02dea Changed logging level to default at INFO instead of WARN.
Changes to StingUtils command line for use in Queue, replacing Queue's use of property files.
Updates to walkers used in existing QScripts to add @Input/@Output.
RMD used in @Required/@Allows now has a new default equal to "any" type.
New QueueGATKExtensions.jar generator for auto wrapping walkers as Queue CommandLineFunctions.
Added hooks to modify the functions that perform the Scattering and Gathering (setting their jar files, other arguments, etc.)
Removed dependency on BroadCore by porting LSF job submitter to scala.
Ivy now pulls down module dependencies from maven.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3984 348d0f76-0448-11de-a6fe-93d51630548a
2010-08-09 16:42:48 +00:00
aaron b3fd145161 fix for a bug deep in the tribble indexing: if you had a single record in the first contig, the second contig's index blocks would point to the wrong file seek location, and you'd see no
features in that contig. Thanks to Mark for finding this.  I'm not rev'ing the index version (which would cause all indexes to be rebuilt), since this seems like a pretty rare edge case.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3865 348d0f76-0448-11de-a6fe-93d51630548a
2010-07-23 18:39:55 +00:00
aaron 9579aace1f updates to code dependent on Tribble, as well as the following Tribble changes:
- makes writing to disk optional for indexes using the indexCreator classes (allow the user to specify the index file, if null don't write it)
- removed some system.out debugging code
- fixed version checking in interval tree 
- made indexes store and return a LinkedHashSet for sequence names (to ensure they've preserved the ordering in the file)
- index creators now read the file before creating the index
- changed the Index.write() method to take a LEDataStream instead of a file
- removed the sequence dictionary code on the header
- added utils for getting LEDataStreams
- added a base Tribble exception




git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3857 348d0f76-0448-11de-a6fe-93d51630548a
2010-07-23 01:56:10 +00:00
aaron 1cba81c16f updates to tribble with fixes for some bugs I've found in some new indexing code.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3842 348d0f76-0448-11de-a6fe-93d51630548a
2010-07-20 22:08:04 +00:00
aaron af6b5f000e updating the Tribble library; added writing of indexes to the index interface for working with the tree index.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3836 348d0f76-0448-11de-a6fe-93d51630548a
2010-07-20 07:02:08 +00:00
aaron 250ab70fed update the Tribble library too.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3827 348d0f76-0448-11de-a6fe-93d51630548a
2010-07-19 05:00:37 +00:00
hanna b6af17b82d Rev Picard with new IndexedFastaSequenceFile patch.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3700 348d0f76-0448-11de-a6fe-93d51630548a
2010-07-01 03:05:43 +00:00
aaron dff4c06763 Rev'ing Tribble with a special version that has excluded VCF 3.3
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3640 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-25 18:20:51 +00:00
hanna 4840ef6d3e Another rev of picard for /dev/null writing.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3627 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-24 19:22:47 +00:00
hanna c32f9d78ae Rev picard again, this time for error writing to /dev/null.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3626 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-24 04:08:26 +00:00
aaron 54ae0b8e4e some updates to tribble for the svn commit that will follow
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3621 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-23 20:20:07 +00:00
hanna 26d51bbe14 Another round of optimizations from Alec. Switching the header merger to
an IdentityHashMap provides another 10x+ performance boost over his previous
optimization for us.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3616 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-23 14:54:58 +00:00
hanna 003dd4de3e Rev Picard with performance enhancements.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3615 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-22 22:54:23 +00:00
aaron 5b87a00a5f updating with associated Tribble changes
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3605 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-22 07:54:15 +00:00
depristo 57a13805da GATK now uses a optimized indexing scheme in Tribble. 5x or more performance gain on files with many genotypes. Updated integrationtest that was failing and was clearly wrong. DB=; isn't a valid annotation.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3596 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-19 21:36:41 +00:00
aaron 32f6781ac7 updating tribble with the VCF header changes
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3583 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-18 08:20:44 +00:00
hanna db1383d0b2 Rev the latest version of Picard.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3575 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-16 23:55:07 +00:00
ebanks 8c28be5933 Fixing a VCF bug for Sendu: we weren't emitting flags (booleans) correctly in VCF3.3 (rev'ed tribble for this).
Updated dbsnp/hapmap membership info fields to be flags now instead of ints.
While I was there, I added the change in the Annotator for Jan to force reads to be from a specific sample.



git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3536 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-11 16:42:06 +00:00
aaron e27951ab39 re-updating the VCF code to handle spaces in sample names
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3528 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-10 20:18:34 +00:00
aaron ad98512f6c adding changes so that we look at the headers already loaded by the engine for samples and other VCF utils, and not create readers for each file to get them (this caused Tribble to regerenate indices if the index file can't be written to disk).
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3518 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-09 17:21:12 +00:00
aaron 6febd0291d rev tribble to include some dbsnp clean-up and fixes
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3510 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-09 03:08:31 +00:00
aaron 6d5556939d updating Tribble with a couple of important Tabix fixes, and updating the variant eval integration tests to run each test with both plain vcf and gzipped tabix (added the tabix version
to the vlidation directory), using the same md5sum.



git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3509 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-09 01:47:04 +00:00
hanna c1ecf75dd5 Update to the latest rev of the picard sharding patch. Includes updates reflecting
the imminent move of IlluminaUtil into picard public.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3493 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-06 20:33:21 +00:00
aaron 0b03e28b60 updating the tribble library to include the reference dictionary reading / writing. We now check the dictionaries of any tracks that have them against the reference (all new tribble tracks and out-of-date tracks will have this). Also renamed some classes to be more reflective of their function.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3485 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-04 06:34:26 +00:00