Commit Graph

3624 Commits (5f8a3f95efd335fee13fca7ed6608ca17f970d7f)

Author SHA1 Message Date
aaron 5f8a3f95ef The GT field once again reigns supreme (it must be the first genotype field). Thanks for the catch Eric.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3645 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-25 21:03:05 +00:00
kshakir 894ad354fa Fixed typo in the name of the shell directory.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3644 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-25 20:59:40 +00:00
kshakir 75c98c42b8 Started path of deprecation of Sting's @Argument by splitting the annotation into @Output and @Input. Anything that's not an @Output should be an @Input.
Checked in example qscripts that are basically todo integration tests.
Replaced use of queue @Input/@Output with Sting's new @Input/@Output.  This means you'll now have to doc-ument the annotations.
More work on dependency resolution cycles being created in the graph during scatter/gather.
Filtering nulls to avoid NPE exceptions in scala's 'Collection'.hashCode.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3643 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-25 20:51:13 +00:00
weisburd 147ba68441 Fixed bug with mrnaCoord field - made it count exon positions only, rather than introns & exons
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3642 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-25 19:53:32 +00:00
kshakir ce27ed0d60 Added missing @ClassType to memory limits.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3641 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-25 18:56:35 +00:00
aaron dff4c06763 Rev'ing Tribble with a special version that has excluded VCF 3.3
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3640 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-25 18:20:51 +00:00
aaron d3848745ab moving VCF 3.3 back into the GATK so Guillermo can make changes for VCF 4 output
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3639 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-25 18:20:06 +00:00
aaron b3edb7dc08 two fixes for the VCF 4 parser:
- Allow the "GT" field in genotypes at any point in the genotype string (before we required they be the first key-value pair).
- Fix a bug with the phasing value put into the VariantContext, thanks for the catch Guillermo!

git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3638 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-25 18:01:23 +00:00
weisburd e15fe6858e Disabling test - Will need to update big-tables soon.. will re-enable after updating md5
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3637 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-25 15:43:41 +00:00
aaron efa60e5de5 and add changes to the vcf used in testing
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3636 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-25 02:56:02 +00:00
aaron f9c7803d4e this got left off my last commit
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3635 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-25 02:42:44 +00:00
weisburd 1cb8f51f8c Fixed -t arg
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3634 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-24 23:44:10 +00:00
weisburd 3cd0570c1e Now can run with multiple processes, multiple threads, or both
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3633 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-24 23:25:01 +00:00
weisburd dae3ce2c0f changed log dir
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3632 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-24 23:08:13 +00:00
weisburd fea8054e9e Updated long name for -l to --run-locally
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3631 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-24 22:26:45 +00:00
aaron 682f9b46c6 Two fixes together:
1) Some improvements to the VCF4 parsing, including disabling validation.
2) Reimplemented RefSeq in the new Tribble-style rod system.

git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3630 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-24 22:17:03 +00:00
weisburd 72e669538e Updated arg description for -s
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3629 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-24 22:04:01 +00:00
aaron 62bc7651a8 fix for PSPW with DbSNP mask. Added an integration test for this case.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3628 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-24 19:31:32 +00:00
hanna 4840ef6d3e Another rev of picard for /dev/null writing.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3627 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-24 19:22:47 +00:00
hanna c32f9d78ae Rev picard again, this time for error writing to /dev/null.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3626 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-24 04:08:26 +00:00
corin bcab0eba01 This replaces tearsheet.r, neatens up graphics, and allows the script to be used in R's interactive environment
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3625 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-24 01:02:58 +00:00
aaron 8a9b2f4256 removing the GLF ROD.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3624 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-23 22:51:45 +00:00
asivache 17d2043354 bug fix: now contigs not present in the sequence dictionary are registered properly and do not cause the script to break
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3623 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-23 20:59:38 +00:00
aaron 611d834092 a couple of VCF 4 improvements:
-Validation of INFO and FORMAT fields.
-Conversion to the the correct type for info fields (i.e. allele frequency is now stored as a float instead of a string).
-Checks for CNV style alternate allele encodings( i.e. <INS:ME:L1>), right now we exception out.  Maybe we should just warn the user?
-Tests for the multiple-base polymorphism allele case.



git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3622 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-23 20:21:43 +00:00
aaron 54ae0b8e4e some updates to tribble for the svn commit that will follow
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3621 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-23 20:20:07 +00:00
ebanks f0fc34bb8e Bug fix: N's are allowed in the ref so don't fail when e.g. dbsnp has an N!
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3620 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-23 17:49:14 +00:00
ebanks b6bceb39b0 Fixing up output for performance tests
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3619 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-23 17:00:17 +00:00
chartl 75d4736600 Committing changes to comp overlap for indels. Passes all integration tests; minor changes to MVC walker.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3618 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-23 15:49:13 +00:00
ebanks 9b8775180e Turn on the memory improvement by default (assume the target interval list is sorted, since it is 99.9% of the time). Make the user throw a flag when it's specfically not sorted.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3617 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-23 15:44:55 +00:00
hanna 26d51bbe14 Another round of optimizations from Alec. Switching the header merger to
an IdentityHashMap provides another 10x+ performance boost over his previous
optimization for us.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3616 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-23 14:54:58 +00:00
hanna 003dd4de3e Rev Picard with performance enhancements.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3615 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-22 22:54:23 +00:00
aaron 0cafd3d642 clip VCF alleles for indels: only a single left base, and as many right bases as align before converting to variant context.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3614 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-22 22:42:38 +00:00
aaron 9872b65803 clip to the null allele on the reference string in VCF 4, instead of stopping to perserve one reference base.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3613 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-22 20:52:19 +00:00
ebanks b5df2705c9 -Remove Nway output option
-Remove in-memory sorting
-Default to name-sorting (although we allow coordinate sorting with the --sortInCoordinateOrderEvenThoughItIsHighlyUnsafe flag).

Cleaner, faster code.  Wiki has been updated (including how to use FixMateInformation.jar from Picard).  More changes coming soon.



git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3612 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-22 20:31:55 +00:00
kshakir 30cf78fdc0 Refactoring for a first version of scatter gather api with basic shell script implementations.
Modified build script so that queue is cleaned during "ant clean".



git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3611 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-22 18:39:20 +00:00
aaron 18f62a346d fixing the tests; Bamboo captured the failure in the logs correctly.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3610 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-22 18:38:03 +00:00
aaron a6d3e4bd47 Add code to allow reference alleles with 'N' in VariantContext, but not in the alternate allele(s). Also more updates to the VCF 4 code (fixed parsing for files without genotypes).
This check-in will temperarly break the build (I need to see if Bamboo is correctly returning the log file for the failed builds).  

Will be fixed once Bamboo starts building.

git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3609 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-22 18:26:37 +00:00
ebanks 824c2bbac0 Finishing previous checkin
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3608 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-22 17:21:38 +00:00
ebanks 4727bcda24 Removing Beagle output from UG. Use ProduceBeagleInput walker instead (since it can be run post-filtration and respects the FILTER column).
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3607 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-22 16:56:37 +00:00
aaron 55abfcc52f adding file my VCF4 tests depend on
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3606 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-22 13:39:20 +00:00
aaron 5b87a00a5f updating with associated Tribble changes
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3605 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-22 07:54:15 +00:00
aaron 32f324a009 incremental changes to the VCF4 codec, including allele clipping down to the minimum reference allele; adding unit testing for certain aspects of the parsing. Not ready for prime-time yet.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3604 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-22 06:31:05 +00:00
bthomas de9f1f575f Fixing command line parsing to accept negative number arguments. Command line definitions must now start with a letter or underscore; previously, they could start with a digit.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3603 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-21 21:54:31 +00:00
bthomas 9d6a341d15 Fixing the error messages thrown with bad interval arguments. I simplified the exception handling and made the messages more verbose.
Note: the -L argument takes both interval strings and filenames. If you specify an interval string that is also a file, an error will be thrown to move the file: ie. if you have a file "chr1" in the parent directory, GATK will ask you to move/delete it. But, this only happens with interval string arguments, NOT with intervals that are contained in files, which is a majority of the use case. 



git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3602 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-21 21:49:41 +00:00
bthomas 300a18b85f Updating the way reference data is processed, so GATK creates the .fasta.fai and .dict files automatically. If either (or both) don't exist, GATK will create them in the same folder as the fasta file. If it can't write the file, GATK will fail with a message to create them manually.
Note that this functionality will only work if the directory with the fasta is writeable. GATK will fail if directory is read only and and either the .fasta.fai or .dict files don't exist. In the future, we could have these references be created in memory, but we decided against it this time. 

Locking was also added to ReferenceDataSource so no issues come up while running multiple GATKs on the same reference: we don't want one process to be half-finished and another try to read it. So, you could see error messages related to locking. See ReferenceDataSource.java for explanation of the locking strategy. 



git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3601 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-21 21:42:42 +00:00
ebanks df1cadc4c9 Fix NullPointerException when priority list is left out
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3600 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-21 13:46:54 +00:00
hanna c806ffba5f Switching over DownsamplingLocusIteratorByState -> LocusIteratorByState. Some operations
will not be as fast as they could be because the workflow is currently merge sam records (sharding)
-> split sam records (LocusIteratorByState) -> merge records (LocusIteraotorByState) -> split
records (StratifiedAlignmentContext), but this will be fixed when StratifiedAlignmentContext
is updated to take advantage of the new functionality in ReadBackedPileup.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3599 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-21 02:11:42 +00:00
hanna 1d50fc7087 Misc bug fixes: fix tracking of nInsertions with sample-split pileup constructor. Fix performance
issue building up pileups from pileups of individual sample data.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3598 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-20 20:32:27 +00:00
hanna f18ac069e2 A refactoring / unification of ReadBackedPileup and ReadBackedExtendedEventPileup.
Provides a cleaner interface with extended events inheriting all of the basic RBP
functionality.  Implementation is still slightly messy, but should allow users to
provide separate implementations of methods for sample split pileups and unsplit
pileups for efficiency's sake.
Methods not covered by unit/integration tests have not been sufficiently tested yet.
Unit tests will follow this week.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3597 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-20 04:42:26 +00:00
depristo 57a13805da GATK now uses a optimized indexing scheme in Tribble. 5x or more performance gain on files with many genotypes. Updated integrationtest that was failing and was clearly wrong. DB=; isn't a valid annotation.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3596 348d0f76-0448-11de-a6fe-93d51630548a
2010-06-19 21:36:41 +00:00