Commit Graph

10908 Commits (094db7bf2468352811764d4e98f38d637516a577)

Author SHA1 Message Date
Ryan Poplin 094db7bf24 We now require at least 10 samples to merge variants into complex events in the HC. Added a new population based bam for the complex event integration test. 2012-10-24 14:07:36 -04:00
Mark DePristo 6e421a72d6 Add more exhaustive unit tests for input errors to NanoScheduler
-- Resolves issue GSA-515 / Nanoscheduler GSA-605 / Seems that -nct may deadlock as not reproducible
-- It seems that it's not an input error problem (or at least cannot be provoked with unit tests)
-- I'll keep an eye on this later
2012-10-23 20:11:29 -04:00
Mark DePristo f838815343 Updating MD5s for confidence ref site estimation in IndependentAllelesDiploidExactAFCalc
-- Included logic to only add priors for alleles with sufficient evidence to be called polymorphic.  If no alleles are poly make sure to add priors of first allele
2012-10-23 06:47:53 -04:00
Mark DePristo 15b28e61cd Retiring TraverseReads and TraverseLoci after testing confirms nano scheduler version in single threaded version is fine
-- There's been no report of problems with the nano scheduled version of TraverseLoci and TraverseReads, so I'm removing the old versions since they are no longer needed
-- Removing unnecessary intermediate base classes
-- GSA-515 / Nanoscheduler GSA-549 / https://jira.broadinstitute.org/browse/GSA-549
2012-10-22 16:55:06 -04:00
Khalid Shakir fd59e7d5f6 Better error message when generic types are erased from scala collections. 2012-10-22 16:27:31 -04:00
Ryan Poplin 008df54575 Bug fix in GATKSAMRecord.getSoftEnd() for reads that are entirely clipped. 2012-10-22 14:21:52 -04:00
Mark DePristo 90f59803fd MaxAltAlleles now defaults to 6, no more MaxAltAllelesForIndels
-- Updated StandardCallerArgumentCollection to remove MaxAltAllelesForIndels. Previous argument is deprecated with meaningful doc message for people to use maxAltAlleles
-- All constructores, factory methods, and test builders and their users updated to provide just a single argument
-- Updating MD5s for integration tests that change due to genotyping more alleles
-- Adding more alleles to genotyping results in slight changes in the QUAL value for multi-allelic loci where one or more alleles aren't polymorphic.  That's simply due to the way that alternative hypotheses contribute as reference evidence against each true allele.  The effect can be large (new qual = old qual / 2 in one case here).
-- If we want more precision in our estimates we could decide (Eric, should we discuss?) to actually separately do a discovery phase in the genotyping, eliminate all variants not considered polymorphic, and then do a final round of calling to get the exact QUAL value for only those that are segregating.  This would have the value of having the QUAL stay constant as more alleles are genotyped, at the cost of some code complexity increase and runtime.  Might be worth it through
2012-10-22 13:47:56 -04:00
Khalid Shakir 97dc3664c9 Fixed yet another NPE related to the ArgumentTypeDescriptor vs. ArgumentMatchValue. Added integration test based on GSA-621. 2012-10-22 12:05:32 -04:00
Eric Banks ccae6a5b92 Fixed the RR bug I (knowingly) introduced last week: turns out we can't trust a context size's worth of data from the previous marking. I think Mauricio warned me about this but I forgot. 2012-10-22 11:48:34 -04:00
Mark DePristo 9f2851d769 Updating UnifiedGenotyperGeneralPloidyIntegrationTest following rebasing
-- Created a JIRA ticket https://jira.broadinstitute.org/browse/GSA-623 for Guillermo to look at the differences as the multi-allelic nature of many sites seems to change with the new more protected infrastructure.  This may be due to implementation issues in the pooled caller, problems with my interface, or could be a genuine improvement.
2012-10-21 20:23:11 -04:00
Mark DePristo eb6c9a1a79 Disable EfficiencyMonitoringThreadFactoryUnitTest
-- This is no longer a core GATK activity, and the tests need to run for so long (2 min each) that it's just too painful to run them.  Should be re-eabled if we come to care about this capability again, or if we can run these tests all in parallel in the future.
2012-10-21 12:43:46 -04:00
Mark DePristo 5296de8251 Fix UnifiedArgumentCollection constructor logic error
-- The old way of overloading constructors and calling super didn't work (might have been a consequence of merge).  This is the right way to do the copy constructor with the call to super()
2012-10-21 12:43:46 -04:00
Mark DePristo d21e42608a Updating integration tests for minor changes due to switching to EXACT_INDEPENDENT model by default 2012-10-21 12:43:46 -04:00
Mark DePristo 6b6caf8e3a Bugfix for indel DP calculations using reduced reads
-- Adding tests for SNP and indel calling on reduced BAM
2012-10-21 12:42:32 -04:00
Mark DePristo 0fcd358ace Original EXACT model implementation lives, providing another reference (bi-allelic only) EXACT model
-- Potentially a very fast implementation (it's very clean) but restricted to the biallelic case
-- A starting point for future bi-allelic only optimized (logless) or generalized (bi-allelic general ploidy) implementations
-- Added systematic unit tests covering this implementation, and comparing it to others
-- Uncovered a nasty normalization bug in StateTracker that was capping our likelihoods at 0, even after summing up multiple likelihoods, which is just not safe to do and was causing us to lose likelihood in some cases
-- Removed the restriction that a likelihood be <= 0 in StateTracker, and the protection for these cases in GeneralPloidyExactAFCalc which just wasn't right
2012-10-21 12:42:31 -04:00
Mark DePristo 0fb8274507 Fix contact on sorting of AFCalcResults
-- The thing that must be sorted is the pre-theta^N list, which is not checked in the routine that applies the theta^N prior.
2012-10-21 12:42:31 -04:00
Mark DePristo eaffb814d3 IndependentExactAFCalc is now the default EXACT model implementation
-- Changed UG / HC to use this one via the StandardCallerArgumentCollection
-- Update the AFCalcFactory.Calculation to have a getDefault() value instead of having a duplicate entry in the enums
2012-10-21 12:42:31 -04:00
Mark DePristo 326f429270 Bugfixes to make new AFCalc system pass integrationtests
-- GeneralPloidyExactAFCalc turns -Infinity values into -Double.MAX_VALUE, so our calculations pass unit tests
-- Bugfix for GeneralPloidyGenotypeLikelihoodsCalculationModel, return a null VC when the only allele we get from our final alleles to use method is the reference base
-- Fix calculation of reference posteriors when P(AF == 0) = 0.0 and P(AF == 0) = X for some meaningful value of X.  Added unit test to ensure this behavior is correct
-- Fix horrible sorting bug in IndependentAllelesDiploidExactAFCalc that applied the theta^N priors in the wrong order.  Add contract to ensure this doesn't ever happen again
-- Bugfix in GLBasedSampleSelector, where VCs without any polymorphic alleles were being sent to the exact model
--
2012-10-21 12:42:31 -04:00
Mark DePristo 695cf83675 More docs and contracts for classes in genotyper.afcalc
-- Future protection of the output of GeneralPloidyExactAFCalc, which produces in some cases bad likelihoods (positive values)
2012-10-21 12:42:31 -04:00
Mark DePristo 99c9031cb4 Merge AFCalcResultTracker into StateTracker, cleanup
-- These two classes were really the same, and now they are actually the same!
-- Cleanuped the interfaces, removed duplicate data
-- Added lots of contracts, some of which found numerical issues with GeneralPloidyExactAFCalc (which have been patched over but not fixed)
-- Moved goodProbability and goodProbabilityVector utilities to MathUtils.  Very useful for contracts!
2012-10-21 12:42:31 -04:00
Mark DePristo 9c63cee9fc Moving pnrm to UnifiedArgumentCollection so it's available with the HaplotypeCaller 2012-10-21 12:42:31 -04:00
Guillermo del Angel e9b7324dc1 Merge branch 'master' of ssh://gsa3/humgen/gsa-scr1/gsa-engineering/git/unstable 2012-10-21 12:38:49 -04:00
Guillermo del Angel 67b9e7319e Fix for integration tests: new criterion in AF exact calculation model to trim alleles based on likelihoods does produce better results and resulting alleles changed in 2 sites at integration tests (and all subsequent sites after this had minor annotation differences due to RankSum dithering) 2012-10-21 12:38:33 -04:00
Eric Banks 0616b98551 Not sure why we were setting the UAC variables instead of the simpleUAC ones when that's what we wanted. 2012-10-21 08:26:26 -04:00
Eric Banks d44d5b8275 Fix RawHapMapCodec so that it can build indexes. Minor fixes to VCF codec. 2012-10-21 01:29:59 -04:00
Eric Banks 841a906f21 Adding a hidden (for now) argument to UG (and HC) that tells the caller that the incoming samples are contaminated by N% and to fix it by aggressively down-sampling all alleles. This actually works. Yes, you read that right: given that we know what N is, we can make good calls on bams that have N% contamination. Only hooked up for SNPS right now. No tests added yet. 2012-10-20 23:31:56 -04:00
Eric Banks 2c624f76c8 Refactoring the Unified (and Standard) Argument Collections because it was really ugly that the subclass had to do all the cloning for the super class. The clone() method is really not recommended best practice in Java anyways, so I changed it so that we use standard overloaded constructors. Confirmed that the Haplotype Caller --help docs do not include UG-specific arguments. 2012-10-20 20:35:54 -04:00
Ryan Poplin a647f1e076 Refactoring the PairHMM util class to allow for multiple implementations which can be specified by the callers via an enum argument. Adding an optimized PairHMM implementation which caches per-read calculations as well as a logless implementation which drastically reduces the runtime of the HMM while also increasing the precision of the result. In the HaplotypeCaller we now lexicographically sort the haplotypes to take maximal benefit of the haplotype offset optimization which only recalculates the HMM matrices after the first differing base in the haplotype. Many thanks to Mauricio for all the initial groundwork for these optimizations. The change to the one HC integration test is in the fourth decimal of HaplotypeScore. 2012-10-20 16:38:18 -04:00
David Roazen 25e7dcc46f Performance tests: only need 1 iteration now that we're no longer running on the farm 2012-10-19 22:11:59 -04:00
Joel Thibault 45f64425a3 Update read metrics per shard rather than locus 2012-10-19 15:29:01 -04:00
Joel Thibault 637e0cf151 CountReads does not permit the use of output files 2012-10-19 15:29:01 -04:00
Joel Thibault a5333006bb Mark @Output as required 2012-10-19 15:29:01 -04:00
Yossi Farjoun 6557b7e364 Qscript for validating the pooledCaller against a single validation set 2012-10-19 14:39:57 -04:00
Khalid Shakir 2ef456d51a Added explicit @ClassType annotations to @Argument for Option[Int] or Option[Double] since scala seems to change the reflected type to Option[Object] on some systems.
Changed ReflectionUtils.getGenericTypes' order of looking for @ClassType since the primitive generic wasn't completely erased, only changed to Object which is incorrect.
More fixes to @Arguments labeled as java.io.File via incorrect @Input annotation.
Put in a default undocumented implementation of @Argument doc() to match the one added to @Input.
2012-10-19 13:20:29 -04:00
Eric Banks 4622896312 Oops, killed contracts 2012-10-19 13:04:05 -04:00
Eric Banks 9c088fe3fe Actually a better implementation of GATKSAMRecord.getSoftStart(). Last commit was all wrong. Oops. 2012-10-19 12:41:24 -04:00
Eric Banks f7bd4998fc No need for dummy GLs 2012-10-19 12:13:59 -04:00
Eric Banks f08e5a44da Better implementation of GATKSAMRecord.getSoftStart() 2012-10-19 12:11:18 -04:00
Eric Banks deca564aef Merge branch 'master' of ssh://gsa2/humgen/gsa-scr1/gsa-engineering/git/unstable 2012-10-19 12:01:49 -04:00
Eric Banks d3cf37dfaf Bug fix for general ploidy model: when choosing the most likely alternate allele(s), you need to weight the likelihood mass by the ploidy of the specific alleles (otherwise all alt alleles will have the same probability). This fixes Yossi's issue with pooled validation calling. This may brek integration tests, but I will leave that to GdA to handle. 2012-10-19 12:01:45 -04:00
Eric Banks 27d8d3f51e RR optimization: don't recalculate the entire bitset of variant sites for every read added to the sliding window. Instead, reuse as much of the previously calculated bitset as you can (basically from the window start until the start of the new read minus the context size). In some awfully performing regions this cuts down the runtime in half, although in others this doesn't seem to help much (so clearly something else is going on). Note that I still need to fix one last bug here, but it's almost done. 2012-10-19 11:59:34 -04:00
Khalid Shakir 403654d40a Fixed null checkes in ArgumentTypeDescriptor due to ArgumentMatchValue updates.
Fixed @Arguments such as scatter count that were labeled as java.io.File via incorrect @Input annotation.
2012-10-18 16:57:15 -04:00
Guillermo del Angel a4184716d8 Merge branch 'master' of ssh://gsa3/humgen/gsa-scr1/gsa-engineering/git/unstable 2012-10-18 15:42:31 -04:00
Guillermo del Angel 3db38c5a93 Bug fix: inbreeding coeff shouldn't be computed in ref-only sites 2012-10-18 15:42:14 -04:00
Ami Levy Moonshine a0381f15af Merge branch 'master' of ssh://gsa2.broadinstitute.org/humgen/gsa-scr1/gsa-engineering/git/unstable 2012-10-18 14:49:34 -04:00
Ami Levy Moonshine f665c6ceb9 minor chnage (add 129 to the dbSNP name 2012-10-18 14:48:37 -04:00
Ryan Poplin b4e69239dd In order to be considered an informative read in the PerReadAlleleLikelihoodMap it has to be informative compared to all other alleles not just the worst allele. Also, fixing a bug when there is only one allele in the map. 2012-10-18 14:31:15 -04:00
Mauricio Carneiro 3504f71b6b Fixing a null pointer exception bug for DEV-10 2012-10-18 13:58:38 -04:00
Ami Levy Moonshine 262c84a459 (1) Add RR step to the general calling script. \n (2) add more modularity to the script 2012-10-18 10:47:17 -04:00
Mark DePristo d3fc797cfe SelectVariants is actually *NOT* NanoSchedulable 2012-10-18 10:42:20 -04:00