Commit Graph

11734 Commits (d067c7f136142beca07b7bd2d539a4a4ff0b2061)

Author SHA1 Message Date
Eric Banks d067c7f136 Resolving merge conflicts 2013-01-30 10:47:59 -05:00
Eric Banks 9025567cb8 Refactoring the SimpleGenomeLoc into the now public utility UnvalidatingGenomeLoc and the RR-specific FinishedGenomeLoc.
Moved the merging utility methods into GenomeLoc and moved the unit tests around accordingly.
2013-01-30 10:45:29 -05:00
Mark DePristo 4852c7404e GenomeLocs are already comparable, so I'm removing the less complete GenomeLocComparator class and updating ReduceReads and CompressionStash to use built-in comparator 2013-01-30 10:12:27 -05:00
Mark DePristo 45603f58cd Refactoring and unit testing GenomeLocParser
-- Moved previously inner class to MRUCachingSAMSequenceDictionary, and unit test to 100% coverage
-- Fully document all functions in GenomeLocParser
-- Unit tests for things like parsePosition (shocking it wasn't tested!)
-- Removed function to specifically create GenomeLocs for VariantContexts.  The fact that you must incorporate END attributes in the context means that createGenomeLoc(Feature) works correctly
-- Depreciated (and moved functionality) of setStart, setStop, and incPos to GenomeLoc
-- Unit test coverage at like 80%, moving to 100% with next commit
2013-01-30 09:47:47 -05:00
Mark DePristo 8562bfaae1 Optimize GenomeLocParser.createGenomeLoc
-- The new version is roughly 2x faster than the previous version.  The key here was to cleanup the workflow for validateGenomeLoc and remove the now unnecessary synchronization blocks from the CachingSequencingDictionary, since these are now thread local variables
-- #resolves https://jira.broadinstitute.org/browse/GSA-724
2013-01-30 09:47:47 -05:00
Mark DePristo 69dd5cc902 AutoFormattingTimeUnitTest should be in utils 2013-01-30 09:47:47 -05:00
Mark DePristo 92c5635e19 Cleanup, document, and unit test ActiveRegion
-- All functions tested.  In the testing / review I discovered several bugs in the ActiveRegion routines that manipulate reads.  New version should be correct
-- Enforce correct ordering of supporting states in constructor
-- Enforce read ordering when adding reads to an active region in add
-- Fix bug in HaplotypeCaller map with new updating read spans.  Now get the full span before clipping down reads in map, so that variants are correctly placed w.r.t. the full reference sequence
-- Encapsulate isActive field with an accessor function
-- Make sure that all state lists are unmodifiable, and that the docs are clear about this
-- ActiveRegion equalsExceptReads is for testing only, so make it package protected
-- ActiveRegion.hardClipToRegion must resort reads as they can become out of order
-- Previous version of HC clipped reads but, due to clipping, these reads could no longer overlap the active region.  The old version of HC kept these reads, while the enforced contracts on the ActiveRegion detected this was a problem and those reads are removed.  Has a minor impact on PLs and RankSumTest values
-- Updating HaplotypeCaller MD5s to reflect changes to ActiveRegions read inclusion policy
2013-01-30 09:47:12 -05:00
Mauricio Carneiro 3d9a83c759 BaseCoverageDistributions should be 'by reference'
otherwise we miss all the 0 coverage spots.
2013-01-29 22:37:44 -05:00
David Roazen e02dac03aa Fix use of fully-qualified package name in the HybridSelectionPipeline 2013-01-29 21:07:16 -05:00
David Roazen 6449c320b4 Fix the CachingIndexedFastaSequenceFileUnitTest
BaseUtils.convertIUPACtoN() no longer throws a UserException,
since it's in org.broadinstitute.variant
2013-01-29 21:07:16 -05:00
Mauricio Carneiro 29fd536c28 Updating licenses manually
Please check that your commit hook is properly pointing at ../../private/shell/pre-commit

Conflicts:
	public/java/test/org/broadinstitute/variant/VariantBaseTest.java
2013-01-29 17:27:53 -05:00
Mauricio Carneiro 0be5c66b61 Fixing return value of the license check script
- it was doing the right thing, but if the last statement didn't fail, bamboo wouldn't fail.
2013-01-29 17:26:39 -05:00
David Roazen a536e1da84 Move some VCF/VariantContext methods back to the GATK based on feedback
-Moved some of the more specialized / complex VariantContext and VCF utility
 methods back to the GATK.

-Due to this re-shuffling, was able to return things like the Pair class back
 to the GATK as well.
2013-01-29 16:56:55 -05:00
Mauricio Carneiro 581df64197 Updating Licensing scripts not to panic when removing files
- Licensing script was freaking out when a file was being removed from the repo
2013-01-29 16:47:53 -05:00
Ami Levy-Moonshine a1908a0eca Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-29 16:33:20 -05:00
Ami Levy-Moonshine 4aaef495c6 correct the help message 2013-01-29 16:33:12 -05:00
Eric Banks e4ec899a87 First pass at adding unit tests for the RR framework: I have added 3 tests and all 3 uncovered RR bugs!
One of the fixes was critical: SlidingWindow was not converting between global and relative positions correctly.
Besides not being correct, it was resulting in a massive slow down of the RR traversal.
That fix definitely breaks at least one of the integration tests, but it's not worth changing md5s now because I'll be
changing things all over RR for the next few days, so I am going to let that test fail indefinitely until I can confirm
general correctness of the tool.
2013-01-29 15:51:07 -05:00
Guillermo del Angel 1d5b29e764 Unit tests for repeat covariates: generate 100 random reads consisting of tandem repeat units of random content and size, and check that covariates match expected values at all positions in reads.
Fixed corner case where value of covariate at border between 2 tandem repeats of different length/content wasn't consistent
2013-01-29 15:23:02 -05:00
Guillermo del Angel c11197e361 Refactored repeat covariates to eliminate duplicated code - now all inherit from basic RepeatCovariate abstract class. Comprehensive unit tests coming... 2013-01-29 10:10:24 -05:00
Ryan Poplin 35543b9cba updating BQSR integration test values for the PR half of BQSR. 2013-01-29 09:47:57 -05:00
Ryan Poplin bf25196a0b Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-28 22:33:13 -05:00
Ryan Poplin e9c3a0acdf fix typo 2013-01-28 22:18:58 -05:00
Ryan Poplin 1f254d29df Don't set the empirical quality when reading in the recal table because then we won't be using the new quality estimates for the prior since the value is cached. 2013-01-28 22:16:43 -05:00
Ami Levy-Moonshine a8a68697f1 Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-28 20:18:51 -05:00
Guillermo del Angel ff799cc79a Fixed bad merge 2013-01-28 20:04:25 -05:00
Guillermo del Angel 5995f01a01 Big intermediate commit (mostly so that I don't have to go again through merge/rebase hell) in expanding BQSR capabilities. Far from done yet:
a) Add option to stratify CalibrateGenotypeLikelihoods by repeat - will add integration test in next push.
b) Simulator to produce BAM files with given error profile - for now only given SNP/indel error rate can be given. A bad context can be specified and if such context is present then error rate is increased to given value.
c) Rewrote RepeatLength covariate to do the right thing - not fully working yet, work in progress.
d) Additional experimental covariates to log repeat unit and combined repeat unit+length. Needs code refactoring/testing
2013-01-28 19:55:46 -05:00
Ami Levy-Moonshine 3f5c2e4989 Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-28 19:04:52 -05:00
Ami Levy-Moonshine c103623cf6 bug fix in my new function at SampleUtils.java 2013-01-28 19:04:39 -05:00
Mauricio Carneiro c2e580b252 Fixing SplitReads chopping unit tests
New contract for splitreads return the whole read if requested to chop more than or the exact size of the read now reflected in the unit test
2013-01-28 16:21:49 -05:00
Ryan Poplin d665a8ba0c The Bayesian calculation of Qemp in the BQSR is now hierarchical. This fixes issues in which the covariate bins were very sparse and the prior estimate being used was the original quality score. This resulted in large correction factors for each covariate which breaks the equation. There is also now a new option, qlobalQScorePrior, which can be used to ignore the given (very high) quality scores and instead use this value as the prior. 2013-01-28 15:56:33 -05:00
Tad Jordan 8777e02aa5 R issue in Queue fixed.
GSA-721
2013-01-28 14:42:20 -05:00
Ryan Poplin aab160372a No need to sort the BQSR tables by default. 2013-01-28 11:26:01 -05:00
David Roazen f63f27aa13 org.broadinstitute.variant refactor, part 2
-removed sting dependencies from test classes
-removed org.apache.log4j dependency
-misc cleanup
2013-01-28 09:03:46 -05:00
David Roazen 1599c9a20e Remove the ability to package GATK/Queue Lite from the build system
Still need to reconfigure Bamboo to open-source protected, but this
can be done during the 2.4 release freeze.
2013-01-28 02:42:37 -05:00
David Roazen 3744d1a596 Collapse the downsampling fork in the GATK engine
With LegacyLocusIteratorByState deleted, the legacy downsampling implementation
was already non-functional. This commit removes all remaining code in the
engine belonging to the legacy implementation.
2013-01-28 01:50:30 -05:00
Mauricio Carneiro 5003deafb6 Fixing split-reads unit tests
The new implementation calls for the number of bases to chop, not the chop index anymore, so 0 is no longer appropriate.
2013-01-27 23:38:46 -05:00
Mauricio Carneiro 1aee8f205e Tool to calculate per base coverage distribution
GSATDG-29 #resolve
2013-01-27 23:38:46 -05:00
Mark DePristo 63913d516f Add join call to Progress meter unit test so we actually know the daemon thread has finished 2013-01-27 16:52:45 -05:00
Mark DePristo f5473285d5 Update CountReadsInActiveRegions md5 2013-01-27 14:35:55 -05:00
Mark DePristo 14d8afe413 Remove startSearchAt state variable from ActivityProfile
-- New algorithm will only try to create an active region if there's at least maxREgionSize + propagation distance states in the list.  When that's true, we are guaranteed to actually find a region.  So this algorithm is not only truly correct but as super fast, as we only ever do the search for the end of the region when we will certainly find one, and actually generate a region.
2013-01-27 14:10:08 -05:00
Mark DePristo c97a361b5d Added realistic BandPassFilterUnitTest that ensures quality results for 1000G phase I VCF and NA12878 VCF
-- Helped ID more bugs in the ActivityProfile, necessitating a new algorithm for popping off active regions.  This new algorithm requires that at least maxRegionSize + prob. propagation distance states have been examined.  This ensures that the incremental results are the same as you get reading in an entire profile and running getRegions on the full profile
-- TODO is to remove incremental search start algorithm, as this is no longer necessary, and nicely eliminates a state variable I was always uncomfortable with
2013-01-27 14:10:08 -05:00
Mark DePristo 72b2e77eed Linearize the findEndOfRegion algorithm in ActivityProfile, radically improving its performance
-- Previous algorithm was O(N^2)
-- #resolve GSA-723 https://jira.broadinstitute.org/browse/GSA-723
2013-01-27 14:10:06 -05:00
Mark DePristo 0fb238b61e TraverseActiveRegions Optimizations and Bugfixes: make sure to record position of current locus to discharge active regions when there's no data
-- Now records the position of the current locus, as well as that of the last read.  Necessary when passing through regions with no reads.  The previous version would keep accumulating empty active regions, and never discharge them until end of traversal (if there was no reads in the future) or until a read was finally found
-- Protected a call to logger.debug with if ( logger.isDebugEnabled()) to avoid a lot of overhead in writing unseen debugger logging information
2013-01-27 14:10:06 -05:00
Mark DePristo 804caf7a45 HaplotypeCaller Optimization: return a inactive (p = 0.0) activity if the context has no bases in the pileup
-- Allows us to avoid doing a lot of misc. work to set up the genotype when we don't have any data to genotype.  Valuable in the case where we are passing through large regions without any data
2013-01-27 14:10:06 -05:00
Mark DePristo 93d88cdc68 Optimization: LocusReferenceView now passes along the contig index to createGenomeLoc, speeding up their creation
-- Also cleaned up some unused methods
2013-01-27 14:10:06 -05:00
Mark DePristo 52a28968a9 ART optimization: BandPassActivityProfile only applies the gaussian filter if the state probability > 0 2013-01-27 14:10:06 -05:00
Mauricio Carneiro 705cccaf63 Making SplitReads output FastQ's instead of BAM
- eliminates one step in my pipeline
   - BAM is too finicky and maintaining parameters that wouldn't be useful was becoming a headache, better avoided.
2013-01-27 02:36:31 -05:00
Mauricio Carneiro ae38cf3f72 Adding read directionality to SplitReads
- directionality only influences 'chop' operation (since split will maintain all bases of the original read)
   - added directional unit test

GSATDG-25 #resolved
2013-01-26 22:25:56 -05:00
Mauricio Carneiro 6ea7133d95 Updating licenses of latest moved files 2013-01-26 13:46:52 -05:00
Mauricio Carneiro ef4cc742e5 Fixing the licensing scripts
- Fixed shell glob limitation that was failing license updates on big commits
	- Hook will now force user to re-commit after updating the licenses (pre-commit hook can't update and commit in the same process)
	- Moved all scripts to bash/zsh
	- Separated the license utilities in a separate python module to avoid copying code

GSATDG-28 #resolve
2013-01-26 13:42:49 -05:00