Commit Graph

3345 Commits (9985f82a7af60dbbdf0c1343a02bd6b10ada218c)

Author SHA1 Message Date
David Roazen 9985f82a7a Move BaseUtils back to the GATK by request, along with associated utility methods 2013-01-30 13:09:44 -05:00
Mark DePristo 1ff78679ca UnitTesting example for copying
-- Example combinatorial unit tests, plus unit tests that create reads and bam files, pileups, variant context (from scratch and from a file), and genome locs
2013-01-30 11:19:08 -05:00
Eric Banks d067c7f136 Resolving merge conflicts 2013-01-30 10:47:59 -05:00
Eric Banks 9025567cb8 Refactoring the SimpleGenomeLoc into the now public utility UnvalidatingGenomeLoc and the RR-specific FinishedGenomeLoc.
Moved the merging utility methods into GenomeLoc and moved the unit tests around accordingly.
2013-01-30 10:45:29 -05:00
Mark DePristo 4852c7404e GenomeLocs are already comparable, so I'm removing the less complete GenomeLocComparator class and updating ReduceReads and CompressionStash to use built-in comparator 2013-01-30 10:12:27 -05:00
Mark DePristo 45603f58cd Refactoring and unit testing GenomeLocParser
-- Moved previously inner class to MRUCachingSAMSequenceDictionary, and unit test to 100% coverage
-- Fully document all functions in GenomeLocParser
-- Unit tests for things like parsePosition (shocking it wasn't tested!)
-- Removed function to specifically create GenomeLocs for VariantContexts.  The fact that you must incorporate END attributes in the context means that createGenomeLoc(Feature) works correctly
-- Depreciated (and moved functionality) of setStart, setStop, and incPos to GenomeLoc
-- Unit test coverage at like 80%, moving to 100% with next commit
2013-01-30 09:47:47 -05:00
Mark DePristo 8562bfaae1 Optimize GenomeLocParser.createGenomeLoc
-- The new version is roughly 2x faster than the previous version.  The key here was to cleanup the workflow for validateGenomeLoc and remove the now unnecessary synchronization blocks from the CachingSequencingDictionary, since these are now thread local variables
-- #resolves https://jira.broadinstitute.org/browse/GSA-724
2013-01-30 09:47:47 -05:00
Mark DePristo 69dd5cc902 AutoFormattingTimeUnitTest should be in utils 2013-01-30 09:47:47 -05:00
Mark DePristo 92c5635e19 Cleanup, document, and unit test ActiveRegion
-- All functions tested.  In the testing / review I discovered several bugs in the ActiveRegion routines that manipulate reads.  New version should be correct
-- Enforce correct ordering of supporting states in constructor
-- Enforce read ordering when adding reads to an active region in add
-- Fix bug in HaplotypeCaller map with new updating read spans.  Now get the full span before clipping down reads in map, so that variants are correctly placed w.r.t. the full reference sequence
-- Encapsulate isActive field with an accessor function
-- Make sure that all state lists are unmodifiable, and that the docs are clear about this
-- ActiveRegion equalsExceptReads is for testing only, so make it package protected
-- ActiveRegion.hardClipToRegion must resort reads as they can become out of order
-- Previous version of HC clipped reads but, due to clipping, these reads could no longer overlap the active region.  The old version of HC kept these reads, while the enforced contracts on the ActiveRegion detected this was a problem and those reads are removed.  Has a minor impact on PLs and RankSumTest values
-- Updating HaplotypeCaller MD5s to reflect changes to ActiveRegions read inclusion policy
2013-01-30 09:47:12 -05:00
David Roazen 6449c320b4 Fix the CachingIndexedFastaSequenceFileUnitTest
BaseUtils.convertIUPACtoN() no longer throws a UserException,
since it's in org.broadinstitute.variant
2013-01-29 21:07:16 -05:00
Mauricio Carneiro 29fd536c28 Updating licenses manually
Please check that your commit hook is properly pointing at ../../private/shell/pre-commit

Conflicts:
	public/java/test/org/broadinstitute/variant/VariantBaseTest.java
2013-01-29 17:27:53 -05:00
David Roazen a536e1da84 Move some VCF/VariantContext methods back to the GATK based on feedback
-Moved some of the more specialized / complex VariantContext and VCF utility
 methods back to the GATK.

-Due to this re-shuffling, was able to return things like the Pair class back
 to the GATK as well.
2013-01-29 16:56:55 -05:00
Ami Levy-Moonshine a1908a0eca Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-29 16:33:20 -05:00
Ami Levy-Moonshine 4aaef495c6 correct the help message 2013-01-29 16:33:12 -05:00
Ryan Poplin bf25196a0b Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-28 22:33:13 -05:00
Ryan Poplin e9c3a0acdf fix typo 2013-01-28 22:18:58 -05:00
Ami Levy-Moonshine a8a68697f1 Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-28 20:18:51 -05:00
Guillermo del Angel 5995f01a01 Big intermediate commit (mostly so that I don't have to go again through merge/rebase hell) in expanding BQSR capabilities. Far from done yet:
a) Add option to stratify CalibrateGenotypeLikelihoods by repeat - will add integration test in next push.
b) Simulator to produce BAM files with given error profile - for now only given SNP/indel error rate can be given. A bad context can be specified and if such context is present then error rate is increased to given value.
c) Rewrote RepeatLength covariate to do the right thing - not fully working yet, work in progress.
d) Additional experimental covariates to log repeat unit and combined repeat unit+length. Needs code refactoring/testing
2013-01-28 19:55:46 -05:00
Ami Levy-Moonshine 3f5c2e4989 Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-28 19:04:52 -05:00
Ami Levy-Moonshine c103623cf6 bug fix in my new function at SampleUtils.java 2013-01-28 19:04:39 -05:00
Ryan Poplin d665a8ba0c The Bayesian calculation of Qemp in the BQSR is now hierarchical. This fixes issues in which the covariate bins were very sparse and the prior estimate being used was the original quality score. This resulted in large correction factors for each covariate which breaks the equation. There is also now a new option, qlobalQScorePrior, which can be used to ignore the given (very high) quality scores and instead use this value as the prior. 2013-01-28 15:56:33 -05:00
Tad Jordan 8777e02aa5 R issue in Queue fixed.
GSA-721
2013-01-28 14:42:20 -05:00
David Roazen f63f27aa13 org.broadinstitute.variant refactor, part 2
-removed sting dependencies from test classes
-removed org.apache.log4j dependency
-misc cleanup
2013-01-28 09:03:46 -05:00
David Roazen 3744d1a596 Collapse the downsampling fork in the GATK engine
With LegacyLocusIteratorByState deleted, the legacy downsampling implementation
was already non-functional. This commit removes all remaining code in the
engine belonging to the legacy implementation.
2013-01-28 01:50:30 -05:00
Mark DePristo 63913d516f Add join call to Progress meter unit test so we actually know the daemon thread has finished 2013-01-27 16:52:45 -05:00
Mark DePristo 14d8afe413 Remove startSearchAt state variable from ActivityProfile
-- New algorithm will only try to create an active region if there's at least maxREgionSize + propagation distance states in the list.  When that's true, we are guaranteed to actually find a region.  So this algorithm is not only truly correct but as super fast, as we only ever do the search for the end of the region when we will certainly find one, and actually generate a region.
2013-01-27 14:10:08 -05:00
Mark DePristo c97a361b5d Added realistic BandPassFilterUnitTest that ensures quality results for 1000G phase I VCF and NA12878 VCF
-- Helped ID more bugs in the ActivityProfile, necessitating a new algorithm for popping off active regions.  This new algorithm requires that at least maxRegionSize + prob. propagation distance states have been examined.  This ensures that the incremental results are the same as you get reading in an entire profile and running getRegions on the full profile
-- TODO is to remove incremental search start algorithm, as this is no longer necessary, and nicely eliminates a state variable I was always uncomfortable with
2013-01-27 14:10:08 -05:00
Mark DePristo 72b2e77eed Linearize the findEndOfRegion algorithm in ActivityProfile, radically improving its performance
-- Previous algorithm was O(N^2)
-- #resolve GSA-723 https://jira.broadinstitute.org/browse/GSA-723
2013-01-27 14:10:06 -05:00
Mark DePristo 0fb238b61e TraverseActiveRegions Optimizations and Bugfixes: make sure to record position of current locus to discharge active regions when there's no data
-- Now records the position of the current locus, as well as that of the last read.  Necessary when passing through regions with no reads.  The previous version would keep accumulating empty active regions, and never discharge them until end of traversal (if there was no reads in the future) or until a read was finally found
-- Protected a call to logger.debug with if ( logger.isDebugEnabled()) to avoid a lot of overhead in writing unseen debugger logging information
2013-01-27 14:10:06 -05:00
Mark DePristo 93d88cdc68 Optimization: LocusReferenceView now passes along the contig index to createGenomeLoc, speeding up their creation
-- Also cleaned up some unused methods
2013-01-27 14:10:06 -05:00
Mark DePristo 52a28968a9 ART optimization: BandPassActivityProfile only applies the gaussian filter if the state probability > 0 2013-01-27 14:10:06 -05:00
Mauricio Carneiro 705cccaf63 Making SplitReads output FastQ's instead of BAM
- eliminates one step in my pipeline
   - BAM is too finicky and maintaining parameters that wouldn't be useful was becoming a headache, better avoided.
2013-01-27 02:36:31 -05:00
Mauricio Carneiro 6ea7133d95 Updating licenses of latest moved files 2013-01-26 13:46:52 -05:00
Ami Levy-Moonshine 99cb8d68e9 Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-25 16:07:38 -05:00
Mark DePristo b8c0b05785 Add contract to ensure that getAdapterBoundary returns the right result
-- Also renamed the function to getAdaptorBoundary for consistency across the codebase
2013-01-25 16:05:17 -05:00
Mark DePristo e445c71161 LIBS optimization for adapter clipping
-- GATKSAMRecords now cache the result of the getAdapterBoundary, allowing us to avoid repeating a lot of work in LIBS
-- Added unittests to cover adapter clipping
2013-01-25 16:05:17 -05:00
Ami Levy-Moonshine b4447cdca2 In cases where one uses VariantContextUtils.GenotypeMergeType.REQUIRE_UNIQUE we used to verify that the samples names are unique in VariantContextUtils.simpleMerge for each VCs. It couse to a bug that was reported on the forum (when a VCs had 2 VC from the same sample).
Now we will check it only in CombineVariants.init using the headers. A new function was added to SamplesUtils with unitTests in CVunitTest.java.
2013-01-25 15:49:51 -05:00
Ami Levy-Moonshine fc22a5c71c Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-25 11:47:38 -05:00
Ami Levy-Moonshine eaf6279d48 adding RBP to the general calling pipeline and few other small changes to it (to make it run with the current bundel file names 2013-01-25 11:47:30 -05:00
Mark DePristo 008b617577 Cleanup the getLIBS function in LocusIterator
-- Now throws an UnsupportedOperationException in the base class.  Only LocusView implements this function and actually returns the LIBS
2013-01-25 11:07:28 -05:00
Eric Banks 6dd0e1ddd6 Pulled out the --regenotype functionality from SelectVariants into its own tool: RegenotypeVariants.
This allows us to move SelectVariants into the public suite of tools now.
2013-01-25 09:42:04 -05:00
Mark DePristo c7a29b1d39 Fixed NPE in ActiveRegionUnitTest by allowing null supporting states in ActiveRegion 2013-01-24 13:48:00 -05:00
Mark DePristo 592f90aaef ActivityProfile now cuts intelligently at the best local minimum when in a larger than max size active region
-- This new algorithm is essential to properly handle activity profiles that have many large active regions generated from lots of dense variant events.  The new algorithm passes unit tests and passes visualize visual inspection of both running on 1000G and NA12878
-- Misc. commenting of the code
-- Updated ActiveRegionExtension to include a min active region size
-- Renamed ActiveRegionExtension to ActiveRegionTraversalParameters, as it carries more than just the traversal extension now
2013-01-24 13:48:00 -05:00
Mark DePristo c96b64973a Soft clip probability propagation is capped by the MAX_PROB_PROPAGATION_DISTANCE, which is 50 bp 2013-01-24 13:48:00 -05:00
Mark DePristo 0c94e3d96e Adaptively compute the band pass filter from the sigma, up to a maximum size of 50 bp
-- Previously we allowed band pass filter size to be specified along with the sigma.  But now that sigma is controllable from walkers and from the command line, we instead compute the filter size given the kernel from the sigma, including all kernel points with p > 1e-5 in the kernel.  This means that if you use a smaller kernel you get a small band size and therefore faster ART
-- Update, as discussed with Ryan, the sigma and band size to 17 bp for HC (default ART wide) and max band size of 50 bp
2013-01-24 13:47:59 -05:00
Mark DePristo 9e43a2028d Making band pass filter size, sigma, active region max size and extension all accessible from the command line 2013-01-24 13:47:59 -05:00
Mark DePristo cd91e365f4 Optimize getCurrentContigLength and getLocForOffset in ActivityProfile 2013-01-24 13:47:59 -05:00
Eric Banks 6790e103e0 Moving lots of walkers back from protected to public (along with several of the VA annotations).
Let's see whether Mauricio's automatic git hook really works!
2013-01-24 11:42:49 -05:00
Mark DePristo ee8039bf25 Fix trivial call in unit test 2013-01-23 13:51:58 -05:00
Mark DePristo 09edc6baeb TraverseActiveRegions now writes out very nice active region and activity profile IGV formatted files 2013-01-23 13:46:01 -05:00