Commit Graph

2098 Commits (39bc9e999d8215486708a77a8cff31b9084f7dca)

Author SHA1 Message Date
Mark DePristo b8b2b9b2de ManagingReferenceOrderedView optimization: don't allow a fresh RefMetaDataTracker in the frequent case where there's no reference meta data 2013-01-14 16:30:16 -05:00
Mark DePristo 7eea6b8f92 ReservoirDownsampler optimizations
-- Add an option to not allocate always ArrayLists of targetSampleSize, but rather the previous size + MARGIN.  This helps for LIBS as most of the time we don't need nearly so much space as we allow
-- consumeFinalizedItems returns an empty list if the reservior is empty, which it often true for our BAM files with low coverage
-- Allow empty sample lists for SamplePartitioner as these are used by the RefTraversals and other non-read based traversals

Make the reservoir downsampler use a linked list, rather than a fixed sized array list, in the expectFewOverflows case
2013-01-14 16:30:16 -05:00
Mark DePristo a4334a67e0 SamplePartitioner optimizations and bugfixes
-- Use a linked hash map instead of a hash map since we want to iterate through the map fairly often
-- Ensure that we call doneSubmittingReads before getting reads for samples.  This function call fell out before and since it wasn't enforced I only noticed the problem while writing comments
-- Don't make unnecessary calls to contains for map.  Just use get() and check that the result is null
-- Use a LinkedList in PassThroughDownsampler, since this is faster for add() than the existing ArrayList, and we were's using random access to any resulting
2013-01-14 16:30:16 -05:00
Mark DePristo e88dae2758 LocusIteratorByState operates natively on GATKSAMRecords now
-- Updated code to reflect this new typing
2013-01-11 15:17:18 -05:00
Mark DePristo 94cb50d3d6 Retire LegacyLocusIteratorByState
-- Left in the remaining infrastructure for David to remove, but the legacy downsampler is no longer a functional option in the GATK
2013-01-11 15:17:18 -05:00
Mark DePristo cc0c1b752a Delete old LocusIteratorByState, leaving only new LIBS and legacy 2013-01-11 15:17:18 -05:00
Mark DePristo 6a91902aa2 Fix final merge conflicts 2013-01-11 15:17:18 -05:00
Mark DePristo b9a33d3c66 Split original and optimized ART into largely independent pieces
-- Allows us to cleanly run old and new art, which now have different traversal behavior (on purpose).  Split unit tests as well.
2013-01-11 15:17:18 -05:00
Mark DePristo 02130dfde7 Cleanup ART
-- Initialize routine captures essential information for running the traversal
2013-01-11 15:17:17 -05:00
Mark DePristo 9b2be795a7 Initial working version of new ActiveRegionTraversal based on the LocusIteratorByState read stream
-- Implemented as a subclass of TraverseActiveRegions
-- Passes all unit tests
-- Will be very slow -- needs logical fixes
2013-01-11 15:17:17 -05:00
Mark DePristo 2c38310868 Create LIBS using new AlignmentStateMachine infrastructure
-- Optimizations to AlignmentStateMachine
-- Properly count deletions.  Added unit test for counting routines
-- AlignmentStateMachine.java is no longer recursive
-- Traversals now use new LIBS, not the old one
2013-01-11 15:17:17 -05:00
Mark DePristo 0ac4352614 LIBS can now (optionally) track the unique reads it uses from the underlying read iterator
-- This capability is essential to provide an ordered set of used reads to downstream users of LIBS, such as ART, who want an efficient way to get the reads used in LIBS
-- Vastly expanded the multi-read, multi-sample LIBS unit tests to make sure this capability is working
-- Added createReadStream to ArtificialSAMUtils that makes it relatively easy to create multi-read, multi-sample read streams for testing
2013-01-11 15:17:16 -05:00
Mark DePristo b3ecfbfce8 Refactor LIBS into component parts, expand unit tests, some code cleanup
-- Split out all of the inner classes of LIBS into separate independent classes
-- Split / add unit tests for many of these components.
-- Radically expand unit tests for SAMRecordAlignmentState (the lowest level piece of code) making sure at least some of it works
-- No need to change unit tests or integration tests.  No change in functionality.
-- Added (currently disabled) code to track all submitted reads to LIBS, but this isn't accessible or tested
2013-01-11 15:17:16 -05:00
Mark DePristo b2990497e2 Refactor LIBS into utils.locusiterator before refactoring 2013-01-11 15:17:16 -05:00
Mauricio Carneiro 9ed922d562 Updating licenses to Eric's last commit
- for now we're still running the script by hand, soon automated solution will be in place.

GSATDG-5
2013-01-11 14:33:00 -05:00
Eric Banks e7906713d9 Moving some random walkers back to public as requested by Mark. Mauricio will the licenses get updated automatically? 2013-01-11 02:03:43 -05:00
Ami Levy-Moonshine 352cb831d0 Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-10 21:27:06 -05:00
Ami Levy-Moonshine fac0bce916 add RunCoveredByNSamplesSites; changes in CoveredByNSamplesSites so it can work in parallel; also, move it to diagnostics 2013-01-10 21:26:49 -05:00
Mauricio Carneiro 2a4ccfe6fd Updated all JAVA file licenses accordingly
GSATDG-5
2013-01-10 17:06:41 -05:00
Eric Banks 4fa439d89e Move some classes back to public because they are used in the engine. Move some test classes to protected. We should have no more public->protected dependancies now 2013-01-09 11:06:10 -05:00
Eric Banks 676e79542a Bring CombineVariants back to public since it's used for SG. I needed to break ChromosomeCountConstants out of ChromosomeCounts to make this work. 2013-01-09 10:39:48 -05:00
Ami Levy-Moonshine 15ca5015cd Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-08 21:53:36 -05:00
Ami Levy-Moonshine d6071728e8 add new walker to find sites with good coverage 2013-01-08 17:10:38 -05:00
Eric Banks 264cc9e78d Resolve protected->public dependencies for BQSR by wrapping the BQSR-specific arguments in a new class.
Instead of the GATK Engine creating a new BaseRecalibrator (not clean), it just keeps track of the arguments (clean).

There are still some dependency issues, but it looks like they are related to Ami's code.  Need to look into it further.
2013-01-08 16:23:29 -05:00
Eric Banks f0bd1b5ae5 Okay, all public->protected dependencies are gone except for the BQSR arguments. I'll need to think through this but should be able to make that work too. 2013-01-08 15:46:32 -05:00
Eric Banks 35d9bd377c Moved (nearly) all Walkers from public to protected and removed GATKLite utils 2013-01-07 14:42:40 -05:00
Tad Jordan 04e3978b04 Fixed VariantEval tests
-Added sorting by rows to VariantEval
2013-01-07 12:45:32 -05:00
Ami Levy-Moonshine d3c2c97fb2 Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-06 23:35:47 -05:00
Ami Levy-Moonshine c554d9db25 add TODO 2013-01-06 23:04:38 -05:00
Ami Levy-Moonshine 81eef3aa37 merge development branchs of log-less HMM and FastGatherer to master 2013-01-06 23:01:58 -05:00
Eric Banks 0249e1f497 Resolving merge conflicts from VCF move 2013-01-06 14:32:31 -05:00
Eric Banks 8822b8e7c8 Moving HelpConstants out of HelpUtils so that we stop getting these ProgramElementDoc errors when com.sun.javadoc cannot load on a user's system. 2013-01-06 14:30:45 -05:00
Eric Banks 52067f0549 Handle merge conflicts 2013-01-06 12:29:12 -05:00
Eric Banks b73d72fe94 update docs for LEftAlignVariants 2013-01-06 01:56:57 -05:00
Mark DePristo 2ab55e4ee7 Fixing bug in TraverseDuplicates.printProgress call: only passes in single location of genome loc 2013-01-05 12:50:27 -05:00
Mark DePristo 69bf70c42e Cleanup and more unit tests for RecalibrationTables in BQSR
-- Added unit tests for combining RecalibrationTables.  As a side effect now has serious tests for incrementDatumOrPutIfNecessary
-- Removed unnecessary enum.index system from RecalibrationTables.
-- Moved what were really static utility methods out of RecalibrationEngine and into RecalUtils.
2013-01-05 12:50:27 -05:00
Chris Hartl 9df30880cb Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-04 17:15:22 -05:00
Joel Thibault 01738e70c3 Archive the experimental Active Region Traversals 2013-01-04 17:05:31 -05:00
Chris Hartl 7b7efa0fff Add in the AAL as an experimental covariate, in case it's wanted. 2013-01-04 16:47:26 -05:00
Chris Hartl 41bc416b65 Remove AAL and update MD5s. 2013-01-04 16:46:14 -05:00
Eric Banks bce6fce58d Resolving merge conflicts after Mark's latest push 2013-01-04 14:46:39 -05:00
Eric Banks dd7f5e2be7 Hooking up the Bayesian estimate code for calculating Qemp in BQSR; various fixes after adding unit tests. 2013-01-04 14:43:11 -05:00
Ami Levy-Moonshine 80b531f695 emit all sites where more than 90% of the samples have good coverage 2013-01-04 14:27:50 -05:00
Tad Jordan fe06912a87 Removed sorting by row from walkers 2013-01-04 11:52:33 -05:00
Mark DePristo 810e2da1d4 Cleanup and unit tests for EventType and ReadRecalibrationInfo in BQSR
-- Added unit tests for EventType and ReadRecalibrationInfo
-- Simplified interface of EventType.  Previously this enum carried an index with it, but this is redundant with the enum.ordinal function.  Now just using that function instead.
2013-01-04 11:39:25 -05:00
Mark DePristo a5901cdd20 Bugfix for printProgress in TraverseReadsNano
-- Must provide a single bp position (1:10) not the range of the read (1:1-50).  ProgressMeter now checks at runtime for this problem as well.
2013-01-04 11:39:24 -05:00
Mark DePristo bbdf9ee91b BQSR cleanup: merge Advanced and Standard recalibration engine into just the RecalibrationEngine
-- As we are no longer maintaining a public/protected system we need only have one RecalibrationEngine.
-- Misc. code cleanup and docs along the way
2013-01-04 11:39:24 -05:00
Mark DePristo 7df47418d8 BQSR optimization: make RecalibrationTables thread-local, and merge results in onTraversalDone
-- With the newer, faster BQSR, scaling was limited by the NestedIntegerArray.  The solution to this is to make the entire table thread-local, so that each nct thread has its own data and doesn't have any collisions.
-- Removed the previous partial solution of having a thread-local quality score table
-- Added a new argument -lowMemory
2013-01-04 11:39:24 -05:00
Joel Thibault 319d651e4a Initial updates for ActiveRegionShard 2013-01-03 17:00:13 -05:00
Joel Thibault e7553545ef Initial updates for ReadShard 2013-01-03 17:00:13 -05:00