Commit Graph

1179 Commits (392b5cbcdfd5200f04d0b26f9e73d16399e17769)

Author SHA1 Message Date
Eric Banks 392b5cbcdf The CachingIndexedFastaSequenceFile now automatically converts IUPAC bases to Ns and errors out on other non-standard bases.
This way walkers won't see anything except the standard bases plus Ns in the reference.
Added option to turn off this feature (to maintain backwards compatibility).

As part of this commit I cleaned up the BaseUtils code by adding a Base enum and removing all of the static indexes for
each of the bases.  This uncovered a bug in the way the DepthOfCoverage walker counts deletions (it was counting Ns instead!) that isn't covered by tests.  Fortunately that walker is being deprecated soon...
2013-01-16 10:22:43 -05:00
Mark DePristo 3c37ea014b Retire original TraverseActiveRegion, leaving only the new optimized version
-- Required some updates to MD5s, which was unexpected, and will be sorted out later with more detailed unit tests
2013-01-15 10:24:45 -05:00
Mark DePristo 39bc9e999d Add a test to LocusIteratorByState to ensure that we aren't holding reads anywhere
-- Run an iterator with 100Ks of reads, each carrying MBs of byte[] data, through LIBS, all starting at the same position.  Will crash with an out-of-memory error if we're holding reads anywhere in the system.
-- Is there a better way to test this behavior?
2013-01-14 16:30:16 -05:00
Mark DePristo 5a5422e4f8 Refactor PerSampleReadStates into a separate class
-- No longer update the total counts in each per-sample state manager, but instead return delta counts that are updated by the overall ReadStateManager
-- One step on the way to improving the underlying representation of the data in PerSampleReadStateManager
-- Make LocusIteratorByState final
2013-01-14 16:30:16 -05:00
Mark DePristo 19288b007d LIBS bugfix: kept reads now only (correctly) includes reads that at least passed the reservoir
-- Added unit tests to ensure this behavior is correct
2013-01-14 16:30:16 -05:00
Mark DePristo 83fcc06e28 LIBS optimizations and performance tools
-- Made LIBSPerformance a full featured CommandLineProgram, and it can be used to assess the LIBS performance by reading a provided BAM
-- ReadStateManager now provides a clean interface to iterate in sample order the per-sample read states, allowing us to avoid many map.get calls
-- Moved updateReadStates to ReadStateManager
-- Removed the unnecessary wrapping of an iterator in ReadStateManager
-- readStatesBySample is now a LinkedHashMap so that iteration occurs in LIBS sample order, allowing us to avoid many unnecessary calls to map.get iterating over samples.  Now those are just map native iterations
-- Restructured collectPendingReads for simplicity, removing redundant and consolidating common range checks.  The new piece is code is much clearer and avoids several unnecessary function calls
2013-01-14 16:30:15 -05:00
Mark DePristo ec05ecef60 getAdaptorBoundary returns an int, not an Integer, as this was taking 30% of the allocation effort for LIBS 2013-01-14 16:30:15 -05:00
Mark DePristo e88dae2758 LocusIteratorByState operates natively on GATKSAMRecords now
-- Updated code to reflect this new typing
2013-01-11 15:17:18 -05:00
Mark DePristo 94cb50d3d6 Retire LegacyLocusIteratorByState
-- Left in the remaining infrastructure for David to remove, but the legacy downsampler is no longer a functional option in the GATK
2013-01-11 15:17:18 -05:00
Mark DePristo cc0c1b752a Delete old LocusIteratorByState, leaving only new LIBS and legacy 2013-01-11 15:17:18 -05:00
Mark DePristo bd03511e35 Updating AlignmentStateMachinePerformance to include some more useful performance assessments 2013-01-11 15:17:18 -05:00
Mark DePristo 9e23c592e6 ReadBackedPileup cleanup
-- Only ReadBackedPileupImpl (concrete class) and ReadBackedPileup (interface) live, moved all functionality of AbstractReadBackedPileup into the impl
-- ReadBackedPileupImpl was literally a shell class after we removed extended events.  A few bits of code cleanup and we reduced a bunch of class complexity in the gatk
-- ReadBackedPileups no longer accept pre-cached values (size, nMapQ reads, etc) but now lazy load these values as needed
-- Created optimized calculation routines to iterator over all of the reads in the pileup in whatever order is most efficient as well.
-- New LIBS no longer calculates size, n mapq, and n deletion reads while making pileups.
-- Added commons-collections for IteratorChain
2013-01-11 15:17:18 -05:00
Mark DePristo b9a33d3c66 Split original and optimized ART into largely independent pieces
-- Allows us to cleanly run old and new art, which now have different traversal behavior (on purpose).  Split unit tests as well.
2013-01-11 15:17:18 -05:00
Mark DePristo 02130dfde7 Cleanup ART
-- Initialize routine captures essential information for running the traversal
2013-01-11 15:17:17 -05:00
Mark DePristo 9b2be795a7 Initial working version of new ActiveRegionTraversal based on the LocusIteratorByState read stream
-- Implemented as a subclass of TraverseActiveRegions
-- Passes all unit tests
-- Will be very slow -- needs logical fixes
2013-01-11 15:17:17 -05:00
Mark DePristo 8b83f4d6c7 Near final cleanup of PileupElement
-- All functions documented and unit tested
-- New constructor interface
-- Cleanup some uses of old / removed functionality
2013-01-11 15:17:17 -05:00
Mark DePristo fb9eb3d4ee PileupElement and LIBS cleanup
-- function to create pileup elements in AlignmentStateMachine and LIBS
-- Cleanup pileup element constructors, directing users to LIBS.createPileupFromRead() that really does the right thing
2013-01-11 15:17:17 -05:00
Mark DePristo 2f2a592c8e Contracts and documentation for AlignmentStateMachine and LocusIteratorByState
-- Add more unit tests for both as well
2013-01-11 15:17:17 -05:00
Mark DePristo cc1d259cac Implement get Length and Bases of OfImmediatelyFollowingIndel in PileupElement
-- Added unit tests for this behavior.  Updated users of this code
2013-01-11 15:17:17 -05:00
Mark DePristo 2c38310868 Create LIBS using new AlignmentStateMachine infrastructure
-- Optimizations to AlignmentStateMachine
-- Properly count deletions.  Added unit test for counting routines
-- AlignmentStateMachine.java is no longer recursive
-- Traversals now use new LIBS, not the old one
2013-01-11 15:17:17 -05:00
Mark DePristo 80d9b7011c Complete rewrite of low-level machinery of LIBS, not hooked up
-- AlignmentStateMachine does what SAMRecordAlignmentState should really do.  It's correct in that it's more accurate than the LIB_position tests themselves.  This is a non-broken, correct implementation.  Needs cleanup, contracts, etc.
-- This version is like 6x slower than the original implementation (according to the google caliper benchmark here).  Obvious optimizations for future commit
2013-01-11 15:17:16 -05:00
Mark DePristo 0ac4352614 LIBS can now (optionally) track the unique reads it uses from the underlying read iterator
-- This capability is essential to provide an ordered set of used reads to downstream users of LIBS, such as ART, who want an efficient way to get the reads used in LIBS
-- Vastly expanded the multi-read, multi-sample LIBS unit tests to make sure this capability is working
-- Added createReadStream to ArtificialSAMUtils that makes it relatively easy to create multi-read, multi-sample read streams for testing
2013-01-11 15:17:16 -05:00
Mark DePristo b3ecfbfce8 Refactor LIBS into component parts, expand unit tests, some code cleanup
-- Split out all of the inner classes of LIBS into separate independent classes
-- Split / add unit tests for many of these components.
-- Radically expand unit tests for SAMRecordAlignmentState (the lowest level piece of code) making sure at least some of it works
-- No need to change unit tests or integration tests.  No change in functionality.
-- Added (currently disabled) code to track all submitted reads to LIBS, but this isn't accessible or tested
2013-01-11 15:17:16 -05:00
Mark DePristo 2e5d38fd0e Updating to latest google caliper code 2013-01-11 15:17:16 -05:00
Mark DePristo b2990497e2 Refactor LIBS into utils.locusiterator before refactoring 2013-01-11 15:17:16 -05:00
Mauricio Carneiro 2a4ccfe6fd Updated all JAVA file licenses accordingly
GSATDG-5
2013-01-10 17:06:41 -05:00
Eric Banks 4fa439d89e Move some classes back to public because they are used in the engine. Move some test classes to protected. We should have no more public->protected dependancies now 2013-01-09 11:06:10 -05:00
Eric Banks b099e2b4ae Moving integration tests to protected 2013-01-08 09:34:08 -05:00
Eric Banks 35d9bd377c Moved (nearly) all Walkers from public to protected and removed GATKLite utils 2013-01-07 14:42:40 -05:00
Eric Banks b4e7b3d691 Fixed precision problem in the Bayesian calculation of Qemp: we need to cap below max integer because the MathUtils code add +1.
Added unit tests for handling large number of observations.
2013-01-07 13:07:36 -05:00
Eric Banks ef638489d5 Fixing BQSR gatherer test to keep up to date with latest changes 2013-01-06 14:07:59 -05:00
Eric Banks 52067f0549 Handle merge conflicts 2013-01-06 12:29:12 -05:00
Eric Banks bf25e151ff Handle long->int precision in Bayesian estimate 2013-01-06 12:26:32 -05:00
Mark DePristo b403c269e9 Make multi-threaded progress meter daemon unit test more robust 2013-01-05 12:59:18 -05:00
Mark DePristo 69bf70c42e Cleanup and more unit tests for RecalibrationTables in BQSR
-- Added unit tests for combining RecalibrationTables.  As a side effect now has serious tests for incrementDatumOrPutIfNecessary
-- Removed unnecessary enum.index system from RecalibrationTables.
-- Moved what were really static utility methods out of RecalibrationEngine and into RecalUtils.
2013-01-05 12:50:27 -05:00
Chris Hartl 9df30880cb Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-04 17:15:22 -05:00
Joel Thibault 01738e70c3 Archive the experimental Active Region Traversals 2013-01-04 17:05:31 -05:00
Chris Hartl 41bc416b65 Remove AAL and update MD5s. 2013-01-04 16:46:14 -05:00
Eric Banks bce6fce58d Resolving merge conflicts after Mark's latest push 2013-01-04 14:46:39 -05:00
Eric Banks dd7f5e2be7 Hooking up the Bayesian estimate code for calculating Qemp in BQSR; various fixes after adding unit tests. 2013-01-04 14:43:11 -05:00
Joel Thibault ab5526b372 More TODOs 2013-01-04 14:09:02 -05:00
Tad Jordan fe06912a87 Removed sorting by row from walkers 2013-01-04 11:52:33 -05:00
Mark DePristo 810e2da1d4 Cleanup and unit tests for EventType and ReadRecalibrationInfo in BQSR
-- Added unit tests for EventType and ReadRecalibrationInfo
-- Simplified interface of EventType.  Previously this enum carried an index with it, but this is redundant with the enum.ordinal function.  Now just using that function instead.
2013-01-04 11:39:25 -05:00
Mark DePristo 1ba8d47a81 Unit tests for ProgressMeterDaemon 2013-01-04 11:39:24 -05:00
Mark DePristo fbee4c11f1 Unit tests for ProgressMeterData 2013-01-04 11:39:23 -05:00
Joel Thibault 319d651e4a Initial updates for ActiveRegionShard 2013-01-03 17:00:13 -05:00
Joel Thibault e7553545ef Initial updates for ReadShard 2013-01-03 17:00:13 -05:00
Joel Thibault 14a3ac0e3c Enable the use of alternate shards 2013-01-03 17:00:13 -05:00
Joel Thibault 47e620dfbc Create BAM index to test shard boundaries 2013-01-03 17:00:12 -05:00
Tad Jordan c1ba12d71a Added unit test for outputting sorted GATKReport Tables
- Made few small modifications to code
- Replaced the two arguments in GATKReportTable constructor with an enum used to specify way of sorting the table
2013-01-03 16:53:59 -05:00