Commit Graph

11751 Commits (b875ff4c8d88dbe16905d84b49e83cd19cbaa4c8)

Author SHA1 Message Date
Ami Levy-Moonshine b875ff4c8d Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-30 18:44:03 -05:00
Ami Levy-Moonshine 670bf1e094 few changes in the generalCallingPipeline 2013-01-30 18:43:53 -05:00
Ryan Poplin 438c98035b Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-30 17:12:28 -05:00
Ryan Poplin bb29bd7df7 Use base List and Map types in the HaplotypeCaller when possible. 2013-01-30 17:09:27 -05:00
Mark DePristo b707331332 Encrypt GATK AWS keys using the GATK private key, and decrypt as needed as a resource when uploading to AWS logs
-- Has the overall effect that the GATK user AWS keys are no longer visible in the gatk source as plain text.  This will stop AWS from emailing me (they crawl the web looking for keys)
-- Added utility EncryptAWSKeys that takes as command line arguments the GATK user AWS access and secret keys, encrypts them with the GATK private key, and writes out the resulting file to resources in phonehome.
-- GATKRunReport now decrypts as needed these keys using the GATK public key as resources in the GATK bundle
-- Refactored the essential function of Resource (reading the resource) from IOUtils into the class itself.  Now how to get the data in the resouce is straightforward
-- Refactored md5 calculation code from a byte[] into Utils.  Added unit tests
-- Committing the encrypted AWS keys
-- #resolves https://jira.broadinstitute.org/browse/GSA-730
2013-01-30 16:42:23 -05:00
Ryan Poplin 5f4a063def Breaking up my massive commits into smaller pieces that I can successfully merge and digest. This one enables downsampling in the HaplotypeCaller (by lowering the default dcov to 20) and removes my long-standing, temporary region-based downsampling. 2013-01-30 16:14:07 -05:00
David Roazen 591df2be44 Move additional VariantContext utility methods back to the GATK
Thanks to Eric for his feedback
2013-01-30 13:58:17 -05:00
Ryan Poplin ff8ba03249 Updating BQSR integration test md5s to reflect the updates to the hierarchicalBayesianQualityEstimate function 2013-01-30 13:30:18 -05:00
Ryan Poplin 85dabd321f Adding unit tests for hierarchicalBayesianQualityEstimate function 2013-01-30 13:26:07 -05:00
Ryan Poplin 07fe3dd1ef Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-30 13:19:24 -05:00
David Roazen 9985f82a7a Move BaseUtils back to the GATK by request, along with associated utility methods 2013-01-30 13:09:44 -05:00
Mark DePristo 51a322fb54 Remove FindContaminatingReadGroups walker and associated untested utils.NamedTable code 2013-01-30 12:51:50 -05:00
Ryan Poplin 2967776458 The Empirical quality column in the recalibration report can't be compared in the BQSRGatherer because the value is calculated using the Bayesian estimate with different priors. This value should never be used from a recalibration report anyway except during plotting. 2013-01-30 12:28:14 -05:00
Mark DePristo 1ff78679ca UnitTesting example for copying
-- Example combinatorial unit tests, plus unit tests that create reads and bam files, pileups, variant context (from scratch and from a file), and genome locs
2013-01-30 11:19:08 -05:00
Eric Banks d067c7f136 Resolving merge conflicts 2013-01-30 10:47:59 -05:00
Eric Banks 9025567cb8 Refactoring the SimpleGenomeLoc into the now public utility UnvalidatingGenomeLoc and the RR-specific FinishedGenomeLoc.
Moved the merging utility methods into GenomeLoc and moved the unit tests around accordingly.
2013-01-30 10:45:29 -05:00
Mark DePristo 4852c7404e GenomeLocs are already comparable, so I'm removing the less complete GenomeLocComparator class and updating ReduceReads and CompressionStash to use built-in comparator 2013-01-30 10:12:27 -05:00
Ryan Poplin 59311aeea2 Getting back null values from the tables is perfectly reasonable if those covariates don't appear in your table. Need to handle them gracefully. 2013-01-30 10:06:14 -05:00
Ryan Poplin e7d7d70247 Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-30 10:01:06 -05:00
Mark DePristo 45603f58cd Refactoring and unit testing GenomeLocParser
-- Moved previously inner class to MRUCachingSAMSequenceDictionary, and unit test to 100% coverage
-- Fully document all functions in GenomeLocParser
-- Unit tests for things like parsePosition (shocking it wasn't tested!)
-- Removed function to specifically create GenomeLocs for VariantContexts.  The fact that you must incorporate END attributes in the context means that createGenomeLoc(Feature) works correctly
-- Depreciated (and moved functionality) of setStart, setStop, and incPos to GenomeLoc
-- Unit test coverage at like 80%, moving to 100% with next commit
2013-01-30 09:47:47 -05:00
Mark DePristo 8562bfaae1 Optimize GenomeLocParser.createGenomeLoc
-- The new version is roughly 2x faster than the previous version.  The key here was to cleanup the workflow for validateGenomeLoc and remove the now unnecessary synchronization blocks from the CachingSequencingDictionary, since these are now thread local variables
-- #resolves https://jira.broadinstitute.org/browse/GSA-724
2013-01-30 09:47:47 -05:00
Mark DePristo 69dd5cc902 AutoFormattingTimeUnitTest should be in utils 2013-01-30 09:47:47 -05:00
Mark DePristo 92c5635e19 Cleanup, document, and unit test ActiveRegion
-- All functions tested.  In the testing / review I discovered several bugs in the ActiveRegion routines that manipulate reads.  New version should be correct
-- Enforce correct ordering of supporting states in constructor
-- Enforce read ordering when adding reads to an active region in add
-- Fix bug in HaplotypeCaller map with new updating read spans.  Now get the full span before clipping down reads in map, so that variants are correctly placed w.r.t. the full reference sequence
-- Encapsulate isActive field with an accessor function
-- Make sure that all state lists are unmodifiable, and that the docs are clear about this
-- ActiveRegion equalsExceptReads is for testing only, so make it package protected
-- ActiveRegion.hardClipToRegion must resort reads as they can become out of order
-- Previous version of HC clipped reads but, due to clipping, these reads could no longer overlap the active region.  The old version of HC kept these reads, while the enforced contracts on the ActiveRegion detected this was a problem and those reads are removed.  Has a minor impact on PLs and RankSumTest values
-- Updating HaplotypeCaller MD5s to reflect changes to ActiveRegions read inclusion policy
2013-01-30 09:47:12 -05:00
Mauricio Carneiro 3d9a83c759 BaseCoverageDistributions should be 'by reference'
otherwise we miss all the 0 coverage spots.
2013-01-29 22:37:44 -05:00
David Roazen e02dac03aa Fix use of fully-qualified package name in the HybridSelectionPipeline 2013-01-29 21:07:16 -05:00
David Roazen 6449c320b4 Fix the CachingIndexedFastaSequenceFileUnitTest
BaseUtils.convertIUPACtoN() no longer throws a UserException,
since it's in org.broadinstitute.variant
2013-01-29 21:07:16 -05:00
Mauricio Carneiro 29fd536c28 Updating licenses manually
Please check that your commit hook is properly pointing at ../../private/shell/pre-commit

Conflicts:
	public/java/test/org/broadinstitute/variant/VariantBaseTest.java
2013-01-29 17:27:53 -05:00
Mauricio Carneiro 0be5c66b61 Fixing return value of the license check script
- it was doing the right thing, but if the last statement didn't fail, bamboo wouldn't fail.
2013-01-29 17:26:39 -05:00
David Roazen a536e1da84 Move some VCF/VariantContext methods back to the GATK based on feedback
-Moved some of the more specialized / complex VariantContext and VCF utility
 methods back to the GATK.

-Due to this re-shuffling, was able to return things like the Pair class back
 to the GATK as well.
2013-01-29 16:56:55 -05:00
Mauricio Carneiro 581df64197 Updating Licensing scripts not to panic when removing files
- Licensing script was freaking out when a file was being removed from the repo
2013-01-29 16:47:53 -05:00
Ami Levy-Moonshine a1908a0eca Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-29 16:33:20 -05:00
Ami Levy-Moonshine 4aaef495c6 correct the help message 2013-01-29 16:33:12 -05:00
Eric Banks e4ec899a87 First pass at adding unit tests for the RR framework: I have added 3 tests and all 3 uncovered RR bugs!
One of the fixes was critical: SlidingWindow was not converting between global and relative positions correctly.
Besides not being correct, it was resulting in a massive slow down of the RR traversal.
That fix definitely breaks at least one of the integration tests, but it's not worth changing md5s now because I'll be
changing things all over RR for the next few days, so I am going to let that test fail indefinitely until I can confirm
general correctness of the tool.
2013-01-29 15:51:07 -05:00
Ryan Poplin cba89e98ad Refactoring the Bayesian empirical quality estimates to be in a single unit-testable function. 2013-01-29 15:50:46 -05:00
Guillermo del Angel 1d5b29e764 Unit tests for repeat covariates: generate 100 random reads consisting of tandem repeat units of random content and size, and check that covariates match expected values at all positions in reads.
Fixed corner case where value of covariate at border between 2 tandem repeats of different length/content wasn't consistent
2013-01-29 15:23:02 -05:00
Guillermo del Angel c11197e361 Refactored repeat covariates to eliminate duplicated code - now all inherit from basic RepeatCovariate abstract class. Comprehensive unit tests coming... 2013-01-29 10:10:24 -05:00
Ryan Poplin 35543b9cba updating BQSR integration test values for the PR half of BQSR. 2013-01-29 09:47:57 -05:00
Ryan Poplin bf25196a0b Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-28 22:33:13 -05:00
Ryan Poplin e9c3a0acdf fix typo 2013-01-28 22:18:58 -05:00
Ryan Poplin 1f254d29df Don't set the empirical quality when reading in the recal table because then we won't be using the new quality estimates for the prior since the value is cached. 2013-01-28 22:16:43 -05:00
Ami Levy-Moonshine a8a68697f1 Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-28 20:18:51 -05:00
Guillermo del Angel ff799cc79a Fixed bad merge 2013-01-28 20:04:25 -05:00
Guillermo del Angel 5995f01a01 Big intermediate commit (mostly so that I don't have to go again through merge/rebase hell) in expanding BQSR capabilities. Far from done yet:
a) Add option to stratify CalibrateGenotypeLikelihoods by repeat - will add integration test in next push.
b) Simulator to produce BAM files with given error profile - for now only given SNP/indel error rate can be given. A bad context can be specified and if such context is present then error rate is increased to given value.
c) Rewrote RepeatLength covariate to do the right thing - not fully working yet, work in progress.
d) Additional experimental covariates to log repeat unit and combined repeat unit+length. Needs code refactoring/testing
2013-01-28 19:55:46 -05:00
Ami Levy-Moonshine 3f5c2e4989 Merge branch 'master' of github.com:broadinstitute/gsa-unstable 2013-01-28 19:04:52 -05:00
Ami Levy-Moonshine c103623cf6 bug fix in my new function at SampleUtils.java 2013-01-28 19:04:39 -05:00
Mauricio Carneiro c2e580b252 Fixing SplitReads chopping unit tests
New contract for splitreads return the whole read if requested to chop more than or the exact size of the read now reflected in the unit test
2013-01-28 16:21:49 -05:00
Ryan Poplin d665a8ba0c The Bayesian calculation of Qemp in the BQSR is now hierarchical. This fixes issues in which the covariate bins were very sparse and the prior estimate being used was the original quality score. This resulted in large correction factors for each covariate which breaks the equation. There is also now a new option, qlobalQScorePrior, which can be used to ignore the given (very high) quality scores and instead use this value as the prior. 2013-01-28 15:56:33 -05:00
Tad Jordan 8777e02aa5 R issue in Queue fixed.
GSA-721
2013-01-28 14:42:20 -05:00
Ryan Poplin aab160372a No need to sort the BQSR tables by default. 2013-01-28 11:26:01 -05:00
David Roazen f63f27aa13 org.broadinstitute.variant refactor, part 2
-removed sting dependencies from test classes
-removed org.apache.log4j dependency
-misc cleanup
2013-01-28 09:03:46 -05:00