893c9c85faAdded previous optimization to diploid (non-pool) model and shaved off 20% of runtime from it. Moved out some common functionality to joint estimate parent class.
ebanks
2009-12-28 21:20:48 +0000
92e3682991Moved NHashMap to sting/utils
rpoplin
2009-12-28 20:57:32 +0000
562db45fa5Sites that were marked NO_DINUC no longer get dinuc-corrected but are still recalibrated using the other available covariates. Solid cycle is now the same as Illumina cycle pending an analysis that looks at the effect of PrimerRoundCovariate. Solid color space methods cleaned up to reduce number of calls to read.getAttribute(). Polished NHashMap sort method in preparation for move to core/utils. Added additional plots in AnalyzeCovariates to look at reported quality as a function of the covariate.
rpoplin
2009-12-28 20:19:37 +0000
2a704e83dfReads now have new traversal flag: generateExtendedEvents(). Support added to GenomeAnalysisEngine and Walker. This is a silent and transparent framework change that no existing code is going to see. The actual code that makes use of the new flag (which is false by default) will be committed separately...
asivache
2009-12-28 19:52:44 +0000
c8d0e6e004Optimization to pooled calculation model: stop calculating P(D|AF) if we are beyond the max likelihood such that subsequent likelihoods won't factor into the confidence score. Also, use new Pileup interface. Pooled calling now takes less than half the time it used to.
ebanks
2009-12-28 18:39:55 +0000
b1ac4b81d5Optimization: look up diploid genotypes from a static matrix instead of creating them on the fly (with String.format); bases no longer need to be ordered appropriately
ebanks
2009-12-28 17:28:51 +0000
57516582c2Converter from HapMap chip genotype data to VCF added; HapMapGenotypeROD adjusted to not convert from Hg18 to b36 formatting of contigs
andrewk
2009-12-28 01:36:08 +0000
d2770f380cWriting calls to standard out now works again (it got broken when we introduced parallelization)
ebanks
2009-12-27 04:36:45 +0000
0571d9dcb9Point MAX_QUAL_SCORE to SAMUtils.MAX_PHRED_SCORE. Also, array size for caches should be max score + 1.
ebanks
2009-12-24 20:47:32 +0000
438d21842aThe new recalibrator had been mimicking the behavior of the old one in that if there was no dinuc available (following a no-call base or at either end of a read), it didn't try to recalibrate. Now that Ryan has modularized the system, we no longer need to skip the base completely (we just need to skip the dinuc value)... which is good because the Picard people complained after realizing that cycle #1 never got recalibrated.
ebanks
2009-12-24 20:41:29 +0000
aaf674d9dbCleaned up this annotation. Still experimental. As of now, it's not useful. More analysis is needed to determine how to handle cases where UG is unsure whether a sample is het or hom.
ebanks
2009-12-24 03:06:46 +0000
2bd0b1bbf7After further review, it's unclear that my patch in RecalDataManager was the right choice. Reverting. Also updating other IntervalCleanerIntegrationTest failures that were masked by my first patch.
hanna
2009-12-24 00:32:33 +0000
98c268483eFixed issues with the integration tests: 1) sam-jdk apparently no longer supports custom tags with type int[] values. 2) BAM output for indel cleaner integration test changed in a way that's so subtle it can't be seen after converting the output to .sam.
hanna
2009-12-23 23:12:22 +0000
d1d6b79673Update picard public and private to match new SAM update.
hanna
2009-12-23 22:34:12 +0000
b134e0052fadded changes to the code to allow different types of interval merging,
aaron
2009-12-23 21:59:14 +0000
159778416cIn TableRecalibrationWalker, update UQ tag if it was present in the original SAMRecord. This required a new sam.jar, which caused some other files to need to be changed.
alecw
2009-12-23 21:42:36 +0000
87ff2b15d4First step in introducing a patch to Picard: create our ideal interface into the BAM file for sharding. This commit can iterate over the BAM file, pulling out information about the blocks in the file without actually loading or decompressing the reads.
hanna
2009-12-23 21:35:08 +0000
770093a40eOops - forgot to check this one in.
ebanks
2009-12-23 19:53:28 +0000
dc968798612 separate changes which both affect lots of UG integration md5s, so I'm committing them together:
ebanks
2009-12-23 19:03:56 +0000
872a9d1c7bI'm making this change now (as opposed to waiting until Monday) to honor Tim's request. The cycle covariate is now first/second of pair aware. I'm taking it on faith from both Chris Hartl (waiting on slides from him) and Tim that this is the right thing to do. We'll have Ryan confirm it all next week. The only change is that if a read is the second of a pair, we multiple the cycle by -1 (a simple way of separating its index from that of its mate). Of course, this broke all integration tests.
ebanks
2009-12-23 16:26:43 +0000
e29e8e52b9Multithreading support for the unified genotyper. Tests on a 10Mbase region on pilot 1 show a 6.8x improvement when running 8 ways parallel.
hanna
2009-12-23 00:48:06 +0000
164a94a3d0Modified the walker documentation so that the stray punctuation wouldn't cause the GATK to stop parsing the help documenation early (aka I changed one word).
kiran
2009-12-22 20:50:01 +0000
4ee6a478e3Creates a table of reference allele percentage and alternate allele percentage at Hapmap-chip sites in a BAM file.
kiran
2009-12-22 20:43:44 +0000
66eeb4a552Adding a package description for making a version of the GATK that only includes the FixBAMSortOrderTag walker.
aaron
2009-12-22 00:35:24 +0000
0d890e1bf0Rework Eric's output management code given that the behavior of the UG changes drastically depending on its output format. Current implementation is probably a bit overkill-ish and we can whittle this down to what's absolutely necessary. Writing VCFs to the 'out' protected printstream may not work at this moment.
hanna
2009-12-22 00:33:43 +0000
f448a263e9The cleaner now cleans duplicate reads (instead of ignoring them) - although it doesn't include them for scoring ref or alt consenses
ebanks
2009-12-21 21:01:55 +0000
cf303810d3VCF reader now creates the correct type of header line for each header type
ebanks
2009-12-21 20:39:06 +0000
c0c9f6c8b6Create package that contains just what Picard needs.
alecw
2009-12-21 18:48:27 +0000
e06dfe44c4Check for null platform (even when the read group isn't null) and assign it the default platform if it is
ebanks
2009-12-21 07:01:41 +0000
87e5a41964Fixed a bug that accounted for a bunch of my remaining mis-cleaned indels. Also, slightly optimized the cleaner by using readBases (instead of readString) and caching cigar element lengths.
ebanks
2009-12-21 05:46:16 +0000
b780ffb34aAdd a getFormat() method to get the output format from the writer. The need for this call suggests that I may be thinking about the typing of the GenotypeWriter object the wrong way.
hanna
2009-12-21 01:46:26 +0000
11cbfcec9cGet rid of backlink from ArgumentDefinitions to ArgumentSources. This will help in the future with multiple source -> single definition mapping sets.
hanna
2009-12-21 00:39:36 +0000
9e53c06328First revision of command-line argument support for GenotypeWriter. Also, fixed the damn build.
hanna
2009-12-20 19:19:23 +0000
566b556b50Give user ability to turn off max allowed interval size
ebanks
2009-12-20 03:20:22 +0000
a5f75cbfd4The previous commit broke the build, so this is a temporary patch to get it to compile. ConcordanceTruthTable should use enums (esp. now that all of the concordance variables need to be public), but VariantEval will need to be rewritten soon anyways so I'll just push it off until then.
ebanks
2009-12-20 02:34:41 +0000
ee8bcdc61dPooledConcordance calculations have been reformatted and bugs fixed. Now properly handles monomorphic sites. Also works with -G option now, correctly
depristo
2009-12-19 23:22:36 +0000
9bf2d12c64Misc. improvements to the LMW code. Support for emitting all sites, regardless of genotype. Min and max quality scores.
depristo
2009-12-19 23:20:57 +0000
7e0f69dab5Changed the GLF record to store it's contig name and position in each record instead of in the Reader. Integration tests all stay the same.
aaron
2009-12-18 22:54:56 +0000
80b3eb85faFixed curiously epic failure in read-backed pileup: size() mismatched the numReads-numDeletions at that locus in the case where includeReadsWithDeletionsAtLoci == false, causing failures including bad output from pileup walker. Also fixed up ValidatingPileup to run with the new ReadBackedPileup instead of just compiling successfully.
hanna
2009-12-18 22:52:44 +0000
fdf542c214The CycleCovariate for 454 data is now the TACG flow cycle. That is, each flow grabs all the T's, A's, C's, and G's in order in a single cycle. This is changed from incrementing the cycle whenever there is a discontinuous nucleotide along the direction of the read.
rpoplin
2009-12-18 22:39:51 +0000
c39675d2c1VCFTool.java got left off of the last commit
aaron
2009-12-18 21:33:53 +0000
4ea31fd949Pushed header initialization out of the GenotypeWriter constructors and into a writeHeader method, in preparation for parallelization.
ebanks
2009-12-18 19:16:41 +0000
eeddf0d08eAdding sample utils for convenience methods to pull out samples from e.g. SAMFileHeader or Genotype objects
ebanks
2009-12-18 18:51:21 +0000
79b997f43dMinor fix to getValue (thanks Ryan!)
chartl
2009-12-18 15:45:51 +0000
9971a8da9aadding a check to the RodVCF to ensure that records are in-order in the underlying VCF file.
aaron
2009-12-18 15:24:45 +0000
38563bbc2dThe values used to be integers (-1 for unpaired, 0 for unmapped, 1 for first, 2 for second); but i switched to strings before commit so it was more clear. Forgot to update the OTHER getValue method.
chartl
2009-12-18 15:05:14 +0000
7b5e332ff3Added - PairedQualityScoreCountsWalker: counts quality scores (e.g. as a histogram) on first reads of a pair and second reads of a pair. Turns out there's a consistent difference in quality scores; even after recalibrating without the pair ordering as a covariate (there's a bit of averaging -- but not as much as I initially thought).
chartl
2009-12-18 15:01:01 +0000
4f59bfd513Updates to the various GenotypeWriters to make them do simple things like write records (plus allow GLFReader to close). Adding first pass of stub and storage classes for the GenotypeWriters so that UG can be parallelizable. Not hooked up yet, so UG is unchanged. The mergeInto() code in the storage class is ugly, but it's all Tribble's fault. We can clean it up later if this whole thing works.
ebanks
2009-12-18 07:20:23 +0000
1cde4161b7Fixed another test
ebanks
2009-12-18 05:05:03 +0000
94f5edb68a1. Fixed VCFGenotypeRecord bug (it needs to emit fields in the order specified by the GenotypeFormatString) 2. isNoCall() added to Genotype interface so that we can distinguish between ref and no calls (all we had before was isVariant()) 3. Added Hardy-Weinberg annotation; still experimental - not working yet so don't use it. 4. Move 'output type' argument out of the UnifiedArgumentCollection and into the UnifiedGenotyper, in preparation for parallelization. 5. Improved some of the UG integration tests.
ebanks
2009-12-18 04:14:14 +0000
98839193b7compatibility with VCF lib's switch to GenomeLoc.
jmaguire
2009-12-18 00:52:48 +0000
8787dd4c5eVarious and sundry additions to VCF tools. Some useful to the general public, some one-offs.
jmaguire
2009-12-18 00:35:45 +0000
6fbf77be95Updating the two solid_recal_mode options to also change the previous base since solid aligner prefers single color mismatch alignments over true SNP alignments. COUNT_AS_MISMATCH mode has been removed completely. The default mode is now SET_Q_ZERO.
rpoplin
2009-12-17 20:07:26 +0000
07f1859290Added integration test for running the recalibrator with no index.
hanna
2009-12-17 19:10:53 +0000
c75ec67f84When called as a standalone, VariantAnnotator now emits samples in sorted (as opposed to random) order in VCFs.
ebanks
2009-12-17 19:01:08 +0000
aa86f3710dUpdating HomopolymerCovariate to only count the consecutive previous bases. I left in the code but commented out for if somebody wants to worry about carry forward homopolymer problems.
rpoplin
2009-12-17 18:25:09 +0000
d72d3322391) changed to search specifically for D and I cigar elements (and to process properly/ignore H,S,P elements) and print out only intervals that encompass actual indels. There's still one interval per read (at most) generated, which is the smallest intervals that covers ALL indels (D or I elements) present in the read; 2) if an interval (thus the original read itself and indels in it) sticks beyond the end of the chromosome, the read is ignored and this interval is NOT printed into the output; instead, a warning is printed to STDOUT (should we send it to logger.warn() instead?
asivache
2009-12-16 23:29:07 +0000
5b78354efdFixed NPE in index check with RefWalkers.
hanna
2009-12-16 22:37:45 +0000
e6127cd6c5Temporary hack for Tim Fennell: introduce a sharding strategy that stuffs all data into a single shard for cases when the index file isn't available. Works for the case in question, but is not guaranteed to work in general. Will be replaced once the new sharding system comes online.
hanna
2009-12-16 21:55:42 +0000
bb92e31118Optimizations: 1. push the ReadBackedPileup filtering up into the ReadFilters for read-based filters 2. stop querying the cigar for its length (just do it once)
ebanks
2009-12-16 21:39:58 +0000
36875fca89Update documentation in the new help system
andrewk
2009-12-16 21:33:12 +0000
ee47eb4367Make filters used available to the walker via getToolkit().
hanna
2009-12-16 21:26:04 +0000
4e7e0432a2Updated SNP calling power from coverage tools to work with new UnifiedGenotyper and DepthOfCoverage tools.
andrewk
2009-12-16 20:44:30 +0000
f5e547ed6eAdd ability for flat file table parsing module to skip ahead to first occurence of a regular expression (use case: consistently parsing DepthOfCoverage output for histogram section of file across file format changes)
andrewk
2009-12-16 20:38:50 +0000
b626fc0684Joint Estimate is now the default calculation model. Reworked all of the integration tests so that they're now more comprehensive, cover more of what we wan to test, and don't take forever to run.
ebanks
2009-12-16 19:41:02 +0000
bf76019f22Minor change to coverage evalution script, to update for new file format and add output fields
andrewk
2009-12-16 18:06:08 +0000
e051311e8cAdded convenience methods in RodVCF to pull out all of the VCF data from the VCFRecord (e.g. getID(), getSamples(), getInfoValues())
ebanks
2009-12-16 17:58:41 +0000
bb312814a2UG is now officially in the business of making good SNP calls (as opposed to being hyper-aggressive in its calls and expecting the end-user to filter). Bad/suspicious bases/reads (high mismatch rate, low MQ, low BQ, bad mates) are now filtered out by default (and not used for the annotations either), although this can all be turned off.
ebanks
2009-12-16 17:28:09 +0000
af440943a4Fixing a bug that Steven uncovered; we had an abigous contract for peek() in PushbackIterator, and SeekableRODIterator wasn't checking to see if it's PushbackIterator hasNext() was true before calling peek().
aaron
2009-12-15 23:04:40 +0000
1035abc85fAdd minimum base quality thresholding to depth of coverage via getBaseAndMappingFilteredPileup
andrewk
2009-12-15 22:58:30 +0000
4148991d81Now also encodes amino acids, includes documentation.
sjia
2009-12-15 20:26:56 +0000
9b0bdbbf29Fix for homopolymer bug: ref was lowercase, alt allele was uppercase, so alt != ref. Yuck. This is a temporary fix - pushed more elegant solution over to Matt.
ebanks
2009-12-15 19:02:23 +0000
a810586418Check-in without javadoc = smackdown
depristo
2009-12-15 15:32:39 +0000
b234019cf5Readded locus printing suppression to DoC walker (and removed unused import from UG)
ebanks
2009-12-15 14:50:56 +0000
0d2a761460Bugfix for minBaseQuality to ignore deletion reads. LocusMismatch walker now allows us to skip every nths eligable site
depristo
2009-12-15 14:38:39 +0000
bf7bab754eMade getPileupWithoutMappingQualityZeroReads() and getPileupWithoutDeletions() more efficient, per Mark's cue.
ebanks
2009-12-15 04:35:21 +0000
874552ff75Pull the genotype (and genotype quality) calculation out of the VCF code and into the Genotyper. [Also, enable Mark's new UG arguments]
ebanks
2009-12-15 04:29:28 +0000
2cbc85cc7amin mapping quality and min base quality arguments for UG
depristo
2009-12-15 03:57:27 +0000
rpoplin
ebanks
asivache
andrewk
hanna
aaron
alecw
kiran
depristo
chartl
jmaguire
sjia