11f91b3c95Reverting Eric's previous change because it killed the PG tag in the output bam file header. Added a new -compress command line argument to set the compression level of the output bam file.
rpoplin
2010-01-11 19:02:56 +0000
62dd2fa5beFixing another bug in solid recal regarding negative strand reads. The isInconsistentColorSpace method incorrectly used the inconsistent tag added by parseColorSpace, the inconsistent tag is in the direction of the read like the color space tag, and not in the direction of the reference like everything else. This affects the recalibrated quality scores but the improvment in SNP calling performance is minor when using the default UG settings (min base quality 10).
rpoplin
2010-01-11 14:28:52 +0000
971834ca90Added a walker to the vcf tools compilation: one that combines vcf records. Both merges and unions are supported (see documentation... when it gets written this week). Also, moved some code that pulls samples out of rods from VCFUtils into SampleUtils.
ebanks
2010-01-10 06:45:11 +0000
80af0f2f54Changed the OUTPUT_BAM_FILE argument from String to SAMFileWriter and removed the call to close().
ebanks
2010-01-10 03:45:54 +0000
7893aaefe9Updates to chunk iteration. Includes the return of the dreaded *2.java files; hopefully I can find a way to kill these off before the Picard patch is ready.
hanna
2010-01-08 20:20:56 +0000
fcce77c245Added -beagle option to emit likelihoods file for use with the BEAGLE imputation engine; still experimental. (Also converted getPileup -> getBasePileup)
ebanks
2010-01-08 18:41:04 +0000
9cbae53ee1Bug fixes for both SET_Q_ZERO and REMOVE_REF_BIAS solid recal modes regarding proper handling of negative strand reads. These changes yield a minor improvment in HapMap sensitivity.
rpoplin
2010-01-08 15:19:22 +0000
dfcd5ce25bFixed broken test
ebanks
2010-01-08 06:13:01 +0000
d5ab002449Curiously, it seems I never set the default base quality used by the Genotyper to 10. It's done now.
ebanks
2010-01-08 06:02:01 +0000
b468369dfa-UG's call into VariantAnnotator now uses the full alignment context (as opposed to the filtered one) -MQ0 annotation is now standard again -Added AC and AN annotations to VCF output
ebanks
2010-01-08 05:40:42 +0000
f587ff46afTile is now a standard covariate. By default the TileCovariate returns -1 if tile can't be derived from the read's name. Added a new command line option -throwTileException which will force TileCovariate to throw an exception if tile can't be derived for a read. Singleton covariates, such as any read group without tile info, must be skipped over in TableRecalibration so that the sequential formulation doesn't apply the same correction more than once. TileCovariate class has been added to the Early Access package.
rpoplin
2010-01-07 22:51:41 +0000
d01bde36a4Make sure that reference view holds enough bases to pass full-length deleted sequence to the walker's map() function in extended event mode (this addresses the problem of a deletion crossing the shard's boundary, so that an attempt to extract deleted bases results in a crash)
asivache
2010-01-07 22:37:22 +0000
e9bc85c188Now has methods that allow to 1) check if a location is within the bounds of the reference view; 2) expand reference view (i.e. expand the bounds and reload the reference sequence) in order to accomodate specified location. The second method can be called directly since it performs a check and if the location is already within the bounds, then returns immediately. The costly ref sequence reloading occurs only when the location is not fully contained within the current bounds.
asivache
2010-01-07 22:35:17 +0000
7f91b4d824Bug fix. It would be nice if we could extract ROD annotations for the whole length of an extended event (indel), and we tried... But alas, it does not work with the current ROD system (after extracting length on ref > 1 ROD data for a deletion, rod iterator crashes on the attempt to re-load annotations for next reference base)
asivache
2010-01-07 21:30:55 +0000
5f58492401A rogue QualityUtils.MAX_REASONABLE_Q_SCORE managed to get through my previous bug fix. It should instead check the command line -maxQ argument.
rpoplin
2010-01-07 21:17:39 +0000
c7a8dffa89Check for division by 0 in annotations
ebanks
2010-01-07 19:27:15 +0000
9a658e6b18-Fixed VCF header line bug -Added useful trim() method for Strings for characters other than whitespace
ebanks
2010-01-07 17:51:41 +0000
b643a513bbMinor interface change for VCFGenotypeRecord.
ebanks
2010-01-07 16:48:09 +0000
431e9c2c8bAdd dbSNP ID to VCF output records
andrewk
2010-01-07 15:30:04 +0000
076481f786Fixes to mergeVCF -- now correctly supports merging of filter fields. Also removed incorrect hasFilteringCodes() function. Updated intergration tests
depristo
2010-01-07 14:50:13 +0000
cea544871dFixed an issue with recalibrating original quality scores above Q40. There is a new option -maxQ which sets the maximum quality score possible for when a RecalDatum tries to compute its quality score from the mismatch rate. The same option was added to AnalyzeCovariates to help with plotting q scores above Q40. Added an integration test which makes use of this new -maxQ option.
rpoplin
2010-01-07 13:50:30 +0000
6c739e30e01. Removing an old version of the Genotype interface which is no longer being used. Needed to do this now so that the naming conflicts would cease.
ebanks
2010-01-07 05:51:10 +0000
a9245a58e2Fix for incorrect exception throwing in VCFRecord. It is reasonable to ask for the non-ref allele freq at all ref sites. Was only passing in tests because isReference was broken
depristo
2010-01-07 01:18:30 +0000
7215526810Fix to isReference() in VCFRecord. Change to VariantCounter to correctly counter only non-genotype variants, as well as update to VariantEvalWalker
depristo
2010-01-07 00:03:29 +0000
6c4ac9e663Updated HapMap2VCF to use the VCFGenotypeWriterAdapter interface; fixed bug in VCFParameters that affects VariantsToVCF and HapMap2VCF when reference is lower-cased; added integration test for HapMap2VCF that checks for the lower-case issue by testing against Hg18 region that has lower-cased bases
andrewk
2010-01-06 21:27:11 +0000
576594eda2clean-up of the GATK paper genotyper, and better output formatting for the simple call format we emit.
aaron
2010-01-06 20:54:56 +0000
7e3e714d3cMoving experimental annotations from core to oneoffs
chartl
2010-01-06 19:34:10 +0000
a32245f7d2Modifications: QualityUtils - Stole the BaseUtils code for flipping reads around and applied it to quality scores SecondBaseSkew - Nothing's really different, just a commented line
chartl
2010-01-06 19:18:07 +0000
eb899741e1reverting last changes. no cacheing
asivache
2010-01-06 18:59:37 +0000
a17d725c35Cache pileup bases and mapping quals after first call to getBases() and getMappingQuals(), respectively. Subsequent calls to these method will return cached arrays.
asivache
2010-01-06 18:05:00 +0000
d6fb19bb67Don't hard-code base qual max
ebanks
2010-01-06 17:21:44 +0000
75809100c6Use inheritance so that shared code isn't duplicated between the RecalDatums
rpoplin
2010-01-06 16:45:16 +0000
dfe160ff77Minor changes (additional info calculated)
chartl
2010-01-06 16:41:01 +0000
fdd14e1a01Proposed interface for VariantContext. It's currently an interface so it doesn't break the build...
ebanks
2010-01-06 16:31:39 +0000
e011a1b6f8Cut the memory footprint of the RecalDatum in half to improve performance of CountCovariates when run with many covariates.
rpoplin
2010-01-06 16:12:27 +0000
370a365147Small runtime improvement in TableRecalibration.
rpoplin
2010-01-06 15:51:12 +0000
b745c2f8d7Fix for Jared: don't blow up if there are no samples in the input (since that's allowed) - but warn the user just in case.
ebanks
2010-01-06 15:37:06 +0000
090212c1c2Home for retired duplicateQuals analysis utility
depristo
2010-01-06 15:05:00 +0000
1e462419datrivial code restructuing, and commented out failed attempt to support sample selection with VCF. VariantEval2 go go go
depristo
2010-01-06 15:04:27 +0000
f857159343useful convenience function to get a genotype associated with a particular sample
depristo
2010-01-06 15:03:07 +0000
34519b3e3bBetter printing support for false positives and false negatives in concordance tables
depristo
2010-01-06 15:02:40 +0000
1c90e6a954More informative error message in AnalyzeCovariates and cleanup
rpoplin
2010-01-06 14:56:29 +0000
bca3d1b943useful convenience function to get a genotype associated with a particular sample
depristo
2010-01-06 14:53:56 +0000
ec774f62beSome checking to protect the BasicGenotype
depristo
2010-01-06 14:53:24 +0000
71ecbe75d7AnalyzeCovariates would crash with 'too many open files' exception when spawning Rscript jobs for every read group at once. It now waits for some to finish before spawning the rest.
rpoplin
2010-01-06 14:19:02 +0000
588006ee92Now supports strings in command line for farm submission
depristo
2010-01-06 13:15:40 +0000
21a50eedb5Simple extension to VariantEval: --includeFilteredRecords will now keep filtered VCF records so you can see what the entire call set looks like. Looking forward to VariantEval v2 from Kiran.
depristo
2010-01-06 12:59:09 +0000
8d13597a27Temporary command-line support to enable rod walkers, if you know what you are doing this is safe.
depristo
2010-01-06 12:15:36 +0000
d8351cb9fcGive Annotations access to rod data.
ebanks
2010-01-05 18:53:01 +0000
331d6db783Reduce file handle usage.
hanna
2010-01-05 18:03:01 +0000
8b087305f3Added back the MQ0 annotation - however, it's not yet standard (since mq0 reads are filtered out by default in the genotyper). But it'll work when using the Annotator as a standalone. While I'm at it, change getPileup to getBasePileup to remove all of the deprecation warnings.
ebanks
2010-01-05 17:07:19 +0000
a4b69d0adfMisc bug fixes.
hanna
2010-01-05 14:48:19 +0000
9fb6533549new -a option does fast merging of already sorted files
depristo
2010-01-05 13:55:39 +0000
0a6bd5a270CycleCovariate is now one-based so that 0 and -0 don't collide with each other. Solid recal modes now only change the inconsistent base and the previous base (along the direction of the read) instead of both the bases before and after. Removed estimatedNumberOfBins from the Covariate interface because it wasn't being used.
rpoplin
2010-01-04 20:52:15 +0000
ed2fff13aa-Misc improvements to VCF code -Small fix to callset concordance
ebanks
2010-01-04 02:28:47 +0000
29c129acedAdded very primitive read fishing walker with lots of hard coding. Fixed bugs encountered when testing read fishing in Ecoli.
hanna
2010-01-04 00:54:57 +0000
7b702b086fYou don't need to be bi-allelic to have a non-ref alt allele frequnecy, but you do have to be a variant.
ebanks
2010-01-03 22:02:39 +0000
b668d32cf1Updated the min mapping quality and min base quality defaults to be 10 in both cases (and updated all integration tests) as suggested by Mark.
ebanks
2010-01-03 21:31:04 +0000
b6ecc9e151Support for ad-hoc reference sequences. Also reenabled BWA/Java integration test, which was commented out and the data backing it up deleted without my knowledge. Unfortunately, since the data was deleted, I had to regenerate the data and a new md5. Hopefully the aligner output is still correct.
hanna
2010-01-02 20:19:14 +0000
ad549eacfdNow that we changed how deletions are represented, got to update MD5...
asivache
2009-12-30 22:00:58 +0000
46362ce532In extended event lines, now prints deletions in verbose format as well (e.g. "-AAT")
asivache
2009-12-30 21:57:20 +0000
a18e31f5b8If alignment context at the locus holds extended event, get rod metadata and (importantly) reference bases for the whole span of the event (if it is a deletion that is, insertions still have length 0 on the ref!)
asivache
2009-12-30 21:56:25 +0000
a41cb0701bNow can generate verbose String representation of deletions (e.g. "-AAT") if reference bases are provided as an argument to getEventStringWithCounts().
asivache
2009-12-30 21:54:50 +0000
89791d730eCompute and cache the length of the longest deletion observed at the site; ReadBackedExtendedEventPileup now has a getter to access that value.
asivache
2009-12-30 21:19:39 +0000
9c41ac252fDisable testSingleBPFailure - getReferenceContext() now whould agree to accept length > 1 genome locs as its argument, so there's nothing to test...
asivache
2009-12-30 21:12:00 +0000
8932e67325Removed sanity check that required GenomeLoc argument to be strictly 1-base long. We need to relax this in order to be able to pass around a reference context containing full-length chunk of deleted reference bases
asivache
2009-12-30 20:14:08 +0000
497ae700c4A rethink of the existing BAM block extraction code: rather than working in chunk space directly, stream data in block space, converting to chunk space on demand.
hanna
2009-12-30 18:19:51 +0000
80658fd99eAnalyzeCovariates gets the same performance improvements as the recalibrator. NHashMap class is removed completely.
rpoplin
2009-12-30 18:10:10 +0000
9b2733a54aMisc clean up in the recalibrator related to the nested hash map implementation. CountCovariates no longer creates the full flattened set of keys and iterates over them. The output csv file is in sorted order by default now but there is a new option -unsorted which can be used to save a little bit of run time.
rpoplin
2009-12-30 16:58:04 +0000
4aeb50c87dAdded: integration test for extended pileup (with indels included)
asivache
2009-12-29 23:02:23 +0000
c928347c0cExtended event pileups are more verbose now: following a sequence of 'D','I', and '.' symbols, actual distinct events are listed along with their counts (example: +AAA:3,+AAC:1 for the total of 4 indel observations with 3 reads showing +AAA and one read showing +AAC)
asivache
2009-12-29 22:44:18 +0000
8330058216method added: getEventStringsWithCounts() Returns list of Pairs <String,Integer>, where each pair consists of a unique indel event observed at the site and the total number of observations of that event. String representation for insertions is verbose (e.g. +ACT), while deletions are represented as "5D" (since read backed pileup has no reference information, so we can not get actual sequence of deleted bases)
asivache
2009-12-29 22:41:58 +0000
cf3e59eb4aback to archive
asivache
2009-12-29 22:00:38 +0000
295d16572esynch; will go back to archive in a sec
asivache
2009-12-29 22:00:03 +0000
e286313b67Fix for reads that have insertion as their last (mapped) cigar elements (i.e. not followed by M)
asivache
2009-12-29 21:13:16 +0000
05deb8796bSimplify handling of reference sequence for unmapped reads. Improvement made based on a suggestion from Alec.
hanna
2009-12-29 21:06:20 +0000
96c4929b3cRecalibrator now uses NestedHashMap instead of NHashMap. The keys are now nested hash maps instead of Lists of Comparables. These results in a big speed up (thanks Tim!). There is still a little bit of clean up to do, but everything works now.
rpoplin
2009-12-29 21:01:32 +0000
7826e144a1forgot to update md5s
depristo
2009-12-29 20:31:29 +0000
bfd6bf9ec5PileupWalker just got a new option: --showIndelPileups. When this option is used, two lines are printed for every genomic location that has indels associated with it: first line is a conventional base pileup, the second line is an "extended event" (indel) pileup. The refence base in that second line is always set to "E" (for Extended), and the pileup string contains I,D,. symbols for insertion, deletion, noevent, respectively. Only this simple short format for indel pileups is implemented so far.
asivache
2009-12-29 20:16:34 +0000
9652692019Modified to enable locus traversals firing additional calls to walker's map() with alignment context filled with extended events (indels). Walker should override generateExtendedEvents() to return true, and it should make sure that it catches those additional indel pileups and processes them differently, as needed. If there are indels associated with a specific reference base, TWO map() calls will be issued in locus traversal at that location: first one will have a context filled with a regular base pileup, the second call will provide the context filled with indel pileup (pileup elements will have insertion, deletion, or noevent type associated with them and will also carry information about the full length of the event and inserted bases).
asivache
2009-12-29 20:13:25 +0000
06eb576924Can now be constructed with either base pileup or extended event (indel) pileup; has query methods checking what kind of pileup is served by the context, and getter methods return the appropriate pileup. TODO: while it is impossible right now to create a context that contains both types of pileups simultaneously, this restriction is only weakly enforced through the lack of appropriate constructor. Either we keep it this way, or some getters may become ambiguous and have to be fixed!
asivache
2009-12-29 20:07:29 +0000
f445745c56Pileup element and corresponding container class tweaked for representing pileups of extended events (indels) at a given locus. There's some redundancy with PileupElement and ReadBackedPileup (should we rename them to BasePileupElement and ReadBackedBasePileup?), so that abstracting a basic interface/abstract base from these classes can be considered in the future
asivache
2009-12-29 20:03:39 +0000
87e863b48dRemoved used routines in duputils; duplicatequals to archive; docs for new duplicate traversal code; general code cleanup; bug fixes for combineduplicates; integration tests for combine duplicates walker
depristo
2009-12-29 19:46:29 +0000
5fdf17fccbRemoved the VCF "NS" annotation (which wasn't working for pooled calls anyways) since it's ambiguous and not useful.
ebanks
2009-12-29 17:30:47 +0000
e32174fbc4UnifiedGenotyper now works without -varout or -vf set.
hanna
2009-12-29 16:46:24 +0000
b125571a98Intermediate check in: transfer responsibility of wrapping the GenotypeWriter around the output stream to the output management code. Currently, will not work when neither -varout nor -vf are specified, but should work in all other cases.
hanna
2009-12-29 16:11:11 +0000
aeb34758e6Adding a validation stringency to the VCF writers (which defaults to STRICT). If set to SILENT, it will not throw an exception for (reasonable) off-spec requests but will instead ignore such requests and silently move on.
ebanks
2009-12-29 15:33:53 +0000
29a3d9b47aAnalyzeCovariates also has to skip over NO_DINUC
rpoplin
2009-12-29 14:36:05 +0000
a34c2442c0moved hard-coded file paths to the oneKGLocation, validationDataLocation, and seqLocation variables setup in the BaseTest.
aaron
2009-12-29 07:40:48 +0000
9d263b2565Integration tests for count duplicates walker validated on a TCGA hybrid capture lane.
depristo
2009-12-28 23:57:25 +0000
fcc80e8632Completely rewritten duplicate traversal, more free of bugs, with integration tests for count duplicates walker validated on a TCGA hybrid capture lane.
depristo
2009-12-28 23:56:49 +0000
4617052b3cFor Alec, and others at the Broad who want to run our unit/integration tests off of gsa1/gsa2: put a ceiling on the amount of memory that integration tests can use. Reduce the memory footprint of the fasta reader test.
hanna
2009-12-28 23:42:46 +0000
b5e5e27225New versions of picard-private, sam and picard jars for TileCovariate and regeneration of NM tag
alecw
2009-12-28 22:18:55 +0000
d4ee999ef9Creates files supplemental to the reference sequence, consumed by BWA. ANN - Alternate form of the sequence dictionary. Should be created from a sequence dictionary with full contig names. AMB - A map of 'holes' in the genome, aka runs of non-ACGTacgt bases. This skeletal implementation always reports no holes.
hanna
2009-12-28 21:40:44 +0000
rpoplin
chartl
ebanks
hanna
asivache
andrewk
depristo
aaron
alecw