beb8a83ca4
Multiple inputs / outputs for rules.
...
Cleanup.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3464 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-30 08:28:16 +00:00
e96bcf2128
GSA-333 Replaced all accidental usages of getCanonicalPath (which should never be used) with getAbsolutePath.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3463 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-28 22:27:05 +00:00
eb25e41111
minor update to new tribble name
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3462 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-28 20:23:25 +00:00
ffeb3fd80d
Thanks to Guillermo, I found a bug in the Unified Genotyper output: GL was posteriors instead of likelihoods. Not a huge deal because the
...
priors were flat, but fixed nonetheless.
Also, needed to update Tribble.
Minor updates to the Beagle input maker.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3461 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-28 19:28:26 +00:00
4e268ef6ac
Removing the Variant Recalibration Performance test because it isn't ready yet.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3460 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-28 18:27:25 +00:00
522dd7a5b2
Adding the variantrecalibration classes.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3459 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-28 18:21:27 +00:00
2014837f8a
VariantOptimizer package is moved to core, renamed as VariantRecalibration, and added to the binary release package. VariantOptimizer walker is renamed to GenerateVariantClustersWalker and ApplyVariantClustersWalker renamed to VariantRecalibrator. Integration tests added, performance tests still to be done.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3458 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-28 18:20:18 +00:00
871cf0f4f6
Call out ROD types by there record type, instead of the codec type (which was clumsy). So instead of:
...
@Requires(value={},referenceMetaData=@RMD(name="eval",type= VCFCodec.class))
you'd say:
@Requires(value={},referenceMetaData=@RMD(name="eval",type= VCFRecord.class))
Which is more in-line with what was done before. All instances in the existing codebase should be switched over.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3457 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-28 14:52:44 +00:00
98350da177
rev tribble
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3456 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-28 14:52:39 +00:00
cc2bf549c8
Removing my unnecessary optimization. 10 lines later in the code the same optimization was applied. A monumental waste of time.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3455 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-28 14:10:48 +00:00
a4d834cc01
fixing the test I broke
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3454 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-28 02:06:20 +00:00
6485e8383d
Trivial change to retrigger broken build that really isn't broken
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3453 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-27 23:33:46 +00:00
f2e7582cfc
Reorganization of SW code for clarity. Totally failure at raw optimization. Discovered that ~50% of reads being cleaned were perfect reference matches. New code comes with flag to look at NM field and not clean perfect matches. Can we turned off with command line option (needed for 1KG bams with bad NM fields). Going to rerun cleaning jobs due to accidentally rebuilding of stable codebase and loss of 2 days of runtime.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3452 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-27 23:16:00 +00:00
e1b0aefb29
fix for parallelism bug
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3451 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-27 22:16:14 +00:00
cded9ec985
adding a command line option, -etd (enable threaded debugging), that uses a custom thread pool class to catch exceptions thrown inside of a thread.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3450 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-27 21:57:56 +00:00
0e3c76ab84
GSA-333 Meant to use getAbsolutePath and not deref symlinks.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3449 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-27 21:23:02 +00:00
e2674671e7
The liftover code needs to *hard filter* records whose reference changes (since they no longer adhere to the VCF spec as they don't match the new reference - and can't be converted to VariantContext).
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3448 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-27 19:22:47 +00:00
ff4a0764df
Read error rate is now parallelizable
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3447 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-27 19:00:09 +00:00
99aae4ac0b
ended not done for LSF queue submission.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3446 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-26 22:16:48 +00:00
dfc36c1e95
Restructuring of the mandatory read filters for traversals. Now everything uses ReadFilters, even for the required filters like being mapped for LocusWalkers. Statistics now tracked for each read filter used during the traversal and info emitted in INFO at the end.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3445 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-26 22:12:25 +00:00
3873dccb35
First fully functional (though preliminary) version of walker that takes an input VCF and outputs a Beagle .bgl file that can be used for missing genotype calls/haplotype imputation. For now, only supported input format is likelihood format for unrelated individuals.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3444 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-26 21:03:23 +00:00
f9efc1248c
VariantEvalWalker now takes indels if you throw the -dels flag. IndelLengthHistogram appears to be working properly, it is turned off by default (as it is experimental) but you can turn it on in your own repository.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3443 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-26 20:03:14 +00:00
a0bf1753f9
A script to sanity check that a release is runnable.
...
Not fancy - just pulls out the performance tests in 20sec chunks. Only a temporary solution.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3442 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-26 17:45:42 +00:00
058441fa39
Trivial renaming of test
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3441 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-26 16:56:42 +00:00
0265199ce4
First pass at an IndelLengthHistogram module for variant annotator. Off by default. Will be tested shortly (have to commit, so I can check out in another directory, so that compiling won't kill all my jobs running on LSF)
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3440 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-26 15:04:39 +00:00
a2fab07258
fixed the build problem: there were two copies of the AnnotatorInputTable Codec and Feature in two different spots.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3439 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-26 14:47:15 +00:00
5928047d8b
Optimization of reference window calculation to us bytes not char and no uppercasing since reference and read bases are always uppercase now. Should remove some ~5% of runtime of UG.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3438 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-26 14:10:26 +00:00
88a06ad81f
Changes to Depth of Coverage:
...
- For speedup in large number of samples, base counts are done on a per read group level, then
merged into counts on larger partitions (samples, libraries, etc)
+ passed all integration tests before next item
- Added additional summary item, a coverage threshold. Set by (possibly multiple) -ct flags,
the summary outputs will have columns for "%_bases_covered_to_X"; both per sample, and
per sample per interval summary files are effected (thus md5s changed for these)
NOTE:
This is the last revision that will include the per-gene summary files. Once DesignFileGenerator is sufficiently general, and has integration tests, it will be moved to core and the per-gene summary from Depth of Coverage will be retired.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3437 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-26 03:39:22 +00:00
0607f76a15
commenting out this test until I can figure out what the hell is going on with the codecs.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3436 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-26 01:12:10 +00:00
ed4d8ddd05
Refactoring and sugar to give lower level access to the job graph.
...
Will add more sugar / glue depending on how much of a graph the python generator outputs.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3435 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-25 22:52:29 +00:00
062b316881
Better Exception message when can't find annotation value in variant recalibrator.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3434 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-25 21:15:50 +00:00
bf530d23de
Variant Recalibrator now makes use of a prior on known/novel status as well as on allele frequency spectrum. The VariantOptimizer walker now clusters with all variants but gives more weight to knowns / hapmap / 1KG / MQ1 sites. The weights are all optional command line arguments. We no longer assign default values to annotations that are malformed. The walkers will crash with exception so as to not cover up potential issues. We only produce titv-less clusters now, and so the titv argument in VO was removed and the WithoutTiTv string that gets added to the cluster file is removed. The wiki is updated to show new example commands.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3433 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-25 21:08:31 +00:00
ae6c014884
Fixed UG parallelization bug. Better integration test to catch this in the future.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3432 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-25 21:03:45 +00:00
434e920da9
Oops, forgot to update integration tests
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3431 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-25 20:37:45 +00:00
772f558ae0
Massive change to the indel realigner code. We now properly deal with soft-clipped reads. Also, improved left-alignment code.
...
Small change for Ryan to get hard-clipped reads working for the recalibrator.
PLEASE DO NOT RELEASE THIS WEEK. I still have some more testing to do and need Mark to run WG jobs.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3430 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-25 20:04:33 +00:00
f3e2aae570
add experimental support for tabix files (for any of our Tribble rod types), as long as they end in .gz and can be read by the tabix reader.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3429 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-25 04:44:46 +00:00
f7c9f131ea
revisioning tribble to version 85, which includes tabix and bug fixes
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3428 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-25 04:28:32 +00:00
8db7c97c4d
Moved AnnotatorInputTableFeature and Codec to org.broadinstitute.sting.gatk.refdata.features.annotator
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3427 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-24 14:38:54 +00:00
4aa749c709
Moved AnnotatorInputTableFeature and Codec to org.broadinstitute.sting.gatk.refdata.features.annotator
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3426 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-24 14:38:07 +00:00
aca3bcb193
Moved AnnotatorInputTableFeature and Codec to org.broadinstitute.sting.gatk.refdata.features.annotator
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3425 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-24 14:37:17 +00:00
64ed770250
Moved AnnotatorInputTableFeature and Codec to org.broadinstitute.sting.gatk.refdata.features.annotator
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3424 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-24 14:36:28 +00:00
42fa12d9e4
without the *, the package system only took directly referenced files (which pulled in VCF and BED) but skipped dbsnp. Epic star fail...
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3423 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-24 05:19:57 +00:00
ee3f2eb1d0
Don't output traversal reduce result in the logger. In many cases, the reduce
...
result is tangential to the product of the analysis and having the logger always
emit it can confuse the output (such as in the new reduceByInterval
DepthOfCoverage walker). If users want to emit it, they can choose not override
onTraversalDone, or override onTraversalDone and write results to the output
stream / logger / whatever their choice.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3422 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-23 22:41:43 +00:00
a40e64e47b
A downsampling validator. Compares the generated pileup passed in from the alignment context to the reads,
...
passed in as a Tribble SAM text feature. If the generated pileup contains a valid set of reads according to
the downsampling rules, the test passes.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3421 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-23 21:49:54 +00:00
e9ee55d7dd
A cleaned up functioning early, early access version of Queue for others to play with and provide feedback about next steps.
...
Current version only has syntatic sugar for accessing the graph via rules ex. "bam" -> "bam.bai", "samtools index ${bam}" and DOES NOT have sugar for constructing your own graph.
Usage info on the internal wiki at https://iwww.broadinstitute.org/gsa/wiki/index.php/Queue
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3420 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-23 20:21:09 +00:00
a280a0ff0d
a) Made HaplotypeScore default annotation. This changed several integration tests, whose MD5 is now updated.
...
b) Disabled BaseQualRankSumTest, the returned p-values differ wildly from Matlab/R-provided ones, cause TBD.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3419 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-21 22:25:17 +00:00
f32a32269c
minor change for eric
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3418 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-21 20:13:36 +00:00
b10950c691
Simple performance optimization -- cache the number of reads in the locus hanger.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3417 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-21 19:26:16 +00:00
355396109b
Bug fix to avoid build failure (class changed under me??)
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3416 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-21 18:48:56 +00:00
1753d07b02
Added AnnotationByAlleleFrequencyWalker - walker takes an input vcf, a reference vcf and a list of annotations (with the -A argument). For each site present in both VCF's, it outputs the given annotations into the screen as well as allele frequency. Since HapMap vcf reference doesn't include AF in annotations, it computes it from Chromosome, Het and HomVar counts.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3415 348d0f76-0448-11de-a6fe-93d51630548a
2010-05-21 18:31:34 +00:00
kshakir
ebanks
rpoplin
aaron
depristo
chartl
delangel
weisburd
hanna