687fd477ff
Just some code I want to freeze. If you ever need to estimate the % of bases covered by exon, given an interval list, give it to getTargetedGenes. Not the best name for this function, but I don't expect anyone to use it but me.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3111 348d0f76-0448-11de-a6fe-93d51630548a
2010-04-01 20:21:50 +00:00
ac9c335cd2
This is a python job dispatcher I've been using, which builds on Mark's FarmJob utility, and an example script of how I'm using it. Basically I wrote it to smartly break up analysis over an interval list, givin a maximum number of bases per job, a list of available queues, and a limit on each queue. It handles going over these limits in three ways:
...
1) [default]: Fail
- No jobs are actually spawned
2) Space
- User provides a string of the form A:B:C where A is the number of days to wait before scheduling jobs over
the queue limits; B the number of hours, C the number of minutes. Exceeding the queue limits again will
increment the space by another A:B:C
3) Stop-Resume
- Spawns the maximum number of jobs, and writes a file describing the next job, and a hash code of the remaining jobs.
The next time the script is run, it spawns the next set of jobs starting with what's written in the file. If the
hash code (and thus the command string) changes between runs, the dispatcher fails-fast.
The base job dispatcher class is also capable of dealing with dependencies if it is used correctly.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3102 348d0f76-0448-11de-a6fe-93d51630548a
2010-03-31 19:53:13 +00:00
dc802aa26f
Moved CoverageStatistics to core. This will be (soon) renamed DepthOfCoverage; so please use CoverageStatistics
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3090 348d0f76-0448-11de-a6fe-93d51630548a
2010-03-29 13:32:00 +00:00
08d9ae403d
better farm commands, and simple utility to convert ucsc repeat masks to interval lists
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3040 348d0f76-0448-11de-a6fe-93d51630548a
2010-03-19 13:11:06 +00:00
4bdc3b2784
automatic generation of individual and individual set import files
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3001 348d0f76-0448-11de-a6fe-93d51630548a
2010-03-15 10:36:33 +00:00
d9b12b468f
Adding default filter info
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3000 348d0f76-0448-11de-a6fe-93d51630548a
2010-03-15 10:05:46 +00:00
196bca6819
Script to split concordance files into their constituent sets and calculate summary stats from a concordance file - SNPs called and number in dbSNP
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2992 348d0f76-0448-11de-a6fe-93d51630548a
2010-03-12 22:20:44 +00:00
9298e13201
Make annotated VCF not be broken
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2906 348d0f76-0448-11de-a6fe-93d51630548a
2010-03-01 23:22:41 +00:00
5f9c3f3884
Outputing annotated VCF to the current directory instead of attempting to write in the directory next to the original vcf.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2869 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-22 21:31:24 +00:00
217deb9809
Changed the INFO field delimiter from a comma to a semicolon
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2847 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-16 20:44:57 +00:00
0e4b5ad9c6
Check to ensure sample status is "Complete" before writing out the bam file
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2844 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-16 15:36:42 +00:00
e491b42951
Dumb little script that grabs Picard metrics (alignment, hybrid selection, insert size) from picard_aggregation given the path to the bam file; zips them up, and spits them out; for use with Firehose
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2824 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-11 14:09:30 +00:00
60f05379a7
fix typo. Check explicitly that fingerprint files exist.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2821 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-10 21:29:54 +00:00
d16a1b5645
Simple python script for generating a firehose-parseable text file from MS-Dos formatted TSV spreadsheets (of the type that we get from project management). Will be deprecated in a few weeks with the advent of direct BSP ID entries, but useful until then.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2820 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-10 20:25:40 +00:00
57a168c0db
Added a header crediting the python script as a source. Looping over an arbitrary number of headers.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2804 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-07 03:38:30 +00:00
61a67cdce4
Moving file up a directory for dependencies
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2798 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-05 19:36:13 +00:00
58456822ab
Two perl scripts (from Kristian Cibulskis) and one python script for annotating VCF files with the information generated by the cancer MAF annotation tool.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2797 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-05 19:25:46 +00:00
e964660df3
snpSelector now supports min and max q scores.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2751 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-31 19:38:34 +00:00
7b3c34d210
keeping a backup
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2750 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-31 15:36:25 +00:00
0fb032a436
Quick script that changes "chr#" to "#" and "chrM" to "MT" and moves mitochondria to the end of the vcf; in accordance with the 1KG reference.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2727 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-28 21:59:33 +00:00
4990139b60
A collection of python objects that are useful for VCF validation. Use 'em or don't.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2679 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-25 18:44:10 +00:00
cf46e3c85f
Valuable series of commands
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2641 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-20 13:58:59 +00:00
8226f4aa12
minor cleanup
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2616 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-16 20:23:20 +00:00
dfe160ff77
Minor changes (additional info calculated)
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2522 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-06 16:41:01 +00:00
588006ee92
Now supports strings in command line for farm submission
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2507 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-06 13:15:40 +00:00
9fb6533549
new -a option does fast merging of already sorted files
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2500 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-05 13:55:39 +00:00
89f3ee614a
minor printing fix
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2492 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-30 22:14:50 +00:00
fcc80e8632
Completely rewritten duplicate traversal, more free of bugs, with integration tests for count duplicates walker validated on a TCGA hybrid capture lane.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2458 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-28 23:56:49 +00:00
4e7e0432a2
Updated SNP calling power from coverage tools to work with new UnifiedGenotyper and DepthOfCoverage tools.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2378 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-16 20:44:30 +00:00
f5e547ed6e
Add ability for flat file table parsing module to skip ahead to first occurence of a regular expression (use case: consistently parsing DepthOfCoverage output for histogram section of file across file format changes)
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2377 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-16 20:38:50 +00:00
bf76019f22
Minor change to coverage evalution script, to update for new file format and add output fields
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2375 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-16 18:06:08 +00:00
0d2a761460
Bugfix for minBaseQuality to ignore deletion reads. LocusMismatch walker now allows us to skip every nths eligable site
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2357 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-15 14:38:39 +00:00
56467df49a
minor improvements to snpSelector to work with hapmap chip VCF files
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2343 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-13 17:59:32 +00:00
b2dfe85648
Better support for reading truth file
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2307 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-10 12:16:05 +00:00
6a4118ad3c
grr, ought to actually assign it to the TRUTH_CALLS variable
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2302 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-09 23:31:46 +00:00
987fced151
Should read truth data from the parser options rather than direct from args
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2301 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-09 23:26:26 +00:00
8825211fdb
Adding this to subversion so it's protected
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2299 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-09 21:26:17 +00:00
2632cb6b58
minor improvements to snp selector
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2275 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-07 03:37:14 +00:00
b817db0962
Syzygy has a default LOD score of 0.91 on bases with no coverage, this is problematic. Set the minimum lod threshold to 1 because I just don't want to see that codswallop.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2268 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-04 23:29:14 +00:00
0753315156
updates to the python snp selector -- now sorts info fields and we stop printing unnecessary debugging info in vcf2table
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2265 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-04 22:16:02 +00:00
0f89a38473
forgot to commit this earlier
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2264 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-04 22:10:16 +00:00
c1263e841c
stop printing the debug info -- hurr
...
Also it turns out that sometimes there can be a call with zero total non-I/non-D bases -- so add one to numerator and denominator to prevent divide by zero error
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2262 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-04 16:17:38 +00:00
0c2d6d7e41
A brute-force script to convert Syzygy lod-score calls files into a proper VCF -- with some useful annotations.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2261 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-04 16:07:06 +00:00
2c7cb912f0
Bug fixes for mixed none/valued attributes. also now assigns fake float values for display, if requested, for covariates using the -plottable flag
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2253 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-03 23:52:35 +00:00
dbb8b86ed1
Minor updates to correctly handle emitting FN calls
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2231 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-02 22:53:17 +00:00
67179e2412
Initial checkin of AnalyzeCovariates.java which replaces analyzeRecalQuals_1KG.py and is updated to use the new Covariates system. It creates similar plots of residual error for each covariate that was used in the calculation. There is also an option to filter out base qualities below a given threshold.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2215 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-02 16:47:35 +00:00
8a87d5add1
misc. bug fixes
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2212 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-02 14:36:03 +00:00
c93d37d9fb
continuing improvements in output of snpSelector
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2198 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-01 15:42:06 +00:00
2ea93385be
Better support for comparison to truth. Now emits FP rates for each covariate if a truth file is provided. Also now writes out a detailed recal.log file that can be parsed directly into R
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2179 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-29 22:20:40 +00:00
662bbbd53b
Awful stupid bug. This will use up one of my bad code offsets.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2178 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-29 20:09:33 +00:00
fa2d564f2c
And the compulsory one-second-later fix -- better handling of arguments (e.g. for callng from outside of /trunk/python/)
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2177 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-29 20:02:43 +00:00
45673d7851
A quick and dirty script that, given a list of input VCF files, will output a new VCF file which looks identical to the first VCF file of the input list, except that the info field has been updated to reflect the union of all the INFO annotations across the VCF files
...
Note: this is primarily for use with two files with mostly disjoint annotations. It views "SB=2.5" as a different info field than "SB=2.2" and so will output as info "SB=2.5;SB=2.2". That is, it compares the full field string, rather than only the field name.
Usage:
./mergeVCFInfoFields I=[comma-delimited list of files] O=[output file]
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2176 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-29 20:01:29 +00:00
65da04ca85
Now uses the theoretically correct relationship between SNP FP and TP ratios for Illumina data. maxQ score for a snp is now 60
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2168 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-25 22:08:12 +00:00
03342c1fdd
Restructuring and interface change to ReadBackedPileup. We now lower support the Pileup interface, the BasicPileup static methods, and the ReadBackedPileup class. Now everything is a ReadBackedPileup and all methods to manipulate pileups are off of it. Also provides the recommended iterable() interface of pileup elements so you can use the syntax for (PileupElement p : pileup) and access directly from p.getBase() and p.getQual() and p.getSecondBase(). Only a few straggler walkers use the old style interface -- but those walkers will be retired soon. Documentation coming in the AM. Please everyone use the new syntax, it's safer, and will be more efficient as soon as the LocusIteratorByState directly emits the ReadBackedPileup for the Alignment context, as opposed to the current interface. In the process of the change over, discovered several bugs in the second-best base code due to things getting out of sync, but these changes were resolved manually. All other integrationtests passed without modification.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2154 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-25 03:51:41 +00:00
bc35a34f60
More informative printing, no longer prints tons of NaN warnings
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2139 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-24 15:45:48 +00:00
da7de9960b
General bug fixes for snpSelector. More robust error checking and handling of NaN values.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2106 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-21 14:48:29 +00:00
52494d8176
cleanup of SNP selector -- ready for some additional testing
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2042 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-13 21:46:31 +00:00
1a4d071d37
Better snpSelector, plus VCFmerge tool
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2022 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-11 22:02:57 +00:00
3990c6d950
snpSelector v3 -- bootstrapping support and VCF output
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2004 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-09 22:48:51 +00:00
f777c806d6
snpSelector v2 -- code refactoring and support for comparison with known truth. Looks great.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1986 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-07 19:32:12 +00:00
7cb51dbc31
snpSelector v1 -- and supporting changes to VCF reader
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1983 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-06 23:00:46 +00:00
eca0942644
Oops. Let's make sure only to write calls that the pool supports to the auxiliary vcf files.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1974 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-04 17:14:55 +00:00
fc17e75759
Put this puppy through its paces. Eliminated the sorting and header-handling stuff; that isn't the purvey of this script and should be handled downstream or by a script wrapper.
...
I also secretly handled another pesky overlow exception. Occasionally Syzygy could report lods of like -1000; e.g. posterior probabilities of one in one (((googol) googol) googol) googol which of course makes python blow up. Now we safely output an accurate posterior.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1971 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-04 06:05:45 +00:00
3d9195f8b6
Added - converter from expanded summary to VCF (beautiful thing, really)
...
Removed - the ugly hackjob that was expanded summary to Geli
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1970 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-03 22:20:47 +00:00
d60c632099
Minor output improvement
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1965 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-03 13:20:55 +00:00
44ea55d338
Useful library for parsing VCF files, plus a general VCF->table converter
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1964 348d0f76-0448-11de-a6fe-93d51630548a
2009-11-03 13:14:04 +00:00
99337df929
Now looks up and propagates Syzygy's LOD scores into the appropriate field (so variantfiltration can adjust lod scores accurately)
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1950 348d0f76-0448-11de-a6fe-93d51630548a
2009-10-30 21:13:03 +00:00
7654051aee
Faster grepping
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1948 348d0f76-0448-11de-a6fe-93d51630548a
2009-10-30 16:59:17 +00:00
4319ff0610
A python script that will convert pooled expanded summary files (from Jason Flannick's pipeline) into .geli files
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1947 348d0f76-0448-11de-a6fe-93d51630548a
2009-10-30 16:39:57 +00:00
c1e1d910cb
simple monitor for watching pilot 1 call progress
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1769 348d0f76-0448-11de-a6fe-93d51630548a
2009-10-06 13:04:53 +00:00
de9f2b11da
Detects unmapped (no bai) bam files and doesn't blow up
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1725 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-25 12:56:28 +00:00
8349004414
Generalize the regexp for analysis files
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1714 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-24 03:17:41 +00:00
3a341b2f06
Fixes for VariantEval for genotyping mode
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1659 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-18 21:01:43 +00:00
d191e02c88
Automated parsing stats from VariantEval and outputting stats to "*.oneline_stats" files; needed to do larger culling of predictions vs. actual SNP call for Pilot 3 lanes
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1620 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-14 23:40:11 +00:00
e06e31d99f
Many generalization improvements - parameters, files as options - to script that runs pieces for predicting SNP calling performance for given SNP calling coverage
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1619 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-14 23:37:53 +00:00
7eb21e55c1
Added die_on_fail which outputs an error message and stops execution if a farm or terminal command fails
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1618 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-14 23:30:13 +00:00
6e13a36059
Framework for ROD walkers -- totally experiment and not working right now
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1600 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-12 19:13:15 +00:00
70ec37661c
Fix merger command
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1584 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-11 13:13:23 +00:00
45c794d066
pipeline is complete
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1583 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-11 13:09:37 +00:00
8c33dd2393
enable job names
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1582 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-11 13:08:53 +00:00
fc0d9578f6
better feedback now
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1579 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-10 12:43:45 +00:00
c988205884
Notes for Aaron in SSG
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1576 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-10 03:18:51 +00:00
ec0f6f23c7
LocusIterationByState is now the system deafult. Fixed Aaron's build problem
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1552 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-09 01:28:05 +00:00
2a6f3a03c9
update script to put pilot1 bams directly onto hphome
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1547 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-08 14:41:35 +00:00
e716f9337d
A few more additions; almost done...
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1541 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-07 01:50:22 +00:00
5dbba6711c
Lots of changes: (I'll send email out in a sec)
...
1) Moved various disparate concordance / set splitting functionalities to a new parent tool which works like VariantFiltration (i.e. people can write various modules that fit inside and can be run though it).
2) Fixed up argument parsing in VariantFiltration to use key=value format so we don't accidentally mox up values (like I had been doing).
3) Have indel rod print samples
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1540 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-07 01:12:09 +00:00
1c3d67f0f3
Improvements to the CountCovariates and TableRecablirator, as well as regression tests for SLX and 454 data
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1539 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-04 22:26:57 +00:00
3ac5ac066f
Checking in Michael's DoC parameterization script;
...
this functionality will eventually be moved into VariantFiltration
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1515 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-03 15:07:49 +00:00
d804a119dc
script to run the complete pilot2 pipeline: from cleaning to calling to filtering
...
[not quite finished though]
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1512 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-03 14:35:55 +00:00
b01ac9de0c
High performance LocusIterator implementation. Now with greatly reduced memory impact and 2x (and more potentially) speed ups of raw locus iteration. General performance improvements to SSG with empirical probs. You can enable high-performance locus iteration with the -LIBS arg. It's still testing but passes validing pileup.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1510 348d0f76-0448-11de-a6fe-93d51630548a
2009-09-03 03:06:25 +00:00
2402dcd4c9
Give usage message if no arguments provided.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1483 348d0f76-0448-11de-a6fe-93d51630548a
2009-08-31 00:28:43 +00:00
ee05ddde16
Added command line options to make the barcode analysis script executable by end users.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1455 348d0f76-0448-11de-a6fe-93d51630548a
2009-08-24 21:15:09 +00:00
0e7c158949
I've pulled out the functionality of the analyzer into a single python file which doesn't require all of the irrelevant config parameters (which would cause problems for external users). I'll release this and the simple config file to 1KG for use in analyzing recalibration efforts.
...
Please note that this is literally my first foray into the wonderful world of python. There could very well be a much more elegant way of releasing the script to external users without having to duplicate the file. If so, anyone out there should (please) feel free to do so in a second release; but, for now, this needs to be online by tomorrow morning.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1404 348d0f76-0448-11de-a6fe-93d51630548a
2009-08-11 02:56:43 +00:00
afccbc44ec
Script that performs all the processing steps from raw Illumina reads through to analysis of barcoding and hybrid selection efficience as documented in the GATK tutorial; can automatically run all steps in series on the farm.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1354 348d0f76-0448-11de-a6fe-93d51630548a
2009-07-31 00:22:53 +00:00
eb4b9a743a
Script that runs most of the steps involved in validating the CoverageEval system that predicts performance for given depth of sequencing coverage across a genome.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1353 348d0f76-0448-11de-a6fe-93d51630548a
2009-07-31 00:18:45 +00:00
efd0fd1f0a
Short python script that takes paired-end BAMs and aligns them with BWA. Referenced in GSA wiki tutorial
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1351 348d0f76-0448-11de-a6fe-93d51630548a
2009-07-31 00:04:10 +00:00
1c648a2d5f
Skip compiled python files (*.pyc) in svn status output
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1346 348d0f76-0448-11de-a6fe-93d51630548a
2009-07-30 21:45:23 +00:00
d665d9714f
By default now writes output to JOBID.lsf.output instead of going to email -- based on recommendations from the cancer group
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1325 348d0f76-0448-11de-a6fe-93d51630548a
2009-07-28 13:18:58 +00:00
00f9bcd6d1
CoverageEval.py tool right before some major changes to the core of the code
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1293 348d0f76-0448-11de-a6fe-93d51630548a
2009-07-22 16:58:23 +00:00
702cdd087f
Actually listens to justPrint now
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1253 348d0f76-0448-11de-a6fe-93d51630548a
2009-07-15 16:52:46 +00:00
c25f84a01c
Regression: we lost our hack to work around BAM files with index problems (affects BAM files created before 23 Apr 2009 and traversed by interval). Added the hack back in, along with a much more explicit comment about why its there.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1248 348d0f76-0448-11de-a6fe-93d51630548a
2009-07-15 14:41:37 +00:00
84d407ff3f
Fixing odd merge problem with VariantEval -- better cluster analysis (no cumsum), rodVariant is now an AllelicVariant
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1239 348d0f76-0448-11de-a6fe-93d51630548a
2009-07-14 18:53:27 +00:00
c8fcecbc6f
Added ParseDCCSequenceData.py to repository and made changes that allow an analysis of quantity of sequence data by platform and project, moved table / record system to a new module called FlatFileTable.py and built that into ParseDCCSequenceData and CoverageEval.py; changed lod threshold in CoverageEvalWalker.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1201 348d0f76-0448-11de-a6fe-93d51630548a
2009-07-08 22:04:26 +00:00
d3daecfc4d
Added unit tests for function in ListUtils to randomly sample lists with replacement, updated AlleleFrequencyEstimate to provide a callType of HomRef, HetSNP, HomSNP, update indices in CoverageEval.py, and made a lot of changes to CoverageWalker biggest one being that it directly calls SingleSampleGenotyper instead of implementing some parts of SSG itself.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1189 348d0f76-0448-11de-a6fe-93d51630548a
2009-07-08 02:05:40 +00:00
f5b00c20d0
Updated python files
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1182 348d0f76-0448-11de-a6fe-93d51630548a
2009-07-07 14:15:39 +00:00
dcb8892568
Lot of code for coverage evaluation tools including first version of python script to evaluate the downsampled SSG callls made and the java code to make all the calls at Hapmap chip sites at various downsampling levels; ListUtils contains functions for randomnly subsetting lists (with replacement) which are useful for subsetting the same elements in both the reads and the offsets lists of a LocusWalker
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1162 348d0f76-0448-11de-a6fe-93d51630548a
2009-07-03 08:07:02 +00:00
5289230eb8
Version 0.2.1 (released) of the TableRecalibrator
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1108 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-25 22:50:55 +00:00
caf5aef0f8
Much improved python analysis routines, as well as easier / more correct merging utility. Better R scripts, which now close recalibration data by the confidence of the quality score itself
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1081 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-24 01:12:35 +00:00
c9e6cb72e1
Major improvements to python analysis code -- now computes a host of statistics about quality scores from the recal_data.csv file emitted by countcovariates. Includes average Q scores, medians, modes, stdev, coefficients of variation, RSME, and % bases > q10, q20, q30. Can finally quantify and compare the improvement of quality score recalibration.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1064 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-20 19:50:37 +00:00
9e26550b0d
Apprach v2. Added python analysis script, so java no longer must be used to analyses quality score data. About to refactor out lots of unneeded code
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1063 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-20 16:00:23 +00:00
8ac40e8e2d
Updated version of the recalibration tool
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1060 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-19 17:45:47 +00:00
17a5b50ea4
Script that aligns paired-end BAMs using BWA.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1042 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-18 18:14:58 +00:00
260fd0dc45
Trivial change
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@1000 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-12 19:11:28 +00:00
fb7ba47fff
Now does really neightbor distance calculation, as well as true snp cluster counting
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@998 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-12 16:29:26 +00:00
1fb241a8b8
Now supports resume and dry runningRecalQual.py
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@996 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-11 23:31:59 +00:00
5440dd13df
Preparation for point release of read calibrator: no artificial heap size limit, no duplicate dbsnp records.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@986 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-11 18:39:33 +00:00
e77dfe9983
Allow script to be easily modified to support different platforms.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@955 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-09 16:06:57 +00:00
7fa84ea157
10x speedup of recalibration walker
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@954 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-09 15:39:40 +00:00
5fa3f7ed3a
Added absolute path bug fix for Mark.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@949 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-09 02:25:17 +00:00
127c321d0a
Cut over to 1kG version of fasta / reference. Updated doc with latest version of tool summary.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@940 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-08 21:11:44 +00:00
596773e6c6
Cleanup.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@931 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-07 20:25:08 +00:00
e6aa058ec4
Tighten up error handling a bit.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@920 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-06 03:40:50 +00:00
819862e04e
major restructuring of generalized variant analysis framework. Now trivally easy to add additional analyses. Easy partitioning of all analyses by features, such as singleton status. Now has transition/transversional bias, counting, dbSNP coverage, HWE violation, selecting of variants by presence/absense in dbs. Also restructured the ROD system to make it easier to add tracks. Also, added the interval track -- if you provide an interval list, then the system autoatmically makese this available to you as a bound rod -- you can always find out where you are in the interval at every site. Python scripts improved to handle more merging, etc, into population snps.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@918 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-05 23:34:37 +00:00
050d55cdb0
Basic graph support for testing.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@916 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-05 21:04:01 +00:00
2035d7dfd3
Revert some debug code in RecalQual.py. Make LogisticRegression easier to Ctrl-C out of.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@904 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-05 01:53:48 +00:00
61ae00c7bf
Lots of cleanup.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@903 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-05 01:26:10 +00:00
9689bb3331
Very early draft of script integrating the covariant counting / logistic regression. Deleted some unused code and spurious debug info.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@902 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-04 22:52:11 +00:00
40bc4ae39a
The building blocks for segmenting covariate counting data by read group.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@899 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-04 19:55:24 +00:00
67112c79a1
More robust individual genotypes to population script
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@893 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-04 00:12:31 +00:00
7755476d36
Updated coverter to reflect change in contig ordering in Geli files
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@888 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-03 10:05:28 +00:00
080af519cb
Added R script and uncommented a line in recal_qual.py
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@886 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-03 03:15:45 +00:00
b2eb724456
First commit of recalibration master control script for recalibrating quality scores.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@885 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-03 02:17:10 +00:00
3998085e4b
more and better python scripts for dealing with calls
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@881 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-02 20:37:19 +00:00
587d07da00
Merged functionality of two python scripts into LogRegression.py, some clarity updates to covariate and regression java files.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@876 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-02 16:55:05 +00:00
ae2eddec2d
Improving, yet again, the merging of bam files
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@874 348d0f76-0448-11de-a6fe-93d51630548a
2009-06-02 13:31:12 +00:00
543c68cdd8
First version of individual geli files to population SNPS
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@865 348d0f76-0448-11de-a6fe-93d51630548a
2009-05-31 15:29:10 +00:00
6adef28b97
Now supports automatic merging by population
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@864 348d0f76-0448-11de-a6fe-93d51630548a
2009-05-31 15:28:44 +00:00
e0803eabd9
enabled underlying filtering of zero mapping quality reads, vastly improves system performance
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@853 348d0f76-0448-11de-a6fe-93d51630548a
2009-05-29 14:51:08 +00:00
c72601322a
now returns the farm id when submitting a job!
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@825 348d0f76-0448-11de-a6fe-93d51630548a
2009-05-26 22:23:24 +00:00
04e51c8d1d
Better version of MergeBAMBatch -- more options for creating the file
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@787 348d0f76-0448-11de-a6fe-93d51630548a
2009-05-21 22:26:19 +00:00
3b1f84e15b
Slightly improved interface to merging utility for multiple bam files
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@757 348d0f76-0448-11de-a6fe-93d51630548a
2009-05-20 12:54:41 +00:00
e9f85ef920
Better merge support
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@748 348d0f76-0448-11de-a6fe-93d51630548a
2009-05-18 21:18:51 +00:00
9dec783a82
Actually writes out a good header now
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@744 348d0f76-0448-11de-a6fe-93d51630548a
2009-05-18 13:34:52 +00:00
8e9e2f4502
Revised ROD system. Split the system in Basic type and interface. Enabled more control over rod accessing, including an initialize() function to fetch headers and other options from the file. Added general tabular rod, which has a named columns and supports a map<String,String> interface. Comes with shiny new Junit system for RODs. Also, added simple python script for accessing picard data.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@716 348d0f76-0448-11de-a6fe-93d51630548a
2009-05-14 21:06:28 +00:00
de1c282e62
Reference-ordered data relies on bugs in the old command-line argument system to work. Update the ROD system to from -B track1 type1 file1 track2 type2 file2 to -B track1,type1,file1 -B track2,type2,file2.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@640 348d0f76-0448-11de-a6fe-93d51630548a
2009-05-08 15:28:19 +00:00
30218ee31a
Better validation scripts and data
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@562 348d0f76-0448-11de-a6fe-93d51630548a
2009-04-29 17:40:07 +00:00
58b2578c44
Several changes to CovariateCounter walker to print more tables (called vs. observed Q scores), bug fixes to LogisticRecalibrationWalker and LogisticRegressor, and print string functionality added to Pair.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@550 348d0f76-0448-11de-a6fe-93d51630548a
2009-04-28 00:37:48 +00:00
3739682bef
Actually has working version of the python script to merge multiple bam files
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@530 348d0f76-0448-11de-a6fe-93d51630548a
2009-04-24 19:15:55 +00:00
40a2b3eeb3
Basic logistic regression support for calibrating qualities; mostly for Andrew to experiment with
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@529 348d0f76-0448-11de-a6fe-93d51630548a
2009-04-24 19:09:50 +00:00
38c2f73457
LogRegression.py script that converts parameter files for each dinucleotide regression into one file to be read in by correction script.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@528 348d0f76-0448-11de-a6fe-93d51630548a
2009-04-24 18:31:26 +00:00
chartl
depristo
andrewk
kshakir
kiran
rpoplin
ebanks
hanna