Commit Graph

171 Commits (4e18c54bb8ff0e3ef435d5c238ccf283b799440c)

Author SHA1 Message Date
depristo 7902db616e Marginally more useful output
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3201 348d0f76-0448-11de-a6fe-93d51630548a
2010-04-20 14:45:14 +00:00
chartl 4eba9bffc1 Grabs average SNP calls, mismatch rate, aligned reads, and other important lane metrics from a SQUID export and summarizes them across multiple margins (lane numbers, flowcells, samples, libraries)
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3193 348d0f76-0448-11de-a6fe-93d51630548a
2010-04-17 03:09:05 +00:00
depristo 7973806716 interim update
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3173 348d0f76-0448-11de-a6fe-93d51630548a
2010-04-15 14:25:02 +00:00
chartl 2e4377b1cf Awesome: JobDispatcher can now dispatch jobs by gene from a target .design file found in /seq/references.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3170 348d0f76-0448-11de-a6fe-93d51630548a
2010-04-14 18:17:41 +00:00
weisburd 04c22a6640 Added handling of UCSC and NCBI reference sequences
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3165 348d0f76-0448-11de-a6fe-93d51630548a
2010-04-14 14:40:31 +00:00
weisburd 2183f10a1d Script for validating and converting text files into the tabular format required for GenomicAnnotator -B inputs
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3156 348d0f76-0448-11de-a6fe-93d51630548a
2010-04-13 13:35:10 +00:00
chartl fab31e1d53 Check in so I don't lose this code -- spawning of jobs by genes
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3137 348d0f76-0448-11de-a6fe-93d51630548a
2010-04-08 16:18:40 +00:00
chartl 27fb6f7594 Make sure to convert non-integer chromosomes (M,X,Y) back from their corresponding integer representations (0,23,24) when writing in .bed format
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3119 348d0f76-0448-11de-a6fe-93d51630548a
2010-04-03 20:01:21 +00:00
chartl 687fd477ff Just some code I want to freeze. If you ever need to estimate the % of bases covered by exon, given an interval list, give it to getTargetedGenes. Not the best name for this function, but I don't expect anyone to use it but me.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3111 348d0f76-0448-11de-a6fe-93d51630548a
2010-04-01 20:21:50 +00:00
chartl ac9c335cd2 This is a python job dispatcher I've been using, which builds on Mark's FarmJob utility, and an example script of how I'm using it. Basically I wrote it to smartly break up analysis over an interval list, givin a maximum number of bases per job, a list of available queues, and a limit on each queue. It handles going over these limits in three ways:
1) [default]: Fail
  - No jobs are actually spawned

2) Space
  - User provides a string of the form A:B:C where A is the number of days to wait before scheduling jobs over
    the queue limits; B the number of hours, C the number of minutes. Exceeding the queue limits again will
    increment the space by another A:B:C

3) Stop-Resume
 - Spawns the maximum number of jobs, and writes a file describing the next job, and a hash code of the remaining jobs.
   The next time the script is run, it spawns the next set of jobs starting with what's written in the file. If the 
   hash code (and thus the command string) changes between runs, the dispatcher fails-fast.

The base job dispatcher class is also capable of dealing with dependencies if it is used correctly.



git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3102 348d0f76-0448-11de-a6fe-93d51630548a
2010-03-31 19:53:13 +00:00
chartl dc802aa26f Moved CoverageStatistics to core. This will be (soon) renamed DepthOfCoverage; so please use CoverageStatistics
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3090 348d0f76-0448-11de-a6fe-93d51630548a
2010-03-29 13:32:00 +00:00
depristo 08d9ae403d better farm commands, and simple utility to convert ucsc repeat masks to interval lists
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3040 348d0f76-0448-11de-a6fe-93d51630548a
2010-03-19 13:11:06 +00:00
chartl 4bdc3b2784 automatic generation of individual and individual set import files
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3001 348d0f76-0448-11de-a6fe-93d51630548a
2010-03-15 10:36:33 +00:00
chartl d9b12b468f Adding default filter info
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@3000 348d0f76-0448-11de-a6fe-93d51630548a
2010-03-15 10:05:46 +00:00
andrewk 196bca6819 Script to split concordance files into their constituent sets and calculate summary stats from a concordance file - SNPs called and number in dbSNP
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2992 348d0f76-0448-11de-a6fe-93d51630548a
2010-03-12 22:20:44 +00:00
andrewk 9298e13201 Make annotated VCF not be broken
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2906 348d0f76-0448-11de-a6fe-93d51630548a
2010-03-01 23:22:41 +00:00
kshakir 5f9c3f3884 Outputing annotated VCF to the current directory instead of attempting to write in the directory next to the original vcf.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2869 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-22 21:31:24 +00:00
kiran 217deb9809 Changed the INFO field delimiter from a comma to a semicolon
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2847 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-16 20:44:57 +00:00
chartl 0e4b5ad9c6 Check to ensure sample status is "Complete" before writing out the bam file
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2844 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-16 15:36:42 +00:00
chartl e491b42951 Dumb little script that grabs Picard metrics (alignment, hybrid selection, insert size) from picard_aggregation given the path to the bam file; zips them up, and spits them out; for use with Firehose
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2824 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-11 14:09:30 +00:00
chartl 60f05379a7 fix typo. Check explicitly that fingerprint files exist.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2821 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-10 21:29:54 +00:00
chartl d16a1b5645 Simple python script for generating a firehose-parseable text file from MS-Dos formatted TSV spreadsheets (of the type that we get from project management). Will be deprecated in a few weeks with the advent of direct BSP ID entries, but useful until then.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2820 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-10 20:25:40 +00:00
kshakir 57a168c0db Added a header crediting the python script as a source. Looping over an arbitrary number of headers.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2804 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-07 03:38:30 +00:00
andrewk 61a67cdce4 Moving file up a directory for dependencies
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2798 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-05 19:36:13 +00:00
andrewk 58456822ab Two perl scripts (from Kristian Cibulskis) and one python script for annotating VCF files with the information generated by the cancer MAF annotation tool.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2797 348d0f76-0448-11de-a6fe-93d51630548a
2010-02-05 19:25:46 +00:00
depristo e964660df3 snpSelector now supports min and max q scores.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2751 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-31 19:38:34 +00:00
depristo 7b3c34d210 keeping a backup
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2750 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-31 15:36:25 +00:00
chartl 0fb032a436 Quick script that changes "chr#" to "#" and "chrM" to "MT" and moves mitochondria to the end of the vcf; in accordance with the 1KG reference.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2727 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-28 21:59:33 +00:00
chartl 4990139b60 A collection of python objects that are useful for VCF validation. Use 'em or don't.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2679 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-25 18:44:10 +00:00
depristo cf46e3c85f Valuable series of commands
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2641 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-20 13:58:59 +00:00
depristo 8226f4aa12 minor cleanup
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2616 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-16 20:23:20 +00:00
chartl dfe160ff77 Minor changes (additional info calculated)
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2522 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-06 16:41:01 +00:00
depristo 588006ee92 Now supports strings in command line for farm submission
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2507 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-06 13:15:40 +00:00
depristo 9fb6533549 new -a option does fast merging of already sorted files
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2500 348d0f76-0448-11de-a6fe-93d51630548a
2010-01-05 13:55:39 +00:00
depristo 89f3ee614a minor printing fix
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2492 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-30 22:14:50 +00:00
depristo fcc80e8632 Completely rewritten duplicate traversal, more free of bugs, with integration tests for count duplicates walker validated on a TCGA hybrid capture lane.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2458 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-28 23:56:49 +00:00
andrewk 4e7e0432a2 Updated SNP calling power from coverage tools to work with new UnifiedGenotyper and DepthOfCoverage tools.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2378 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-16 20:44:30 +00:00
andrewk f5e547ed6e Add ability for flat file table parsing module to skip ahead to first occurence of a regular expression (use case: consistently parsing DepthOfCoverage output for histogram section of file across file format changes)
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2377 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-16 20:38:50 +00:00
andrewk bf76019f22 Minor change to coverage evalution script, to update for new file format and add output fields
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2375 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-16 18:06:08 +00:00
depristo 0d2a761460 Bugfix for minBaseQuality to ignore deletion reads. LocusMismatch walker now allows us to skip every nths eligable site
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2357 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-15 14:38:39 +00:00
depristo 56467df49a minor improvements to snpSelector to work with hapmap chip VCF files
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2343 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-13 17:59:32 +00:00
depristo b2dfe85648 Better support for reading truth file
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2307 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-10 12:16:05 +00:00
chartl 6a4118ad3c grr, ought to actually assign it to the TRUTH_CALLS variable
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2302 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-09 23:31:46 +00:00
chartl 987fced151 Should read truth data from the parser options rather than direct from args
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2301 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-09 23:26:26 +00:00
chartl 8825211fdb Adding this to subversion so it's protected
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2299 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-09 21:26:17 +00:00
depristo 2632cb6b58 minor improvements to snp selector
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2275 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-07 03:37:14 +00:00
chartl b817db0962 Syzygy has a default LOD score of 0.91 on bases with no coverage, this is problematic. Set the minimum lod threshold to 1 because I just don't want to see that codswallop.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2268 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-04 23:29:14 +00:00
depristo 0753315156 updates to the python snp selector -- now sorts info fields and we stop printing unnecessary debugging info in vcf2table
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2265 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-04 22:16:02 +00:00
chartl 0f89a38473 forgot to commit this earlier
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2264 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-04 22:10:16 +00:00
chartl c1263e841c stop printing the debug info -- hurr
Also it turns out that sometimes there can be a call with zero total non-I/non-D bases -- so add one to numerator and denominator to prevent divide by zero error



git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@2262 348d0f76-0448-11de-a6fe-93d51630548a
2009-12-04 16:17:38 +00:00