Commit Graph

5819 Commits (cf3dbfee979d1f0bbb3fa0140615fd2bc4b73abe)

Author SHA1 Message Date
rpoplin d925f76edc Cutting down on the number of info lines in VQSR so that I can read the warning messages
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5810 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-16 13:35:51 +00:00
delangel 5a7444e186 First step in refactoring UG way of storing indel likelihoods - main motive is that rank sum annotations require per-read quality or likelihood information, and even the question "what allele of a variant is present in a read" which is trivial for SNPs may not be that straightforward for indels.
This step just changes storage of likelihoods so now we have, instead of an internal matrix, a class member which stores, as a hash table, a mapping from pileup element to an (allele, likelihood) pair. There's no functional change aside from internal data storage.
As a bonus, we get for free a 2-3x improvement in speed in calling because redundant likelihood computations are removed.
Next step will hook this up to, and redefine annotation engine interaction with UG for indel case.




git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5809 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-15 23:04:11 +00:00
depristo 3ccc08ace4 Now emits siteType = {SNP,INDEL}. Doesn't work (and may never actually work) for indels under current extended event system.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5808 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-15 19:16:09 +00:00
depristo 75db4705ab Added splitContextByReadGroup() and fixed bug in getPileupForReadGroup() that resulted in a NPE when no reads where present for a read group.
Added doc string for getNBoundRodTracks()

Intermediate commit for CalibrateGenotypeLikelihoods and GenotypeConcordanceTable, so I have a record of my work.  Not ready for public consumption.  Really looking forward to making local commits so I can track my progress without needing to push incomplete functionality up to the server.

git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5807 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-15 17:36:07 +00:00
depristo 9423652ad8 Computes how well a genotype chip covers a reference panel
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5806 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-14 15:07:28 +00:00
depristo 5e9c0d00c6 Simple R script to visualize geontype likelihood accuracy
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5805 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-14 15:05:55 +00:00
delangel fa75efb6ac Backing off - need to change pileup interface for rank sum tests before indels can be annotated with them
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5804 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-13 21:54:54 +00:00
asivache befbcd274b Computes additional stats we want to use later for filtering: median and mad for indel position with respect to starts and ends of all the reads that support it
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5803 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-13 21:19:58 +00:00
asivache 5c889580c4 Change of logic: if "read" (sequence 2) sticks out beyond the boundary of the ref (sequence 1) it is aligned to, the extra bases on the left or on the right will be softclipped in the cigar generated for such an alignment, rather than added to the firts/last M block. This also affects alignment offset: if read starts before the ref (used to be represented by a negative offset), the cigar now will start with S, and the returned offset (alignment start) will be 0.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5802 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-13 21:12:54 +00:00
delangel d4ca8d94fa Trivial change to allow indels to be annotated by rank rum tests
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5801 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-13 20:24:08 +00:00
kshakir 95fc6c0a83 Changed VR tranches from old 0.1-10 to new 100 to 90.
Using hapmap training and truth based on wiki.
Explicitly setting the ts_filter_level even though 99.0 is the default.
Recal file path now ends with with .recal.
Added ar's vcf input.
Omni rod name now omni instead of 1kg.
The VR RodBind tags had spaces in them.
Was passing both the full intervals and the chunk intervals to chunk jobs.
Switched back to chr20 for default since the VR crashes on small intervals sets with "MESSAGE: Matrix is singular."
Log files names based on the file paths + .out.
Added eval statifications by sample based on the Hybrid Selection / Whole Exome pipeline.

git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5800 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-13 14:38:56 +00:00
kshakir 08c13f3944 Using embedded GATK.
Hardcoded the reference and dbsnp since the training rods are also hardcoded, for now.
Changed freeze/chr20 to wg/chr20/cent1 to also test the heaviest known shard.
Other cleanup.
TODO: Memory command line options or have the script figure it out using FLS or similar.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5799 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-12 23:19:49 +00:00
hanna 03452c15c0 Cleanup GATKBAMIndex unit test to allow a more efficient access pattern for
FindLargeShards.  Runtime of FindLargeShards on papuan dataset is now 75min.
GATK proper should benefit as well, although the benefits might be so small
as to not be measurable.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5798 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-12 21:50:33 +00:00
dheiman 9e08a699c6 Corrected memory handling and jobName formatting issues
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5797 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-12 17:47:56 +00:00
depristo db1f9af679 Now supports multiple records in allele at sites that genotype as reference
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5796 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-12 17:36:27 +00:00
chartl 66c8fa5c48 James P says this change worked for him, so I'm committing it.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5795 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-12 16:55:18 +00:00
rpoplin a22e98a2c4 Yikes. Fixing the build
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5794 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-12 01:52:35 +00:00
rpoplin 40797f9d45 Ensuring a minimum number of variants when clustering with bad variants. Better error message when Matrix library fails to calculate inverse.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5793 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-12 01:48:37 +00:00
kshakir a20d257773 Generating extensions for org.broadinstitute.sting.gatk.datasources.reads.utilities, including FindLargeShards.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5792 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-12 00:49:31 +00:00
kshakir ec443e89cf Added pass-throughs for -Djava.io.tmpdir to javac and testng.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5791 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-11 20:56:35 +00:00
carneiro fb1be2653c A succint walker that reports GC content by interval. Taking down two old implementations of the same thing from oneoffs. Documentation added to the wiki.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5790 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-11 18:53:11 +00:00
depristo 9a1d0d7076 Simple bug fix to allow multiple records at same site when genotyping given alleles. Takes only the first record (respecting filters, SNP type, etc), and issues a warning if there is more than one valid record at a site
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5789 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-11 14:17:14 +00:00
dheiman 16db86e6cb Grid Engine backend to GATK-Queue, initial commit of implementation
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5788 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-11 13:21:45 +00:00
ebanks dfdef2d29b PLEASE READ ME! In order to prepare for the upcoming changes to VCF4, we felt it was best to split up the vcf3 and vcf4 codecs (vcf4 is not backwards compatible to vcf3 and certain changes are too complex to handle in both codecs). Using the 'VCF' rod type in the GATK will now throw a UserException for vcf3.2 or vcf3.3 files telling you to use the 'VCF3' type instead (and vice versa). Integration/unit tests have been updated. For programmers: note that there is currently a lot of code duplication in the two codecs (although I pulled out the easy stuff to a VCFCodecUtils class); however WE ARE FREEZING THE VCF3 CODEC AND WILL NO LONGER MAKE CHANGES TO IT. All updates/improvements will be targetted to the vcf4 codec only as vcf3 is there only to be able to read legacy files. People should really be using vcf4 files only.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5787 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-11 12:07:44 +00:00
delangel 852e555c00 Fix broken functionality from previous commit.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5786 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-10 18:38:25 +00:00
ebanks 8d47d2e813 Fix for Tim. It was possible for the constrained mate fixer to dump its cache in them middle of a given realignment (so the IndelRealigner was playing by the rules). No longer possible.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5785 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-10 16:48:24 +00:00
ebanks fbe7974094 Renaming for consistency
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5784 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-10 16:36:39 +00:00
delangel 3c364279f4 Add simple ability to create "X out of N" combined files: if a site is present in at least X input rods, it gets output, otherwise it's skipped, controlled with argument -minN.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5783 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-10 15:27:18 +00:00
hanna f275be6968 A 'fat shard' finder. Cranks through the indices of a BAM file or list of
BAM files looking for outliers (outliers right now are defined naively  as 
shards whose sizes are more than 5 stddevs away from the mean).  Runs in
13 minutes per chromosome on 707 low pass whole genome BAMs -- not great, but
much faster than running UG on the same region to discover anomalies.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5782 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-10 12:56:47 +00:00
kshakir 3ffc2ccd81 Implemented broad specific LSF requirement in the LSF job runner ahead of GridEngine check in by dheiman.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5781 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-09 22:14:04 +00:00
kshakir 7d21350a17 Fixed import.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5780 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-09 18:07:40 +00:00
asivache 0861451726 Print on multiple rows in standalone command line mode when the sequences are too long
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5779 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-09 13:51:00 +00:00
ebanks bf40351094 Minor update
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5778 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-08 03:48:37 +00:00
ebanks 15c7bd82a5 Fix for IndelRealigner memory problem. Now the Constrained mate fixing writer is told whether a read has been modified and, if it wasn't, can dump it when the cache needs to get flushed at places with tons of coverage.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5777 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-06 19:34:41 +00:00
rpoplin d8a761bbbd Warn the user if trying to train with too few variants
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5776 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-06 17:47:58 +00:00
hanna c2e8c460cb Factor out all testing dependencies into a separate test configuration and
only download that test configuration when running unit/integration tests.
This means that the build will (hopefully) never break because it can't
fetch a file that isn't required for the GATK to run.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5775 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-05 22:42:11 +00:00
rpoplin 1d11e88899 Adding another example call set to GATK resource bundle for use in VQSR wiki tutorial
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5774 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-05 21:16:33 +00:00
rpoplin b94d8dae17 Removing requirement of providing known track in VQSR for the non-humans. Updating placement of legend on tranche plot.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5773 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-05 20:24:06 +00:00
fromer 04f156d86b Removed extraneous import
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5772 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-05 18:51:03 +00:00
delangel 7d7ce6cf00 Two embarassing bug fixes:
a) Forgot to convert from phred to log-prob when computing gap penalties from recal table.
b) Forgot to uncomment code to correctly deal with hard-clipped bases in a read. But because of this, had to do a short term workaround to at least temporarily return class from hardClipAdaptorSequence to GATKSAMRecord. Otherwise, I get exceptions when casting because somehow some reads in HiSeq get to be SAMRecord (which GATKSAMRecord inherits from) but some reads get to be BAMRecords (which can't be cast into GATKSAMRecord), not sure why.



git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5771 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-05 17:08:34 +00:00
hanna 45d8634522 Intermediate commit: bring Google Caliper into our private repository (even
though sonatype is back up).  This will tide us over until I figure out how
to add caliper to test configuration, so that it's only swapped in when we
actually run our unit / performance tests.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5770 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-05 14:33:14 +00:00
kshakir 4d08d39849 Moved some of the java to scala conversions from production to test code as it's not needed in production and slows down the code.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5769 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-05 04:11:15 +00:00
kshakir 28b897d5de Fixed O(N^2) operation when scattering interval files.
Cleaned up intervals contig count function.

git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5768 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-05 03:32:35 +00:00
carneiro 3882d1b9c0 fixing the build \o/
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5767 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-05 00:57:49 +00:00
kshakir 8ad547e6c2 Fixed another interval bug where dividing up N intervals into N parts wasn't working.
Minor updates to the FCPTest to match the changes due to using the old indel caller.


git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5766 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-04 20:49:35 +00:00
rpoplin 825682f58c oops, putting the script back into a sensible state
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5765 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-04 20:17:05 +00:00
rpoplin b5ab2274f6 Committing the base qscript I used to make the Phase1 Project Consensus. Does per-population cleaning and simplifyBAM, and then per-analysis-panel calling with genotype given alleles. Combines info fields using the panel with max AC.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5764 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-04 20:13:26 +00:00
corin b4654b0f47 Status messages to user added
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5763 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-04 20:10:47 +00:00
corin bcc688c1e9 small formatting change
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5762 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-04 20:01:20 +00:00
corin 1410327901 Cmd line argument reference fix
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5761 348d0f76-0448-11de-a6fe-93d51630548a
2011-05-04 20:00:14 +00:00