zzh
/
gatk-3.8
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Prep MuTect2 and ContEst for release 

Renamed M2 to MuTect2
    Renamed ContaminationWalker to ContEst
    Refactored related tests and usages (including in Queue scripts)
    Moved M2 and ContEst + accompanying classes from private to protected
    Made QSS a StandardSomaticAnnotation (new annotation group/interface) to prevent it from being sucked in with the rest of the StandardAnnotation group
This commit is contained in:
Geraldine Van der Auwera 2015-11-20 23:12:44 -05:00
parent 5a3756410c
commit bf875974d1
26 changed files with 5519 additions and 19 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

283
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/AssignSomaticStatus.java 100644
View File

@ -0,0 +1,283 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer;
import org.broadinstitute.gatk.utils.commandline.Argument;
import org.broadinstitute.gatk.utils.commandline.ArgumentCollection;
import org.broadinstitute.gatk.utils.commandline.Output;
import org.broadinstitute.gatk.engine.arguments.StandardVariantContextInputArgumentCollection;
import org.broadinstitute.gatk.utils.contexts.AlignmentContext;
import org.broadinstitute.gatk.utils.contexts.ReferenceContext;
import org.broadinstitute.gatk.utils.refdata.RefMetaDataTracker;
import org.broadinstitute.gatk.engine.walkers.RodWalker;
import org.broadinstitute.gatk.engine.walkers.TreeReducible;
import org.broadinstitute.gatk.utils.MathUtils;
import org.broadinstitute.gatk.utils.QualityUtils;
import org.broadinstitute.gatk.engine.SampleUtils;
import org.broadinstitute.gatk.utils.exceptions.UserException;
import org.broadinstitute.gatk.engine.GATKVCFUtils;
import org.broadinstitute.gatk.utils.variant.GATKVariantContextUtils;
import htsjdk.variant.variantcontext.Genotype;
import htsjdk.variant.variantcontext.VariantContext;
import htsjdk.variant.variantcontext.VariantContextBuilder;
import htsjdk.variant.variantcontext.VariantContextUtils;
import htsjdk.variant.variantcontext.writer.VariantContextWriter;
import htsjdk.variant.vcf.*;
import java.util.*;
/**
* Assigns somatic status to a set of calls
*/
public class AssignSomaticStatus extends RodWalker<Integer, Integer> implements TreeReducible<Integer> {
@ArgumentCollection
protected StandardVariantContextInputArgumentCollection variantCollection = new StandardVariantContextInputArgumentCollection();
@Argument(shortName="n", fullName="normalSample", required=true, doc="The normal sample")
public String normalSample;
@Argument(shortName="t", fullName="tumorSample", required=true, doc="The tumor sample")
public String tumorSample;
@Argument(shortName="somaticPriorQ", fullName="somaticPriorQ", required=false, doc="Phred-scaled probability that a site is a somatic mutation")
public byte somaticPriorQ = 60;
@Argument(shortName="somaticMinLOD", fullName="somaticMinLOD", required=false, doc="Phred-scaled min probability that a site should be called somatic mutation")
public byte somaticMinLOD = 1;
@Argument(shortName="minimalVCF", fullName="minimalVCF", required=false, doc="If provided, the attributes of the output VCF will only contain the somatic status fields")
public boolean minimalVCF = false;
@Output
protected VariantContextWriter vcfWriter = null;
private final String SOMATIC_LOD_TAG_NAME = "SOMATIC_LOD";
private final String SOMATIC_AC_TAG_NAME = "SOMATIC_AC";
private final String SOMATIC_NONREF_TAG_NAME = "SOMATIC_NNR";
private final Set<String> samples = new HashSet<String>(2);
/**
* Parse the familial relationship specification, and initialize VCF writer
*/
public void initialize() {
List<String> rodNames = new ArrayList<String>();
rodNames.add(variantCollection.variants.getName());
Map<String, VCFHeader> vcfRods = GATKVCFUtils.getVCFHeadersFromRods(getToolkit(), rodNames);
Set<String> vcfSamples = SampleUtils.getSampleList(vcfRods, GATKVariantContextUtils.GenotypeMergeType.REQUIRE_UNIQUE);
// set up tumor and normal samples
if ( !vcfSamples.contains(normalSample) )
throw new UserException.BadArgumentValue("--normalSample", "the normal sample " + normalSample + " doesn't match any sample from the input VCF");
if ( !vcfSamples.contains(tumorSample) )
throw new UserException.BadArgumentValue("--tumorSample", "the tumor sample " + tumorSample + " doesn't match any sample from the input VCF");
logger.info("Normal sample: " + normalSample);
logger.info("Tumor sample: " + tumorSample);
Set<VCFHeaderLine> headerLines = new HashSet<VCFHeaderLine>();
headerLines.addAll(GATKVCFUtils.getHeaderFields(this.getToolkit()));
headerLines.add(new VCFInfoHeaderLine(VCFConstants.SOMATIC_KEY, 0, VCFHeaderLineType.Flag, "Is this a confidently called somatic mutation"));
headerLines.add(new VCFInfoHeaderLine(SOMATIC_LOD_TAG_NAME, 1, VCFHeaderLineType.Float, "log10 probability that the site is a somatic mutation"));
headerLines.add(new VCFInfoHeaderLine(SOMATIC_AC_TAG_NAME, 1, VCFHeaderLineType.Integer, "Allele count of samples with somatic event"));
headerLines.add(new VCFInfoHeaderLine(SOMATIC_NONREF_TAG_NAME, 1, VCFHeaderLineType.Integer, "Number of samples with somatic event"));
samples.add(normalSample);
samples.add(tumorSample);
vcfWriter.writeHeader(new VCFHeader(headerLines, samples));
}
private double log10pNonRefInSamples(final VariantContext vc, final String sample) {
return log10PLFromSamples(vc, sample, false);
}
private double log10pRefInSamples(final VariantContext vc, final String sample) {
return log10PLFromSamples(vc, sample, true);
}
private double log10PLFromSamples(final VariantContext vc, final String sample, boolean calcRefP) {
Genotype g = vc.getGenotype(sample);
double log10pSample = -1000;
if ( ! g.isNoCall() ) {
final double[] gLikelihoods = MathUtils.normalizeFromLog10(g.getLikelihoods().getAsVector());
log10pSample = Math.log10(calcRefP ? gLikelihoods[0] : 1 - gLikelihoods[0]);
log10pSample = Double.isInfinite(log10pSample) ? -10000 : log10pSample;
}
return log10pSample;
}
private int calculateTumorAC(final VariantContext vc) {
int ac = 0;
switch ( vc.getGenotype(tumorSample).getType() ) {
case HET: ac += 1; break;
case HOM_VAR: ac += 2; break;
case NO_CALL: case UNAVAILABLE: case HOM_REF: break;
}
return ac;
}
private int calculateTumorNNR(final VariantContext vc) {
int nnr = 0;
switch ( vc.getGenotype(tumorSample).getType() ) {
case HET: case HOM_VAR: nnr += 1; break;
case NO_CALL: case UNAVAILABLE: case HOM_REF: break;
}
return nnr;
}
/**
* P(somatic | D)
* = P(somatic) * P(D | somatic)
* = P(somatic) * P(D | normals are ref) * P(D | tumors are non-ref)
*
* P(! somatic | D)
* = P(! somatic) * P(D | ! somatic)
* = P(! somatic) *
* * ( P(D | normals are non-ref) * P(D | tumors are non-ref) [germline]
* + P(D | normals are ref) * P(D | tumors are ref)) [no-variant at all]
*
* @param vc
* @return
*/
private double calcLog10pSomatic(final VariantContext vc) {
// walk over tumors
double log10pNonRefInTumors = log10pNonRefInSamples(vc, tumorSample);
double log10pRefInTumors = log10pRefInSamples(vc, tumorSample);
// walk over normals
double log10pNonRefInNormals = log10pNonRefInSamples(vc, normalSample);
double log10pRefInNormals = log10pRefInSamples(vc, normalSample);
// priors
double log10pSomaticPrior = QualityUtils.qualToErrorProbLog10(somaticPriorQ);
double log10pNotSomaticPrior = Math.log10(1 - QualityUtils.qualToErrorProb(somaticPriorQ));
double log10pNotSomaticGermline = log10pNonRefInNormals + log10pNonRefInTumors;
double log10pNotSomaticNoVariant = log10pRefInNormals + log10pRefInTumors;
double log10pNotSomatic = log10pNotSomaticPrior + MathUtils.log10sumLog10(new double[]{log10pNotSomaticGermline, log10pNotSomaticNoVariant});
double log10pSomatic = log10pSomaticPrior + log10pNonRefInTumors + log10pRefInNormals;
double lod = log10pSomatic - log10pNotSomatic;
return Double.isInfinite(lod) ? -10000 : lod;
}
/**
* For each variant in the file, determine the phasing for the child and replace the child's genotype with the trio's genotype
*
* @param tracker the reference meta-data tracker
* @param ref the reference context
* @param context the alignment context
* @return null
*/
@Override
public Integer map(RefMetaDataTracker tracker, ReferenceContext ref, AlignmentContext context) {
if (tracker != null) {
for ( VariantContext vc : tracker.getValues(variantCollection.variants, context.getLocation()) ) {
vc = vc.subContextFromSamples(samples);
if ( !vc.isPolymorphicInSamples() )
continue;
double log10pSomatic = calcLog10pSomatic(vc);
// write in the somatic status probability
Map<String, Object> attrs = new HashMap<String, Object>(); // vc.getAttributes());
if ( ! minimalVCF ) attrs.putAll(vc.getAttributes());
attrs.put(SOMATIC_LOD_TAG_NAME, log10pSomatic);
if ( log10pSomatic > somaticMinLOD ) {
attrs.put(VCFConstants.SOMATIC_KEY, true);
attrs.put(SOMATIC_NONREF_TAG_NAME, calculateTumorNNR(vc));
attrs.put(SOMATIC_AC_TAG_NAME, calculateTumorAC(vc));
}
final VariantContextBuilder builder = new VariantContextBuilder(vc).attributes(attrs);
VariantContextUtils.calculateChromosomeCounts(builder, false);
VariantContext newvc = builder.make();
vcfWriter.add(newvc);
}
return null;
}
return null;
}
/**
* Provide an initial value for reduce computations.
*
* @return Initial value of reduce.
*/
@Override
public Integer reduceInit() {
return null;
}
/**
* Reduces a single map with the accumulator provided as the ReduceType.
*
* @param value result of the map.
* @param sum accumulator for the reduce.
* @return accumulator with result of the map taken into account.
*/
@Override
public Integer reduce(Integer value, Integer sum) {
return null;
}
@Override
public Integer treeReduce(Integer sum1, Integer sum2) {
return reduce(sum1, sum2);
}
}

190
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/BaseQualitySumPerAlleleBySample.java 100644
View File

@ -0,0 +1,190 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer;
import htsjdk.variant.variantcontext.Allele;
import htsjdk.variant.variantcontext.Genotype;
import htsjdk.variant.variantcontext.GenotypeBuilder;
import htsjdk.variant.variantcontext.VariantContext;
import htsjdk.variant.vcf.VCFFormatHeaderLine;
import htsjdk.variant.vcf.VCFHeaderLineCount;
import htsjdk.variant.vcf.VCFHeaderLineType;
import org.apache.log4j.Logger;
import org.broadinstitute.gatk.tools.walkers.annotator.interfaces.AnnotatorCompatible;
import org.broadinstitute.gatk.tools.walkers.annotator.interfaces.GenotypeAnnotation;
import org.broadinstitute.gatk.tools.walkers.annotator.interfaces.StandardSomaticAnnotation;
import org.broadinstitute.gatk.tools.walkers.cancer.m2.MuTect2;
import org.broadinstitute.gatk.utils.QualityUtils;
import org.broadinstitute.gatk.utils.contexts.AlignmentContext;
import org.broadinstitute.gatk.utils.contexts.ReferenceContext;
import org.broadinstitute.gatk.utils.exceptions.GATKException;
import org.broadinstitute.gatk.utils.genotyper.MostLikelyAllele;
import org.broadinstitute.gatk.utils.genotyper.PerReadAlleleLikelihoodMap;
import org.broadinstitute.gatk.utils.refdata.RefMetaDataTracker;
import org.broadinstitute.gatk.utils.sam.GATKSAMRecord;
import org.broadinstitute.gatk.utils.sam.ReadUtils;
import org.broadinstitute.gatk.utils.variant.GATKVCFConstants;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
/**
* Sum of evidence in reads supporting each allele for each sample
*
* <p>In the domain of somatic variants, a variant call can be supported by a few high quality reads. The
* BaseQualitySumPerAlleleBySample annotation aims to give the user an estimate of the quality of the evidence supporting
* a variant.</p>
*
* <h3>Notes</h3>
* BaseQualitySumPerAlleleBySample is called and used by MuTect2 for variant filtering. This annotation is applied to SNPs
* and INDELs. Qualities are not literal base qualities, but instead are derived from the per-allele likelihoods derived
* from the assembly engine.
*
* <h3>Caveats</h3>
* <ul>
* <li>At this time, BaseQualitySumPerAlleleBySample can only be called from MuTect2</li>
* </ul>
*/
public class BaseQualitySumPerAlleleBySample extends GenotypeAnnotation implements StandardSomaticAnnotation {
private final static Logger logger = Logger.getLogger(BaseQualitySumPerAlleleBySample.class);
private boolean walkerIdentityCheckWarningLogged = false;
public List<String> getKeyNames() { return Arrays.asList(GATKVCFConstants.QUALITY_SCORE_SUM_KEY); }
public void annotate(final RefMetaDataTracker tracker,
final AnnotatorCompatible walker,
final ReferenceContext ref,
final AlignmentContext stratifiedContext,
final VariantContext vc,
final Genotype g,
final GenotypeBuilder gb,
final PerReadAlleleLikelihoodMap alleleLikelihoodMap) {
// Can only call from MuTect2
if ( !(walker instanceof MuTect2) ) {
if ( !walkerIdentityCheckWarningLogged ) {
if ( walker != null )
logger.warn("Annotation will not be calculated, can only be called from MuTect2, not " + walker.getClass().getName());
else
logger.warn("Annotation will not be calculated, can only be called from MuTect2");
walkerIdentityCheckWarningLogged = true;
}
return;
}
if ( g == null || !g.isCalled() || ( stratifiedContext == null && alleleLikelihoodMap == null) )
return;
if (alleleLikelihoodMap != null) {
annotateWithLikelihoods(alleleLikelihoodMap, vc, gb);
}
}
protected void annotateWithLikelihoods(final PerReadAlleleLikelihoodMap perReadAlleleLikelihoodMap, final VariantContext vc, final GenotypeBuilder gb) {
final ArrayList<Double> refQuals = new ArrayList<>();
final ArrayList<Double> altQuals = new ArrayList<>();
// clean up
fillQualsFromLikelihoodMap(vc.getAlleles(), vc.getStart(), perReadAlleleLikelihoodMap, refQuals, altQuals);
double refQualSum = 0;
for(Double d : refQuals) { refQualSum += d; }
double altQualSum = 0;
for(Double d : altQuals) { altQualSum += d; }
gb.attribute(GATKVCFConstants.QUALITY_SCORE_SUM_KEY, new Integer[]{ (int) refQualSum, (int) altQualSum});
}
public List<VCFFormatHeaderLine> getDescriptions() {
return Arrays.asList(new VCFFormatHeaderLine(getKeyNames().get(0), VCFHeaderLineCount.A, VCFHeaderLineType.Integer, "Sum of base quality scores for each allele"));
}
// from rank sum test */
protected void fillQualsFromLikelihoodMap(final List<Allele> alleles,
final int refLoc,
final PerReadAlleleLikelihoodMap likelihoodMap,
final List<Double> refQuals,
final List<Double> altQuals) {
for ( final Map.Entry<GATKSAMRecord, Map<Allele,Double>> el : likelihoodMap.getLikelihoodReadMap().entrySet() ) {
final MostLikelyAllele a = PerReadAlleleLikelihoodMap.getMostLikelyAllele(el.getValue());
if ( ! a.isInformative() )
continue; // read is non-informative
final GATKSAMRecord read = el.getKey();
if ( isUsableRead(read) ) {
final Double value = getBaseQualityForRead(read, refLoc);
if ( value == null )
continue;
if ( a.getMostLikelyAllele().isReference() )
refQuals.add(value);
else if ( alleles.contains(a.getMostLikelyAllele()) )
altQuals.add(value);
}
}
}
protected boolean isUsableRead(final GATKSAMRecord read) {
return !( read.getMappingQuality() == 0 ||
read.getMappingQuality() == QualityUtils.MAPPING_QUALITY_UNAVAILABLE );
}
protected Double getBaseQualityForRead(final GATKSAMRecord read, final int refLoc) {
return (double)read.getBaseQualities()[ReadUtils.getReadCoordinateForReferenceCoordinateUpToEndOfRead(read, refLoc, ReadUtils.ClippingTail.RIGHT_TAIL)];
}
}

197
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/OxoGReadCounts.java 100644
View File

@ -0,0 +1,197 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer;
import htsjdk.variant.variantcontext.Allele;
import htsjdk.variant.variantcontext.Genotype;
import htsjdk.variant.variantcontext.GenotypeBuilder;
import htsjdk.variant.variantcontext.VariantContext;
import htsjdk.variant.vcf.VCFFormatHeaderLine;
import htsjdk.variant.vcf.VCFHeaderLineCount;
import htsjdk.variant.vcf.VCFHeaderLineType;
import org.apache.log4j.Logger;
import org.broadinstitute.gatk.tools.walkers.annotator.interfaces.AnnotatorCompatible;
import org.broadinstitute.gatk.tools.walkers.annotator.interfaces.GenotypeAnnotation;
import org.broadinstitute.gatk.tools.walkers.annotator.interfaces.StandardAnnotation;
import org.broadinstitute.gatk.tools.walkers.cancer.m2.MuTect2;
import org.broadinstitute.gatk.utils.QualityUtils;
import org.broadinstitute.gatk.utils.contexts.AlignmentContext;
import org.broadinstitute.gatk.utils.contexts.ReferenceContext;
import org.broadinstitute.gatk.utils.exceptions.GATKException;
import org.broadinstitute.gatk.utils.genotyper.MostLikelyAllele;
import org.broadinstitute.gatk.utils.genotyper.PerReadAlleleLikelihoodMap;
import org.broadinstitute.gatk.utils.refdata.RefMetaDataTracker;
import org.broadinstitute.gatk.utils.sam.GATKSAMRecord;
import org.broadinstitute.gatk.utils.sam.ReadUtils;
import org.broadinstitute.gatk.utils.variant.GATKVCFConstants;
import org.broadinstitute.gatk.utils.variant.GATKVCFHeaderLines;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
/**
* Count of read pairs in the F1R2 and F2R1 configurations supporting the reference and alternate alleles
*
* <p>This is an annotation that gathers information about the read pair configuration for the reads supporting each
* allele. It can be used along with downstream filtering steps to identify and filter out erroneous variants that occur
* with higher frequency in one read pair orientation.</p>
*
* <h3>References</h3>
* <p>For more details about the mechanism of oxoG artifact generation, see <a href='http://www.ncbi.nlm.nih.gov/pubmed/23303777' target='_blank'>
* "Discovery and characterization of artefactual mutations in deep coverage targeted capture sequencing data due to oxidative DNA damage during sample preparation."
* by Costello et al.</a></p>
*
* <h3>Caveats</h3>
* <ul>
* <li>At present, this annotation can only be called from MuTect2</li>
* <li>The FOXOG annotation is only calculated for SNPs</li>
* </ul>
*/
public class OxoGReadCounts extends GenotypeAnnotation {
private final static Logger logger = Logger.getLogger(OxoGReadCounts.class);
private boolean walkerIdentityCheckWarningLogged = false;
Allele refAllele;
Allele altAllele;
public List<String> getKeyNames() {
return Arrays.asList(GATKVCFConstants.OXOG_ALT_F1R2_KEY, GATKVCFConstants.OXOG_ALT_F2R1_KEY, GATKVCFConstants.OXOG_REF_F1R2_KEY, GATKVCFConstants.OXOG_REF_F2R1_KEY, GATKVCFConstants.OXOG_FRACTION_KEY);
}
public void annotate(final RefMetaDataTracker tracker,
final AnnotatorCompatible walker,
final ReferenceContext ref,
final AlignmentContext stratifiedContext,
final VariantContext vc,
final Genotype g,
final GenotypeBuilder gb,
final PerReadAlleleLikelihoodMap alleleLikelihoodMap) {
// Can only call from MuTect2
if ( !(walker instanceof MuTect2) ) {
if ( !walkerIdentityCheckWarningLogged ) {
if ( walker != null )
logger.warn("Annotation will not be calculated, can only be called from MuTect2, not " + walker.getClass().getName());
else
logger.warn("Annotation will not be calculated, can only be called from MuTect2");
walkerIdentityCheckWarningLogged = true;
}
return;
}
if (g == null || !g.isCalled() || (stratifiedContext == null && alleleLikelihoodMap == null))
return;
refAllele = vc.getReference();
altAllele = vc.getAlternateAllele(0);
if (alleleLikelihoodMap != null) {
annotateWithLikelihoods(alleleLikelihoodMap, vc, gb);
}
}
protected void annotateWithLikelihoods(final PerReadAlleleLikelihoodMap perReadAlleleLikelihoodMap, final VariantContext vc, final GenotypeBuilder gb) {
int ALT_F1R2, ALT_F2R1, REF_F1R2, REF_F2R1;
ALT_F1R2 = ALT_F2R1 = REF_F1R2 = REF_F2R1 = 0;
double numerator, denominator;
for ( final Map.Entry<GATKSAMRecord, Map<Allele,Double>> el : perReadAlleleLikelihoodMap.getLikelihoodReadMap().entrySet() ) {
final MostLikelyAllele a = PerReadAlleleLikelihoodMap.getMostLikelyAllele(el.getValue());
if ( ! a.isInformative() || ! isUsableRead(el.getKey()))
continue; // read is non-informative or MQ0
if (a.getAlleleIfInformative().equals(refAllele, true) && el.getKey().getReadPairedFlag()) {
if (el.getKey().getReadNegativeStrandFlag() == el.getKey().getFirstOfPairFlag())
REF_F2R1++;
else
REF_F1R2++;
}
else if (a.getAlleleIfInformative().equals(altAllele,true) && el.getKey().getReadPairedFlag()){
if (el.getKey().getReadNegativeStrandFlag() == el.getKey().getFirstOfPairFlag())
ALT_F2R1++;
else
ALT_F1R2++;
}
}
denominator = ALT_F1R2 + ALT_F2R1;
Double fOxoG = null;
if (vc.isSNP() && denominator > 0) {
if (refAllele.equals(Allele.create((byte) 'C', true)) || refAllele.equals(Allele.create((byte) 'A', true)))
numerator = ALT_F2R1;
else
numerator = ALT_F1R2;
fOxoG = (float) numerator / denominator;
}
gb.attribute(GATKVCFConstants.OXOG_ALT_F1R2_KEY, new Integer(ALT_F1R2));
gb.attribute(GATKVCFConstants.OXOG_ALT_F2R1_KEY, new Integer(ALT_F2R1));
gb.attribute(GATKVCFConstants.OXOG_REF_F1R2_KEY, new Integer(REF_F1R2));
gb.attribute(GATKVCFConstants.OXOG_REF_F2R1_KEY, new Integer(REF_F2R1));
gb.attribute(GATKVCFConstants.OXOG_FRACTION_KEY, fOxoG);
}
public List<VCFFormatHeaderLine> getDescriptions() {
return Arrays.asList(GATKVCFHeaderLines.getFormatLine(GATKVCFConstants.OXOG_ALT_F1R2_KEY),
GATKVCFHeaderLines.getFormatLine(GATKVCFConstants.OXOG_ALT_F2R1_KEY),
GATKVCFHeaderLines.getFormatLine(GATKVCFConstants.OXOG_REF_F1R2_KEY),
GATKVCFHeaderLines.getFormatLine(GATKVCFConstants.OXOG_REF_F2R1_KEY),
GATKVCFHeaderLines.getFormatLine(GATKVCFConstants.OXOG_FRACTION_KEY));
}
protected boolean isUsableRead(final GATKSAMRecord read) {
return !( read.getMappingQuality() == 0 ||
read.getMappingQuality() == QualityUtils.MAPPING_QUALITY_UNAVAILABLE );
}
}

186
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/contamination/AnnotatePopulationAFWalker.java 100755
View File

@ -0,0 +1,186 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer.contamination;
import org.broadinstitute.gatk.utils.commandline.Argument;
import org.broadinstitute.gatk.utils.commandline.Input;
import org.broadinstitute.gatk.utils.commandline.Output;
import org.broadinstitute.gatk.utils.commandline.RodBinding;
import org.broadinstitute.gatk.utils.contexts.AlignmentContext;
import org.broadinstitute.gatk.utils.contexts.ReferenceContext;
import org.broadinstitute.gatk.utils.refdata.RefMetaDataTracker;
import org.broadinstitute.gatk.engine.samples.Sample;
import org.broadinstitute.gatk.engine.walkers.DataSource;
import org.broadinstitute.gatk.engine.walkers.Requires;
import org.broadinstitute.gatk.engine.walkers.RodWalker;
import org.broadinstitute.gatk.engine.walkers.TreeReducible;
import htsjdk.variant.vcf.VCFHeader;
import htsjdk.variant.vcf.VCFHeaderLine;
import org.broadinstitute.gatk.utils.exceptions.UserException;
import htsjdk.variant.variantcontext.Allele;
import htsjdk.variant.variantcontext.Genotype;
import htsjdk.variant.variantcontext.VariantContext;
import htsjdk.variant.variantcontext.VariantContextBuilder;
import htsjdk.variant.variantcontext.writer.VariantContextWriter;
import java.util.*;
/**
* Given a input VCF representing a collection of populations, split the input into each population, and annotate each record with population allele frequencies
*/
// @Requires(DataSource.SAMPLE) <- require the sample data when this works
public class AnnotatePopulationAFWalker extends RodWalker<Integer, Integer> implements TreeReducible<Integer> {
// control the output
@Output(doc="File to which variants should be written",required=true)
protected VariantContextWriter writer = null;
// our mapping of population to sample list
private final Map<String, List<Sample>> popMapping = new LinkedHashMap<String, List<Sample>>();
@Input(fullName="population", shortName = "pop", doc="the VCF containing large populations of samples", required=true)
public RodBinding<VariantContext> pop;
// either load the lanes into our name list, or the samples, depending on the command line parameters
public void initialize() {
// get the sample information
for (Sample sp: getToolkit().getSampleDB().getSamples())
if (sp.getOtherPhenotype() != null) {
if (!popMapping.containsKey(sp.getOtherPhenotype()))
popMapping.put(sp.getOtherPhenotype(),new ArrayList<Sample>());
popMapping.get(sp.getOtherPhenotype()).add(sp);
}
// this is a stop-gap until the @Requires tag is working with sample information
if (popMapping.size() == 0)
throw new UserException.BadInput("we require a sample file that contains population information. Please see the wiki about how to supply one");
// setup our VCF
// TODO: add code to get the samples from the input VCF, if they set 'preserveGenotypes' above
Set<VCFHeaderLine> hInfo = new HashSet<VCFHeaderLine>();
VCFHeader vcfHeader = new VCFHeader(hInfo);
writer.writeHeader(vcfHeader);
}
// boilerplate code - the standard reduce function for integers
@Override public Integer reduceInit() { return 0; }
@Override public Integer reduce(Integer value, Integer sum) { return(value + sum); }
public Integer treeReduce(Integer lhs, Integer rhs) { return lhs + rhs; }
@Override
public Integer map(RefMetaDataTracker tracker, ReferenceContext ref, AlignmentContext context) {
if (tracker == null) return 0;
// get the variant contexts, and return if we have anything other than one record at this site
Collection<VariantContext> vcs = tracker.getValues(pop);
if (vcs.size() != 1) return 0;
VariantContext originalVC = vcs.iterator().next();
if (!originalVC.isSNP()) return 0;
VariantContext vc = originalVC;
// get the list of alleles
List<Allele> vcAlleles = vc.getAlleles();
// setup the mapping of allele to population map
Map<String,Map<Allele,String>> popToAlleleFreq = new LinkedHashMap<String,Map<Allele,String>>();
// initialize all pops
Map<Allele,Integer> allPopAC = new LinkedHashMap<Allele,Integer>();
int allPopTotal = 0;
for (Allele a : vcAlleles) allPopAC.put(a,0);
// find the sub-population allele frequencies, and annotate them
for (Map.Entry<String, List<Sample>> pop : popMapping.entrySet()) {
Map<Allele,Integer> thisPopAC = new LinkedHashMap<Allele,Integer>();
int total = 0;
for (Allele a : vcAlleles) thisPopAC.put(a,0);
for (Sample s : pop.getValue()) {
Genotype g = vc.getGenotype(s.getID());
if (g == null) continue;
for (Allele a : vcAlleles) {
int count = a.length();
total += count;
thisPopAC.put(a,thisPopAC.get(a) + count);
allPopTotal += count;
allPopAC.put(a, allPopAC.get(a) + count);
}
}
Map<Allele,String> thisPopAF = new LinkedHashMap<Allele,String>();
for (Map.Entry<Allele,Integer> entry : thisPopAC.entrySet())
thisPopAF.put(entry.getKey(),String.format("%1.5f", (total == 0) ? 0 : (double)entry.getValue()/(double)total));
popToAlleleFreq.put(pop.getKey(),thisPopAF);
}
// add the all pops value as well
Map<Allele, String> allPopAF = new LinkedHashMap<Allele, String>();
for (Map.Entry<Allele,Integer> entry : allPopAC.entrySet())
allPopAF.put(entry.getKey(), String.format("%1.5f", (allPopTotal == 0) ? 0 : (double)entry.getValue()/(double)allPopTotal));
popToAlleleFreq.put("ALL", allPopAF);
// add the population af annotations
VariantContextBuilder vcb = new VariantContextBuilder(vc);
Map<String,Object> popToAlleleFreqAsObject = new LinkedHashMap<String,Object>();
for (Map.Entry<String,Map<Allele,String>> mp : popToAlleleFreq.entrySet()) {
popToAlleleFreqAsObject.put(mp.getKey(),(Object)mp.getValue());
}
vcb.attributes(popToAlleleFreqAsObject);
writer.add(vc);
return 1;
}
}

729
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/contamination/ContEst.java 100755
View File

@ -0,0 +1,729 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer.contamination;
import htsjdk.samtools.SAMReadGroupRecord;
import htsjdk.samtools.util.StringUtil;
import org.broadinstitute.gatk.engine.CommandLineGATK;
import org.broadinstitute.gatk.engine.walkers.*;
import org.broadinstitute.gatk.tools.walkers.genotyper.afcalc.AFCalculatorProvider;
import org.broadinstitute.gatk.tools.walkers.genotyper.afcalc.FixedAFCalculatorProvider;
import org.broadinstitute.gatk.utils.commandline.*;
import org.broadinstitute.gatk.engine.GenomeAnalysisEngine;
import org.broadinstitute.gatk.utils.contexts.AlignmentContext;
import org.broadinstitute.gatk.utils.contexts.ReferenceContext;
import org.broadinstitute.gatk.utils.help.DocumentedGATKFeature;
import org.broadinstitute.gatk.utils.help.HelpConstants;
import org.broadinstitute.gatk.utils.sam.SAMReaderID;
import org.broadinstitute.gatk.utils.refdata.RefMetaDataTracker;
import org.broadinstitute.gatk.tools.walkers.genotyper.UnifiedArgumentCollection;
import org.broadinstitute.gatk.tools.walkers.genotyper.UnifiedGenotypingEngine;
import org.broadinstitute.gatk.tools.walkers.genotyper.VariantCallContext;
import org.broadinstitute.gatk.utils.GenomeLoc;
import org.broadinstitute.gatk.utils.exceptions.GATKException;
import org.broadinstitute.gatk.utils.exceptions.UserException;
import org.broadinstitute.gatk.utils.pileup.ReadBackedPileup;
import htsjdk.variant.variantcontext.*;
import java.io.*;
import java.util.*;
/**
* Estimate cross-sample contamination
*
* This tool determine the percent contamination of an input bam by sample, by lane, or in aggregate across all the input reads.
*
* <h3>Usage examples</h3>
* <p>These are example commands that show how to run ContEst for typical use cases. Square brackets ("[ ]")
* indicate optional arguments. Note that parameter values and/or resources shown here may not be the latest recommended; see the Best Practices documentation for detailed recommendations. </p>
*
* <br />
* <h4>Contamination estimation using a VCF containing the normal sample's genotypes (as might be derived from a genotyping array)</h4>
* <pre>
* java
* -jar GenomeAnalysisTK.jar \
* -T ContEst \
* -R reference.fasta \
* -I tumor.bam \
* --genotypes normalGenotypes.vcf \
* --popFile populationAlleleFrequencies.vcf \
* -L populationSites.interval_list
* [-L targets.interval_list] \
* -isr INTERSECTION \
* -o output.txt
* </pre>
*
* <br />
* <h4>Contamination estimation using the normal BAM for genotyping on-the-fly</h4>
* <pre>
* java
* -jar GenomeAnalysisTK.jar \
* -T ContEst \
* -R reference.fasta \
* -I:eval tumor.bam \
* -I:genotype normal.bam \
* --popFile populationAlleleFrequencies.vcf \
* -L populationSites.interval_list
* [-L targets.interval_list] \
* -isr INTERSECTION \
* -o output.txt
* </pre>
*
*<h3>Output</h3>
* A text file containing estimated percent contamination, as well as error bars on this estimate.
*
* <h3>Notes</h3>
* Multiple modes are supported simultaneously, e.g. contamination by sample and readgroup can be computed in the same run.
*/
@DocumentedGATKFeature( groupName = HelpConstants.DOCS_CAT_QC, extraDocs = {CommandLineGATK.class} )
@Allows(value = {DataSource.READS, DataSource.REFERENCE})
@Requires(value = {DataSource.READS, DataSource.REFERENCE}, referenceMetaData = @RMD(name = "genotypes", type = VariantContext.class))
@By(DataSource.READS)
public class ContEst extends RodWalker<Map<String, Map<String, ContaminationStats>>, ContaminationResults> {
// ------------------------------------------------------------------------------------------------------------------------------------------------------
// Some constants we use
// ------------------------------------------------------------------------------------------------------------------------------------------------------
/** what type of run stats would we like: */
public enum ContaminationRunType {
SAMPLE, // calculate contamination for each sample
READGROUP, // for each read group
META // for all inputs as a single source
}
// ------------------------------------------------------------------------------------------------------------------------------------------------------
// inputs
// ------------------------------------------------------------------------------------------------------------------------------------------------------
// the genotypes ROD; this contains information about the genotypes from our sample
@Input(fullName="genotypes", shortName = "genotypes", doc="the genotype information for our sample", required=false)
public RodBinding<VariantContext> genotypes;
// the population information; the allele frequencies for each position in known populations
@Input(fullName="popfile", shortName = "pf", doc="the variant file containing information about the population allele frequencies", required=true)
public RodBinding<VariantContext> pop;
// ------------------------------------------------------------------------------------------------------------------------------------------------------
// outputs and args
// ------------------------------------------------------------------------------------------------------------------------------------------------------
@Output
PrintStream out; // the general output of the tool
@Argument(fullName = "min_qscore", required = false, doc = "threshold for minimum base quality score")
public int MIN_QSCORE = 20;
@Argument(fullName = "min_mapq", required = false, doc = "threshold for minimum mapping quality score")
public int MIN_MAPQ = 20;
@Argument(fullName = "trim_fraction", doc = "at most, what fraction of sites should be trimmed based on BETA_THRESHOLD", required = false)
public double TRIM_FRACTION = 0.01;
@Argument(fullName = "beta_threshold", doc = "threshold for p(f>=0.5) to trim", required = false)
public double BETA_THRESHOLD = 0.95;
@Argument(shortName = "llc", fullName = "lane_level_contamination", doc = "set to META (default), SAMPLE or READGROUP to produce per-bam, per-sample or per-lane estimates", required = false)
private Set<ContaminationRunType> laneStats = null;
@Argument(shortName = "sn", fullName = "sample_name", doc = "The sample name; used to extract the correct genotypes from mutli-sample truth vcfs", required = false)
private String sampleName = "unknown";
@Argument(shortName = "pc", fullName = "precision", doc = "the degree of precision to which the contamination tool should estimate (e.g. the bin size)", required = false)
private double precision = 0.1;
@Argument(shortName = "br", fullName = "base_report", doc = "Where to write a full report about the loci we processed", required = false)
public PrintStream baseReport = null;
@Argument(shortName = "lf", fullName = "likelihood_file", doc = "write the likelihood values to the specified location", required = false)
public PrintStream likelihoodFile = null;
@Argument(shortName = "vs", fullName = "verify_sample", doc = "should we verify that the sample name is in the genotypes file?", required = false)
public boolean verifySample = false;
@Argument(shortName = "mbc", fullName = "minimum_base_count", doc = "what minimum number of bases do we need to see to call contamination in a lane / sample?", required = false)
public Integer minBaseCount = 500;
@Argument(shortName = "population", fullName = "population", doc = "evaluate contamination for just a single contamination population", required = false)
public String population = "CEU";
@Argument(shortName = "gm", fullName = "genotype_mode", doc = "which approach should we take to getting the genotypes (only in array-free mode)", required = false)
public SeqGenotypeMode genotypeMode = SeqGenotypeMode.HARD_THRESHOLD;
// ------------------------------------------------------------------------------------------------------------------------------------------------------
// hidden arguments
// ------------------------------------------------------------------------------------------------------------------------------------------------------
@Hidden
@Argument(fullName = "trim_interval", doc = "progressively trim from 0 to TRIM_FRACTION by this interval", required = false)
public double TRIM_INTERVAL = 0;
@Hidden
@Argument(fullName = "min_site_depth", required = false, doc = "minimum depth at a site to consider in calculation")
public int MIN_SITE_DEPTH = 0;
@Hidden
@Argument(fullName = "fixed_epsilon_qscore", required = false, doc = "use a constant epsilon (phred scale) for calculation")
public Byte FIXED_EPSILON = null;
@Hidden
@Argument(fullName = "min_genotype_depth", required = false, doc = "what minimum depth is required to call a site in seq genotype mode")
public int MIN_GENOTYPE_DEPTH_FOR_SEQ = 50;
@Hidden
@Argument(fullName = "min_genotype_ratio", required = false, doc = "the ratio of alt to other bases to call a site a hom non-ref variant")
public double MIN_GENOTYPE_RATIO = 0.80;
@Hidden
@Argument(fullName = "min_genotype_llh", required = false, doc = "the min log likelihood for UG to call a genotype")
public double MIN_UG_LOG_LIKELIHOOD = 5;
// ------------------------------------------------------------------------------------------------------------------------------------------------------
// global variables to the walker
// ------------------------------------------------------------------------------------------------------------------------------------------------------
private static final Map<Integer,Allele> alleles = new HashMap<Integer,Allele>(); // the set of alleles we work with
private boolean verifiedSampleName = false; // have we yet verified the sample name?
private final Map<String, ContaminationRunType> contaminationNames = new LinkedHashMap<String, ContaminationRunType>(); // a list, containing the contamination names, be it read groups or bam file names
private static String[] ALL_POPULATIONS = new String[]{"ALL", "CHD", "LWK", "CHB", "CEU", "MXL", "GIH", "MKK", "TSI", "CLM", "GBR", "ASW", "YRI", "IBS", "FIN", "PUR", "JPT", "CHS"};
private String[] populationsToEvaluate;
// variables involved in the array-free mode
private boolean useSequencingGenotypes = false; // if false we're using the sequencing geneotypes; otherwise we require array genotypes
public static final String EVAL_BAM_TAG = "eval";
public static final String GENOTYPE_BAM_TAG = "genotype";
String evalSample = null;
String genotypeSample = null;
// counts for each of the possible combinations
int totalSites = 0;
int countPopulationSites = 0;
int countGenotypeNonHomVar = 0;
int countGenotypeHomVar = 0;
int countPassCoverage = 0;
int countResults = 0;
public enum SeqGenotypeMode { HARD_THRESHOLD, UNIFIED_GENOTYPER }
// create our list of allele characters for conversion
static {
alleles.put(0,Allele.create((byte) 'A'));
alleles.put(1,Allele.create((byte) 'C'));
alleles.put(2,Allele.create((byte) 'G'));
alleles.put(3,Allele.create((byte) 'T'));
}
// a bunch of setup to initialize the walker
public void initialize() {
// set the genotypes source - figure out what to do if we're not using arrays
if (genotypes == null || !genotypes.isBound()) {
logger.info("Running in sequencing mode");
useSequencingGenotypes = true;
// if were not using arrays, we need to figure out what samples are what
for(SAMReaderID id : getToolkit().getReadsDataSource().getReaderIDs()) {
if (id.getTags().getPositionalTags().size() == 0)
throw new UserException.BadInput("BAMs must be tagged with " + GENOTYPE_BAM_TAG + " and " + EVAL_BAM_TAG + " when running in array-free mode. Please see the ContEst documentation for more details");
// now sort out what tags go with what bam
for (String tag : id.getTags().getPositionalTags()) {
if (GENOTYPE_BAM_TAG.equalsIgnoreCase(tag)) {
try {
if (getToolkit().getReadsDataSource().getHeader(id).getReadGroups().size() == 0)
throw new RuntimeException("No Read Groups found for Genotyping BAM -- Read Groups are Required in sequencing genotype mode!");
genotypeSample = getToolkit().getReadsDataSource().getHeader(id).getReadGroups().get(0).getSample();
} catch (NullPointerException npe) {
throw new UserException.BadInput("Unable to fetch read group from the bam files tagged with " + GENOTYPE_BAM_TAG);
}
} else if (EVAL_BAM_TAG.equalsIgnoreCase(tag)) {
try {
if (getToolkit().getReadsDataSource().getHeader(id).getReadGroups().size() == 0)
throw new RuntimeException("No Read Groups found for Genotyping BAM -- Read Groups are Required in sequencing genotype mode!");
evalSample = getToolkit().getReadsDataSource().getHeader(id).getReadGroups().get(0).getSample();
} catch (NullPointerException npe) {
throw new UserException.BadInput("Unable to fetch read group from the bam files tagged with " + EVAL_BAM_TAG);
}
} else {
throw new UserException.BadInput("Unable to process " + tag + " tag, it's not either of the two accepted values: " + GENOTYPE_BAM_TAG + " or " + EVAL_BAM_TAG);
}
}
}
if (evalSample == null || genotypeSample == null)
throw new UserException.BadInput("You must provide both a " + GENOTYPE_BAM_TAG + " tagged bam and a " + EVAL_BAM_TAG + " tagged bam file. Please see the ContEst documentation");
} else {
logger.info("Running in array mode");
}
if (laneStats == null) {
laneStats = new HashSet<ContaminationRunType>();
laneStats.add(ContaminationRunType.META);
}
for (ContaminationRunType type : laneStats) {
if (type == ContaminationRunType.READGROUP) {
for (SAMReadGroupRecord name : getToolkit().getSAMFileHeader().getReadGroups())
this.contaminationNames.put(name.getId(),ContaminationRunType.READGROUP);
} else if (type == ContaminationRunType.SAMPLE) {
for (SAMReadGroupRecord name : getToolkit().getSAMFileHeader().getReadGroups())
this.contaminationNames.put(name.getSample(),ContaminationRunType.SAMPLE);
} else if (type == ContaminationRunType.META)
this.contaminationNames.put("META",ContaminationRunType.META);
else
throw new IllegalArgumentException("Unknown type name " + laneStats);
}
if (baseReport != null)
baseReport.println("lane\tchrom\tposition\trs_id\tref\tfreq_major_allele\tfreq_minor_allele\tgeli_gt\tmaf\tmajor_allele_counts\tminor_allele_counts\ta_counts\tc_counts\tg_counts\tt_counts");
this.populationsToEvaluate = (population == null || "EVERY".equals(population)) ? ALL_POPULATIONS : new String[]{population};
}
/**
* our map function, which emits a contamination stats for each of the subgroups (lanes, samples, etc) that we encounter
*
* @param tracker the reference meta data tracker, from which we get the array truth data
* @param ref the reference information at this position
* @param context the read context, where we get the alignment data
* @return a mapping of our subgroup name to contamination estimate
*/
@Override
public Map<String, Map<String, ContaminationStats>> map(RefMetaDataTracker tracker, ReferenceContext ref, AlignmentContext context) {
totalSites++;
if (tracker == null) return null;
if (context == null) return null;
VariantContext popVC = tracker.getFirstValue(pop);
byte referenceBase = ref.getBase();
if (popVC == null) return null;
countPopulationSites++;
Genotype genotype = getGenotype(tracker,context,ref,useSequencingGenotypes);
// only use homozygous sites
if (genotype == null || !genotype.isHomVar()) {
countGenotypeNonHomVar++;
return null;
} else {
countGenotypeHomVar++;
}
// only use non-reference sites
byte myBase = genotype.getAllele(0).getBases()[0];
String rsNumber = "";
// our map of contamination results
Map<String, Map<String, ContaminationStats>> stats = new HashMap<String, Map<String, ContaminationStats>>();
// get the base pileup. This is only really required when we have both a genotyping and EVAL_BAM_TAG tagged bams
// becuase we only want contamination estimates drawn from the eval tagged bam
ReadBackedPileup defaultPile;
if (this.useSequencingGenotypes)
defaultPile = context.getBasePileup().getPileupForSample(evalSample);
else
defaultPile = context.getBasePileup();
// if we're by-lane, get those stats
for (Map.Entry<String, ContaminationRunType> namePair : contaminationNames.entrySet()) {
ReadBackedPileup pile;
if (namePair.getValue() == ContaminationRunType.READGROUP)
pile = defaultPile.getPileupForReadGroup(namePair.getKey());
else if (namePair.getValue() == ContaminationRunType.META)
pile = defaultPile;
else if (namePair.getValue() == ContaminationRunType.SAMPLE)
pile = defaultPile.getPileupForSample(namePair.getKey());
else
throw new IllegalStateException("Unknown state, contamination type = " + laneStats + " is unsupported");
if (pile != null) {
ReadBackedPileup filteredPile =
pile.getBaseAndMappingFilteredPileup(MIN_QSCORE, MIN_MAPQ);
byte[] bases = filteredPile.getBases();
// restrict to sites that have greater than our required total depth
if (bases.length < MIN_SITE_DEPTH) {
continue;
} else {
countPassCoverage++;
}
byte[] quals;
if (FIXED_EPSILON == null) {
quals = filteredPile.getQuals();
} else {
quals = new byte[bases.length];
Arrays.fill(quals, FIXED_EPSILON);
}
Map<String, ContaminationStats> results =
calcStats(referenceBase,
bases,
quals,
myBase,
rsNumber,
popVC,
baseReport,
context.getLocation(),
precision,
namePair.getKey(),
populationsToEvaluate);
if (results.size() > 0) {
countResults++;
stats.put(namePair.getKey(), results);
}
}
}
// return our collected stats
return stats;
}
/**
* get the genotype for the sample at the current position
* @param tracker the reference meta data (RODs)
* @param context the reads
* @param referenceContext the reference information
* @param useSeq are we using sequencing to get our genotypes
* @return a genotype call, which could be null
*/
private Genotype getGenotype(RefMetaDataTracker tracker, AlignmentContext context, ReferenceContext referenceContext, boolean useSeq) {
if (!useSeq) {
Genotype g = getGenotypeFromArray(tracker, this.genotypes,this.verifiedSampleName,this.verifySample,this.sampleName);
if (g != null) this.verifiedSampleName = true;
return g;
} else {
return getGenotypeFromSeq(
context,
referenceContext,
this.alleles,
this.genotypeMode,
this.MIN_GENOTYPE_RATIO,
this.MIN_GENOTYPE_DEPTH_FOR_SEQ,
this.MIN_UG_LOG_LIKELIHOOD,
this.genotypeSample,
this.sampleName,
this.getToolkit());
}
}
static Genotype getGenotypeFromSeq(AlignmentContext context,
ReferenceContext referenceContext,
Map<Integer, Allele> alleles,
SeqGenotypeMode genotypeMode,
double minGenotypeRatio,
int minGenotypingDepth,
double minGenotypingLOD,
String genotypingSample,
String sampleName,
GenomeAnalysisEngine toolKit) {
ReadBackedPileup pileup = context.getBasePileup().getPileupForSample(genotypingSample);
if (pileup == null || pileup.isEmpty()) return null;
// which genotyping mode are we using
if (genotypeMode == SeqGenotypeMode.HARD_THRESHOLD) {
if (sum(pileup.getBaseCounts()) < minGenotypingDepth) return null;
int[] bases = pileup.getBaseCounts();
int mx = maxPos(bases);
int allGenotypes = sum(bases);
String refBase = String.valueOf((char)referenceContext.getBase());
if (bases[mx] / (float)allGenotypes >= minGenotypeRatio && !refBase.equals(alleles.get(mx).getBaseString())) {
List<Allele> al = new ArrayList<Allele>();
al.add(alleles.get(mx));
GenotypeBuilder builder = new GenotypeBuilder(sampleName, al);
return builder.make();
}
} else if (genotypeMode == SeqGenotypeMode.UNIFIED_GENOTYPER) {
UnifiedArgumentCollection basicUAC = new UnifiedArgumentCollection();
UnifiedGenotypingEngine engine = new UnifiedGenotypingEngine(basicUAC, FixedAFCalculatorProvider.createThreadSafeProvider(toolKit, basicUAC, logger),toolKit);
AlignmentContext contextSubset = new AlignmentContext(context.getLocation(),pileup,0,true);
List<VariantCallContext> callContexts = engine.calculateLikelihoodsAndGenotypes(null, referenceContext, contextSubset);
if (callContexts != null && callContexts.size() == 1)
for (Genotype g : callContexts.get(0).getGenotypes()){
if (g.isCalled() && g.isHomVar() && g.getLog10PError() > minGenotypingLOD)
return g;
}
}
else {
throw new GATKException("Unknown genotyping mode, being an enum this really shouldn't be seen ever.");
}
return null;
}
// utils
private static int sum(int[] a) {int sm = 0; for (int i : a) {sm = sm + i;} return sm;}
private static int maxPos(int[] a) {int mx = 0; for (int i = 0;i < a.length; i++) {if (a[i] > a[mx]) mx = i;} return mx;}
private static Genotype getGenotypeFromArray(RefMetaDataTracker tracker, RodBinding<VariantContext> genotypes, boolean verifiedSampleName, boolean verifySample, String sampleName) {
// get the truthForSample and the hapmap information for this site; if either are null we can't move forward
Collection<VariantContext> truths = tracker.getValues(genotypes);
if (truths == null || truths.size() == 0) return null;
VariantContext truthForSample = truths.iterator().next();
// verify that the sample name exists in the input genotype file
if (!verifiedSampleName && verifySample) {
if (!truthForSample.getSampleNames().contains(sampleName))
throw new UserException.BadInput("The sample name was set to " + sampleName + " but this sample isn't in your genotypes file. Please Verify your sample name");
verifiedSampleName = true;
}
GenotypesContext gt = truthForSample.getGenotypes();
// if we are supposed to verify the sample name, AND the sample doesn't exist in the genotypes -- skip this site
if (verifySample && !gt.containsSample(sampleName)) return null;
// if the sample doesn't exist in genotypes AND there is more than one sample in the genotypes file -- skip this site
if (!gt.containsSample(sampleName) && gt.size() != 1) return null;
// if there is more than one sample in the genotypes file, get it by name. Otherwise just get the sole sample genotype
return gt.size() != 1 ? gt.get(sampleName) : gt.get(0);
}
private static class PopulationFrequencyInfo {
private byte majorAllele;
private byte minorAllele;
private double minorAlleleFrequency;
private PopulationFrequencyInfo(byte majorAllele, byte minorAllele, double minorAlleleFrequency) {
this.majorAllele = majorAllele;
this.minorAllele = minorAllele;
this.minorAlleleFrequency = minorAlleleFrequency;
}
public byte getMajorAllele() {
return majorAllele;
}
public byte getMinorAllele() {
return minorAllele;
}
public double getMinorAlleleFrequency() {
return minorAlleleFrequency;
}
}
private static PopulationFrequencyInfo parsePopulationFrequencyInfo(VariantContext variantContext, String population) {
PopulationFrequencyInfo info = null;
List<String> values = (List<String>) variantContext.getAttribute(population);
if (values != null) {
byte majorAllele = 0;
byte minorAllele = 0;
double maf = -1;
for (String str : values) {
// strip off the curly braces and trim whitespace
if (str.startsWith("{")) str = str.substring(1, str.length());
if (str.contains("}")) str = str.substring(0, str.indexOf("}"));
str = str.trim();
String spl[] = str.split("=");
byte allele = (byte) spl[0].trim().charAt(0);
double af = Double.valueOf(spl[1].trim());
if (af <= 0.5 && minorAllele == 0) {
minorAllele = allele;
maf = af;
} else {
majorAllele = allele;
}
}
info = new PopulationFrequencyInfo(majorAllele, minorAllele, maf);
}
return info;
}
/**
* Calculate the contamination values per division, be it lane, meta, sample, etc
* @param referenceBase the reference base
* @param bases the bases seen
* @param quals and the bases qual values
* @param myAllele the allele we have (our hom var genotype allele)
* @param rsNumber the dbsnp number if available
* @param popVC the population variant context from hapmap
* @param baseReport if we're writing a base report, write it here
* @param loc our location
* @param precision the percision we're aiming for
* @param lane the lane name information
* @param pops our pops to run over
* @return a mapping of each target population to their estimated contamination
*/
private static Map<String, ContaminationStats> calcStats(byte referenceBase,
byte[] bases,
byte[] quals,
byte myAllele,
String rsNumber,
VariantContext popVC,
PrintStream baseReport,
GenomeLoc loc,
Double precision,
String lane,
String[] pops) {
int[] alts = new int[4];
int total = 0;
// get the depth ratio we are aiming for
for (byte base : bases) {
if (base == 'A' || base == 'a') alts[0]++;
if (base == 'C' || base == 'c') alts[1]++;
if (base == 'G' || base == 'g') alts[2]++;
if (base == 'T' || base == 't') alts[3]++;
total++;
}
Map<String, ContaminationStats> ret = new HashMap<String, ContaminationStats>();
for (String pop : pops) {
PopulationFrequencyInfo info = parsePopulationFrequencyInfo(popVC, pop);
double alleleFreq = info.getMinorAlleleFrequency();
if (alleleFreq > 0.5) {
throw new RuntimeException("Minor allele frequency is greater than 0.5, this is an error; we saw AF of " + alleleFreq);
}
int majorCounts = alts[getBaseIndex(info.getMajorAllele())];
int minorCounts = alts[getBaseIndex(info.getMinorAllele())];
int otherCounts = total - majorCounts - minorCounts;
// only use sites where this is the minor allele
if (myAllele == info.minorAllele) {
if (pops.length == 1) {
if (baseReport != null) {
baseReport.print(
StringUtil.join("\t",
lane,
loc.getContig(),
"" + loc.getStart(),
rsNumber,
"" + (char) referenceBase,
"" + (char) info.getMajorAllele(),
"" + (char) info.getMinorAllele(),
"" + (char) info.getMinorAllele() + "" + (char) info.getMinorAllele(),
String.format("%1.4f", alleleFreq),
"" + majorCounts,
"" + minorCounts));
for (long cnt : alts)
baseReport.print("\t" + cnt);
baseReport.println();
}
}
ContaminationEstimate est = new ContaminationEstimate(precision, alleleFreq, bases, quals, info.getMinorAllele(), info.getMajorAllele(), pop, loc);
ret.put(pop, new ContaminationStats(loc, 1, alleleFreq, minorCounts, majorCounts, otherCounts, alts, est));
}
}
return ret;
}
private static int getBaseIndex(byte base) {
if (base == 'A' || base == 'a') return 0;
if (base == 'C' || base == 'c') return 1;
if (base == 'G' || base == 'g') return 2;
if (base == 'T' || base == 't') return 3;
return -1;
}
// create a ContaminationResults to store the run information
@Override
public ContaminationResults reduceInit() {
return new ContaminationResults(precision);
}
@Override
public ContaminationResults reduce(Map<String, Map<String, ContaminationStats>> value, ContaminationResults sum) {
if (value != null)
sum.add(value);
return sum;
}
/**
* on traversal done, output all the stats to the appropriate files
*
* @param result the results of our contamination estimate
*/
public void onTraversalDone(ContaminationResults result) {
// filter out lanes / samples that don't have the minBaseCount
Map<String, Map<String, ContaminationStats>> cleanedMap = new HashMap<String, Map<String, ContaminationStats>>();
for (Map.Entry<String, Map<String, ContaminationStats>> entry : result.getStats().entrySet()) {
Map<String, ContaminationStats> newMap = new HashMap<String, ContaminationStats>();
Map<String, ContaminationStats> statMap = entry.getValue();
for (String popKey : statMap.keySet()) {
ContaminationStats stat = statMap.get(popKey);
if (stat.getBasesMatching() + stat.getBasesMismatching() >= minBaseCount) newMap.put(popKey, stat);
}
if (newMap.size() > 0)
cleanedMap.put(entry.getKey(), newMap);
else
out.println("Warning: We're throwing out lane " + entry.getKey() + " since it has fewer than " + minBaseCount +
" read bases at genotyped positions");
}
// output results at the end, based on the input parameters
result.setStats(cleanedMap);
result.outputReport(precision, out, TRIM_FRACTION, TRIM_INTERVAL, BETA_THRESHOLD);
if (likelihoodFile != null) result.writeCurves(likelihoodFile);
logger.info("Total sites: " + totalSites);
logger.info("Population informed sites: " + countPopulationSites);
logger.info("Non homozygous variant sites: " + countGenotypeNonHomVar);
logger.info("Homozygous variant sites: " + countGenotypeHomVar);
logger.info("Passed coverage: " + countPassCoverage);
logger.info("Results: " + countResults);
}
}

234
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/contamination/ContaminationEstimate.java 100755
View File

@ -0,0 +1,234 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer.contamination;
import org.broadinstitute.gatk.utils.GenomeLoc;
import org.broadinstitute.gatk.utils.collections.Pair;
import java.util.Arrays;
/**
* a class that estimates and stores the contamination values for a site.
*/
class ContaminationEstimate {
private final double precision; // to what precision do we want to run; e.g. if set to 1, we run using 1% increments
private final double[] bins; // the bins representing the discrete contamination levels we're evaluating
private double populationFit = 0.0;
private String popultationName = "";
private static double[] precalculatedEpsilon;
private int arrayAlleleObservations = 0;
private int alternateAlleleObservations = 0;
// precalculate the 128 values of epsilon that are possible
static {
precalculatedEpsilon = new double[Byte.MAX_VALUE+1];
for(int i=0; i <= (int)Byte.MAX_VALUE; i++) {
precalculatedEpsilon[i] = Math.pow(10.0,-1.0*(((double)i)/10.0));
}
}
/**
* create the contamination estimate, given:
* @param precision the precision value, to what level are we calculating the contamination
*/
public ContaminationEstimate(double precision,
double maf,
byte[] bases,
byte[] quals,
byte arrayAllele,
byte hapmapAlt,
String popName,
GenomeLoc locus
) {
// setup the bins to the correct precision
this.precision = precision;
bins = new double[(int)Math.ceil(100/precision)+1];
if (maf == 0) maf = 0.00001;
popultationName = popName;
Arrays.fill(bins,0.0); // just to make sure we don't have any residual values
// convert the quals
double[] realQuals = new double[quals.length];
int qIndex = 0;
for (byte qual : quals) {realQuals[qIndex++] = Math.pow(10.0,-1.0*(qual/10.0));}
// check our inputs
if (maf > 1.0 || maf < 0.0) throw new IllegalArgumentException("Invalid allele Freq: must be between 0 and 1 (inclusive), maf was " + maf + " for population " + popName);
// calculate the contamination for each bin
int qualOffset = 0;
for (byte base : bases) {
if (base == arrayAllele) { arrayAlleleObservations++; }
if (base == hapmapAlt) { alternateAlleleObservations++; }
double epsilon = precalculatedEpsilon[quals[qualOffset++]];
for (int index = 0; index < bins.length; index++) {
double contaminationRate = (1.0 - (double) index / (double) bins.length);
if (base == arrayAllele) {
bins[index] += Math.log((1.0 - contaminationRate) * (1.0 - epsilon) +
contaminationRate * ((maf) * (1.0 - epsilon) + (1.0 - maf) * (epsilon/3.0)));
populationFit += Math.log(epsilon);
} else if(hapmapAlt == base) {
bins[index] += Math.log((1.0 - contaminationRate) * (epsilon / 3.0) +
contaminationRate * ((maf) * (epsilon/3.0) + (1.0 - maf) * (1.0 - epsilon)));
populationFit += Math.log(maf + epsilon);
}
}
}
}
public double[] getBins() {
return bins;
}
public void setPopulationFit(double populationFit) {
this.populationFit = populationFit;
}
public double getPopulationFit() {
return populationFit;
}
public String getPopultationName() {
return popultationName;
}
public static class ConfidenceInterval {
private double start;
private double stop;
private double contamination;
private double maxLikelihood;
double[] newBins;
public ConfidenceInterval(double bins[], double intervalArea) {
// make a copy of the bins in non-log space
int maxIndex = 0;
for (int x = 0; x < bins.length; x++) if (bins[x] > bins[maxIndex]) maxIndex = x;
newBins = new double[bins.length];
maxLikelihood = bins[maxIndex];
int index = 0;
double total = 0.0;
for (double d : bins) {
newBins[index] = Math.pow(10,(bins[index] - bins[maxIndex]));
total += newBins[index];
index++;
}
for (int x = 0; x < newBins.length; x++) {
newBins[x] = newBins[x] / total;
}
double areaUnderCurve = 0;
int leftIndex = maxIndex;
int rightIndex = maxIndex;
while (areaUnderCurve < 0.95) {
// if the "left" bin is bigger, and can be moved, move it
if (newBins[leftIndex] >= newBins[rightIndex] && leftIndex > 0) {
leftIndex--;
} else {
// otherwise move the right bin if possible
if (rightIndex < bins.length - 1) {
rightIndex++;
} else {
// and if not move the left bin, or die
if (leftIndex > 0) {
leftIndex--;
} else {
throw new RuntimeException("Error trying to compute confidence interval");
}
}
}
areaUnderCurve = 0.0;
for (int x = leftIndex; x <= rightIndex; x++)
areaUnderCurve += newBins[x];
}
start = (bins.length - rightIndex) * (100.0/bins.length);
stop = (bins.length - leftIndex) * (100.0/bins.length);
contamination = (bins.length - maxIndex) * (100.0/bins.length);
}
public double getStart() {
return start;
}
public double getStop() {
return stop;
}
public double getContamination() {
return contamination;
}
public double getMaxLikelihood() {
return maxLikelihood;
}
public String toString() {
return contamination + "[" + start + " - " + stop + "] log likelihood = " + maxLikelihood;
}
}
}

304
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/contamination/ContaminationResults.java 100755
View File

@ -0,0 +1,304 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer.contamination;
import org.apache.commons.math.MathException;
import org.apache.commons.math.distribution.BetaDistribution;
import org.apache.commons.math.distribution.BetaDistributionImpl;
import org.broadinstitute.gatk.utils.GenomeLoc;
import org.broadinstitute.gatk.utils.Utils;
import java.io.*;
import java.util.*;
/**
* our contamination results object; this object aggregates the results of the contamination run over lanes, samples,
* or whatever other divisor we've used on the read data
*/
public class ContaminationResults {
public static class ContaminationData implements Comparable<ContaminationData> {
private GenomeLoc site;
private long basesMatching = 0l;
private long basesMismatching = 0l;
private double mismatchFraction = -1d;
private double[] bins;
private double p;
public long getBasesMatching() {
return basesMatching;
}
public long getBasesMismatching() {
return basesMismatching;
}
public double getMismatchFraction() {
return mismatchFraction;
}
public double[] getBins() {
return bins;
}
public double getP() {
return p;
}
public ContaminationData(GenomeLoc site, long basesMatching, long basesMismatching, double[] bins) {
this.site = site;
this.basesMatching = basesMatching;
this.basesMismatching = basesMismatching;
this.bins = bins;
long totalBases = this.basesMatching + this.basesMismatching;
if (totalBases != 0) {
this.mismatchFraction = (double)this.basesMismatching / (double) totalBases;
}
int a = (int) this.getBasesMismatching() + 1;
int b = (int) this.getBasesMatching() + 1;
BetaDistribution dist = new BetaDistributionImpl(a,b);
try {
this.p = 1.0d - dist.cumulativeProbability(0.5d);
} catch (MathException me) {
throw new RuntimeException("Error! - " + me.toString());
}
}
public int compareTo(ContaminationData other) {
return -Double.compare(this.getP(), other.getP());
}
@Override
public String toString() {
return "ContaminationData{" +
"site=" + site +
", basesMatching=" + basesMatching +
", basesMismatching=" + basesMismatching +
", mismatchFraction=" + mismatchFraction +
'}';
}
}
// what precision are we using in our calculations
private final double precision;
// a map of our contamination targets and their stats
// key: aggregation entity ("META", sample name, or lane name)
// value: ContaminationStats (whcih
private Map<String,Map<String, ContaminationStats>> stats = new HashMap<String,Map<String, ContaminationStats>>();
public ContaminationResults(double precision) {
this.precision = precision;
}
Map<String, Map<String, List<ContaminationData>>> storedData = new HashMap<String, Map<String, List<ContaminationData>>>();
/**
* add to the stats
*
* @param newAggregationStats a mapping of the stat name to their statistics collected
*/
public void add(Map<String, Map<String, ContaminationStats>> newAggregationStats) {
// for each aggregation level
for (String aggregationKey : newAggregationStats.keySet()) {
Map<String, ContaminationStats> populationContaminationStats = newAggregationStats.get(aggregationKey);
// a new way of doing this... store all the data until the end...
if (!storedData.containsKey(aggregationKey)) { storedData.put(aggregationKey, new HashMap<String, List<ContaminationData>>()); }
for (String pop : populationContaminationStats.keySet()) {
ContaminationStats newStats = populationContaminationStats.get(pop);
// if it exists... just merge it
if (!storedData.get(aggregationKey).containsKey(pop)) {
storedData.get(aggregationKey).put(pop, new ArrayList<ContaminationData>());
}
double[] newData = new double[newStats.getContamination().getBins().length];
System.arraycopy(newStats.getContamination().getBins(),0,newData,0,newStats.getContamination().getBins().length);
storedData.get(aggregationKey).get(pop).add(new ContaminationData(newStats.getSite(), newStats.getBasesMatching(), newStats.getBasesMismatching(), newData));
}
// merge the sets
if (stats.containsKey(aggregationKey)) {
// and for each population
for (String pop : populationContaminationStats.keySet()) {
ContaminationStats newStats = populationContaminationStats.get(pop);
// if it exists... just merge it
if (stats.get(aggregationKey).containsKey(pop)) {
stats.get(aggregationKey).get(pop).add(newStats);
} else {
stats.get(aggregationKey).put(pop, newStats);
}
}
} else {
stats.put(aggregationKey, populationContaminationStats);
}
}
}
/**
* output the contamination data, and return the contamination data
* @param out the output source
* @return the contamination value
*/
public void outputReport(double precision, PrintStream out, double fractionToTrim, double trimInterval, double betaThreshold) {
out.println("name\tpopulation\tpopulation_fit\tcontamination\tconfidence_interval_95_width\tconfidence_interval_95_low\tconfidence_interval_95_high\tsites");
for (Map.Entry<String,Map<String, ContaminationStats>> entry : stats.entrySet()) {
for (ContaminationStats stats : entry.getValue().values()) {
String aggregationLevel = entry.getKey();
String population = stats.getContamination().getPopultationName();
List<ContaminationData> newStats = storedData.get(aggregationLevel).get(population);
String pm = "%3." + Math.round(Math.log10(1/precision)) +"f";
int bins = newStats.iterator().next().getBins().length;
int maxTrim = (int) Math.floor((double)(newStats.size()) * fractionToTrim);
// sort the collection
Collections.sort(newStats);
List<ContaminationData> data = new ArrayList<ContaminationData>(newStats);
// trim sites with > 95% p of being > 0.5 f (based on beta distribution)
int trimmed = 0;
for(Iterator<ContaminationData> i = data.iterator(); trimmed < maxTrim && i.hasNext();) {
ContaminationData x = i.next();
if (x.getP() >= betaThreshold) {
System.out.println("Trimming " + x.toString() + " with p(f>=0.5) >= " + betaThreshold + " with a value of " + x.getP());
i.remove();
trimmed++;
}
}
double[][] matrix = new double[bins][data.size()];
for (int i = 0; i<bins; i++) {
for (int j=0; j<data.size(); j++) {
matrix[i][j] = data.get(j).getBins()[i];
}
}
// now perform the sum
double[] output = new double[bins];
for (int i = 0; i<bins; i++) {
double[] binData = matrix[i];
// remove the top and bottom
output[i] = 0;
for (int x = 0; x < binData.length; x++) {
output[i] += binData[x];
}
}
double[] newTrimmedStats = output;
// get the confidence interval, at the set width
ContaminationEstimate.ConfidenceInterval newInterval = new ContaminationEstimate.ConfidenceInterval(newTrimmedStats, 0.95);
out.println(
String.format("%s\t%s\t%s\t"+pm+"\t"+pm+"\t"+pm+"\t"+pm+"\t"+"%d",
aggregationLevel,
population,
"n/a",
newInterval.getContamination(),
(newInterval.getStop() - newInterval.getStart()),
newInterval.getStart(),
newInterval.getStop(),
data.size())
);
}
}
}
public void writeCurves(PrintStream out) {
boolean outputBins = false;
for (Map.Entry<String, Map<String, ContaminationStats>> entry : stats.entrySet()) {
for (ContaminationStats stats : entry.getValue().values()) {
if (!outputBins) {
String[] bins = new String[stats.getContamination().getBins().length];
for (int index = 0; index < stats.getContamination().getBins().length; index++)
bins[index] = String.valueOf(100.0 * (1 - (double) index / stats.getContamination().getBins().length));
outputBins = true;
out.print("name,pop,");
out.println(Utils.join(",",bins));
}
String[] bins = new String[stats.getContamination().getBins().length];
int index = 0;
for (double value : stats.getContamination().getBins())
bins[index++] = String.valueOf(value);
out.print(entry.getKey()+",\""+stats.getContamination().getPopultationName()+"\",");
out.println(Utils.join(",", bins));
}
}
}
public Map<String, Map<String, ContaminationStats>> getStats() {
return Collections.unmodifiableMap(stats);
}
public void setStats(Map<String, Map<String,ContaminationStats>> stats) {
this.stats = stats;
}
}

125
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/contamination/ContaminationStats.java 100755
View File

@ -0,0 +1,125 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer.contamination;
import org.broadinstitute.gatk.utils.GenomeLoc;
import org.omg.PortableInterceptor.SYSTEM_EXCEPTION;
/**
* a class that tracks our contamination stats; both the estimate of contamination, as well as the number of sites and other
* run-specific data
*/
public class ContaminationStats {
final static int ALLELE_COUNT = 4;
private GenomeLoc site;
private int numberOfSites = 0;
private double sumOfAlleleFrequency = 0.0;
private long basesFor = 0l;
private long basesAgainst = 0l;
private long basesOther = 0l;
private ContaminationEstimate contaminationEstimate;
private final int[] alleleBreakdown;
public ContaminationStats(GenomeLoc site, int numberOfSites, double sumOfAlleleFrequency, long basesFor, long basesAgainst, long basesOther, int alleleBreakdown[], ContaminationEstimate estimate) {
this.site = site;
this.numberOfSites = numberOfSites;
this.sumOfAlleleFrequency = sumOfAlleleFrequency;
this.basesFor = basesFor;
this.basesAgainst = basesAgainst;
this.contaminationEstimate = estimate;
if (alleleBreakdown.length != ALLELE_COUNT) throw new IllegalArgumentException("Allele breakdown should have length " + ALLELE_COUNT);
this.alleleBreakdown = alleleBreakdown;
}
public int getNumberOfSites() {
return numberOfSites;
}
public double getMinorAlleleFrequency() {
return sumOfAlleleFrequency /(double)numberOfSites;
}
public long getBasesMatching() {
return basesFor;
}
public long getBasesOther() {
return basesOther;
}
public long getBasesMismatching() {
return basesAgainst;
}
public ContaminationEstimate getContamination() {
return this.contaminationEstimate;
}
public GenomeLoc getSite() {
return site;
}
public void add(ContaminationStats other) {
if (other == null) return;
this.numberOfSites += other.numberOfSites;
this.sumOfAlleleFrequency += other.sumOfAlleleFrequency;
this.basesOther += other.basesOther;
this.basesFor += other.basesFor;
this.basesAgainst += other.basesAgainst;
for (int x = 0; x < ALLELE_COUNT; x++) this.alleleBreakdown[x] += other.alleleBreakdown[x];
for (int i = 0; i < this.contaminationEstimate.getBins().length; i++) {
this.contaminationEstimate.getBins()[i] += other.contaminationEstimate.getBins()[i];
}
this.contaminationEstimate.setPopulationFit(this.contaminationEstimate.getPopulationFit() +other.contaminationEstimate.getPopulationFit());
}
}

77
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/Dream_Evaluations.md 100644
View File

@ -0,0 +1,77 @@
# Dream Challenge Evaluation
In order to evaluate the performance of M2, we use two sets of data from the SMC DREAM Challenge. Specifically challenges #3 and #4.
All scripts referenced here are relative to the current working directory of ```
/dsde/working/mutect/dream_smc```
### Current Performance (Unmasked)
From the output of the evaluation method
(gsa-unstable 7/13/15, commit:9e93a70)
|set | subset | type | sensitivity | specificity | accuracy |
|----|--------|------|-------------|-------------|----------|
|SMC 3|chr21|SNP|0.935897435897|0.935897435897|0.935897435897|
|SMC 3|chr21|INDEL|0.904255319149|0.977011494253|0.940633406701|
|SMC 3|wgs|SNP|0.930532709098|0.955188985583|0.94286084734|
|SMC 3|wgs|INDEL|0.902139907396|0.970516962843|0.93632843512|
|SMC 4|chr21|SNP|0.769607843137|0.969135802469|0.869371822803|
|SMC 4|chr21|INDEL|0.771241830065|0.991596638655|0.88141923436|
|SMC 4|wgs|SNP|0.764507007622|0.975374480433|0.869940744028|
|SMC 4|wgs|INDEL|0.768634634353|0.989389679877|0.879012157115|
### How To Run
The SCALA script for running M2 can be found in the gsa-unstable repository under ```private/gatk-tools-private/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2```
First, chose the appropriate settings (runnable as environment variables here)
```
QUEUE_JAR=<your-queue-jar>
OUT_VCF=<your-output-vcf>
GSA_UNSTABLE_HOME=<path-to-your-gsa-unstable-checkout>
# for Dream 3
NORMAL_BAM=/dsde/working/mutect/dream_smc/bams/synthetic.challenge.set3.normal.bam
TUMOR_BAM=/dsde/working/mutect/dream_smc/bams/synthetic.challenge.set3.tumor.bam
# for Dream 4
NORMAL_BAM=/dsde/working/mutect/dream_smc/bams/synthetic.challenge.set4.normal.bam
TUMOR_BAM=/dsde/working/mutect/dream_smc/bams/synthetic.challenge.set4.tumor.bam
# for WGS
INTERVALS=/dsde/working/mutect/dream_smc/bams/wgs_calling_regions.v1.interval_list
# for chromosome 21 only
INTERVALS=/dsde/working/mutect/ts/c21_wgs_calling_regions.v1.interval_list
TEMPDIR=/broad/hptmp/kcibul/mutect
```
and then run the following Queue command
```
java \
-Djava.io.tmpdir=$TEMPDIR \
-jar $QUEUE_JAR \
-S $GSA_UNSTABLE_HOME/private/gatk-tools-private/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/run_M2_dream.scala \
--job_queue gsa -qsub -jobResReq virtual_free=5G -startFromScratch \
-sc 200 \
-normal $NORMAL_BAM \
-tumor $TUMOR_BAM \
-L $INTERVALS \
-o $OUT_VCF \
-run
```
### How To Evaluate
Run the following
```
/dsde/working/mutect/dream_smc/dream_eval.pl [3|4] [wgs|21] [SNV|INDEL] input.vcf
```
where
- [3|4] the dream challenge round
- [wgs|21] evaluate the whole genome, or just a subset (chromosome 21)
- [SNV|INDEL] evaulate SNV (SNPs) or INDELS

117
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/M2ArgumentCollection.java 100644
View File

@ -0,0 +1,117 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer.m2;
import org.broadinstitute.gatk.tools.walkers.haplotypecaller.AssemblyBasedCallerArgumentCollection;
import org.broadinstitute.gatk.utils.commandline.Advanced;
import org.broadinstitute.gatk.utils.commandline.Argument;
public class M2ArgumentCollection extends AssemblyBasedCallerArgumentCollection {
@Advanced
@Argument(fullName="m2debug", shortName="m2debug", doc="Print out very verbose M2 debug information", required = false)
public boolean M2_DEBUG = false;
/**
* Artifact detection mode is used to prepare a panel of normals. This maintains the specified tumor LOD threshold,
* but disables the remaining pragmatic filters. See M2 usage examples for more information.
*/
@Advanced
@Argument(fullName = "artifact_detection_mode", required = false, doc="Enable artifact detection for creating panels of normals")
public boolean ARTIFACT_DETECTION_MODE = false;
/**
* This is the tumor LOD threshold to output the variant in the VCF, although it may be filtered
*/
@Argument(fullName = "initial_tumor_lod", required = false, doc = "Initial LOD threshold for calling tumor variant")
public double INITIAL_TUMOR_LOD_THRESHOLD = 4.0;
@Argument(fullName = "initial_normal_lod", required = false, doc = "Initial LOD threshold for calling normal variant")
public double INITIAL_NORMAL_LOD_THRESHOLD = 0.5;
/**
* Only variants with tumor LODs exceeding this thresholds can pass filtration
*/
@Argument(fullName = "tumor_lod", required = false, doc = "LOD threshold for calling tumor variant")
public double TUMOR_LOD_THRESHOLD = 6.3;
/**
* This is a measure of the minimum evidence to show that a variant observed in the tumor is not also present in its normal
*/
@Argument(fullName = "normal_lod", required = false, doc = "LOD threshold for calling normal non-germline")
public double NORMAL_LOD_THRESHOLD = 2.2;
/**
* The LOD threshold for the normal is typically made more strict if the variant has been seen in dbSNP (i.e. another
* normal sample). We thus require MORE evidence that a variant is NOT seen in this tumor's normal if it has been observed as a germline variant before.
*/
@Argument(fullName = "dbsnp_normal_lod", required = false, doc = "LOD threshold for calling normal non-variant at dbsnp sites")
public double NORMAL_DBSNP_LOD_THRESHOLD = 5.5;
/**
* This argument is used for the M2 internal "alt_allele_in_normal" filter
**/
@Argument(fullName = "max_alt_alleles_in_normal_count", required = false, doc="Threshold for maximum alternate allele counts in normal")
public int MAX_ALT_ALLELES_IN_NORMAL_COUNT = 2;
/**
* This argument is used for the M2 internal "alt_allele_in_normal" filter
*/
@Argument(fullName = "max_alt_alleles_in_normal_qscore_sum", required = false, doc="Threshold for maximum alternate allele quality score sum in normal")
public int MAX_ALT_ALLELES_IN_NORMAL_QSCORE_SUM = 20;
/**
* This argument is used for the M2 internal "alt_allele_in_normal" filter
*/
@Argument(fullName = "max_alt_allele_in_normal_fraction", required = false, doc="Threshold for maximum alternate allele fraction in normal")
public double MAX_ALT_ALLELE_IN_NORMAL_FRACTION = 0.03;
}

157
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/M2_Contamination_Evaluation.md 100644
View File

@ -0,0 +1,157 @@
# M2 Contamination Correction Evaluation
In order to evaluate the efficacy of the contamination correction in M2 (adapted from HaplotypeCaller), we created synthetic data consisting of the four intra-run CRSP NA12878 replicates, each contaminated with 1-5% of the HCC-1143 normal.
### Creating Artificial Contamination Data
``` QUEUE_JAR=<your-queue-jar>
GSA_UNSTABLE_HOME=<path-to-your-gsa-unstable-checkout>
BASELINE_BAM=/crsp/picard_aggregation/000007820918/SM-612V3/current/SM-612V3.bam
CONTAMINANT_BAM=/seq/tier3b/picard_aggregation/C970/HCC1143_BL/v1/HCC1143_BL.bam
```
```
java -jar $QUEUE_JAR \
-S $GSA_UNSTABLE_HOME/private/gatk-queue-extensions-internal/src/main/qscripts/org/broadinstitute/gatk/queue/qscripts/dev/CreateSyntheticContaminationScript.scala \
-b2 CONTAMINANT_BAM -b1 BASELINE_BAM \
-f 0.01 -f 0.02 -f 0.03 -f 0.04 -f 0.05
```
Repeat for the other three NA12878 replicates.
BASELINE_BAM=/crsp/picard_aggregation/000007820818/SM-612V4/current/SM-612V4.bam
BASELINE_BAM=/crsp/picard_aggregation/000007820718/SM-612V5/current/SM-612V5.bam
BASELINE_BAM=/crsp/picard_aggregation/000007820618/SM-612V6/current/SM-612V6.bam
Use ContEst to get the contamination estimate in the data to be passed into M2. (Note that for these data, the ContEst estimate is on the order of 1% higher than the value used to generate the contaminated data.)
TEMPDIR=/broad/hptmp/$USER
BAM1=HCC1143_BL.small.0.04.contaminated.with.SM-612V3.small.0.96.bam
BAM2=/crsp/picard_aggregation/000007820818/SM-612V4/current/SM-612V4.bam
OUT_TXT=ContEst_0.04HCC1143inNA12878.txt
java -Djava.io.tmpdir=$TEMPDIR \
-Xmx512m -jar /xchip/tcga/gdac_prod/applications/process_mgmt/firehose_task_registry/cga/ContaminationAnalysis/broadinstitute.org/cancer.genome.analysis/00262/107//Queue-1.4-437-g6b8a9e1-svn-35362.jar \
-S /xchip/tcga/gdac_prod/applications/process_mgmt/firehose_task_registry/cga/ContaminationAnalysis/broadinstitute.org/cancer.genome.analysis/00262/107//ContaminationPipeline.scala \
-reference /seq/references/Homo_sapiens_assembly19/v1/Homo_sapiens_assembly19.fasta -interval /xchip/cga/reference/hg19/whole_exome_agilent_1.1_refseq_plus_3_boosters_plus_10bp_padding_minus_mito.Homo_sapiens_assembly19.targets.interval_list \
-out $OUT_TXT \
-bam $BAM1 -nbam $BAM2 \
-array /xchip/gcc_data/results2/production/TEENY/BySample/hg19.vcf.txt/TEENY_p_TCGA_302_303_N_GenomeWideSNP_6_F04_1344608.hg19.vcf.txt.store/verstore.00000.TEENY_p_TCGA_302_303_N_GenomeWideSNP_6_F04_1344608.hg19.vcf.txt \
-pop /xchip/cga/reference/hg19/hg19_population_stratified_af_hapmap_3.3.fixed.vcf -faf true -run -array_interval /xchip/cga/reference/hg19/SNP6.hg19.interval_list
BAM1=HCC1143_BL.small.0.03.contaminated.with.SM-612V3.small.0.97.bam
BAM2=/crsp/picard_aggregation/000007820818/SM-612V4/current/SM-612V4.bam
OUT_TXT=ContEst_0.03HCC1143inNA12878.txt
BAM1=HCC1143_BL.small.0.02.contaminated.with.SM-612V3.small.0.98.bam
BAM2=/crsp/picard_aggregation/000007820818/SM-612V4/current/SM-612V4.bam
OUT_TXT=ContEst_0.02HCC1143inNA12878.txt
BAM1=HCC1143_BL.small.0.01.contaminated.with.SM-612V3.small.0.99.bam
BAM2=/crsp/picard_aggregation/000007820818/SM-612V4/current/SM-612V4.bam
OUT_TXT=ContEst_0.01HCC1143inNA12878.txt
And so on for the other replicates.
ContEst estimates for the four replicates at the five contamination levels are as follows:
|Sample|Input Contamination Level|ContEst Estimate|
|------|-------------------------|----------------|
|SM-612V3|0.01|0.016|
|SM-612V3|0.02|0.030|
|SM-612V3|0.03|0.042|
|SM-612V3|0.04|0.055|
|SM-612V3|0.05|0.067|
|SM-612V4|0.01|0.015|
|SM-612V4|0.02|0.028|
|SM-612V4|0.03|0.039|
|SM-612V4|0.04|0.051|
|SM-612V4|0.05|0.063|
|SM-612V5|0.01|0.016|
|SM-612V5|0.02|0.030|
|SM-612V5|0.03|0.042|
|SM-612V5|0.04|0.054|
|SM-612V5|0.05|0.066|
|SM-612V6|0.01|0.015|
|SM-612V6|0.02|0.027|
|SM-612V6|0.03|0.040|
|SM-612V6|0.04|0.051|
|SM-612V6|0.05|0.062|
###Prepare the inputs for the normal-normal calling script
Create a list of all contamination levels for each replicate
ls -1 HCC*contam*V3*.bam > HCC1143withNA12878_3.bams.list
ls -1 HCC*contam*V4*.bam > HCC1143withNA12878_4.bams.list
ls -1 HCC*contam*V5*.bam > HCC1143withNA12878_5.bams.list
ls -1 HCC*contam*V6*.bam > HCC1143withNA12878_6.bams.list
Create a list of the other, uncontaminated normals to call against
ls -1 /humgen/gsa-hpprojects/NA12878Collection/bams/crsp_ice_validation/SM-612V[^37D].bam > /dsde/working/mutect/laura/contamination/NA12878_not3.list
ls -1 /humgen/gsa-hpprojects/NA12878Collection/bams/crsp_ice_validation/SM-612V[^47D].bam > /dsde/working/mutect/laura/contamination/NA12878_not4.list
ls -1 /humgen/gsa-hpprojects/NA12878Collection/bams/crsp_ice_validation/SM-612V[^57D].bam > /dsde/working/mutect/laura/contamination/NA12878_not5.list
ls -1 /humgen/gsa-hpprojects/NA12878Collection/bams/crsp_ice_validation/SM-612V[^67D].bam > /dsde/working/mutect/laura/contamination/NA12878_not6.list
###Run the Caller
Run M2 on contaminated bams versus other all other replicates. Run one loop for each contaminated replicate, passing in contamination estimates as given above
java -jar $QUEUE_JAR -S /dsde/working/mutect/laura/contamination/Qscript_M2_normalNormalLoop.scala -normal /dsde/working/mutect/laura/contamination/NA12878_not3.list -tumor /dsde/working/mutect/laura/contamination/HCC1143withNA12878_3.bams.list -o M2_NA12878run3_ -f 0.016 -f 0.03 -f 0.042 -f 0.055 -f 0.067
java -jar $QUEUE_JAR -S /dsde/working/mutect/laura/contamination/Qscript_M2_normalNormalLoop.scala -normal /dsde/working/mutect/laura/contamination/NA12878_not4.list -tumor /dsde/working/mutect/laura/contamination/HCC1143withNA12878_4.bams.list -o M2_NA12878run4_ -f 0.015 -f 0.028 -f 0.039 -f 0.051 -f 0.063
java -jar $QUEUE_JAR -S /dsde/working/mutect/laura/contamination/Qscript_M2_normalNormalLoop.scala -normal /dsde/working/mutect/laura/contamination/NA12878_not5.list -tumor /dsde/working/mutect/laura/contamination/HCC1143withNA12878_5.bams.list -o M2_NA12878run5_ -f 0.016 -f 0.030 -f 0.042 -f 0.054 -f 0.066
java -jar $QUEUE_JAR -S /dsde/working/mutect/laura/contamination/Qscript_M2_normalNormalLoop.scala -normal /dsde/working/mutect/laura/contamination/NA12878_not6.list -tumor /dsde/working/mutect/laura/contamination/HCC1143withNA12878_6.bams.list -o M2_NA12878run6_ -f 0.015 -f 0.027 -f 0.040 -f 0.051 -f 0.062
###Count the False Positives
Pull out passing SNPs not in PON for each contamination level:
for vcf in M2_NA12878run[0-9]_HCC1143_BL.small.0.01.contaminated.with.SM-612V*.bam.vcf
do
bedtools intersect -a $vcf -b ICE.corrected.bed | grep PASS | awk '{ if ( length($4) + length($5) == 2) print $0 }' | wc -l
done
for vcf in M2_NA12878run[0-9]_HCC1143_BL.small.0.02.contaminated.with.SM-612V*.bam.vcf
do
bedtools intersect -a $vcf -b ICE.corrected.bed | grep PASS | awk '{ if ( length($4) + length($5) == 2) print $0 }' | wc -l
done
for vcf in M2_NA12878run[0-9]_HCC1143_BL.small.0.03.contaminated.with.SM-612V*.bam.vcf
do
bedtools intersect -a $vcf -b ICE.corrected.bed | grep PASS | awk '{ if ( length($4) + length($5) == 2) print $0 }' | wc -l
done
for vcf in M2_NA12878run[0-9]_HCC1143_BL.small.0.04.contaminated.with.SM-612V*.bam.vcf
do
bedtools intersect -a $vcf -b ICE.corrected.bed | grep PASS | awk '{ if ( length($4) + length($5) == 2) print $0 }' | wc -l
done
for vcf in M2_NA12878run[0-9]_HCC1143_BL.small.0.05.contaminated.with.SM-612V*.bam.vcf
do
bedtools intersect -a $vcf -b ICE.corrected.bed | grep PASS | awk '{ if ( length($4) + length($5) == 2) print $0 }' | wc -l
done
(I pasted the results from the terminal into Excel because it's just so easy.)
###Comparison Without Downsampling
To run normal-normal contaminated calling without downsampling, the above /dsde/working/mutect/laura/contamination/Qscript_M2_normalNormalLoop.scala commands can be used, passing in -f 0 for each contamination level instead, e.g.:
java -jar $QUEUE_JAR -S /dsde/working/mutect/laura/contamination/Qscript_M2_normalNormalLoop.scala -normal /dsde/working/mutect/laura/contamination/NA12878_not3.list -tumor /dsde/working/mutect/laura/contamination/HCC1143withNA12878_3.bams.list -o M2_NA12878run3_noContam_ -f 0.0 -f 0.0 -f 0.0 -f 0.0 -f 0.0
###Comparison to M1
To run normal-normal contaminated calling using M1, run the above Queue commands using a Queue jar containing MuTect and passing in /dsde/working/mutect/laura/contamination/Qscript_M1_normalNormalLoop.scala instead of Qscript_M2_normalNormalLoop.scala, e.g.:
java -jar $QUEUE_JAR_WITH_M1 -S /dsde/working/mutect/laura/contamination/Qscript_M1_normalNormalLoop.scala -normal /dsde/working/mutect/laura/contamination/NA12878_not3.list -tumor /dsde/working/mutect/laura/contamination/HCC1143withNA12878_3.bams.list -o M1_NA12878run3_ -f 0.016 -f 0.03 -f 0.042 -f 0.055 -f 0.067
(The MuTect-containing Queue jar can be built from the gsa-unstable branch ldg_MuTect1.)
###Latest Results
|M2 SNPs no correction|M2 SNPs with correction|M1 SNPs no correction|M1 SNPs with correction|M2 INDELs no correction|M2 INDELs with correction|
|---------------------|-----------------------|---------------------|-----------------------|-----------------------|-------------------------|
|0%|93|93|181|181|25|25|
|1%|938|258|854|317|68|30|
|2%|2550|464|1941|385|92|21|
|3%|4171|596|3061|515|134|18|
|4%|5513|707|4002|589|162|21|
|5%|6475|794|4854|624|188|29|

35
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/M2_Panel_Of_Normals_Creation.md 100644
View File

@ -0,0 +1,35 @@
# M2 Panel of Normals
In order to reduce false positives, we use a panel of "normal" (ie non-cancer) samples to filter out both germline events as well as systematic noise. The form of the panel is a VCF file, which is produced via a Queue scripts
You must supply:
- the reference (defaults to hg19)
- the intervals to evaluate
- the list of BAM files
### How To Run
The Queue script for producing the PON can be found in the gsa-unstable repository under ```private/gatk-tools-private/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2```
First, chose the appropriate settings (runnable as environment variables here)
```
QUEUE_JAR=<your-queue-jar>
GSA_UNSTABLE_HOME=<path-to-your-gsa-unstable-checkout>
TEMPDIR=/broad/hptmp/$USER
```
and then run the following Queue command
```
java \
-Djava.io.tmpdir=$TEMPDIR \
-jar $QUEUE_JAR \
-S $GSA_UNSTABLE_HOME/private/gatk-tools-private/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/run_M2_ICE_NN.scala \
--job_queue gsa -qsub -jobResReq virtual_free=5G -startFromScratch \
-sc 50 \
--allbams <your-list-of-bams> \
--intervals <your-intervals> \
--outputprefix <your-output-prefix> \
--start_from_scratch --keep_intermediate_outputs \
-run
```
This will produce many VCFs (1 per sample), plus \<your-output-prefix\>.genotypes.vcf and \<your-output-prefix\>.vcf which are the panel of normals VCF both with and without sample-genotype information. Typically the latter is the one used as input to M2, although either will work.

1291
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/MuTect2.java 100644
View File

File diff suppressed because it is too large Load Diff

81
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/NA12878_Evaluations.md 100644
View File

@ -0,0 +1,81 @@
# CRSP ICE NA12878 Specificity Evaluation
In order to evaluate the specificity of M2, we sequenced replicates of NA12878 using ICE (Illumina Content Exomes) all call all pairwise combinations as tumor-normals. By definition, everything called is a false positive.
The target territory is ```/dsde/working/mutect/crsp_nn/whole_exome_illumina_coding_v1.Homo_sapiens_assembly19.targets.no_empty.interval_list```
All scripts referenced here are relative to the current working directory of ```
/dsde/working/mutect/crsp_nn```
### Current M2 Performance
(gsa-unstable 7/13/15, commit:9e93a70)
| type | # of false positives |
|------|----------------------|
|SNP|99|
|INDEL|15|
TODO: write a simple tool to do this more easily
To calculate per pair-counts, run:
```
# for SNPs
for vcf in *.bam.vcf
do
cat $vcf | grep PASS | awk '{ if ( length($4) + length($5) == 2) print $0 }' | wc -l
done
# for INDELs
for vcf in *.bam.vcf
do
cat $vcf | grep PASS | awk '{ if ( length($4) + length($5) != 2) print $0 }' | wc -l
done
```
### Current M1 and Indelocator Performance
For comparison, the M1 & Indelocator calls have been made on this same data set in the Firehose workspace ```CRSP_ICE_NA1878_Production_Analysis``` in the pair set ```NA12878_Replicate_Pairs``` which contains 4 samples and 12 pairwise combinations.
| type | # of false positives |
|------|----------------------|
|SNP|181|
|INDEL|106|
These results can be obtained (from a LSF / CGA node running the FuSE daemon)
```
SNP:
cat /local/cga-fh/cga/CRSP_ICE_NA1878_Production_Analysis/Pair_Set/NA12878_Replicate_Pairs/Pair/*/jobs/capture/mut/calls/latest/*.call_stats.txt | grep KEEP | wc -l
INDEL (need to restrict to target territory):
reuse BEDTools
cat /dsde/working/mutect/crsp_nn/whole_exome_illumina_coding_v1.Homo_sapiens_assembly19.targets.no_empty.interval_list | grep -v "@" | awk '{ print $1 "\t" $2-1 "\t" $3 }' > ice.bed
cat /local/cga-fh/cga/CRSP_ICE_NA1878_Production_Analysis/Pair_Set/NA12878_Replicate_Pairs/Pair/*/jobs/capture/indel/maflite/latest/*.full.maf | grep KEEP | cut -f2-4 | awk '{ print $1 "\t" $2-1 "\t" $3 }' > /tmp/indels.bed
bedtools intersect -wa -a /tmp/ice.bed -b /tmp/indels.bed | wc -l
```
### How To Run
The SCALA script for running M2 can be found in the gsa-unstable repository under ```private/gatk-tools-private/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2```
First, chose the appropriate settings (runnable as environment variables here)
```
QUEUE_JAR=<your-queue-jar>
OUT_VCF=<your-output-vcf>
GSA_UNSTABLE_HOME=<path-to-your-gsa-unstable-checkout>
TEMPDIR=/broad/hptmp/$USER
```
and then run the following Queue command
```
java \
-Djava.io.tmpdir=$TEMPDIR \
-jar $QUEUE_JAR \
-S $GSA_UNSTABLE_HOME/private/gatk-tools-private/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/run_M2_ICE_NN.scala \
-sc 50 \
--job_queue gsa -qsub -jobResReq virtual_free=5G -startFromScratch \
--allbams /humgen/gsa-hpprojects/NA12878Collection/bams/crsp_ice_validation//NA12878.intra.flowcell.replicate.bam_list \
-o <your-output-prefix>
-run
```

558
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/SomaticGenotypingEngine.java 100644
View File

@ -0,0 +1,558 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer.m2;
import com.google.java.contract.Ensures;
import htsjdk.samtools.util.StringUtil;
import htsjdk.variant.variantcontext.*;
import org.apache.log4j.Logger;
import org.broadinstitute.gatk.tools.walkers.genotyper.GenotypeLikelihoodsCalculationModel;
import org.broadinstitute.gatk.tools.walkers.genotyper.afcalc.AFCalculatorProvider;
import org.broadinstitute.gatk.tools.walkers.haplotypecaller.HaplotypeCallerGenotypingEngine;
import org.broadinstitute.gatk.utils.GenomeLoc;
import org.broadinstitute.gatk.utils.GenomeLocParser;
import org.broadinstitute.gatk.utils.commandline.RodBinding;
import org.broadinstitute.gatk.utils.contexts.ReferenceContext;
import org.broadinstitute.gatk.utils.genotyper.MostLikelyAllele;
import org.broadinstitute.gatk.utils.genotyper.PerReadAlleleLikelihoodMap;
import org.broadinstitute.gatk.utils.genotyper.ReadLikelihoods;
import org.broadinstitute.gatk.utils.genotyper.SampleList;
import org.broadinstitute.gatk.utils.haplotype.Haplotype;
import org.broadinstitute.gatk.utils.refdata.RefMetaDataTracker;
import org.broadinstitute.gatk.utils.sam.GATKSAMRecord;
import org.broadinstitute.gatk.utils.sam.ReadUtils;
import org.broadinstitute.gatk.utils.variant.GATKVCFConstants;
import org.broadinstitute.gatk.utils.variant.GATKVariantContextUtils;
import java.util.*;
public class SomaticGenotypingEngine extends HaplotypeCallerGenotypingEngine {
protected M2ArgumentCollection MTAC;
private final static Logger logger = Logger.getLogger(SomaticGenotypingEngine.class);
public SomaticGenotypingEngine(final M2ArgumentCollection configuration, final SampleList samples, final GenomeLocParser genomeLocParser, final AFCalculatorProvider afCalculatorProvider, final boolean doPhysicalPhasing, final M2ArgumentCollection MTAC) {
super(configuration, samples, genomeLocParser, afCalculatorProvider, doPhysicalPhasing);
this.MTAC = MTAC;
}
/**
* Main entry point of class - given a particular set of haplotypes, samples and reference context, compute
* genotype likelihoods and assemble into a list of variant contexts and genomic events ready for calling
*
* The list of samples we're working with is obtained from the readLikelihoods
*
* @param haplotypes Haplotypes to assign likelihoods to
* @param readLikelihoods Map from reads->(haplotypes,likelihoods)
* @param perSampleFilteredReadList Map from sample to reads that were filtered after assembly and before calculating per-read likelihoods.
* @param ref Reference bytes at active region
* @param refLoc Corresponding active region genome location
* @param activeRegionWindow Active window
* @param genomeLocParser GenomeLocParser
* @param activeAllelesToGenotype Alleles to genotype
* @param emitReferenceConfidence whether we should add a &lt;NON_REF&gt; alternative allele to the result variation contexts.
*
* @return A CalledHaplotypes object containing a list of VC's with genotyped events and called haplotypes
*
*/
// @Requires({"refLoc.containsP(activeRegionWindow)", "haplotypes.size() > 0"})
@Ensures("result != null")
// TODO - can this be refactored? this is hard to follow!
public HaplotypeCallerGenotypingEngine.CalledHaplotypes callMutations (
final List<Haplotype> haplotypes,
//final Map<String, PerReadAlleleLikelihoodMap> haplotypeReadMap,
final ReadLikelihoods<Haplotype> readLikelihoods,
final Map<String, Integer> originalNormalReadQualities,
final Map<String, List<GATKSAMRecord>> perSampleFilteredReadList,
final byte[] ref,
final GenomeLoc refLoc,
final GenomeLoc activeRegionWindow,
final GenomeLocParser genomeLocParser,
final RefMetaDataTracker tracker,
final List<VariantContext> activeAllelesToGenotype,
final boolean emitReferenceConfidence,
final String tumorSampleName,
final String matchedNormalSampleName,
final RodBinding<VariantContext> dbsnpRod,
final List<RodBinding<VariantContext>> cosmicRod,
final String DEBUG_READ_NAME
) {
// sanity check input arguments
if (haplotypes == null || haplotypes.isEmpty()) throw new IllegalArgumentException("haplotypes input should be non-empty and non-null, got "+haplotypes);
if (readLikelihoods == null || readLikelihoods.sampleCount() == 0) throw new IllegalArgumentException("readLikelihoods input should be non-empty and non-null, got "+readLikelihoods);
if (ref == null || ref.length == 0 ) throw new IllegalArgumentException("ref bytes input should be non-empty and non-null, got "+ref);
if (refLoc == null || refLoc.size() != ref.length) throw new IllegalArgumentException(" refLoc must be non-null and length must match ref bytes, got "+refLoc);
if (activeRegionWindow == null ) throw new IllegalArgumentException("activeRegionWindow must be non-null, got "+activeRegionWindow);
if (activeAllelesToGenotype == null ) throw new IllegalArgumentException("activeAllelesToGenotype must be non-null, got "+activeAllelesToGenotype);
if (genomeLocParser == null ) throw new IllegalArgumentException("genomeLocParser must be non-null, got "+genomeLocParser);
// Somatic Tumor/Normal Sample Handling
verifySamplePresence(tumorSampleName, readLikelihoods.samples());
final boolean hasNormal = (matchedNormalSampleName != null);
// update the haplotypes so we're ready to call, getting the ordered list of positions on the reference
// that carry events among the haplotypes
final TreeSet<Integer> startPosKeySet = decomposeHaplotypesIntoVariantContexts(haplotypes, readLikelihoods, ref, refLoc, activeAllelesToGenotype);
// Walk along each position in the key set and create each event to be outputted
final Set<Haplotype> calledHaplotypes = new HashSet<>();
final List<VariantContext> returnCalls = new ArrayList<>();
for( final int loc : startPosKeySet ) {
if( loc >= activeRegionWindow.getStart() && loc <= activeRegionWindow.getStop() ) { // genotyping an event inside this active region
final List<VariantContext> eventsAtThisLoc = getVCsAtThisLocation(haplotypes, loc, activeAllelesToGenotype);
if( eventsAtThisLoc.isEmpty() ) { continue; }
// Create the event mapping object which maps the original haplotype events to the events present at just this locus
final Map<Event, List<Haplotype>> eventMapper = createEventMapper(loc, eventsAtThisLoc, haplotypes);
// Sanity check the priority list for mistakes
final List<String> priorityList = makePriorityList(eventsAtThisLoc);
// Merge the event to find a common reference representation
VariantContext mergedVC = GATKVariantContextUtils.simpleMerge(eventsAtThisLoc, priorityList,
GATKVariantContextUtils.FilteredRecordMergeType.KEEP_IF_ANY_UNFILTERED,
GATKVariantContextUtils.GenotypeMergeType.PRIORITIZE, false, false, null, false, false);
if( mergedVC == null ) { continue; }
final int numAlts = mergedVC.getNAlleles()-1;
// final VariantContextBuilder vcb = new VariantContextBuilder(mergedVC);
final GenotypeLikelihoodsCalculationModel.Model calculationModel = mergedVC.isSNP()
? GenotypeLikelihoodsCalculationModel.Model.SNP : GenotypeLikelihoodsCalculationModel.Model.INDEL;
if (emitReferenceConfidence)
mergedVC = addNonRefSymbolicAllele(mergedVC);
final Map<VariantContext, Allele> mergeMap = new LinkedHashMap<>();
mergeMap.put(null, mergedVC.getReference()); // the reference event (null) --> the reference allele
for(int iii = 0; iii < eventsAtThisLoc.size(); iii++) {
mergeMap.put(eventsAtThisLoc.get(iii), mergedVC.getAlternateAllele(iii)); // BUGBUG: This is assuming that the order of alleles is the same as the priority list given to simpleMerge function
}
final Map<Allele, List<Haplotype>> alleleMapper = createAlleleMapper(mergeMap, eventMapper);
if( configuration.DEBUG && logger != null ) {
if (logger != null) logger.info("Genotyping event at " + loc + " with alleles = " + mergedVC.getAlleles());
}
ReadLikelihoods<Allele> readAlleleLikelihoods = readLikelihoods.marginalize(alleleMapper, genomeLocParser.createPaddedGenomeLoc(genomeLocParser.createGenomeLoc(mergedVC), ALLELE_EXTENSION));
//LDG: do we want to do this before or after pulling out overlapping reads?
if (MTAC.isSampleContaminationPresent())
readAlleleLikelihoods.contaminationDownsampling(MTAC.getSampleContamination());
//if (!mergedVC.isBiallelic()) {
// logger.info("[UNSUPPORTED] Detected non-Biallelic VC" + mergedVC.toString());
// continue;
//}
// TODO: once tests are passing, refactor to use the new data structure (not the deprecated one)
// handle overlapping fragments
// TODO: CONFIRM WITH GSA IF IT IS OK TO REMOVE READS FROM THE PRALM (should be... they do it in filterPoorlyModeledReads!)
PerReadAlleleLikelihoodMap tumorPRALM = readAlleleLikelihoods.toPerReadAlleleLikelihoodMap(readAlleleLikelihoods.sampleIndex(tumorSampleName));
filterPRALMForOverlappingReads(tumorPRALM, mergedVC.getReference(), loc, false);
MuTect2.logReadInfo(DEBUG_READ_NAME, tumorPRALM.getLikelihoodReadMap().keySet(), "Present after filtering for overlapping reads");
// extend to multiple samples
//handle existence of secondary alts
double[] afs = estimateAlleleFraction(mergedVC, tumorPRALM);
if( configuration.DEBUG && logger != null ) {
String output = "Calculated allelic fraction at " + loc + " = ";
for (int i = 0; i<afs.length; i++)
output = output + afs[i];
if (logger != null) logger.info(output);
}
double[] tumorGLs = getVariableGenotypeLikelihoods(mergedVC, tumorPRALM, originalNormalReadQualities, afs);
PerReadAlleleLikelihoodMap normalPRALM = null;
double[] normalGLs = null;
if (hasNormal) {
normalPRALM = readAlleleLikelihoods.toPerReadAlleleLikelihoodMap(readAlleleLikelihoods.sampleIndex(matchedNormalSampleName));
filterPRALMForOverlappingReads(normalPRALM, mergedVC.getReference(), loc, true);
MuTect2.logReadInfo(DEBUG_READ_NAME, normalPRALM.getLikelihoodReadMap().keySet(), "Present after filtering for overlapping reads");
double[] diploidAFarray = new double[numAlts];
Arrays.fill(diploidAFarray, 0.5d);
normalGLs = getVariableGenotypeLikelihoods(mergedVC, normalPRALM, originalNormalReadQualities, diploidAFarray);
}
double INIT_NORMAL_LOD_THRESHOLD = -Double.MAX_VALUE;
double NORMAL_LOD_THRESHOLD = -Double.MAX_VALUE;
final int REF_INDEX = 0;
double[] tumorLods = new double[numAlts];
for (int altInd = 0; altInd < numAlts; altInd++) {
tumorLods[altInd] = tumorGLs[altInd+1] - tumorGLs[REF_INDEX];
}
if (configuration.DEBUG && logger != null) {
String output = "Tumor LOD at " + loc + " = ";
for (int i = 0; i<tumorLods.length; i++)
output = output + tumorLods[i];
if (logger != null) logger.info(output);
}
double[] normalLods = new double[numAlts];
if (hasNormal) {
GenomeLoc eventGenomeLoc = genomeLocParser.createGenomeLoc(activeRegionWindow.getContig(), loc);
Collection<VariantContext> cosmicVC = tracker.getValues(cosmicRod, eventGenomeLoc);
Collection<VariantContext> dbsnpVC = tracker.getValues(dbsnpRod, eventGenomeLoc);
// remove the effect of cosmic from dbSNP
boolean germlineAtRisk = (!dbsnpVC.isEmpty() && cosmicVC.isEmpty());
NORMAL_LOD_THRESHOLD = (germlineAtRisk)?MTAC.NORMAL_DBSNP_LOD_THRESHOLD:MTAC.NORMAL_LOD_THRESHOLD;
for (int altInd = 0; altInd < numAlts; altInd++)
normalLods[altInd] = normalGLs[REF_INDEX] - normalGLs[altInd+1];
}
//reconcile multiple alts, if applicable
int numPassingAlts = 0;
int lodInd = 0;
for (int altInd = 0; altInd < numAlts; altInd++) {
if (tumorLods[altInd] >= MTAC.INITIAL_TUMOR_LOD_THRESHOLD && normalLods[altInd] >= MTAC.INITIAL_NORMAL_LOD_THRESHOLD) {
numPassingAlts++;
lodInd = altInd;
}
}
final double tumorLod = tumorLods[lodInd];
final double normalLod = normalLods[lodInd];
VariantContext call = null;
if (tumorLod >= MTAC.INITIAL_TUMOR_LOD_THRESHOLD && normalLod >= MTAC.INITIAL_NORMAL_LOD_THRESHOLD) {
VariantContextBuilder callVcb = new VariantContextBuilder(mergedVC);
if (normalLod < NORMAL_LOD_THRESHOLD) {
callVcb.filter(GATKVCFConstants.GERMLINE_RISK_FILTER_NAME);
}
int haplotypeCount = alleleMapper.get(mergedVC.getAlternateAllele(lodInd)).size();
callVcb.attribute(GATKVCFConstants.HAPLOTYPE_COUNT_KEY, haplotypeCount);
callVcb.attribute(GATKVCFConstants.TUMOR_LOD_KEY, tumorLod);
callVcb.attribute(GATKVCFConstants.NORMAL_LOD_KEY, normalLod);
if (normalLod < NORMAL_LOD_THRESHOLD) {
callVcb.filter(GATKVCFConstants.GERMLINE_RISK_FILTER_NAME);
}
if (numPassingAlts > 1) {
callVcb.filter(GATKVCFConstants.TRIALLELIC_SITE_FILTER_NAME);
}
List<Allele> tumorAlleles = new ArrayList<>();
tumorAlleles.add(mergedVC.getReference());
tumorAlleles.add(mergedVC.getAlternateAllele(lodInd));
GenotypeBuilder tumorGenotype =
new GenotypeBuilder(tumorSampleName, tumorAlleles);
tumorGenotype.attribute(GATKVCFConstants.ALLELE_FRACTION_KEY, afs[lodInd]);
// how should we set the genotype properly here?
List<Allele> refAlleles = new ArrayList<>();
refAlleles.add(mergedVC.getReference());
refAlleles.add(mergedVC.getReference());
List<Genotype> genotypes = new ArrayList<>();
genotypes.add(tumorGenotype.make());
// if we are calling with a normal, add that sample in
if (hasNormal) {
int[] normalCounts = getRefAltCount(mergedVC, normalPRALM);
int[] normalAD = new int[2];
normalAD[REF_INDEX] = normalCounts[REF_INDEX];
normalAD[1] = normalCounts[lodInd+1];
double normalF = (double) normalAD[1] / ((double) normalAD[REF_INDEX] + (double) normalAD[1]);
GenotypeBuilder normalGenotype =
new GenotypeBuilder(matchedNormalSampleName, refAlleles).AD(normalAD);
normalGenotype.attribute(GATKVCFConstants.ALLELE_FRACTION_KEY, normalF);
genotypes.add(normalGenotype.make());
}
//only use alleles found in the tumor (
call = new VariantContextBuilder(callVcb).alleles(tumorAlleles).genotypes(genotypes).make();
}
// how should we be making use of _perSampleFilteredReadList_?
if( call != null ) {
readAlleleLikelihoods = prepareReadAlleleLikelihoodsForAnnotation(readLikelihoods, perSampleFilteredReadList,
genomeLocParser, emitReferenceConfidence, alleleMapper, readAlleleLikelihoods, call);
ReferenceContext referenceContext = new ReferenceContext(genomeLocParser, genomeLocParser.createGenomeLoc(mergedVC.getChr(), mergedVC.getStart(), mergedVC.getEnd()), refLoc, ref);
VariantContext annotatedCall = annotationEngine.annotateContextForActiveRegion(referenceContext, tracker, readAlleleLikelihoods, call, false);
if( call.getAlleles().size() != mergedVC.getAlleles().size() )
annotatedCall = GATKVariantContextUtils.reverseTrimAlleles(annotatedCall);
// maintain the set of all called haplotypes
for ( final Allele calledAllele : call.getAlleles() ) {
final List<Haplotype> haplotypeList = alleleMapper.get(calledAllele);
if (haplotypeList == null) continue;
calledHaplotypes.addAll(haplotypeList);
}
returnCalls.add( annotatedCall );
}
}
}
// TODO: understand effect of enabling this for somatic calling...
final List<VariantContext> phasedCalls = doPhysicalPhasing ? phaseCalls(returnCalls, calledHaplotypes) : returnCalls;
return new CalledHaplotypes(phasedCalls, calledHaplotypes);
//return new CalledHaplotypes(returnCalls, calledHaplotypes);
}
private void verifySamplePresence(String sampleName, List<String> samples) {
if (!samples.contains(sampleName)) {
throw new IllegalArgumentException("Unable to find sample name "+sampleName+"in sample list of " + StringUtil.join(",", samples));
}
}
/** Calculate the genotype likelihoods for variable allele fraction
*
* @param mergedVC input VC
* @param tumorPRALM read likelihoods
* @param originalNormalMQs original MQs, before boosting normals to avoid qual capping
* @param afs allele fraction(s) for alternate allele(s)
*
* @return genotype likelihoods for homRef (index 0) and het for each alternate allele
*/
private double[] getVariableGenotypeLikelihoods(final VariantContext mergedVC, final PerReadAlleleLikelihoodMap tumorPRALM,
final Map<String, Integer> originalNormalMQs, double[] afs) {
double[] genotypeLikelihoods = new double[mergedVC.getNAlleles()];
for(Map.Entry<GATKSAMRecord,Map<Allele, Double>> e : tumorPRALM.getLikelihoodReadMap().entrySet()) {
Map<Allele, Double> m = e.getValue();
Double refLL = m.get(mergedVC.getReference());
if (originalNormalMQs.get(e.getKey().getReadName()) != 0) {
genotypeLikelihoods[0] += Math.log10(Math.pow(10, refLL));
for (int altInd = 0; altInd < mergedVC.getNAlleles()-1; altInd++) {
Double altLL = m.get(mergedVC.getAlternateAllele(altInd));
genotypeLikelihoods[altInd+1] += Math.log10(Math.pow(10, refLL) * (1 - afs[altInd]) + Math.pow(10, altLL) * afs[altInd]);
}
}
}
return genotypeLikelihoods;
}
/**
* Find the allele fractions for each alternate allele
*
* @param vc input VC, for alleles
* @param map read likelihoods
* @return estimated AF for each alt
*/
// FIXME: calculate using the uncertainty rather than this cheap approach
private double[] estimateAlleleFraction(VariantContext vc, PerReadAlleleLikelihoodMap map) {
int[] counts = getRefAltCount(vc, map);
int numAlts = vc.getNAlleles()-1;
double[] afs = new double[numAlts];
int refCount = counts[0];
int altCount;
for(int altInd = 0; altInd < numAlts; altInd++) {
altCount = counts[altInd+1];
afs[altInd] = (double) altCount / ((double) refCount + (double) altCount);
//logger.info("Counted " + refCount + " ref and " + altCount + " alt " );
}
return afs;
}
/**
* Evalutate the most likely allele for each read, if it is in fact informative
*
* @param mergedVC input VC, for alleles
* @param afMap read likelihoods
* @return an array giving the read counts for the ref and each alt allele
*/
// TODO: ensure there are only two alleles in the VC
private int[] getRefAltCount(VariantContext mergedVC, PerReadAlleleLikelihoodMap afMap) {
int counts[] = new int[mergedVC.getNAlleles()];
int REF = 0;
for(Map.Entry<GATKSAMRecord,Map<Allele, Double>> e : afMap.getLikelihoodReadMap().entrySet()) {
Map<Allele, Double> m = e.getValue();
Double rl = m.get(mergedVC.getReference());
for(int altInd=0; altInd<mergedVC.getNAlleles()-1;altInd++) {
Double al = m.get(mergedVC.getAlternateAllele(altInd));
logger.debug("At " + mergedVC.getStart() + ", for read " + e.getKey().getReadName() + ", al = " + al + ", rl = " + rl + ", diff = " + (al - rl));
if (arePairHMMLikelihoodsInformative(rl, al)) {
if (rl > al) {
counts[REF]++;
} else {
counts[altInd+1]++;
logM2Debug("Using " + e.getKey().toString() + " towards alternate allele count");
}
}
}
// if (al >= rl) logger.info("Alt found in " + e.getKey().getReadName());
}
return counts;
}
private void logM2Debug(String s) {
if (MTAC.M2_DEBUG) {
logger.info(s);
}
}
// would have used org.broadinstitute.sting.utils.genotyper.PerReadAlleleLikelihoodMap.getMostLikelyAllele but we have this case where
// there is a read that doesn't overlap the variant site, and thus supports both alleles equally.
private boolean arePairHMMLikelihoodsInformative(double l1, double l2) {
// TODO: should this be parameterized, or simply encoded
double EPSILON = 0.1;
return (Math.abs(l1 - l2) >= EPSILON);
}
private void filterPRALMForOverlappingReads(PerReadAlleleLikelihoodMap pralm, Allele ref, int location, boolean retainMismatches) {
Map<GATKSAMRecord, Map<Allele, Double>> m = pralm.getLikelihoodReadMap();
// iterate through the reads, if the name has been seen before we have overlapping (potentially) fragments, so handle them
Map<String, GATKSAMRecord> nameToRead = new HashMap<>();
Set<GATKSAMRecord> readsToKeep = new HashSet<>();
for(GATKSAMRecord rec : m.keySet()) {
// if we haven't seen it... just record the name and add it to the list of reads to keep
GATKSAMRecord existing = nameToRead.get(rec.getReadName());
if (existing == null) {
nameToRead.put(rec.getReadName(), rec);
readsToKeep.add(rec);
} else {
logM2Debug("Found a paired read for " + rec.getReadName());
// NOTE: Can we use FragmentUtils to do all of this processing (to find overlapping pairs?)
// seems like maybe, but it has some requirements about the order of the reads supplied which may be painful to meet
// TODO: CHECK IF THE READS BOTH OVERLAP THE POSITION!!!!
if ( ReadUtils.isInsideRead(existing, location) && ReadUtils.isInsideRead(rec, location) ) {
MostLikelyAllele existingMLA = pralm.getMostLikelyAllele(pralm.getLikelihoodReadMap().get(existing));
Allele existingAllele = existingMLA.getMostLikelyAllele();
MostLikelyAllele recMLA = pralm.getMostLikelyAllele(pralm.getLikelihoodReadMap().get(rec));
Allele recAllele = recMLA.getMostLikelyAllele();
// if the reads disagree at this position...
if (!existingAllele.equals(recAllele)) {
//... and we're not retaining mismatches, throw them both out
if (!retainMismatches) {
logM2Debug("Discarding read-pair due to disagreement" + rec.getReadName() + " and allele " + existingAllele);
readsToKeep.remove(existing);
//... and we are retaining mismatches, keep the mismatching one
} else {
if (existingAllele.equals(ref)) {
logM2Debug("Discarding read to keep mismatching " + rec.getReadName() + " and allele " + existingAllele);
readsToKeep.remove(existing);
readsToKeep.add(rec);
}
}
// Otherwise, keep the element with the higher quality score
} else {
logM2Debug("Discarding lower quality read of overlapping pair " + rec.getReadName() + " and allele " + existingAllele);
if (existingMLA.getLog10LikelihoodOfMostLikely() < recMLA.getLog10LikelihoodOfMostLikely()) {
readsToKeep.remove(existing);
readsToKeep.add(rec);
}
}
} else {
// although these are overlapping fragments, they don't overlap at the position in question
// so keep the read
readsToKeep.add(rec);
}
}
}
// perhaps moved into PRALM
final Iterator<Map.Entry<GATKSAMRecord, Map<Allele, Double>>> it = m.entrySet().iterator();
while ( it.hasNext() ) {
final Map.Entry<GATKSAMRecord, Map<Allele, Double>> record = it.next();
if(!readsToKeep.contains(record.getKey())) {
it.remove();
logM2Debug("Dropping read " + record.getKey() + " due to overlapping read fragment rules");
}
}
}
// Move to utility class so we can use one shared with HaplotypeCallerGenotypingEngine
private VariantContext addNonRefSymbolicAllele(final VariantContext mergedVC) {
final VariantContextBuilder vcb = new VariantContextBuilder(mergedVC);
final List<Allele> originalList = mergedVC.getAlleles();
final List<Allele> alleleList = new ArrayList<>(originalList.size() + 1);
alleleList.addAll(mergedVC.getAlleles());
alleleList.add(GATKVCFConstants.NON_REF_SYMBOLIC_ALLELE);
vcb.alleles(alleleList);
return vcb.make();
}
}

140
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/create_M2_pon.scala 100644
View File

@ -0,0 +1,140 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer.m2
import java.io.File
import org.broadinstitute.gatk.queue.QScript
import org.broadinstitute.gatk.queue.extensions.gatk._
import org.broadinstitute.gatk.queue.function.CommandLineFunction
import org.broadinstitute.gatk.queue.util.QScriptUtils
import org.broadinstitute.gatk.utils.commandline.{Input, Output}
import org.broadinstitute.gatk.utils.variant.GATKVariantContextUtils.FilteredRecordMergeType
import scala.collection.mutable.ListBuffer
class create_M2_pon extends QScript {
@Argument(shortName = "bams", required = true, doc = "file of all BAM files")
var allBams: String = ""
@Argument(shortName = "o", required = true, doc = "Output prefix")
var outputPrefix: String = ""
@Argument(shortName = "minN", required = false, doc = "minimum number of sample observations to include in PON")
var minN: Int = 2
@Argument(doc="Reference fasta file to process with", fullName="reference", shortName="R", required=false)
var reference = new File("/seq/references/Homo_sapiens_assembly19/v1/Homo_sapiens_assembly19.fasta")
@Argument(doc="Intervals file to process with", fullName="intervals", shortName="L", required=true)
var intervals : File = ""
@Argument(shortName = "sc", required = false, doc = "base scatter count")
var scatter: Int = 10
def script() {
val bams = QScriptUtils.createSeqFromFile(allBams)
val genotypesVcf = outputPrefix + ".genotypes.vcf"
val finalVcf = outputPrefix + ".vcf"
val perSampleVcfs = new ListBuffer[File]()
for (bam <- bams) {
val outputVcf = "sample-vcfs/" + bam.getName + ".vcf"
add( createM2Config(bam, outputVcf))
perSampleVcfs += outputVcf
}
val cv = new CombineVariants()
cv.reference_sequence = reference
cv.memoryLimit = 2
cv.setKey = "null"
cv.minimumN = minN
cv.memoryLimit = 16
cv.filteredrecordsmergetype = FilteredRecordMergeType.KEEP_IF_ANY_UNFILTERED
cv.filteredAreUncalled = true
cv.variant = perSampleVcfs
cv.out = genotypesVcf
// using this instead of "sites_only" because we want to keep the AC info
val vc = new VcfCutter()
vc.inVcf = genotypesVcf
vc.outVcf = finalVcf
add (cv, vc)
}
def createM2Config(bam : File, outputVcf : File): org.broadinstitute.gatk.queue.extensions.gatk.MuTect2 = {
val mutect2 = new org.broadinstitute.gatk.queue.extensions.gatk.MuTect2
mutect2.reference_sequence = reference
mutect2.artifact_detection_mode = true
mutect2.intervalsString :+= intervals
mutect2.memoryLimit = 2
mutect2.input_file = List(new TaggedFile(bam, "tumor"))
mutect2.scatterCount = scatter
mutect2.out = outputVcf
mutect2
}
}
class VcfCutter extends CommandLineFunction {
@Input(doc = "vcf to cut") var inVcf: File = _
@Output(doc = "output vcf") var outVcf: File = _
def commandLine = "cat %s | cut -f1-8 > %s".format(inVcf, outVcf)
}

102
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/run_M2_ICE_NN.scala 100644
View File

@ -0,0 +1,102 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.queue.qscripts.dev
import org.broadinstitute.gatk.queue.QScript
import org.broadinstitute.gatk.queue.extensions.gatk._
import org.broadinstitute.gatk.queue.util.QScriptUtils
class run_M2_ICE_NN extends QScript {
@Argument(shortName = "bams", required = true, doc = "file of all BAM files")
var allBams: String = ""
@Argument(shortName = "o", required = false, doc = "Output prefix")
var outputPrefix: String = ""
@Argument(shortName = "pon", required = false, doc = "Normal PON")
var panelOfNormals: String = "/dsde/working/mutect/panel_of_normals/panel_of_normals_m2_ice_wgs_territory/m2_406_ice_normals_wgs_calling_regions.vcf";
@Argument(shortName = "sc", required = false, doc = "base scatter count")
var scatter: Int = 10
def script() {
val bams = QScriptUtils.createSeqFromFile(allBams)
for (tumor <- bams) {
for (normal <- bams) {
if (tumor != normal) add( createM2Config(tumor, normal, new File(panelOfNormals), outputPrefix))
}
}
}
def createM2Config(tumorBAM : File, normalBAM : File, panelOfNormals : File, outputPrefix : String): M2 = {
val mutect2 = new MuTect2
mutect2.reference_sequence = new File("/seq/references/Homo_sapiens_assembly19/v1/Homo_sapiens_assembly19.fasta")
mutect2.cosmic :+= new File("/xchip/cga/reference/hg19/hg19_cosmic_v54_120711.vcf")
mutect2.dbsnp = new File("/humgen/gsa-hpprojects/GATK/bundle/current/b37/dbsnp_138.b37.vcf")
mutect2.normal_panel :+= panelOfNormals
mutect2.intervalsString :+= new File("/dsde/working/mutect/crsp_nn/whole_exome_illumina_coding_v1.Homo_sapiens_assembly19.targets.no_empty.interval_list")
mutect2.memoryLimit = 2
mutect2.input_file = List(new TaggedFile(normalBAM, "normal"), new TaggedFile(tumorBAM, "tumor"))
mutect2.scatterCount = scatter
mutect2.out = outputPrefix + tumorBAM.getName + "-vs-" + normalBAM.getName + ".vcf"
println("Adding " + tumorBAM + " vs " + normalBAM + " as " + mutect2.out)
mutect2
}
}

89
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/run_M2_dream.scala 100644
View File

@ -0,0 +1,89 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.queue.qscripts.dev
import org.broadinstitute.gatk.queue.QScript
import org.broadinstitute.gatk.queue.extensions.gatk._
class run_M2_dream extends QScript {
@Argument(shortName = "L", required=false, doc = "Intervals file")
var intervalsFile: List[File] = Nil
@Argument(shortName = "normal", required=true, doc = "Normal sample BAM")
var normalBAM: String = ""
@Argument(shortName = "tumor", required=true, doc = "Tumor sample BAM")
var tumorBAM: String = ""
@Argument(shortName = "o", required=true, doc = "Output file")
var outputFile: String = ""
@Argument(shortName = "sc", required=false, doc = "base scatter count")
var scatter: Int = 10
def script() {
val mutect2 = new MuTect2
mutect2.reference_sequence = new File("/seq/references/Homo_sapiens_assembly19/v1/Homo_sapiens_assembly19.fasta")
mutect2.cosmic :+= new File("/xchip/cga/reference/hg19/hg19_cosmic_v54_120711.vcf")
mutect2.dbsnp = new File("/humgen/gsa-hpprojects/GATK/bundle/current/b37/dbsnp_138.b37.vcf")
mutect2.normal_panel :+= new File("/xchip/cga/reference/hg19/wgs_hg19_125_cancer_blood_normal_panel.vcf")
mutect2.intervalsString = intervalsFile
mutect2.memoryLimit = 2
mutect2.input_file = List(new TaggedFile(normalBAM, "normal"), new TaggedFile(tumorBAM, "tumor"))
mutect2.scatterCount = scatter
mutect2.out = outputFile
add(mutect2)
}
}

40
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/haplotypecaller/HaplotypeCaller.java
View File

@ -108,9 +108,17 @@ import java.io.FileNotFoundException;
import java.util.*;
/**
* Call SNPs and indels simultaneously via local assembly of haplotypes in an active region
* Call germline SNPs and indels via local re-assembly of haplotypes
*
* <p>The basic operation of the HaplotypeCaller proceeds as follows: </p>
* <p>The HaplotypeCaller is capable of calling SNPs and indels simultaneously via local de-novo assembly of haplotypes in an active region. In other words, whenever the program encounters a region showing signs of variation, it discards the existing mapping information and completely reassembles the reads in that region. This allows the HaplotypeCaller to be more accurate when calling regions that are traditionally difficult to call, for example when they contain different types of variants close to each other. It also makes the HaplotypeCaller much better at calling indels than position-based callers like UnifiedGenotyper.</p>
<p>In the so-called GVCF mode used for scalable variant calling in DNA sequence data, HaplotypeCaller runs per-sample to generate an intermediate genomic gVCF (gVCF), which can then be used for joint genotyping of multiple samples in a very efficient way, which enables rapid incremental processing of samples as they roll off the sequencer, as well as scaling to very large cohort sizes (e.g. the 92K exomes of ExAC).</p>
<p>In addition, HaplotypeCaller is able to handle non-diploid organisms as well as pooled experiment data. Note however that the algorithms used to calculate variant likelihoods is not well suited to extreme allele frequencies (relative to ploidy) so its use is not recommended for somatic (cancer) variant discovery. For that purpose, use MuTect2 instead.</p>
<p>Finally, HaplotypeCaller is also able to correctly handle the splice junctions that make RNAseq a challenge for most variant callers.</p>
*
* <h3>How HaplotypeCaller works</h3>
*
* <br />
* <h4>1. Define active regions </h4>
@ -159,16 +167,13 @@ import java.util.*;
* Best Practices documentation for detailed recommendations. </p>
*
* <br />
* <h4>Single-sample all-sites calling on DNAseq (for `-ERC GVCF` cohort analysis workflow)</h4>
* <h4>Single-sample GVCF calling on DNAseq (for `-ERC GVCF` cohort analysis workflow)</h4>
* <pre>
* java
* -jar GenomeAnalysisTK.jar
* -T HaplotypeCaller
* -R reference.fasta
* java -jar GenomeAnalysisTK.jar \
* -R reference.fasta \
* -T HaplotypeCaller \
* -I sample1.bam \
* --emitRefConfidence GVCF \
* --variant_index_type LINEAR \
* --variant_index_parameter 128000
* [--dbsnp dbSNP.vcf] \
* [-L targets.interval_list] \
* -o output.raw.snps.indels.g.vcf
@ -176,10 +181,9 @@ import java.util.*;
*
* <h4>Variant-only calling on DNAseq</h4>
* <pre>
* java
* -jar GenomeAnalysisTK.jar
* -T HaplotypeCaller
* -R reference.fasta
* java -jar GenomeAnalysisTK.jar \
* -R reference.fasta \
* -T HaplotypeCaller \
* -I sample1.bam [-I sample2.bam ...] \
* [--dbsnp dbSNP.vcf] \
* [-stand_call_conf 30] \
@ -190,10 +194,9 @@ import java.util.*;
*
* <h4>Variant-only calling on RNAseq</h4>
* <pre>
* java
* -jar GenomeAnalysisTK.jar
* -T HaplotypeCaller
* -R reference.fasta
* java -jar GenomeAnalysisTK.jar \
* -R reference.fasta \
* -T HaplotypeCaller \
* -I sample1.bam \
* [--dbsnp dbSNP.vcf] \
* -stand_call_conf 20 \
@ -210,8 +213,7 @@ import java.util.*;
* </ul>
*
* <h3>Special note on ploidy</h3>
* <p>This tool is able to handle almost any ploidy (except very high ploidies in large pooled experiments); the ploidy
* can be specified using the -ploidy argument for non-diploid organisms.</p>
* <p>This tool is able to handle almost any ploidy (except very high ploidies in large pooled experiments); the ploidy can be specified using the -ploidy argument for non-diploid organisms.</p>
*
* <h3>Additional Notes</h3>
* <ul>

110
protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/utils/cancer/TestingReadUtils.java 100644
View File

@ -0,0 +1,110 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.utils.cancer;
import htsjdk.samtools.SAMFileHeader;
import htsjdk.samtools.SAMReadGroupRecord;
import org.broadinstitute.gatk.utils.contexts.AlignmentContext;
import org.broadinstitute.gatk.utils.GenomeLoc;
import org.broadinstitute.gatk.utils.GenomeLocParser;
import org.broadinstitute.gatk.utils.pileup.ReadBackedPileup;
import org.broadinstitute.gatk.utils.pileup.ReadBackedPileupImpl;
import org.broadinstitute.gatk.utils.sam.ArtificialSAMUtils;
import org.broadinstitute.gatk.utils.sam.GATKSAMReadGroupRecord;
import org.broadinstitute.gatk.utils.sam.GATKSAMRecord;
import java.util.ArrayList;
import java.util.List;
/**
* Created by IntelliJ IDEA.
* User: aaron
* Date: 2/18/12
* Time: 9:33 PM
* To change this template use File | Settings | File Templates.
*/
public class TestingReadUtils {
/**
* generates an alignment context with the following parameters:
* @param totalReads the total number of reads. total - targetReads with have all 'A' bases
* @param targetReads number of reads in the pileup to contain 'T'
* @param firstReadGroup the read group for the 'A' reads
* @param secondReadGroup the read group for the 'T' reads
* @return an alignment context with the reads filled in according to the pattern above. The pileup will be located at the
* first base of the first sequence entry
*/
public static final AlignmentContext generateAlignmentContext(int totalReads, int targetReads, SAMFileHeader header, GenomeLocParser parser, String firstReadGroup, String secondReadGroup) {
List<GATKSAMRecord> reads = new ArrayList<GATKSAMRecord>();
// filled with A bases
for (int i = 0; i < totalReads - targetReads; i++) {
reads.add(addReadGroup(ArtificialSAMUtils.createArtificialRead(header, "Read" + i, 0, 1, 50),firstReadGroup));
}
byte bases[] = new byte[50];
byte quals[] = new byte[50];
for (int i = 0; i < 50; i++) {bases[i] = 'T'; quals[i] = 30;}
for (int i = totalReads - targetReads; i < totalReads; i++) {
reads.add(addReadGroup(ArtificialSAMUtils.createArtificialRead(header, "Read" + i, 0, 1, bases, quals),secondReadGroup));
}
GenomeLoc loc = parser.createGenomeLoc(header.getSequenceDictionary().getSequence(0).getSequenceName(),1,1);
ReadBackedPileup rbp = new ReadBackedPileupImpl(loc, reads, 0);
return new AlignmentContext(loc,rbp);
}
public static final GATKSAMRecord addReadGroup(GATKSAMRecord rec, String readGroupName) {
GATKSAMReadGroupRecord rgRec = new GATKSAMReadGroupRecord(readGroupName);
rgRec.setSample("sample");
rec.setReadGroup(rgRec);
return rec;
}
}

80
protected/gatk-tools-protected/src/test/java/org/broadinstitute/gatk/tools/walkers/cancer/BaseQualitySumPerAlleleBySampleUnitTest.java 100644
View File

@ -0,0 +1,80 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer;
import htsjdk.samtools.SAMFileHeader;
import org.broadinstitute.gatk.utils.QualityUtils;
import org.broadinstitute.gatk.utils.sam.ArtificialSAMUtils;
import org.broadinstitute.gatk.utils.sam.GATKSAMRecord;
import org.testng.Assert;
import org.testng.annotations.Test;
public class BaseQualitySumPerAlleleBySampleUnitTest {
@Test
public void BasicTest() {
BaseQualitySumPerAlleleBySample a = new BaseQualitySumPerAlleleBySample();
final SAMFileHeader header = ArtificialSAMUtils.createArtificialSamHeader(5, 1, 10000);
final GATKSAMRecord read = ArtificialSAMUtils.createArtificialRead(header, "myRead", 0, 1, 76);
read.setMappingQuality(60);
Assert.assertTrue(a.isUsableRead(read));
read.setMappingQuality(0);
Assert.assertFalse(a.isUsableRead(read));
read.setMappingQuality(QualityUtils.MAPPING_QUALITY_UNAVAILABLE);
Assert.assertFalse(a.isUsableRead(read));
}
}

103
protected/gatk-tools-protected/src/test/java/org/broadinstitute/gatk/tools/walkers/cancer/contamination/ContEstIntegrationTest.java 100644
View File

@ -0,0 +1,103 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer.contamination;
import org.broadinstitute.gatk.engine.walkers.WalkerTest;
import org.testng.annotations.Test;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
/**
* Test ContEst with and without a "genotyping array" VCF
*
* @author gauthier
*/
public class ContEstIntegrationTest extends WalkerTest {
public static final String DREAMBamsDirectory = new String("/dsde/working/mutect/dream_smc/bams/");
public static final String ContaminatedBamsDirectory = new String("/dsde/working/mutect/contamination/bams/");
public static final String NormalBamsDirectory = new String("/humgen/gsa-hpprojects/NA12878Collection/bams/crsp_ice_validation/");
public static final String ICEexomeIntervals = new String("/seq/references/HybSelOligos/HybSelOligos/whole_exome_illumina_coding_v1/whole_exome_illumina_coding_v1.Homo_sapiens_assembly19.targets.interval_list");
@Test(enabled = true)
public void testWithArray() {
List<String> md5sums = Arrays.asList("d41d8cd98f00b204e9800998ecf8427e");
WalkerTest.WalkerTestSpec spec = new WalkerTest.WalkerTestSpec(
"-T ContEst" +
" -I " + DREAMBamsDirectory + "synthetic.challenge.set4.tumor.bam" +
" -R " + b37KGReference +
" --popfile " + validationDataLocation + "cancer/hg19_population_stratified_af_hapmap_3.3.fixed.vcf" +
" --genotypes " + validationDataLocation + "cancer/Dream.set3.tumorGTs.vcf" +
" -L " + validationDataLocation + "cancer/SNP6.hg19.interval_list" +
" -L " + ICEexomeIntervals +
" -L 1" +
" -isr INTERSECTION",md5sums);
executeTest("testWithArray",spec);
}
@Test(enabled = true)
public void testArrayFree(){
List<String> md5sums = Arrays.asList("d41d8cd98f00b204e9800998ecf8427e");
WalkerTest.WalkerTestSpec spec = new WalkerTest.WalkerTestSpec(
"-T ContEst" +
" -I:eval " + ContaminatedBamsDirectory + "HCC1143_BL.small.0.05.contaminated.with.SM-612V3.small.0.95.bam" +
" -I:genotype " + NormalBamsDirectory + "SM-612V4.bam" +
" -R " + b37KGReference +
" --popfile " + validationDataLocation + "hg19_population_stratified_af_hapmap_3.3.fixed.vcf" +
" -L " + validationDataLocation + "SNP6.hg19.interval_list" +
" -L " + ICEexomeIntervals +
" -L 1" +
" -isr INTERSECTION",md5sums);
executeTest("testArrayFree",spec);
}
}

134
protected/gatk-tools-protected/src/test/java/org/broadinstitute/gatk/tools/walkers/cancer/contamination/ContEstUnitTest.java 100644
View File

@ -0,0 +1,134 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer.contamination;
import htsjdk.samtools.SAMFileHeader;
import org.broadinstitute.gatk.utils.cancer.TestingReadUtils;
import org.broadinstitute.gatk.utils.BaseTest;
import org.broadinstitute.gatk.engine.GenomeAnalysisEngine;
import org.broadinstitute.gatk.utils.contexts.AlignmentContext;
import org.broadinstitute.gatk.utils.contexts.ReferenceContext;
import org.broadinstitute.gatk.utils.GenomeLoc;
import org.broadinstitute.gatk.utils.GenomeLocParser;
import org.broadinstitute.gatk.utils.sam.ArtificialSAMUtils;
import htsjdk.variant.variantcontext.Allele;
import htsjdk.variant.variantcontext.Genotype;
import org.testng.annotations.Test;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import static org.testng.AssertJUnit.assertTrue;
/**
* Created by IntelliJ IDEA.
* User: aaron
* Date: 2/18/12
* Time: 4:05 PM
* To change this template use File | Settings | File Templates.
*/
public class ContEstUnitTest extends BaseTest {
SAMFileHeader header = ArtificialSAMUtils.createArtificialSamHeader(1,1,2000);
GenomeLocParser parser = new GenomeLocParser(header.getSequenceDictionary());
private static final Map<Integer,Allele> alleles = new HashMap<Integer,Allele>();
String primaryRG = new String("genotype");
static {
alleles.put(0,Allele.create((byte) 'A'));
alleles.put(1,Allele.create((byte) 'C'));
alleles.put(2,Allele.create((byte) 'G'));
alleles.put(3,Allele.create((byte) 'T'));
}
@Test
public void testGettingGenotypes() {
// test that we can get a
Genotype g = testContaminationGenotype(20,80,0.8);
assertTrue(g.isHomVar());
assertTrue("T".equals(g.getGenotypeString()));
g = testContaminationGenotype(19,81,0.8);
assertTrue(g.isHomVar());
assertTrue("T".equals(g.getGenotypeString()));
g = testContaminationGenotype(21,79,0.8);
assertTrue(g == null);
}
private Genotype testContaminationGenotype(int aBases, int tBases, double minGenotypeRatio) {
// setup all the parameters
AlignmentContext context = TestingReadUtils.generateAlignmentContext(aBases + tBases, tBases, header, parser, primaryRG, primaryRG);
GenomeLoc loc = parser.createGenomeLoc(header.getSequenceDictionary().getSequence(0).getSequenceName(),1,1);
ReferenceContext referenceContext = new ReferenceContext(parser,loc,(byte)'A');
ContEst.SeqGenotypeMode genotypeMode = ContEst.SeqGenotypeMode.HARD_THRESHOLD;
int minGenotypingDepth = 50;
double minGenotypingLOD = 5;
Map<String,HashSet<String>> bamReadGroupMapping = new HashMap<String, HashSet<String>>();
HashSet<String> brgm = new HashSet<String>();
brgm.add(primaryRG);
bamReadGroupMapping.put(ContEst.GENOTYPE_BAM_TAG,brgm);
String sampleName = "sample";
GenomeAnalysisEngine toolKit = null;
return ContEst.getGenotypeFromSeq(context,
referenceContext,
alleles,
genotypeMode,
minGenotypeRatio,
minGenotypingDepth,
minGenotypingLOD,
sampleName,
sampleName,
toolKit);
}
}

124
protected/gatk-tools-protected/src/test/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/MuTect2IntegrationTest.java 100644
View File

@ -0,0 +1,124 @@
/*
* By downloading the PROGRAM you agree to the following terms of use:
*
* BROAD INSTITUTE
* SOFTWARE LICENSE AGREEMENT
* FOR ACADEMIC NON-COMMERCIAL RESEARCH PURPOSES ONLY
*
* This Agreement is made between the Broad Institute, Inc. with a principal address at 415 Main Street, Cambridge, MA 02142 (“BROAD”) and the LICENSEE and is effective at the date the downloading is completed (“EFFECTIVE DATE”).
*
* WHEREAS, LICENSEE desires to license the PROGRAM, as defined hereinafter, and BROAD wishes to have this PROGRAM utilized in the public interest, subject only to the royalty-free, nonexclusive, nontransferable license rights of the United States Government pursuant to 48 CFR 52.227-14; and
* WHEREAS, LICENSEE desires to license the PROGRAM and BROAD desires to grant a license on the following terms and conditions.
* NOW, THEREFORE, in consideration of the promises and covenants made herein, the parties hereto agree as follows:
*
* 1. DEFINITIONS
* 1.1 PROGRAM shall mean copyright in the object code and source code known as GATK3 and related documentation, if any, as they exist on the EFFECTIVE DATE and can be downloaded from http://www.broadinstitute.org/gatk on the EFFECTIVE DATE.
*
* 2. LICENSE
* 2.1 Grant. Subject to the terms of this Agreement, BROAD hereby grants to LICENSEE, solely for academic non-commercial research purposes, a non-exclusive, non-transferable license to: (a) download, execute and display the PROGRAM and (b) create bug fixes and modify the PROGRAM. LICENSEE hereby automatically grants to BROAD a non-exclusive, royalty-free, irrevocable license to any LICENSEE bug fixes or modifications to the PROGRAM with unlimited rights to sublicense and/or distribute. LICENSEE agrees to provide any such modifications and bug fixes to BROAD promptly upon their creation.
* The LICENSEE may apply the PROGRAM in a pipeline to data owned by users other than the LICENSEE and provide these users the results of the PROGRAM provided LICENSEE does so for academic non-commercial purposes only. For clarification purposes, academic sponsored research is not a commercial use under the terms of this Agreement.
* 2.2 No Sublicensing or Additional Rights. LICENSEE shall not sublicense or distribute the PROGRAM, in whole or in part, without prior written permission from BROAD. LICENSEE shall ensure that all of its users agree to the terms of this Agreement. LICENSEE further agrees that it shall not put the PROGRAM on a network, server, or other similar technology that may be accessed by anyone other than the LICENSEE and its employees and users who have agreed to the terms of this agreement.
* 2.3 License Limitations. Nothing in this Agreement shall be construed to confer any rights upon LICENSEE by implication, estoppel, or otherwise to any computer software, trademark, intellectual property, or patent rights of BROAD, or of any other entity, except as expressly granted herein. LICENSEE agrees that the PROGRAM, in whole or part, shall not be used for any commercial purpose, including without limitation, as the basis of a commercial software or hardware product or to provide services. LICENSEE further agrees that the PROGRAM shall not be copied or otherwise adapted in order to circumvent the need for obtaining a license for use of the PROGRAM.
*
* 3. PHONE-HOME FEATURE
* LICENSEE expressly acknowledges that the PROGRAM contains an embedded automatic reporting system (“PHONE-HOME”) which is enabled by default upon download. Unless LICENSEE requests disablement of PHONE-HOME, LICENSEE agrees that BROAD may collect limited information transmitted by PHONE-HOME regarding LICENSEE and its use of the PROGRAM. Such information shall include LICENSEE’S user identification, version number of the PROGRAM and tools being run, mode of analysis employed, and any error reports generated during run-time. Collection of such information is used by BROAD solely to monitor usage rates, fulfill reporting requirements to BROAD funding agencies, drive improvements to the PROGRAM, and facilitate adjustments to PROGRAM-related documentation.
*
* 4. OWNERSHIP OF INTELLECTUAL PROPERTY
* LICENSEE acknowledges that title to the PROGRAM shall remain with BROAD. The PROGRAM is marked with the following BROAD copyright notice and notice of attribution to contributors. LICENSEE shall retain such notice on all copies. LICENSEE agrees to include appropriate attribution if any results obtained from use of the PROGRAM are included in any publication.
* Copyright 2012-2015 Broad Institute, Inc.
* Notice of attribution: The GATK3 program was made available through the generosity of Medical and Population Genetics program at the Broad Institute, Inc.
* LICENSEE shall not use any trademark or trade name of BROAD, or any variation, adaptation, or abbreviation, of such marks or trade names, or any names of officers, faculty, students, employees, or agents of BROAD except as states above for attribution purposes.
*
* 5. INDEMNIFICATION
* LICENSEE shall indemnify, defend, and hold harmless BROAD, and their respective officers, faculty, students, employees, associated investigators and agents, and their respective successors, heirs and assigns, (Indemnitees), against any liability, damage, loss, or expense (including reasonable attorneys fees and expenses) incurred by or imposed upon any of the Indemnitees in connection with any claims, suits, actions, demands or judgments arising out of any theory of liability (including, without limitation, actions in the form of tort, warranty, or strict liability and regardless of whether such action has any factual basis) pursuant to any right or license granted under this Agreement.
*
* 6. NO REPRESENTATIONS OR WARRANTIES
* THE PROGRAM IS DELIVERED AS IS. BROAD MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE PROGRAM OR THE COPYRIGHT, EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, WHETHER OR NOT DISCOVERABLE. BROAD EXTENDS NO WARRANTIES OF ANY KIND AS TO PROGRAM CONFORMITY WITH WHATEVER USER MANUALS OR OTHER LITERATURE MAY BE ISSUED FROM TIME TO TIME.
* IN NO EVENT SHALL BROAD OR ITS RESPECTIVE DIRECTORS, OFFICERS, EMPLOYEES, AFFILIATED INVESTIGATORS AND AFFILIATES BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ECONOMIC DAMAGES OR INJURY TO PROPERTY AND LOST PROFITS, REGARDLESS OF WHETHER BROAD SHALL BE ADVISED, SHALL HAVE OTHER REASON TO KNOW, OR IN FACT SHALL KNOW OF THE POSSIBILITY OF THE FOREGOING.
*
* 7. ASSIGNMENT
* This Agreement is personal to LICENSEE and any rights or obligations assigned by LICENSEE without the prior written consent of BROAD shall be null and void.
*
* 8. MISCELLANEOUS
* 8.1 Export Control. LICENSEE gives assurance that it will comply with all United States export control laws and regulations controlling the export of the PROGRAM, including, without limitation, all Export Administration Regulations of the United States Department of Commerce. Among other things, these laws and regulations prohibit, or require a license for, the export of certain types of software to specified countries.
* 8.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
* 8.3 Survival. The following provisions shall survive the expiration or termination of this Agreement: Articles 1, 3, 4, 5 and Sections 2.2, 2.3, 7.3, and 7.4.
* 8.4 Notice. Any notices under this Agreement shall be in writing, shall specifically refer to this Agreement, and shall be sent by hand, recognized national overnight courier, confirmed facsimile transmission, confirmed electronic mail, or registered or certified mail, postage prepaid, return receipt requested. All notices under this Agreement shall be deemed effective upon receipt.
* 8.5 Amendment and Waiver; Entire Agreement. This Agreement may be amended, supplemented, or otherwise modified only by means of a written instrument signed by all parties. Any waiver of any rights or failure to act in a specific instance shall relate only to such instance and shall not be construed as an agreement to waive any rights or fail to act in any other instance, whether or not similar. This Agreement constitutes the entire agreement among the parties with respect to its subject matter and supersedes prior agreements or understandings between the parties relating to its subject matter.
* 8.6 Binding Effect; Headings. This Agreement shall be binding upon and inure to the benefit of the parties and their respective permitted successors and assigns. All headings are for convenience only and shall not affect the meaning of any provision of this Agreement.
* 8.7 Governing Law. This Agreement shall be construed, governed, interpreted and applied in accordance with the internal laws of the Commonwealth of Massachusetts, U.S.A., without regard to conflict of laws principles.
*/
package org.broadinstitute.gatk.tools.walkers.cancer.m2;
import org.broadinstitute.gatk.engine.walkers.WalkerTest;
import org.testng.annotations.Test;
import java.util.*;
public class MuTect2IntegrationTest extends WalkerTest {
final static String REF = hg19Reference;
final static String CCLE_MICRO_TUMOR_BAM = privateTestDir + "HCC1143.cghub.ccle.micro.bam";
final static String CCLE_MICRO_NORMAL_BAM = privateTestDir + "HCC1143_BL.cghub.ccle.micro.bam";
final static String CCLE_MICRO_INTERVALS_FILE = privateTestDir + "HCC1143.cghub.ccle.micro.intervals";
final static String DBSNP=b37dbSNP132;
final static String COSMIC="/xchip/cga/reference/hg19/hg19_cosmic_v54_120711.vcf";
final static String PON="/xchip/cga/reference/hg19/refseq_exome_10bp_hg19_300_1kg_normal_panel.vcf";
final static String DREAM3_TUMOR_BAM = validationDataLocation + "cancer/dream3.integrationtest.tumor.bam";
final static String DREAM3_NORMAL_BAM = validationDataLocation + "cancer/dream3.integrationtest.normal.bam";
final static String DREAM3_TP_INTERVALS_FILE = privateTestDir + "m2_dream3.tp.intervals";
final static String DREAM3_FP_INTERVALS_FILE = privateTestDir + "m2_dream3.fp.intervals";
private void M2Test(String tumorBam, String normalBam, String intervals, String args, String md5) {
final String base = String.format(
"-T MuTect2 --no_cmdline_in_header -dt NONE --disableDithering -alwaysloadVectorHMM -pairHMM LOGLESS_CACHING -ip 50 -R %s --dbsnp %s --cosmic %s --normal_panel %s -I:tumor %s -I:normal %s -L %s",
REF, DBSNP, COSMIC, PON, tumorBam, normalBam, intervals) +
" -o %s ";
final WalkerTestSpec spec = new WalkerTestSpec(base + " " + args, Arrays.asList(md5));
// TODO: do we want to enable this and why? It explodes with
// java.lang.RuntimeException: java.lang.ClassCastException: java.lang.Double cannot be cast to java.lang.String
// at htsjdk.variant.variantcontext.writer.BCF2FieldEncoder$StringOrCharacter.javaStringToBCF2String(BCF2FieldEncoder.java:312)
spec.disableShadowBCF();
executeTest("testM2: args=" + args, spec);
}
@Test
public void testMicroRegression() {
M2Test(CCLE_MICRO_TUMOR_BAM, CCLE_MICRO_NORMAL_BAM, CCLE_MICRO_INTERVALS_FILE, "", "617054c6d056cad7448a463cb8d04a55");
}
/**
* Tests all the True Positive sites in the DREAM 3 data set. We don't necessarily call
* all of these (e.g. we have some FNs) but it's the full set of things we want to be able
* to call, and not regress
*/
@Test
public void testTruePositivesDream3() {
M2Test(DREAM3_TUMOR_BAM, DREAM3_NORMAL_BAM, DREAM3_TP_INTERVALS_FILE, "", "f856432679e43445d2939772be4326cf");
}
/**
* Tests a number of False Positive calls from the DREAM 3 data set. Some of them are not rejected
* (e.g. we have some FPs!) but most are rejected.
*/
@Test
public void testFalsePositivesDream3() {
M2Test(DREAM3_TUMOR_BAM, DREAM3_NORMAL_BAM, DREAM3_FP_INTERVALS_FILE, "", "11357aa543e7c6b2725cd330adba23a0");
}
/*
* Test that contamination downsampling reduces tumor LOD, rejects more variants
*/
@Test
public void testContaminationCorrection() {
M2Test(CCLE_MICRO_TUMOR_BAM, CCLE_MICRO_NORMAL_BAM, CCLE_MICRO_INTERVALS_FILE, "-contamination 0.1", "d7947ddf0240fe06a44621312831f44c");
}
}

24
protected/gatk-tools-protected/src/test/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/how_to_make_dream3_bams.txt 100644
View File

@ -0,0 +1,24 @@
# intervals file (m2_dream3.intervals) was created by merging the following data sets:
# 1. all M2 calls (including failures) from chr21 on the full DREAM 3 challenge data set, after being called
# realigned by BWA-MEM. This is to capture FP, TN and some TP
# 2. all TP events from the DREAM 3 truth data for chr21
#
# Then the following command was run to produce the subsetted BAMs
cat m2_dream3.tp.intervals m2_dream3.fp.intervals > m2_dream3.intervals
export GATK_JAR=/humgen/gsa-hpprojects/GATK/bin/current/GenomeAnalysisTK.jar
export REF=/seq/references/Homo_sapiens_assembly19/v1/Homo_sapiens_assembly19.fasta
java -jar $GATK_JAR -T PrintReads -R $REF -ip 250 -L m2_dream3.intervals \
-I /cga/tcga-gsc/benchmark/data/realignments/synthetic.challenge.set3.tumor/IS3.snv.indel.sv.bam \
-o dream3.integrationtest.tumor.bam
java -jar $GATK_JAR -T PrintReads -R $REF -ip 250 -L m2_dream3.intervals \
-I /cga/tcga-gsc/benchmark/data/realignments/synthetic.challenge.set3.normal/G15512.prenormal.sorted.bam \
-o dream3.integrationtest.normal.bam
mv *.bam /humgen/gsa-hpprojects/GATK/data/Validation_Data/cancer
mv *.bai /humgen/gsa-hpprojects/GATK/data/Validation_Data/cancer

28
public/gatk-tools-public/src/main/java/org/broadinstitute/gatk/tools/walkers/annotator/interfaces/StandardSomaticAnnotation.java 100644
View File

@ -0,0 +1,28 @@
/*
* Copyright 2012-2015 Broad Institute, Inc.
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following
* conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR
* THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package org.broadinstitute.gatk.tools.walkers.annotator.interfaces;
public interface StandardSomaticAnnotation extends AnnotationType {}