41c8552d0a
Added implements HasGenomeLocation to all revelant classes. It's not possible to write generic code for working with objects that support the getLocation() function in HasGenomeLocation. Please, if you have an object that has a location, implement this interface and start using / writing generic functions to sort, compare, etc. these objects.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5031 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-20 12:54:03 +00:00
cacdac3914
Major refactoring of shards. No longer uses interfaces but is now an actual object hierarchy with most of the important and common functionality pushed up to base classes. Eliminated a lot of duplicated code, and the shards are much more understandable now. Also now require a GenomeLocParser to work with their own GenomeLocs.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5030 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-20 12:36:56 +00:00
4d611e53e7
Passing the ADPR R script to FCPTest.
...
Changed the FCP.q to use an InProcessFunction work around the -runDir issue GSA-420.
Tested the FCPTest using the following dotkits and "ant clean pipelinetest -Dpipeline.run=run":
- R-2.11
- Oracle-full-client
- .cx-oracle-5.0.2-python-2.6.5-oracle-full-client-11.1
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5029 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-20 06:08:45 +00:00
7087c2f422
Very simple integration tests for basic VCF streaming functionality.
...
Rather than try to fork the integration test process to get a pipe source
and sink, creates a new named pipe by Runtime.exec()ing the 'mkfifo' shell
command. We'll see whether this proves to be a reliable method for testing
streaming.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5028 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-20 04:38:54 +00:00
acc2f1c9fe
Updated FCPTest to use the new path to fullCallingPipeline.q changed in r5017.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5027 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-19 21:43:43 +00:00
22582960be
Adding the backdrop to the current version of the tearsheet so it is always available
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5026 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-19 20:56:11 +00:00
2824e8224c
removes unused titv argument
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5025 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-19 20:49:12 +00:00
50fcebb0c4
Incorporates tearsheet and plot production with database access into standard pipeline. Note that the following dotkit packages must be run before the adpr will be correctly generated:
...
R-2.10,
Oracle-full-client,
cx-oracle-5.0.2-python-2.6.5-oracle-full-client-11.1
This also removes the unused titv argument
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5024 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-19 20:48:42 +00:00
2b895ffb7f
Updated the HG19 reference from v0 to v1 after the v0 was zeroed out.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5023 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-19 20:30:25 +00:00
af60666f5d
An example template for the shell script used to run the full calling pipeline on the broad system
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5022 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-19 20:28:59 +00:00
dfcd45181a
Some minor tweaks to the tearsheet generator that incorperate the gsalib more universally and create a more accurate output
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5021 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-19 20:27:02 +00:00
c901fb6d70
Now populating the refseq and dbsnp in awk instead of retrieving from firehose.
...
Added refseq table to the pipeline object.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5020 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-19 18:19:10 +00:00
55eb0387ac
Another relevant qscript. I use this one to do thousands of variant recalibration jobs to search for optimal parameters.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5019 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-19 18:17:32 +00:00
82f81a83c2
Mauricio shall not escape my vengeance!
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5018 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-19 15:24:37 +00:00
a463dbcda1
Refactoring the qscript directory; oneoffs, playground, and core
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5017 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-19 15:23:40 +00:00
24bc843ae8
Dynamically change the log message update rate so that short jobs receive frequent updates while longer running jobs receive fewer updates
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5016 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-19 15:09:11 +00:00
7db9601c9d
Checking in the 1000G phase1 cleaning and calling scripts for posterity's sake, but also to show everyone what the current best practices for VQSR training looks like.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5015 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-19 14:32:52 +00:00
bd2af33a16
misc clean up in VQSR
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5014 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-18 21:04:31 +00:00
457c59e737
Use the sites-only HapMap files in the Methods development pipeline
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5013 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-18 20:50:09 +00:00
00453919d2
VQSR now only uses the valid polymorphic sites for training and truth sensitivity calculations. Any number of tracks whose ROD binding begins with the name truth can be used as truth sensitivity tracks.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5012 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-18 20:48:19 +00:00
4bec93e3e4
Permit retrieval of read names for debugging purposes
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5011 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-18 16:09:34 +00:00
f8ba76d87c
Incremental commit for distributed computation. Appears to work but has potential deadlock situation not yet debugged. Do not use yet.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5010 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-17 21:23:09 +00:00
2f4a436719
Throw an exception if no eval rods are specified. If one or more samples are specified, subset the 'all' VariantContext to just the specified samples. This is useful when you want to see what effect dropping certain samples will have on the metrics and you don't want to go through SelectVariants first.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5009 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-17 06:46:10 +00:00
366c3a0b8f
Incompatible chain files are user exceptions
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5008 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-16 05:26:47 +00:00
cc4e193670
Removal of empty directories
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5007 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-15 22:43:51 +00:00
a88708ebfa
Moving GLF code to archive
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5006 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-15 22:42:42 +00:00
797e07b0c3
Archived home of GLF code
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5005 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-15 22:37:20 +00:00
35a4f1e366
.Added VariantEval as an optional step in the pipeline.
...
.Lifted to HapMap 3.3
.Lifted to dbSNP 132 where possible.
.Added the CEU-Trio WEx(hg19) dataset
.Added some options to the pipeline
You can now use :
-dataset WEX
-dataset HiSeq
...
to choose which datasets to run through the pipeline.
You can now without BAQ and indel mask:
-noBAQ
-noMASK
Choose not to run the gold standard comparison analysis:
-skipGoldStandard
Activate the VariantEval walker analysis on the Recalibrated vcf:
-eval
The default behavior is to run exactly like it used to, so this version shouldn't change the way you used to use the pipeline.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5004 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-14 21:55:02 +00:00
579e0d59fa
Rewrote warning message to discourage use of unsafe mode.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5003 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-14 21:32:53 +00:00
2355a55067
Added a QFunction.jobLimitSeconds for experimentation, currently only used as the equivalent of bsub -W.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5002 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-14 19:45:03 +00:00
af31d02a2d
Fix concurrency issue that periodically kills VariantEvalIntegrationTest --
...
a member field of RMDTrackBuilder was getting rebuilt every time it was
called, creating concurrency issues.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5001 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-14 18:52:21 +00:00
73acfa654a
Fixed double-counting bug. Fixed issue where evaluation module with an update2() method wasn't getting called if the comp track was null. Added a column to the output report indicating the table name for easy greppability. Fixed an issue where, if sample-level stratification was not required, the sample-level VCs would be generated anyway.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5000 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-14 14:06:43 +00:00
af1bce3492
Longer wait time for threading test (5 min now) and an assertion to ensure that all jobs finished. Should probably just remove the longer running test so this becoming a non-issue
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4999 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-14 13:09:31 +00:00
afbea9ce59
SharedMemory and SharedFile implementations of GenomeLocProcessingTracker, along with serious unit tests that both pass. Slightly inefficient implementation but sufficient for further testing.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4998 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-14 03:14:24 +00:00
bfbf75fe3e
Fix error in command-line validation: don't ever allow intervaled access to unindexed read stream, no
...
matter what type of traversal it is.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4997 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-14 02:49:04 +00:00
00310c05bb
Fix corner condition that happens when there are indels right at the end of a contig and there's not enough reference to build a haplotype.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4996 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 21:08:22 +00:00
c0031b05ff
Stamp out lazy loading in the PluginManager. This is an attempt to stamp
...
out the non-deterministic VariantEvalIntegrationTest errors we've been seeing.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4995 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 20:58:28 +00:00
2163420942
Updated to reflect QD changes in r4984
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4994 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 20:10:21 +00:00
02dc0f97d1
Remove testWalkerUnitTest; it doesn't actually do anything and just adds
...
extra cruft to the output.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4993 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 19:02:00 +00:00
c4f9b262e5
removing the tech dev pipeline script from the repository to keep the methods development pipeline as the reference script.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4992 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 18:15:55 +00:00
b107c97c1a
Cannot have "=" sign in reason, so change to ":"
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4991 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 17:23:44 +00:00
b4a2112a0d
Added the "previous locus" to interesting sites VCF (locus with respect to which the site is phased)
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4990 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 17:19:20 +00:00
e8f0ae4b09
Renamed and documented some phasing-specific classes to make their purpose clearer to someone browing through the code
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4989 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 16:17:36 +00:00
6b5474a00a
This updates the script to produce a more tearsheet-like output for sample set statistics. Formatting will be updated for aesthetic improvements. There are also several database options that currently pull out misleading information because of changes in sequencing methodology that will be updated to show correct information. Eventually, plot formatting will be updated as well and additional informative plots will be added.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4988 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 15:59:06 +00:00
ffae7bf537
Moved phasing-specific utilities to phasing sub-directory
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4987 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 15:38:20 +00:00
91824f478e
FASTQ directory is gone
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4986 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 15:16:06 +00:00
e3956148ac
removing unused fastqtobam
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4985 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 14:29:32 +00:00
ce3d226183
Reverting back to the old definition of QD because it works better with large numbers of samples. The new QD is relegated to a new annotation: sumGLbyD. Tweaks to the new HaplotypeScore based on evaluation with better QD calculation. The default qual threshold in GenerateVariantClusters is updated to be in line with the variant quality scores coming from the exact model.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4984 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 14:12:30 +00:00
e0092bb160
Experimental feature: change the rate at which log messages appear on-the-fly
...
and enable/disable performance logs from outside the JVM process. Making this
available for the moment; we'll see whether it ends up being useful.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4983 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 04:20:53 +00:00
9e93091e9a
-baqGOP now takes phred scaled scores instead of probabilities in the command line.
...
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@4982 348d0f76-0448-11de-a6fe-93d51630548a
2011-01-13 00:06:38 +00:00
depristo
kshakir
hanna
corin
rpoplin
chartl
fromer
kiran
ebanks
carneiro
delangel