Moved gsalib and queueJobReport.R to embeddable namespaced locations.
Updated packager dependencies/dir to add an @includes which filters the embedded fileset.
RScriptExecutor can now JIT compiles the gsalib.
RScriptExecutor uses ProcessController and sends the Rscript output to java's stdout when run under -l DEBUG.
Refactored ProcessController and IOUtils from Queue to Sting Utils.
Added more unit tests to ProcessController along with a utility class to hard stop OutputStreams at a specified byte count.
Replaced uses of some IOUtils with Apache Commons IO.
ShellJobRunner refactored to use direct ProcessController and now kills jobs on shutdown.
Better QGraph responsiveness on shutdown by using Object.wait() instead of Thread.sleep().
The test was successful -- the packaging error was detected and
prevented from propagating out into the binary/source releases.
This reverts commit 8f2a1462d94ce50fb7c1b1d0c40142b109e9c38e.
This is to test the ability of the new release system
to detect the kinds of packaging errors we couldn't
detect before (in this case, missing codecs). If all goes
as expected, "GSA-Stable: Release" will fail, preventing
the binary release and github from getting updated, while
"GSA-Stable: All Tests" and all of the other old test plans
will pass.
Will revert this in a bit -- if the system works as it should,
our users will never see it until after it's been reverted.
-Added targets to run unit and integration tests on the fully-packaged GATK jar,
and pipeline tests on the fully-packaged Queue jar. Once enabled in Bamboo,
these will provide greatly-enhanced protection against breakage in the binary
release.
-Unconditionally include all of the subset of org.broadinstitute.sting
included in the intermediate jars GenomeAnalysisTK.jar, StingUtils.jar,
etc. in the final, fully-packaged jar. This:
* is necessary to get tests to run on the fully-packaged jar
* decreases the chances of a class that is a runtime-only
dependency getting left out of the binary release
* only slightly increases the size of the binary release
(before: 9352465 bytes, after: 10985482 bytes)
classloading of bcel*.jar/ant-apache-bcel*.jar. Switching instead to manually
specifying a minimal set of packages/classes to include in the vcf.jar via
build.xml, and adding a unit test which creates a limited classloader
only aware of vcf.jar and tribble.jar and tries to use it to load the core
classes in the vcf jar.
Hopefully third time's the charm.
-- the underlying data structure is still present, but until I decide what to do for the extensible system I've completely disabled the subsystem
-- Added code to merge Samples, so that a mostly full record can be merged with a consistent empty record. If the two records are inconsistent, an error is thrown
-- addSample() in Sample.class now invokes mergeSample() when appropriate
-- Validation types are now only STRICT or SILENT
-- Validation code implemented in SampleDBBuilder
-- Extensive unit tests for SampleDBBuilder
To run the same set of tests as the bamboo "All Tests" plan (unit tests,
integration tests, and pipeline tests):
ant alltests
To do the same as above on only the public portion of the codebase:
ant alltests.public
Public-only tests will allow us to check for runtime public -> private
dependencies when bamboo updates the github repository (currently, we
only check for *compile-time* public -> private dependencies).
To compile/run only public tests, append ".public" to the name of an existing
test target:
ant test.public
ant integrationtest.public
ant performancetest.public
ant pipelinetest.public
ant pipelinetestrun.public
To include documentation for hidden features in the generated GATKDocs,
run with -Dgatkdocs.include.hidden=true
I will enable this flag when bamboo generates GATKDocs for unstable.
Contracts remain disabled for non-test build targets. To enable for
non-test targets, run with -Duse.contracts=true. To disable for test
targets, run with -Duse.contracts=false.
Usable targets are now:
ant javadoc (public-only)
ant javadoc.private (public + private)
ant scaladoc (public-only)
ant scaladoc.private (public + private)
As documented in the comments, you need to set the ANT_OPTS environment
variable to -Xmx1G before using the scaladoc targets.
Will modify bamboo to auto-generate these and post them to the web after
successful builds.
Required a significant refactoring of the GATKDoclet, which now has a unified place where the ClassDoc, class, annotation, and handler are all stored together.
Index expanded to use summary() annotation field
UserExceptions, ReadFilters, GATK engine all use the system to generate docs
Doclet expanded to handle lots of new cases
Now uses GATKDoc class to organize documentation for arguments.
Arguments now listed by feature (required, optional, hidden, etc) and link to detailed information about the argument in the html
Lots of code moving between Class and ClassDoc objects. Should be refactored into a single static utility class.
This is very useful and easy. After running any test (for example ant integrationtest) and seeing failures, you can rerun ONLY THE TESTS THAT FAILED by using one of the following commands:
ant failed-integration
ant failed-pipeline
ant failed-test
ant failed-performance
obviously matching whatever tests you were running and got failures on.
This should run only the failed tests, and you can keep using this command until you have fixed everything.
(Thanks to David Roazen for major help with ANT)
The NullPointerException we were seeing when trying to run with contracts enabled was being caused
by an outdated version of the asm library.
To run tests without contracts and disable their compilation, pass in "-Duse.contracts=false" to ant.
Also did some minor unrelated cleanup in build.xml
describe" as an absolute version number (if the repository has at least one
tag), using the raw SHA-1 hash value as a fallback version number in the case
where there are no tags.
Temporarily disabled contracts in integrationtests until we can find the cause
of the new error that's cropping up for Ryan.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5967 348d0f76-0448-11de-a6fe-93d51630548a
Current tribble-129M.jar in SVN does not work with current version of GATK code.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5897 348d0f76-0448-11de-a6fe-93d51630548a
only download that test configuration when running unit/integration tests.
This means that the build will (hopefully) never break because it can't
fetch a file that isn't required for the GATK to run.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5775 348d0f76-0448-11de-a6fe-93d51630548a
Also added the old model of indel calling to the FCP.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5749 348d0f76-0448-11de-a6fe-93d51630548a
Using the VCFWriterATD isCompressed to check if the VCF index will be auto generated.
Tracking BAM and Tribble indexes as @Inputs and @Outputs in generated QFunctions.
Updates to the BamGatherFunction to disable the index during merge when disable_bam_indexing = true.
Made a shortcut for live-running pipelinetest, pipelinetestrun.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5606 348d0f76-0448-11de-a6fe-93d51630548a
extracted from source files that have been modified since the help resource
file was last generated. This significantly speeds up builds where only a few
source files have been modified, at the expense of making clean builds take
slightly longer. Here's some performance data gathered by testing the old and
new versions of extracthelp in isolation and averaging across 10 runs:
old extracthelp, 1 modified source file: 20.1 seconds
new extracthelp, 1 modified source file: 7.2 seconds <-- woohoo! :)
old extracthelp, clean build: 17.8 seconds
new extracthelp, clean build: 20.5 seconds
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5590 348d0f76-0448-11de-a6fe-93d51630548a
Custom gatherer prints out the class name in the logs.
Try to retrieve mail domain from /etc/mailname before falling back to the hostname.
Building oneoff jars during ant oneoffs.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5540 348d0f76-0448-11de-a6fe-93d51630548a
Added a property to allow generating Queue extensions around an external version of the GATK.
Updated argument order in -help.
Restored the ability to use QScript trait imported annotations in constructors.
Removing line feeds from email password files.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5416 348d0f76-0448-11de-a6fe-93d51630548a
Added an empty wrapper for a GridEngine job runner which is only activated when SGE_ROOT is detected.
Refactored a bit more common code into CommandLineJobRunner / JobRunner / FunctionEdge.
Status for analyisNames now includes the number of functions in state.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5399 348d0f76-0448-11de-a6fe-93d51630548a
directory to live anywhere on the filesystem.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5259 348d0f76-0448-11de-a6fe-93d51630548a
functionality for BAM index visualization.
git-svn-id: file:///humgen/gsa-scr1/gsa-engineering/svn_contents/trunk@5222 348d0f76-0448-11de-a6fe-93d51630548a
Khalid Shakir
David Roazen
Matt Hanna
Mark DePristo
Kiran V Garimella
Mauricio Carneiro
ebanks
depristo
hanna
kshakir
droazen
kcibul