Known Issues
List of known bugs and limitations


These are all the issues tagged "bug" in the GATK4 development repository. If you have reported a bug and cannot find it in this list, let us know in the thread where you reported it. Note that we are no longer listing or processing issues in older versions of GATK (3.x and prior) unless there is substantial negative impact.





Created 2017-07-23 | Updated 2017-07-23 | bug


due to recursive implementation of Legendre abscissas in Apache Commons.

@vdauwera @takutosato this is very simple; it just caps the number of subdivisions of the integral to avoid the recursive stack overflow. I tested it on absurdly high coverage (100,000) and reproduced the error with the old code. Whichever one of you gets to this first should review.

While this isn't the most beautiful thing in the world, it will work reasonably while new integration code is pending.



Created 2017-07-21 | Updated 2017-07-21 | bug


We use Gauss-Legendre integration in the strand bias model. The number of subdivisions increases with the read count and for very deep coverage this can cause a stack overflow because, unfortunately, Apache Commons has a very questionable recursive implementation.

The short-term fix is to cap the number of subdivisions. The long-term fix is to write some sort of simple adaptive 1D and 2D quadrature method. This ticket is for the short-term fix.



Created 2017-07-20 | Updated 2017-07-21 | bug NIO PRIORITY_HIGH


I got the following error and log file from BaseRecalibrator:

Using GATK jar /usr/gitc/gatk4/gatk-package-4.beta.1-local.jar
Running:
    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=1 -Dsnappy.disable=true -XX:GCTimeLimit=50 -XX:GCHeapFreeLimit=10 -XX:+PrintFlagsFinal -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+PrintGCDetails -Xloggc:gc_log.log -Xms4000m -jar /usr/gitc/gatk4/gatk-package-4.beta.1-local.jar BaseRecalibrator -R /cromwell_root/broad-references/hg38/v0/Homo_sapiens_assembly38.fasta -I gs://broad-gotc-dev-cromwell-execution/PairedEndSingleSampleWorkflow/66442def-ad3f-4c6c-960e-17578f6b382c/call-SortAndFixSampleBam/CHMI_CHMI3_WGS2.aligned.duplicate_marked.sorted.bam --useOriginalQualities -O CHMI_CHMI3_WGS2.recal_data.csv -knownSites gs://broad-references/hg38/v0/Homo_sapiens_assembly38.dbsnp138.vcf -knownSites /cromwell_root/broad-references/hg38/v0/Mills_and_1000G_gold_standard.indels.hg38.vcf.gz -knownSites /cromwell_root/broad-references/hg38/v0/Homo_sapiens_assembly38.known_indels.vcf.gz -L chr11:1+
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/cromwell_root/tmp.WXYB31
[July 20, 2017 2:18:26 PM UTC] BaseRecalibrator  --useOriginalQualities true --knownSites gs://broad-references/hg38/v0/Homo_sapiens_assembly38.dbsnp138.vcf --knownSites /cromwell_root/broad-references/hg38/v0/Mills_and_1000G_gold_standard.indels.hg38.vcf.gz --knownSites /cromwell_root/broad-references/hg38/v0/Homo_sapiens_assembly38.known_indels.vcf.gz --output CHMI_CHMI3_WGS2.recal_data.csv --intervals chr11:1+ --input gs://broad-gotc-dev-cromwell-execution/PairedEndSingleSampleWorkflow/66442def-ad3f-4c6c-960e-17578f6b382c/call-SortAndFixSampleBam/CHMI_CHMI3_WGS2.aligned.duplicate_marked.sorted.bam --reference /cromwell_root/broad-references/hg38/v0/Homo_sapiens_assembly38.fasta  --mismatches_context_size 2 --indels_context_size 3 --maximum_cycle_value 500 --mismatches_default_quality -1 --insertions_default_quality 45 --deletions_default_quality 45 --low_quality_tail 2 --quantizing_levels 16 --bqsrBAQGapOpenPenalty 40.0 --preserve_qscores_less_than 6 --enableBAQ false --computeIndelBQSRTables false --defaultBaseQualities -1 --interval_set_rule UNION --interval_padding 0 --interval_exclusion_padding 0 --readValidationStringency SILENT --secondsBetweenProgressUpdates 10.0 --disableSequenceDictionaryValidation false --createOutputBamIndex true --createOutputBamMD5 false --createOutputVariantIndex true --createOutputVariantMD5 false --lenient false --addOutputSAMProgramRecord true --addOutputVCFCommandLine true --cloudPrefetchBuffer 40 --cloudIndexPrefetchBuffer -1 --disableBamIndexCaching false --help false --version false --showHidden false --verbosity INFO --QUIET false --use_jdk_deflater false --use_jdk_inflater false --disableToolDefaultReadFilters false
[July 20, 2017 2:18:26 PM UTC] Executing as root@ab23aa1f82e3 on Linux 4.9.0-0.bpo.3-amd64 amd64; OpenJDK 64-Bit Server VM 1.8.0_111-8u111-b14-2~bpo8+1-b14; Version: 4.beta.1
[July 20, 2017 2:22:06 PM UTC] org.broadinstitute.hellbender.tools.walkers.bqsr.BaseRecalibrator done. Elapsed time: 3.65 minutes.
Runtime.totalMemory()=4188012544
htsjdk.samtools.SAMFormatException: Did not inflate expected amount
	at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:147)
	at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:96)
	at htsjdk.samtools.util.BlockCompressedInputStream.inflateBlock(BlockCompressedInputStream.java:537)
	at htsjdk.samtools.util.BlockCompressedInputStream.processNextBlock(BlockCompressedInputStream.java:519)
	at htsjdk.samtools.util.BlockCompressedInputStream.nextBlock(BlockCompressedInputStream.java:455)
	at htsjdk.samtools.util.BlockCompressedInputStream.readBlock(BlockCompressedInputStream.java:445)
	at htsjdk.samtools.util.BlockCompressedInputStream.available(BlockCompressedInputStream.java:194)
	at htsjdk.samtools.util.BlockCompressedInputStream.read(BlockCompressedInputStream.java:326)
	at java.io.DataInputStream.read(DataInputStream.java:149)
	at htsjdk.samtools.util.BinaryCodec.readBytesOrFewer(BinaryCodec.java:404)
	at htsjdk.samtools.util.BinaryCodec.readBytes(BinaryCodec.java:380)
	at htsjdk.samtools.util.BinaryCodec.readBytes(BinaryCodec.java:366)
	at htsjdk.samtools.BAMRecordCodec.decode(BAMRecordCodec.java:209)
	at htsjdk.samtools.BAMFileReader$BAMFileIterator.getNextRecord(BAMFileReader.java:829)
	at htsjdk.samtools.BAMFileReader$BAMFileIndexIterator.getNextRecord(BAMFileReader.java:981)
	at htsjdk.samtools.BAMFileReader$BAMFileIterator.advance(BAMFileReader.java:803)
	at htsjdk.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:797)
	at htsjdk.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:765)
	at htsjdk.samtools.BAMFileReader$BAMQueryFilteringIterator.advance(BAMFileReader.java:1034)
	at htsjdk.samtools.BAMFileReader$BAMQueryFilteringIterator.next(BAMFileReader.java:1024)
	at htsjdk.samtools.BAMFileReader$BAMQueryFilteringIterator.next(BAMFileReader.java:988)
	at htsjdk.samtools.SamReader$AssertingIterator.next(SamReader.java:576)
	at htsjdk.samtools.SamReader$AssertingIterator.next(SamReader.java:548)
	at org.broadinstitute.hellbender.utils.iterators.SamReaderQueryingIterator.loadNextRecord(SamReaderQueryingIterator.java:114)
	at org.broadinstitute.hellbender.utils.iterators.SamReaderQueryingIterator.next(SamReaderQueryingIterator.java:151)
	at org.broadinstitute.hellbender.utils.iterators.SamReaderQueryingIterator.next(SamReaderQueryingIterator.java:29)
	at org.broadinstitute.hellbender.utils.iterators.SAMRecordToReadIterator.next(SAMRecordToReadIterator.java:29)
	at org.broadinstitute.hellbender.utils.iterators.SAMRecordToReadIterator.next(SAMRecordToReadIterator.java:15)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
	at org.broadinstitute.hellbender.engine.ReadWalker.traverse(ReadWalker.java:94)
	at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:838)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:115)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:170)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:189)
	at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:131)
	at org.broadinstitute.hellbender.Main.mainEntry(Main.java:152)
	at org.broadinstitute.hellbender.Main.main(Main.java:230)

I ran the same command again from my computer (not in the cloud) still using the NIO paths and it ran successfully. I've also seen it run successfully when running the same pipeline in the cloud. The only thing I think I've changed is the disk size I'm asking for. I'm in the process of validating the input bam right now.



Created 2017-07-19 | Updated 2017-07-19 | bug Mutect


For example:

lichtens@OncobuntuMk6:~$ more /home/lichtens/work/ob_filtered2.tmp.vcf.summary 
Sample	Artifact_Mode	Artifact_Mode_Complement	OBQ	Num_Artifact_Mode	Num_Artifact_Mode_Filtered	Num_Not_Artifact_Mode	Num_NonRef_Passing_Variants
SAMPLE1Tumor	C>T	G>A	42.68076887123618	82339	0	205577	287916
SAMPLE1Tumor	G>T	C>A	32.01295410609514	36517	0	251399	287916
lichtens@OncobuntuMk6:~$ egrep orientation_bias /home/lichtens/work/ob_filtered2.tmp.vcf | wc -l
26279

Not high priority.

Just stems from the VariantContext now being filtered as well as the genotype. Fix can be placed in OrientationBiasUtils.getGenotypeStream



Created 2017-07-18 | Updated 2017-07-18 | bug Mutect


Using VCFTools to validate:

vcf-validator SAMPLE7T-vs-SAMPLE7N-filtered.vcf

I get a massive amount of messages:

.....snip.......
column SAMPLE7N at 19:49136721 .. Could not validate the float [NaN],FORMAT tag [MPOS] expected different number of values (expected 1, found 2),FORMAT tag [MFRL] expected different number of values (expected 1, found 2),FORMAT tag [MMQ] expected different number of values (expected 1, found 2),FORMAT tag [MCL] expected different number of values (expected 1, found 2),FORMAT tag [MBQ] expected different number of values (expected 1, found 2)
.....snip.......
column SAMPLE7T at 19:45901415 .. FORMAT tag [MBQ] expected different number of values (expected 1, found 2),FORMAT tag [MMQ] expected different number of values (expected 1, found 2),FORMAT tag [MCL] expected different number of values (expected 1, found 2),FORMAT tag [MFRL] expected different number of values (expected 1, found 2),FORMAT tag [MPOS] expected different number of values (expected 1, found 2)
.....snip.......

Sure enough, the header does not match the values for those fields (in the header number="A"), so the validation errors are correct. Not sure what is the deal with FOXOG, but that may not be a big deal.



Created 2017-07-11 | Updated 2017-07-12 | bug


I tried to run VariantRecalibrator using the args echoed from an integration test and found that the resource files weren't listed properly. The command in the test was " --resource known,known=true,prior=10.0:" + getLargeVQSRTestDataDir() + "dbsnp_132_b37.leftAligned.20.1M-10M.vcf" and what came out of the engine was --resource known:/Users/gauthier/workspaces/gatk/src/test/resources/large/VQSR/dbsnp_132_b37.leftAligned.20.1M-10M.vcf, so it lost the known=true and the prior which makes the command line unrunnable. Probably affects #2269 too.

This behavior can be replicated by running any of the VariantRecalibration integration tests and checking the console output.



Created 2017-07-02 | Updated 2017-07-07 | bug NativeLibraries


We are currently plagued with cryptic intermittent failures coming from the BWA and FML bindings in Travis CI. These usually manifest as a simple "exited with code 137" (ie., killed by signal 9) error, but sometimes we get an explicit segfault or out-of-memory error. Examples:

�[31mFAILURE: �[39m�[31mBuild failed with an exception.�[39m
* What went wrong:
Execution failed for task ':test'.
�[33m> �[39mProcess 'Gradle Test Executor 1' finished with non-zero exit value 137
:test[M::bwa_idx_load_from_disk] read 0 ALT contigs
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000715180000, 719847424, 0) failed; error='Cannot allocate memory' (errno=12)

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 719847424 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /home/travis/build/broadinstitute/gatk/hs_err_pid11513.log
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f27ebfe7d9a, pid=11455, tid=0x00007f27e87e5700
#
# JRE version: OpenJDK Runtime Environment (8.0_111-b14) (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
# Java VM: OpenJDK 64-Bit Server VM (25.111-b14 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libfml.6198146539708364717.jnilib+0xed9a]  rld_itr_init+0x4a
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007fd2680a350c, pid=11685, tid=0x00007fd2b02bf700
#
# JRE version: OpenJDK Runtime Environment (8.0_111-b14) (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
# Java VM: OpenJDK 64-Bit Server VM (25.111-b14 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libbwa.5694772191018335324.jnilib+0x850c]  bwa_mem2idx+0xcc

The underlying issue in these cases is likely either "out of memory" or, perhaps in the case of the seg faults, "file not found" or "malformed file", but we could greatly improve our ability to interpret Travis failures if we were more careful about checking return values from system calls. Eg., in the function below from the BWA bindings we could check the return values of the mmap() and calloc() calls, and die with an appropriate error message if they fail:

bwaidx_t* jnibwa_openIndex( int fd ) {
    struct stat statBuf;
    if ( fstat(fd, &statBuf) == -1 ) return 0;
    uint8_t* mem = mmap(0, statBuf.st_size, PROT_READ, MAP_SHARED, fd, 0);
    close(fd);
    bwaidx_t* pIdx = calloc(1, sizeof(bwaidx_t));
    bwa_mem2idx(statBuf.st_size, mem, pIdx);
    pIdx->is_shm = 1;
    mem_fmt_fnc = &fmt_BAMish;
    bwa_verbose = 0;
    return pIdx;
}


Created 2017-06-28 | Updated 2017-07-18 | bug BuildSystem Docker


Currently the script fails when configured to run tests, since it doesn't have access to the large files in the docker image. These need to be downloaded and mounted for the tests to pass, as we do in the docker tests on travis.



Created 2017-06-27 | Updated 2017-06-28 | bug Engine HaplotypeCaller Mutect


@sooheelee has some serious concerns about ReadClipper.hardClipAdaptorSequence(), which is called in Mutect2 and HaplotypeCaller via AssemblyBasedCallerUtils.finalizeRegion(). She thinks that the method being used to find the adaptor boundary for clipping purposes is completely bogus!

This is some pretty old code that was also in the GATK3 versions of these tools, so if it's crazy, then we can at least take "comfort" in the fact that it's been like this for a very long time...



Created 2017-06-26 | Updated 2017-07-05 | bug Spark


Line 154(System.exit(0)) in Main.java will lead a unormal exitcode(143) for a container in yarn cluster mode. Remove this will get the correct output.
final Object result = runCommandLineProgram(program, args); handleResult(result); //System.exit(0);



Created 2017-06-15 | Updated 2017-06-15 | bug


Somehow Kris managed to generate a VCF with an index that doesn't have a properly sorted sequence dictionary: gs://broad-dsde-methods/kcibul/bug_for_louis I think it was with GATK4 SelectVariants (with a version prior to the commandline being put in the header), but I'm not 100% sure.

Generating the index on the fly with GATK3 works fine. I'm not sure if the original tabix index from the GotC pipeline is okay: gs://broad-jg-dev-storage/temp/09C99383.91c5a812-70c5-4526-a3a2-3e99b9cf08fb.g.vcf.gz.tbi

I found this because GATK3 complained about the contig order.



Created 2017-06-08 | Updated 2017-06-24 | bug


I just try to run PrintReadsSpark on cloudera cluster and meet this error.

Command:

$ ./gatk-launch PrintReadsSpark -I NA12878.chr17_69k_70k.dictFix.bam -O /user/yaron/output.bam -- --sparkRunner SPARK --sparkMaster yarn  --num-executors 5 --executor-cores 2 --executor-memory 1g

Results:

Using GATK jar /home/yaron/gatk/build/libs/gatk-spark.jar
Running:
    spark-submit --master yarn --conf spark.kryoserializer.buffer.max=512m --conf spark.driver.maxResultSize=0 --conf spark.driver.userClassPathFirst=true --conf spark.io.compression.codec=lzf --conf spark.yarn.executor.memoryOverhead=600 --conf spark.driver.extraJavaOptions=-DGATK_STACKTRACE_ON_USER_EXCEPTION=true -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=false -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=1 -Dsnappy.disable=true  --conf spark.executor.extraJavaOptions=-DGATK_STACKTRACE_ON_USER_EXCEPTION=true -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=false -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=1 -Dsnappy.disable=true  --num-executors 5 --executor-cores 2 --executor-memory 1g /home/yaron/gatk/build/libs/gatk-spark.jar PrintReadsSpark -I NA12878.chr17_69k_70k.dictFix.bam -O /user/yaron/output.bam --sparkMaster yarn
09:14:13.551 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/yaron/gatk/build/libs/gatk-spark.jar!/com/intel/gkl/native/libgkl_compression.so
[June 8, 2017 9:14:13 AM CST] org.broadinstitute.hellbender.tools.spark.pipelines.PrintReadsSpark  --output /user/yaron/output.bam --input NA12878.chr17_69k_70k.dictFix.bam --sparkMaster yarn  --readValidationStringency SILENT --interval_set_rule UNION --interval_padding 0 --interval_exclusion_padding 0 --bamPartitionSize 0 --disableSequenceDictionaryValidation false --shardedOutput false --numReducers 0 --help false --version false --showHidden false --verbosity INFO --QUIET false --use_jdk_deflater false --use_jdk_inflater false --disableToolDefaultReadFilters false
[June 8, 2017 9:14:13 AM CST] Executing as yaron@dn1 on Linux 4.4.0-31-generic amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_121-b13; Version: 4.alpha.2-281-g752d020-SNAPSHOT
09:14:13.567 INFO  PrintReadsSpark - HTSJDK Defaults.COMPRESSION_LEVEL : 1
09:14:13.567 INFO  PrintReadsSpark - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
09:14:13.567 INFO  PrintReadsSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : false
09:14:13.567 INFO  PrintReadsSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
09:14:13.567 INFO  PrintReadsSpark - Deflater: IntelDeflater
09:14:13.567 INFO  PrintReadsSpark - Inflater: IntelInflater
09:14:13.567 INFO  PrintReadsSpark - Initializing engine
09:14:13.567 INFO  PrintReadsSpark - Done initializing engine
log4j:ERROR A "org.apache.log4j.ConsoleAppender" object is not assignable to a "org.apache.log4j.Appender" variable.
log4j:ERROR The class "org.apache.log4j.Appender" was loaded by
log4j:ERROR [sun.misc.Launcher$AppClassLoader@6d21714c] whereas object of type
log4j:ERROR "org.apache.log4j.ConsoleAppender" was loaded by [org.apache.spark.util.ChildFirstURLClassLoader@6ee12bac].
log4j:ERROR Could not instantiate appender named "console".
log4j:ERROR A "org.apache.log4j.ConsoleAppender" object is not assignable to a "org.apache.log4j.Appender" variable.
log4j:ERROR The class "org.apache.log4j.Appender" was loaded by
log4j:ERROR [sun.misc.Launcher$AppClassLoader@6d21714c] whereas object of type
log4j:ERROR "org.apache.log4j.ConsoleAppender" was loaded by [org.apache.spark.util.ChildFirstURLClassLoader@6ee12bac].
log4j:ERROR Could not instantiate appender named "console".
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
log4j:WARN No appenders could be found for logger (org.apache.spark.SparkContext).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
09:14:26.202 INFO  PrintReadsSpark - Shutting down engine
[June 8, 2017 9:14:26 AM CST] org.broadinstitute.hellbender.tools.spark.pipelines.PrintReadsSpark done. Elapsed time: 0.21 minutes.
Runtime.totalMemory()=494927872
***********************************************************************

A USER ERROR has occurred: Couldn't write file /user/yaron/output.bam because writing failed with exception /user/yaron/output.bam.parts/_SUCCESS: Unable to find _SUCCESS file

***********************************************************************
org.broadinstitute.hellbender.exceptions.UserException$CouldNotCreateOutputFile: Couldn't write file /user/yaron/output.bam because writing failed with exception /user/yaron/output.bam.parts/_SUCCESS: Unable to find _SUCCESS file
        at org.broadinstitute.hellbender.engine.spark.GATKSparkTool.writeReads(GATKSparkTool.java:255)
        at org.broadinstitute.hellbender.tools.spark.pipelines.PrintReadsSpark.runTool(PrintReadsSpark.java:37)
        at org.broadinstitute.hellbender.engine.spark.GATKSparkTool.runPipeline(GATKSparkTool.java:353)
        at org.broadinstitute.hellbender.engine.spark.SparkCommandLineProgram.doWork(SparkCommandLineProgram.java:38)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:116)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:171)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:190)
        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:121)
        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:142)
        at org.broadinstitute.hellbender.Main.main(Main.java:220)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:736)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.nio.file.NoSuchFileException: /user/yaron/output.bam.parts/_SUCCESS: Unable to find _SUCCESS file
        at org.seqdoop.hadoop_bam.util.SAMFileMerger.mergeParts(SAMFileMerger.java:53)
        at org.broadinstitute.hellbender.engine.spark.datasources.ReadsSparkSink.writeReadsSingle(ReadsSparkSink.java:230)
        at org.broadinstitute.hellbender.engine.spark.datasources.ReadsSparkSink.writeReads(ReadsSparkSink.java:152)
        at org.broadinstitute.hellbender.engine.spark.GATKSparkTool.writeReads(GATKSparkTool.java:250)
        ... 18 more

However, I can find that _SUCCESS file exists in output.bam.parts. Could someone tell me what may be the cause? Thanks!

$ hdfs dfs -ls output.bam.parts
Found 3 items
-rw-r--r--   3 yaron yaron          0 2017-06-08 09:14 output.bam.parts/_SUCCESS
-rw-r--r--   3 yaron yaron      62019 2017-06-08 09:14 output.bam.parts/part-r-00000.bam
-rw-r--r--   3 yaron yaron         16 2017-06-08 09:14 output.bam.parts/part-r-00000.bam.splitting-bai


Created 2017-06-06 | Updated 2017-06-07 | bug Mutect


Currently, FilterByOrientation requires replacing a required input's contents with a sed command:

sed -r "s/picard\.analysis\.artifacts\.SequencingArtifactMetrics\\\$PreAdapterDetailMetrics/org\.broadinstitute\.
 hellbender\.tools\.picard\.analysis\.artifacts\.SequencingArtifactMetrics\$PreAdapterDetailMetrics/g" \
 "metrics.pre_adapter_detail_metrics" > "gatk.pre_adapter_detail_metrics"

The required input is the detailed pre-adapter summary metrics from Picard's CollectSequencingArtifactMetrics. The sed command replaces

## METRICS CLASS	picard.analysis.artifacts.SequencingArtifactMetrics$PreAdapterDetailMetrics

with

## METRICS CLASS	org.broadinstitute.hellbender.tools.picard.analysis.artifacts.SequencingArtifactMetrics$PreAdapterDetailMetrics

If this class is not replaced, then the tool errors with:

htsjdk.samtools.SAMException: Could not locate class with name gatk.analysis.artifacts.SequencingArtifactMetrics$PreAdapterDetailMetrics
	at htsjdk.samtools.metrics.MetricsFile.read(MetricsFile.java:356)
	at org.broadinstitute.hellbender.tools.exome.FilterByOrientationBias.onTraversalStart(FilterByOrientationBias.java:102)
	at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:777)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:115)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:170)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:189)
	at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:122)
	at org.broadinstitute.hellbender.Main.mainEntry(Main.java:143)
	at org.broadinstitute.hellbender.Main.main(Main.java:221)
Caused by: java.lang.ClassNotFoundException: gatk.analysis.artifacts.SequencingArtifactMetrics$PreAdapterDetailMetrics
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at htsjdk.samtools.metrics.MetricsFile.loadClass(MetricsFile.java:471)
	at htsjdk.samtools.metrics.MetricsFile.read(MetricsFile.java:353)
	... 8 more

If it is replaced, the tool still errors but with a different error:

java.lang.IllegalArgumentException: Features added out of order: previous (TabixFeature{referenceIndex=0, start=118314029, end=118314036, featureStartFilePosition=1403632633, featureEndFilePosition=-1}) > next (TabixFeature{referenceIndex=0, start=33414233, end=33414234, featureStartFilePosition=1403632876, featureEndFilePosition=-1})
	at htsjdk.tribble.index.tabix.TabixIndexCreator.addFeature(TabixIndexCreator.java:89)
	at htsjdk.variant.variantcontext.writer.IndexingVariantContextWriter.add(IndexingVariantContextWriter.java:170)
	at htsjdk.variant.variantcontext.writer.VCFWriter.add(VCFWriter.java:219)
	at java.util.ArrayList.forEach(ArrayList.java:1249)
	at org.broadinstitute.hellbender.tools.exome.FilterByOrientationBias.onTraversalSuccess(FilterByOrientationBias.java:171)
	at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:781)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:115)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:170)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:189)
	at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:122)
	at org.broadinstitute.hellbender.Main.mainEntry(Main.java:143)
	at org.broadinstitute.hellbender.Main.main(Main.java:221)

It does not matter if I produce the pre-adapter metrics with the latest Picard jar v2.9.2. I get the same error.

I'm using a M2 callset from GATK3. Even so, I don't think I should get the above error?



Created 2017-06-05 | Updated 2017-06-26 | bug HaplotypeCaller


@TianJin297 commented on Fri May 26 2017

When I was running HaplotypeCallorSpark with one of my samples, I got an error as "Duplicate key".

The command I used is "/gatk-protected HaplotypeCallerSpark -I XX_BQSRappliedspark.bam -O XX_525.gvcf -R /curr/data/humann_g1k_v37.2bit --emitRefConfidence BP_RESOLUTION --TMP_DIR tmp"

And it runs on Amazon instance m4.2xlarge.

00:10:41.089 WARN StrandBiasBySample - Annotation will not be calculated, genotype is not called or alleleLikelihoodMap is null
00:10:41.089 WARN DepthPerSampleHC - Annotation will not be calculated, genotype is not called or alleleLikelihoodMap is null
00:10:41.089 WARN StrandBiasBySample - Annotation will not be calculated, genotype is not called or alleleLikelihoodMap is null
00:10:45.460 WARN DepthPerSampleHC - Annotation will not be calculated, genotype is not called or alleleLikelihoodMap is null
00:10:45.460 WARN StrandBiasBySample - Annotation will not be calculated, genotype is not called or alleleLikelihoodMap is null
00:11:09.609 WARN TaskMemoryManager:381 - leak 166.6 MB memory from org.apache.spark.util.collection.ExternalAppendOnlyMap@60a3c432
00:11:09.611 ERROR Executor:91 - Exception in task 15.0 in stage 1.0 (TID 519)
java.lang.IllegalStateException: Duplicate key [B@4e233a3c
at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133)
at java.util.HashMap.merge(HashMap.java:1253)
at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320)
at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.buildGapContinuationPenalties(PairHMMLikelihoodCalculat
ionEngine.java:304)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.computeReadLikelihoods(PairHMMLikelihoodCalculationEngi
ne.java:253)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.computeReadLikelihoods(PairHMMLikelihoodCalculationEngi
ne.java:187)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCallerEngine.callRegion(HaplotypeCallerEngine.java:518)
at org.broadinstitute.hellbender.tools.HaplotypeCallerSpark.lambda$regionToVariants$2(HaplotypeCallerSpark.java:192)
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267)
at java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
at java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:169)
at java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
at scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:42)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
at scala.collection.AbstractIterator.to(Iterator.scala:1336)
at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1336)
at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1336)
at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1336)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$13.apply(RDD.scala:912)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$13.apply(RDD.scala:912)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1899)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1899)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:86)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
00:11:09.632 WARN TaskSetManager:66 - Lost task 15.0 in stage 1.0 (TID 519, localhost): java.lang.IllegalStateException: Duplicate key [B@4e233a3c
at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133)
at java.util.HashMap.merge(HashMap.java:1253)
at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320)
at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.buildGapContinuationPenalties(PairHMMLikelihoodCalculat
ionEngine.java:304)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.computeReadLikelihoods(PairHMMLikelihoodCalculationEngi
ne.java:253)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.computeReadLikelihoods(PairHMMLikelihoodCalculationEngi
ne.java:187)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCallerEngine.callRegion(HaplotypeCallerEngine.java:518)
at org.broadinstitute.hellbender.tools.HaplotypeCallerSpark.lambda$regionToVariants$2(HaplotypeCallerSpark.java:192)
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267)
at java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
at java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:169)
at java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
at scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:42)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
at scala.collection.AbstractIterator.to(Iterator.scala:1336)
at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1336)
at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1336)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$13.apply(RDD.scala:912)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$13.apply(RDD.scala:912)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1899)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1899)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:86)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)

00:11:09.634 ERROR TaskSetManager:70 - Task 15 in stage 1.0 failed 1 times; aborting job
00:11:09.810 WARN TaskSetManager:66 - Lost task 33.0 in stage 1.0 (TID 528, localhost): TaskKilled (killed intentionally)
00:11:24.786 INFO HaplotypeCallerSpark - Shutting down engine
[May 26, 2017 12:11:24 AM UTC] org.broadinstitute.hellbender.tools.HaplotypeCallerSpark done. Elapsed time: 10.58 minutes.
Runtime.totalMemory()=16622026752
org.apache.spark.SparkException: Job aborted due to stage failure: Task 15 in stage 1.0 failed 1 times, most recent failure: Lost task 15.0 in stage 1.0 (TID 519
, localhost): java.lang.IllegalStateException: Duplicate key [B@4e233a3c
at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133)
at java.util.HashMap.merge(HashMap.java:1253)
at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320)
at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.buildGapContinuationPenalties(PairHMMLikelihoodCalculat
ionEngine.java:304)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.computeReadLikelihoods(PairHMMLikelihoodCalculationEngi
ne.java:253)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.computeReadLikelihoods(PairHMMLikelihoodCalculationEngi
ne.java:187)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCallerEngine.callRegion(HaplotypeCallerEngine.java:518)
at org.broadinstitute.hellbender.tools.HaplotypeCallerSpark.lambda$regionToVariants$2(HaplotypeCallerSpark.java:192)
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267)
at java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
at java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:169)
at java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
at scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:42)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
at scala.collection.AbstractIterator.to(Iterator.scala:1336)
at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1336)
at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1336)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$13.apply(RDD.scala:912)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$13.apply(RDD.scala:912)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1899)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1899)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:86)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)

Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1454)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1442)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1441)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1441)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:811)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:811)
at scala.Option.foreach(Option.scala:257)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:811)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1667)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1622)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1611)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:632)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1873)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1886)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1899)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1913)
at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:912)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:358)
at org.apache.spark.rdd.RDD.collect(RDD.scala:911)
at org.apache.spark.api.java.JavaRDDLike$class.collect(JavaRDDLike.scala:360)
at org.apache.spark.api.java.AbstractJavaRDDLike.collect(JavaRDDLike.scala:45)
at org.broadinstitute.hellbender.tools.HaplotypeCallerSpark.writeVariants(HaplotypeCallerSpark.java:205)
at org.broadinstitute.hellbender.tools.HaplotypeCallerSpark.runTool(HaplotypeCallerSpark.java:115)
at org.broadinstitute.hellbender.engine.spark.GATKSparkTool.runPipeline(GATKSparkTool.java:353)
at org.broadinstitute.hellbender.engine.spark.SparkCommandLineProgram.doWork(SparkCommandLineProgram.java:38)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:116)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:179)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:198)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:121)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:142)
at org.broadinstitute.hellbender.Main.main(Main.java:220)
Caused by: java.lang.IllegalStateException: Duplicate key [B@4e233a3c
at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133)
at java.util.HashMap.merge(HashMap.java:1253)
at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320)
at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.buildGapContinuationPenalties(PairHMMLikelihoodCalculat
ionEngine.java:304)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.computeReadLikelihoods(PairHMMLikelihoodCalculationEngi
ne.java:253)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.computeReadLikelihoods(PairHMMLikelihoodCalculationEngi
ne.java:187)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCallerEngine.callRegion(HaplotypeCallerEngine.java:518)
at org.broadinstitute.hellbender.tools.HaplotypeCallerSpark.lambda$regionToVariants$2(HaplotypeCallerSpark.java:192)
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267)
at java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
at java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:169)
at java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
at scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:42)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
at scala.collection.AbstractIterator.to(Iterator.scala:1336)
at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1336)
at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1336)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$13.apply(RDD.scala:912)
at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$13.apply(RDD.scala:912)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1899)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1899)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:86)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)



Created 2017-06-05 | Updated 2017-06-13 | bug HaplotypeCaller Spark


@tomwhite commented on Tue May 23 2017

When running on a 160GB BAM I get:

java.lang.IllegalArgumentException: contig must be non-null and not equal to *, and start must be >= 1
     at org.broadinstitute.hellbender.utils.read.SAMRecordToGATKReadAdapter.setPosition(SAMRecordToGATKReadAdapter.java:89)
     at org.broadinstitute.hellbender.utils.clipping.ClippingOp.applyHARDCLIP_BASES(ClippingOp.java:381)
     at org.broadinstitute.hellbender.utils.clipping.ClippingOp.apply(ClippingOp.java:73)
     at org.broadinstitute.hellbender.utils.clipping.ReadClipper.clipRead(ReadClipper.java:147)
     at org.broadinstitute.hellbender.utils.clipping.ReadClipper.clipRead(ReadClipper.java:128)
     at org.broadinstitute.hellbender.utils.clipping.ReadClipper.hardClipSoftClippedBases(ReadClipper.java:332)
     at org.broadinstitute.hellbender.utils.clipping.ReadClipper.hardClipSoftClippedBases(ReadClipper.java:335)
     at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.AssemblyBasedCallerUtils.finalizeRegion(AssemblyBasedCallerUtils.java:84)
     at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.AssemblyBasedCallerUtils.assembleReads(AssemblyBasedCallerUtils.java:238)
     at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCallerEngine.callRegion(HaplotypeCallerEngine.java:478)
     at org.broadinstitute.hellbender.tools.HaplotypeCallerSpark.lambda$regionToVariants$580(HaplotypeCallerSpark.java:203) 

At first glance this looks like a problem with unmapped reads, but these are filtered out by the tool. So it's more likely to be in the clipping logic. It's hard to diagnose since it doesn't say which read caused it, and it's slow to reproduce as it is running on a large input.

Any thoughts @lbergelson, @droazen?


@lbergelson commented on Sat May 27 2017

@tomwhite Is it possible you could upload the bam file somewhere on google cloud along with the command line you used? It's not obvious to me where the error is being caused. It's painful to debug anything on a 160GB file, but I think we can probably do a binary search on the file and find the bad location pretty quickly. I.e. throw compute at the problem instead of human time...



Created 2017-06-05 | Updated 2017-06-05 | bug Copy Number tools


@asmirnov239 commented on Fri Apr 21 2017

Right now if a user disables MAPPED filter, which is a default filter for CalculateTargetCoverage tool, it will fail with the following uninformative exception (unless somehow all reads are mapped):

java.lang.IllegalArgumentException: the input location cannot be null
	at org.broadinstitute.hellbender.utils.Utils.nonNull(Utils.java:549)
	at org.broadinstitute.hellbender.tools.exome.HashedListTargetCollection.indexRange(HashedListTargetCollection.java:152)
	at org.broadinstitute.hellbender.tools.exome.CalculateTargetCoverage.apply(CalculateTargetCoverage.java:298)

We should guard against it and throw an exception before the traversal starts if MAPPED filter is disabled


@asmirnov239 commented on Fri Apr 21 2017

Since there is no direct API call to access the list of resolved filters(after command line parsing) this bug fix will have to wait until broadinstitute/barclay#38 is merged



Created 2017-06-05 | Updated 2017-06-05 | bug


@sooheelee commented on Fri Feb 17 2017

A fix was implemented for HaplotypeCaller but not ported to GenotypeGVCFs nor CombineGVCFs nor CombineVariants. Although user is using v3.7-0-gcfedb67, my understanding is that these types of fixes will only be worked on in GATK4.


Test data submitted by user can be found at

/humgen/gsa-scr1/pub/incoming/bugrep_jgeibel_1.tar.gz

This includes chicken reference files.


The command the user uses to generate the error is very long because we have many vcfs:

Program Args: -T GenotypeGVCFs -R /usr/users/geibel/chicken/chickenrefgen/galGal5_Dec2015/galGal5.fa --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72631_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72632_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72633_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72634_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72635_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72636_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72637_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72638_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72639_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72640_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72641_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72642_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72643_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72644_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72645_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72646_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72647_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72648_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72649_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72650_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72651_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72652_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72653_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72654_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_WL_72655_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72656_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72657_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72658_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72659_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72660_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72661_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72662_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72663_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72664_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72665_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72666_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72667_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72668_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72669_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72670_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72671_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72672_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72673_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72674_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72675_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72676_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72678_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72680_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72682_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/i_BL_72683_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_AB_0001_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_AR_0002_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_AS_0003_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_BA_0004_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_BK_0005_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_BH_0006_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_CG_0007_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_CS_0008_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_DG_0009_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_FG_0010_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_HO_0011_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_GG_0012_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_GS_0013_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_KW_0014_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_IT_0015_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_KA_0016_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_KG_0017_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_PA_0018_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_LE_0019_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_MA_0020_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_MR_0021_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_OH_0022_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_OR_0023_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_OM_0024_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_NH_0025_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_SB_0026_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_SE_0027_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_SH_0028_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_SA_0029_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_SN_0030_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_TO_0031_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_WT_0032_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_WY_0033_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_YO_0034_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_ZC_0035_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_GV_0036_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_CW_0037_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_DL_0038_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_KS_0039_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_OF_0040_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_WR_0041_chr26.raw.g.vcf --variant /usr/users/geibel/chicken/pool_sequence_nov2016/data/gVCF/pl_RI_0042_chr26.raw.g.vcf -nt 10 --max_genotype_count 1024 -L chr26 --dbsnp /usr/users/geibel/chicken/chickenrefgen/ENSEMBL_20170106/Gallus_gallus.updated.vcf -o /usr/users/geibel/chicken/pool_sequence_nov2016/data/rawVCF/IndandPool_chr26.raw.vcf 

The user actually includes a shell script in the test data bundle called JointGenotyping_chr26.sh.


The error shows:

##### ERROR --
##### ERROR stack trace 
java.lang.IllegalArgumentException: the number of genotypes is too large for ploidy 20 and allele 16: approx. 3247943160
	at org.broadinstitute.gatk.tools.walkers.genotyper.GenotypeLikelihoodCalculators.getInstance(GenotypeLikelihoodCalculators.java:319)
	at org.broadinstitute.gatk.tools.walkers.variantutils.ReferenceConfidenceVariantContextMerger.mergeRefConfidenceGenotypes(ReferenceConfidenceVariantContextMerger.java:461)
	at org.broadinstitute.gatk.tools.walkers.variantutils.ReferenceConfidenceVariantContextMerger.merge(ReferenceConfidenceVariantContextMerger.java:164)
	at org.broadinstitute.gatk.tools.walkers.variantutils.GenotypeGVCFs.map(GenotypeGVCFs.java:302)
	at org.broadinstitute.gatk.tools.walkers.variantutils.GenotypeGVCFs.map(GenotypeGVCFs.java:135)
	at org.broadinstitute.gatk.engine.traversals.TraverseLociNano$TraverseLociMap.apply(TraverseLociNano.java:267)
	at org.broadinstitute.gatk.engine.traversals.TraverseLociNano$TraverseLociMap.apply(TraverseLociNano.java:255)
	at org.broadinstitute.gatk.utils.nanoScheduler.NanoScheduler.executeSingleThreaded(NanoScheduler.java:274)
	at org.broadinstitute.gatk.utils.nanoScheduler.NanoScheduler.execute(NanoScheduler.java:245)
	at org.broadinstitute.gatk.engine.traversals.TraverseLociNano.traverse(TraverseLociNano.java:144)
	at org.broadinstitute.gatk.engine.traversals.TraverseLociNano.traverse(TraverseLociNano.java:92)
	at org.broadinstitute.gatk.engine.traversals.TraverseLociNano.traverse(TraverseLociNano.java:48)
	at org.broadinstitute.gatk.engine.executive.ShardTraverser.call(ShardTraverser.java:98)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
##### ERROR ------------------------------------------------------------------------------------------
##### ERROR A GATK RUNTIME ERROR has occurred (version 3.7-0-gcfedb67):
##### ERROR
##### ERROR This might be a bug. Please check the documentation guide to see if this is a known problem.
##### ERROR If not, please post the error message, with stack trace, to the GATK forum.
##### ERROR Visit our website and forum for extensive documentation and answers to 
##### ERROR commonly asked questions https://software.broadinstitute.org/gatk
##### ERROR
##### ERROR MESSAGE: the number of genotypes is too large for ploidy 20 and allele 16: approx. 3247943160
##### ERROR ------------------------------------------------------------------------------------------



Created 2017-06-05 | Updated 2017-06-05 | bug Copy Number tools


@meganshand commented on Thu Feb 16 2017

Using a tiny bam file that I typically use for testing while running the CNV wdl on the cloud, I got the following errors (the tiny file is here: gs://broad-dsde-methods/takuto/test_files/small_NA12878_hg19.bam):

  1. The output tsv from TumorNormalizeSomaticReadCounts contained NaNs
  2. TumorPerformSeg threw the following error:
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/cromwell_root/tmp
[February 16, 2017 3:23:02 PM UTC] org.broadinstitute.hellbender.tools.exome.PerformSegmentation  --tangentNormalized /cromwell_root/broad-dsde-methods/cromwell-execution-24/TumorOnly/f30dd8c6-eec3-45ba-b7f2-f845d308d59d/call-TumorNormalizeSomaticReadCounts/small_NA12878.tn.tsv --output small_NA12878.seg --log2Input true --alpha 0.01 --nperm 10000 --pmethod hybrid --minWidth 2 --kmax 25 --nmin 200 --eta 0.05 --trim 0.025 --undoSplits none --undoPrune 0.05 --undoSD 3 --help false --version false --verbosity INFO --QUIET false  --use_jdk_deflater false
[February 16, 2017 3:23:02 PM UTC] Executing as root@3addd2d7b373 on Linux 3.16.0-0.bpo.4-amd64 amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_121-b13; Version: Version:c17c8ed-SNAPSHOT
[February 16, 2017 3:23:04 PM UTC] org.broadinstitute.hellbender.tools.exome.PerformSegmentation done. Elapsed time: 0.04 minutes.
Runtime.totalMemory()=185597952
org.broadinstitute.hellbender.utils.R.RScriptExecutorException: 
Rscript exited with 1
Command Line: Rscript -e tempLibDir = '/cromwell_root/tmp/root/Rlib.5210694187065743072';source('/cromwell_root/tmp/root/CBS.8616708738798684646.R'); --args --sample_name=NA12878 --targets_file=/cromwell_root/broad-dsde-methods/cromwell-execution-24/TumorOnly/f30dd8c6-eec3-45ba-b7f2-f845d308d59d/call-TumorNormalizeSomaticReadCounts/small_NA12878.tn.tsv --output_file=small_NA12878.seg --log2_input=TRUE --min_width=2 --alpha=0.01 --nperm=10000 --pmethod=hybrid --kmax=25 --nmin=200 --eta=0.05 --trim=0.025 --undosplits=none --undoprune=0.05 --undoSD=3
Stdout: $sample_name
[1] "NA12878"

$targets_file
[1] "/cromwell_root/broad-dsde-methods/cromwell-execution-24/TumorOnly/f30dd8c6-eec3-45ba-b7f2-f845d308d59d/call-TumorNormalizeSomaticReadCounts/small_NA12878.tn.tsv"

$output_file
[1] "small_NA12878.seg"

$log2_input
[1] "TRUE"

$min_width
[1] 2

$alpha
[1] 0.01

$nperm
[1] 10000

$pmethod
[1] "hybrid"

$kmax
[1] 25

$nmin
[1] 200

$eta
[1] 0.05

$trim
[1] 0.025

$undosplits
[1] "none"

$undoprune
[1] "0.05"

$undoSD
[1] 3

$help
[1] FALSE


Stderr: Error in sort(abs(diff(genomdat)))[1:n.keep] : 
  only 0's may be mixed with negative subscripts
Calls: source ... segment -> inherits -> smooth.CNA -> trimmed.variance
Execution halted

	at org.broadinstitute.hellbender.utils.R.RScriptExecutor.exec(RScriptExecutor.java:163)
	at org.broadinstitute.hellbender.utils.segmenter.RCBSSegmenter.writeSegmentFile(RCBSSegmenter.java:114)
	at org.broadinstitute.hellbender.tools.exome.PerformSegmentation.applySegmentation(PerformSegmentation.java:185)
	at org.broadinstitute.hellbender.tools.exome.PerformSegmentation.doWork(PerformSegmentation.java:180)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:112)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:170)
	at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:189)
	at org.broadinstitute.hellbender.Main.instanceMain(Main.java:96)
	at org.broadinstitute.hellbender.Main.instanceMain(Main.java:103)
	at org.broadinstitute.hellbender.Main.mainEntry(Main.java:116)
	at org.broadinstitute.hellbender.Main.main(Main.java:158)


Created 2017-06-05 | Updated 2017-06-05 | bug Copy Number tools


@samuelklee commented on Thu Jan 19 2017

We've had a few reports of this---let's look into it.


@mbabadi commented on Thu Jan 19 2017

The only thing that comes to my mind is that GetBayesianHetCoverage enables indexing in samtools/htsjdk by default and GetHetCoverage disables it in most cases. Can you gather a few problematic samples?


@samuelklee commented on Thu Jan 19 2017

Yup, got a user on the forum that I'm trying to get more information from. @dlivitz have you run into any samples that you could pass along?


@LeeTL1220 commented on Fri Jan 20 2017

I've had this issue.


@LeeTL1220 commented on Wed Jan 25 2017

This may be related to #865 , since I am having an issue like this as well...

I believe it relates to the number of intervals (i.e. size of the SNP list).

A ton of time is being spent during initialization on DiskBasedBAMFileIndex.getSpanOverlapping and AbstractBamFileIndex.query when you have a long interval list. I am not 100% sure that this is the same issue as here, but symptoms look the same. I saw 50k intervals taking about 50 sec to init, but when I ran 500k intervals I just killed it after half an hour.



Created 2017-06-05 | Updated 2017-06-24 | bug Copy Number tools


@eddiebroad commented on Thu Dec 01 2016

An issue encountered with gatk-protected "SparkGenomeReadCounts" tool is a non-helpful "null" error message. A non-helpful error "null" message was printed by gatk-protected with the command-line below
during the course of trying to use it on/in FireCloud:

+ java -Xmx48g -jar fc-7ac504fc-7fe4-4bc1-89d3-7f16317b8ff4/eddie.jar SparkGenomeReadCounts --outputFile this.entity_id.coverage.tsv --reference fc-e2421839-93d5-4ed5-8861-593f00364e54/Homo_sapiens_assembly19.fasta --input firecloud-tcga-open-access/tutorial/bams/C835.HCC1143_BL.4.bam --binsize 5000
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/cromwell_root/tmp
 .....
......
......
proceeding with flushing remote transports.
***********************************************************************

null

***********************************************************************

To try to make a more helpful error message appear I added a "catch" block after a call to runTool in instanceMainPostParseArgs in file CommandLineProgram.java and got a more helpful message about a missing dictionary file:

    try {
    return runTool();
    } 
    catch(Exception e) {
        e.getStackTrace();
        }

java.lang.RuntimeException: org.broadinstitute.hellbender.exceptions.UserException$MissingReferenceDictFile
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:179)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:204)
at org.broadinstitute.hellbender.Main.instanceMain(Main.java:95)
at org.broadinstitute.hellbender.Main.instanceMain(Main.java:103)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:115)
at org.broadinstitute.hellbender.Main.main(Main.java:152)
Caused by: org.broadinstitute.hellbender.exceptions.UserException$MissingReferenceDictFile
at org.broadinstitute.hellbender.engine.spark.GATKSparkTool.initializeReference(GATKSparkTool.java:409)
at org.broadinstitute.hellbender.engine.spark.GATKSparkTool.initializeToolInputs(GATKSparkTool.java:355)
at org.broadinstitute.hellbender.engine.spark.GATKSparkTool.runPipeline(GATKSparkTool.java:346)
at org.broadinstitute.hellbender.engine.spark.SparkCommandLineProgram.doWork(SparkCommandLineProgram.java:38)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:111)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:170)
... 5 more

To bring about success I added 4 lines of code :

    ln -vs ${ref_fasta} ; 
    ln -vs ${ref_fasta_fai} ;
    ln -vs ${ref_fasta_dict} ;
    FASTA_NAME=`basename ${ref_fasta} `;  

Then, instead of --reference ${ref_fasta} in calling gatk-protected, I put --reference $FASTA_NAME and the "null" exception went away and the program run successfully.


@eddiebroad commented on Thu Dec 01 2016

per @droazen : @achevali @LeeTL1220


@LeeTL1220 commented on Tue Dec 06 2016

@eddiebroad Before this gets assigned, what version of gatk-protected are you using?
Assuming that this is a version we built (despite the name "eddie.jar"): @achevali , can you figure out how you are reporting the error. @droazen are you sure this is not in the engine?


@eddiebroad commented on Wed Dec 07 2016

@LeeTL1220

The original JAVA JAR where I first observed the "null" message I presume was based off commit 3a2bb0d. At the time the project was initiated I think it was the latest commit. The original JAR where the "null" message was observed was gatk-protected-all-3a2bb0d-SNAPSHOT-spark_standalone.jar . Because of the "3a2bb0d" in the JAR file name is why I presume that it's based off commit 3a2bb0d.

From the gatk-protected repo code (and also "gatk" repo) I added some debug/print statements and saved to a differently named JAR "eddie.jar" to help me distinguish my hacking from the original JAR.

The JAVA file where I added the most helpful statements was in CommandLineProgram.java which is actually in "gatk" repo (not "gatk-protected" repo)

If I look at a LOG, I can see "EAS" my initials and see c40e75b which appears to be a more recent commit compared to 3a2bb0d

EAS in main!!!!
EAS to call instanceMain second....
EAS to call instanceMain first....
17:28:40.295 INFO  SparkGenomeReadCounts - EAS ABOUT TO CALL instanceMainPostParseArgs in  instanceMain  in clp.java 
17:28:40.396 INFO  IntelGKLUtils - Trying to load Intel GKL library from:
	jar:file:/cromwell_root/fc-7ac504fc-7fe4-4bc1-89d3-7f16317b8ff4/eddie.jar!/com/intel/gkl/native/libIntelGKL.so
17:28:40.498 INFO  IntelGKLUtils - Intel GKL library loaded from classpath.
[December 1, 2016 5:28:40 PM UTC] org.broadinstitute.hellbender.tools.genome.SparkGenomeReadCounts  --binsize 5000 --outputFile this.entity_id.coverage.tsv --reference Homo_sapiens_assembly19.fasta --input firecloud-tcga-open-access/tutorial/bams/C835.HCC1143_BL.4.bam  --keepXYMT false --readValidationStringency SILENT --interval_set_rule UNION --interval_padding 0 --interval_exclusion_padding 0 --bamPartitionSize 0 --disableSequenceDictionaryValidation false --shardedOutput false --numReducers 0 --sparkMaster local[*] --help false --version false --verbosity INFO --QUIET false --use_jdk_deflater false --disableAllReadFilters false
[December 1, 2016 5:28:40 PM UTC] Executing as root@71bfa07f6996 on Linux 3.16.0-0.bpo.4-amd64 amd64; OpenJDK 64-Bit Server VM 1.8.0_111-8u111-b14-2~bpo8+1-b14; Version: Version:c40e75b-SNAPSHOT
17:28:40.501 INFO  SparkGenomeReadCounts - Defaults.BUFFER_SIZE : 131072

@eddiebroad commented on Wed Dec 07 2016

I want to mention in my case, all the reference files were present (fasta, fai, dict) BUT the dict was in a different directory and NOT in the same directory as the other two files (which were in the same directory) and that's why the dict was not found and that's why the soft link creation fixed the issue.


@droazen commented on Wed Dec 07 2016

@LeeTL1220 It's very possibly in the engine -- if the tool itself gets cleared of blame, feel free to bounce this one back to gatk public.



Created 2017-06-05 | Updated 2017-06-05 | bug Copy Number tools


@samuelklee commented on Sun Nov 27 2016

Possibly due to bad input, but we should throw a better exception regardless:

java.lang.NullPointerException
at org.broadinstitute.hellbender.tools.exome.gcbias.CorrectGCBias.lambda$gcContentsOfTargets$1(CorrectGCBias.java:91)
at java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
at java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1351)
at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:498)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.allMatch(ReferencePipeline.java:454)
at org.broadinstitute.hellbender.tools.exome.gcbias.CorrectGCBias.gcContentsOfTargets(CorrectGCBias.java:91)
at org.broadinstitute.hellbender.tools.exome.gcbias.CorrectGCBias.doWork(CorrectGCBias.java:65)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:108)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:166)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:185)
at org.broadinstitute.hellbender.Main.instanceMain(Main.java:76)
at org.broadinstitute.hellbender.Main.main(Main.java:92)

@samuelklee commented on Mon Nov 28 2016

Looks like the issue was that the coverage files contained additional targets (on the sex chromosomes) that were missing in the list of targets passed.



Created 2017-06-05 | Updated 2017-06-05 | bug Copy Number tools


@samuelklee commented on Wed Aug 10 2016

htsjdk.samtools.SAMFormatException: Unrecognized tag type: 
    at htsjdk.samtools.BinaryTagCodec.readSingleValue(BinaryTagCodec.java:351)
    at htsjdk.samtools.BinaryTagCodec.readTags(BinaryTagCodec.java:282)
    at htsjdk.samtools.BAMRecord.decodeAttributes(BAMRecord.java:313)
    at htsjdk.samtools.BAMRecord.getAttribute(BAMRecord.java:293)
    at org.broadinstitute.hellbender.utils.read.SAMRecordToGATKReadAdapter.getReadGroup(SAMRecordToGATKReadAdapter.java:319)
    at org.broadinstitute.hellbender.engine.filters.ReadFilterLibrary.lambda$static$c202bd10$1(ReadFilterLibrary.java:45)
    at org.broadinstitute.hellbender.engine.filters.ReadFilter.lambda$and$cbfb947b$1(ReadFilter.java:30)
    at org.broadinstitute.hellbender.engine.filters.ReadFilter.lambda$and$cbfb947b$1(ReadFilter.java:30)
    at org.broadinstitute.hellbender.engine.filters.ReadFilter.lambda$and$cbfb947b$1(ReadFilter.java:30)
    at org.broadinstitute.hellbender.engine.filters.ReadFilter.lambda$and$cbfb947b$1(ReadFilter.java:30)
    at org.broadinstitute.hellbender.engine.filters.ReadFilter.lambda$and$cbfb947b$1(ReadFilter.java:30)
    at org.broadinstitute.hellbender.engine.filters.WellformedReadFilter.test(WellformedReadFilter.java:31)
    at org.broadinstitute.hellbender.engine.spark.GATKSparkTool.lambda$getReads$e4b35a40$1(GATKSparkTool.java:196)
    at org.apache.spark.api.java.JavaRDD$$anonfun$filter$1.apply(JavaRDD.scala:78)
    at org.apache.spark.api.java.JavaRDD$$anonfun$filter$1.apply(JavaRDD.scala:78)
    at scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:390)
    at scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:388)
    at scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:388)
    at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
    at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
    at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
    at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:189)
    at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:64)
    at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
    at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
    at org.apache.spark.scheduler.Task.run(Task.scala:89)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

@samuelklee commented on Wed Aug 10 2016

@LeeTL1220 have you seen this during your WGS runs? I got it on a few TCGA BAMs, not sure if they're just bad samples?



Created 2017-06-05 | Updated 2017-06-05 | bug Copy Number tools Segmentation


@samuelklee commented on Tue Aug 09 2016

This combined with an unset random seed was causing AlleleFractionSegmenter.testChromosomesOnDifferentSegments to fail intermittently in #633. @davidbenjamin could you take a look when you get back? This could be fixed easily by a Math.min in the right place, but I don't know if the fact that the memory length seems to blow up is something generic that happens when we collapse to a single state that needs to be addressed more carefully.


@samuelklee commented on Wed Aug 17 2016

@davidbenjamin can you take a quick look? I think it's probably a quick fix.


@davidbenjamin commented on Wed Aug 17 2016

Sure.



Created 2017-06-05 | Updated 2017-06-05 | bug


@personalis commented on Thu Jun 23 2016

We got the following error when running gatk-launch FastqToSam:

java.lang.IllegalArgumentException: Self-suppression not permitted
at java.lang.Throwable.addSuppressed(Throwable.java:1043)
at org.broadinstitute.hellbender.tools.picard.sam.FastqToSam.doWork(FastqToSam.java:163)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:102)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:155)
at org.broadinstitute.hellbender.cmdline.PicardCommandLineProgram.instanceMain(PicardCommandLineProgram.java:61)
at org.broadinstitute.hellbender.Main.instanceMain(Main.java:67)
at org.broadinstitute.hellbender.Main.main(Main.java:82)
Caused by: java.lang.NoClassDefFoundError: org/xerial/snappy/LoadSnappy
at htsjdk.samtools.util.SnappyLoader.(SnappyLoader.java:86)
at htsjdk.samtools.util.SnappyLoader.(SnappyLoader.java:52)
at htsjdk.samtools.util.TempStreamFactory.getSnappyLoader(TempStreamFactory.java:42)
at htsjdk.samtools.util.TempStreamFactory.wrapTempOutputStream(TempStreamFactory.java:74)
at htsjdk.samtools.util.SortingCollection.spillToDisk(SortingCollection.java:223)
at htsjdk.samtools.util.SortingCollection.add(SortingCollection.java:166)
at htsjdk.samtools.SAMFileWriterImpl.addAlignment(SAMFileWriterImpl.java:192)
at htsjdk.samtools.AsyncSAMFileWriter.synchronouslyWrite(AsyncSAMFileWriter.java:36)
at htsjdk.samtools.AsyncSAMFileWriter.synchronouslyWrite(AsyncSAMFileWriter.java:16)
at htsjdk.samtools.util.AbstractAsyncWriter$WriterRunnable.run(AbstractAsyncWriter.java:117)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.xerial.snappy.LoadSnappy
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 11 more

We can find snappy-java in <INST_DIR>/build/install/gatk/lib/snappy-java-1.1.1.7.jar, but it does not have a LoadSnappy class. Renaming the snappy-java jar file so gatk cannot find it allows FastqToSam to run through.


@akiezun commented on Thu Jun 30 2016

thanks for the report. Can you provide the whole commandline you used?


@huangk3 commented on Thu Sep 15 2016

Hi @akiezun I experience the same error when running gate-launch FastqToSam. My command line is:
"./gatk_launch FastqToSam -SM "test" -F1 $fq1 -F2 $fq2 -O test.spark.sam -SO coordinate -R $ref --STRIP_UNPAIRED_MATE_NUMBER true --VALIDATION_STRINGENCY LENIENT -PL ILLUMINA --CREATE_INDEX true"

My Spark version is 2.0.0
Thanks!


@lbergelson commented on Mon Sep 19 2016

@huangk3 Unfortunately Adam moved on to a different job so he's longer working on GATK.

I believe this is the same problem as #2026 and has been patched in gatk public with #2028. You might try using FastqToSam in the public repo, or wait and try a new version of protected that incorporates an updated gatk public (coming soon..)



Created 2017-06-05 | Updated 2017-07-17 | bug HaplotypeCaller


@akiezun commented on Wed Apr 27 2016

tag 39c988c2dc7f669306d246a2191d16af50496640


the created bam file does not have an index created





Priority: Normal   | High