Showing tool doc from version 4.1.2.0 | The latest version is 4.1.2.0

DownsampleSam (Picard)

Downsample a SAM or BAM file.This tool applies a downsampling algorithm to a SAM or BAM file to retain only a (deterministically random) subset of the reads. Reads from the same template (e.g. read-pairs, secondary and supplementary reads) are all either kept or discarded as a unit, with the goal of retaining readsfrom PROBABILITY * input templates. The results will contain approximately PROBABILITY * input reads, however for very small PROBABILITIES this may not be the case. A number of different downsampling strategies are supported using the STRATEGY option: ConstantMemory: Downsamples a stream or file of SAMRecords using a hash-projection strategy such that it can run in constant memory. The downsampling is stochastic, and therefore the actual retained proportion will vary around the requested proportion. Due to working in fixed memory this strategy is good for large inputs, and due to the stochastic nature the accuracy of this strategy is highest with a high number of output records, and diminishes at low output volumes. HighAccuracy: Attempts (but does not guarantee) to provide accuracy up to a specified limit. Accuracy is defined as emitting a proportion of reads as close to the requested proportion as possible. In order to do so this strategy requires memory that is proportional to the number of template names in the incoming stream of reads, and will thus require large amounts of memory when running on large input files. Chained: Attempts to provide a compromise strategy that offers some of the advantages of both the ConstantMemory and HighAccuracy strategies. Uses a ConstantMemory strategy to downsample the incoming stream to approximately the desired proportion, and then a HighAccuracy strategy to finish. Works in a single pass, and will provide accuracy close to (but often not as good as) HighAccuracy while requiring memory proportional to the set of reads emitted from the ConstantMemory strategy to the HighAccuracy strategy. Works well when downsampling large inputs to small proportions (e.g. downsampling hundreds of millions of reads and retaining only 2%. Should be accurate 99.9% of the time when the input contains more than 50,000 templates (read names). For smaller inputs, HighAccuracy is recommended instead.

Usage examples:

Downsample file, keeping about 10% of the reads

java -jar picard.jar DownsampleSam \ I=input.bam \ O=downsampled.bam \ P=0.2

Downsample file, keeping about 2% of the reads

java -jar picard.jar DownsampleSam \ I=input.bam \ O=downsampled.bam \ STRATEGY=Chained \ P=0.02 \ ACCURACY=0.0001

Downsample file, keeping about 0.001% of the reads (may require more memory)

java -jar picard.jar DownsampleSam \ I=input.bam \ O=downsampled.bam \ STRATEGY=HighAccuracy \ P=0.00001 \ ACCURACY=0.0000001

Category Read Data Manipulation


Overview

Summary

This tool applies a downsampling algorithm to a SAM or BAM file to retain only a (deterministically random) subset of the reads. Reads from the same template (e.g. read-pairs, secondary and supplementary reads) are all either kept or discarded as a unit, with the goal of retaining reads from PROBABILITY * (input templates). The results will contain approximately PROBABILITY * (input reads), however for very small probabilities this may not be the case. A number of different downsampling strategies are supported using the #STRATEGY option:
ConstantMemory
Downsamples a stream or file of SAMRecords using a hash-projection strategy such that it can run in constant memory. The downsampling is stochastic, and therefore the actual retained proportion will vary around the requested proportion. Due to working in fixed memory this strategy is good for large inputs, and due to the stochastic nature the accuracy of this strategy is highest with a high number of output records, and diminishes at low output volumes.
HighAccuracy
Attempts (but does not guarantee) to provide accuracy up to a specified limit. Accuracy is defined as emitting a proportion of reads as close to the requested proportion as possible. In order to do so this strategy requires memory that is proportional to the number of template names in the incoming stream of reads, and will thus require large amounts of memory when running on large input files.
Chained
Attempts to provide a compromise strategy that offers some of the advantages of both the ConstantMemory and HighAccuracy strategies. Uses a ConstantMemory strategy to downsample the incoming stream to approximately the desired proportion, and then a HighAccuracy strategy to finish. Works in a single pass, and will provide accuracy close to (but often not as good as) HighAccuracy while requiring memory proportional to the set of reads emitted from the ConstantMemory strategy to the HighAccuracy strategy. Works well when downsampling large inputs to small proportions (e.g. downsampling hundreds of millions of reads and retaining only 2%. Should be accurate 99.9% of the time when the input contains more than 50,000 templates (read names). For smaller inputs, HighAccuracy is recommended instead.
The number of records written can be output to a QualityYieldMetrics metrics file via the #METRICS_FILE.

Usage examples:

Downsample file, keeping about 10% of the reads

 java -jar picard.jar DownsampleSam \
       I=input.bam \
       O=downsampled.bam \
       P=0.1
 

Downsample file, keeping 2% of the reads

 java -jar picard.jar DownsampleSam \
       I=input.bam \
       O=downsampled.bam \
       STRATEGY=Chained \
       P=0.02 \
       ACCURACY=0.0001
 

Downsample file, keeping 0.001% of the reads (may require more memory)

 java -jar picard.jar DownsampleSam \
       I=input.bam \
       O=downsampled.bam \
       STRATEGY=HighAccuracy \
       P=0.00001 \
       ACCURACY=0.0000001
 

DownsampleSam (Picard) specific arguments

This table summarizes the command-line arguments that are specific to this tool. For more details on each argument, see the list further down below the table or click on an argument name to jump directly to that entry in the list.

Argument name(s) Default value Summary
Required Arguments
--INPUT
 -I
null The input SAM or BAM file to downsample.
--OUTPUT
 -O
null The output, downsampled, SAM or BAM file to write.
Optional Tool Arguments
--ACCURACY
 -A
1.0E-4 The accuracy that the downsampler should try to achieve if the selected strategy supports it. Note that accuracy is never guaranteed, but some strategies will attempt to provide accuracy within the requested bounds.Higher accuracy will generally require more memory.
--arguments_file
[] read one or more arguments files and add them to the command line
--help
 -h
false display the help message
--METRICS_FILE
 -M
null The metrics file (of type QualityYieldMetrics) which will contain information about the downsampled file.
--PROBABILITY
 -P
1.0 The probability of keeping any individual read, between 0 and 1.
--RANDOM_SEED
 -R
1 Random seed used for deterministic results. Setting to null will cause multiple invocations to produce different results.
--REFERENCE_SEQUENCE
null The reference sequence file.
--STRATEGY
 -S
ConstantMemory The downsampling strategy to use. See usage for discussion.
--version
false display the version number for this tool
Optional Common Arguments
--COMPRESSION_LEVEL
5 Compression level for all compressed files created (e.g. BAM and VCF).
--CREATE_INDEX
false Whether to create a BAM index when writing a coordinate-sorted BAM file.
--CREATE_MD5_FILE
false Whether to create an MD5 digest for any BAM or FASTQ files created.
--GA4GH_CLIENT_SECRETS
client_secrets.json Google Genomics API client_secrets.json file path.
--MAX_RECORDS_IN_RAM
500000 When writing files that need to be sorted, this will specify the number of records stored in RAM before spilling to disk. Increasing this number reduces the number of file handles needed to sort the file, and increases the amount of RAM needed.
--QUIET
false Whether to suppress job-summary info on System.err.
--TMP_DIR
[] One or more directories with space available to be used by this program for temporary storage of working files
--USE_JDK_DEFLATER
 -use_jdk_deflater
false Use the JDK Deflater instead of the Intel Deflater for writing compressed output
--USE_JDK_INFLATER
 -use_jdk_inflater
false Use the JDK Inflater instead of the Intel Inflater for reading compressed input
--VALIDATION_STRINGENCY
STRICT Validation stringency for all SAM files read by this program. Setting stringency to SILENT can improve performance when processing a BAM file in which variable-length data (read, qualities, tags) do not otherwise need to be decoded.
--VERBOSITY
INFO Control verbosity of logging.
Advanced Arguments
--showHidden
false display hidden arguments

Argument details

Arguments in this list are specific to this tool. Keep in mind that other arguments are available that are shared with other tools (e.g. command-line GATK arguments); see Inherited arguments above.


--ACCURACY / -A

The accuracy that the downsampler should try to achieve if the selected strategy supports it. Note that accuracy is never guaranteed, but some strategies will attempt to provide accuracy within the requested bounds.Higher accuracy will generally require more memory.

double  1.0E-4  [ [ -∞  ∞ ] ]


--arguments_file / NA

read one or more arguments files and add them to the command line

List[File]  []


--COMPRESSION_LEVEL / NA

Compression level for all compressed files created (e.g. BAM and VCF).

int  5  [ [ -∞  ∞ ] ]


--CREATE_INDEX / NA

Whether to create a BAM index when writing a coordinate-sorted BAM file.

Boolean  false


--CREATE_MD5_FILE / NA

Whether to create an MD5 digest for any BAM or FASTQ files created.

boolean  false


--GA4GH_CLIENT_SECRETS / NA

Google Genomics API client_secrets.json file path.

String  client_secrets.json


--help / -h

display the help message

boolean  false


--INPUT / -I

The input SAM or BAM file to downsample.

R File  null


--MAX_RECORDS_IN_RAM / NA

When writing files that need to be sorted, this will specify the number of records stored in RAM before spilling to disk. Increasing this number reduces the number of file handles needed to sort the file, and increases the amount of RAM needed.

Integer  500000  [ [ -∞  ∞ ] ]


--METRICS_FILE / -M

The metrics file (of type QualityYieldMetrics) which will contain information about the downsampled file.

File  null


--OUTPUT / -O

The output, downsampled, SAM or BAM file to write.

R File  null


--PROBABILITY / -P

The probability of keeping any individual read, between 0 and 1.

double  1.0  [ [ -∞  ∞ ] ]


--QUIET / NA

Whether to suppress job-summary info on System.err.

Boolean  false


--RANDOM_SEED / -R

Random seed used for deterministic results. Setting to null will cause multiple invocations to produce different results.

Integer  1  [ [ -∞  ∞ ] ]


--REFERENCE_SEQUENCE / NA

The reference sequence file.

File  null


--showHidden / -showHidden

display hidden arguments

boolean  false


--STRATEGY / -S

The downsampling strategy to use. See usage for discussion.

The --STRATEGY argument is an enumerated type (Strategy), which can have one of the following values:

HighAccuracy
ConstantMemory
Chained

Strategy  ConstantMemory


--TMP_DIR / NA

One or more directories with space available to be used by this program for temporary storage of working files

List[File]  []


--USE_JDK_DEFLATER / -use_jdk_deflater

Use the JDK Deflater instead of the Intel Deflater for writing compressed output

Boolean  false


--USE_JDK_INFLATER / -use_jdk_inflater

Use the JDK Inflater instead of the Intel Inflater for reading compressed input

Boolean  false


--VALIDATION_STRINGENCY / NA

Validation stringency for all SAM files read by this program. Setting stringency to SILENT can improve performance when processing a BAM file in which variable-length data (read, qualities, tags) do not otherwise need to be decoded.

The --VALIDATION_STRINGENCY argument is an enumerated type (ValidationStringency), which can have one of the following values:

STRICT
LENIENT
SILENT

ValidationStringency  STRICT


--VERBOSITY / NA

Control verbosity of logging.

The --VERBOSITY argument is an enumerated type (LogLevel), which can have one of the following values:

ERROR
WARNING
INFO
DEBUG

LogLevel  INFO


--version / NA

display the version number for this tool

boolean  false


Return to top


See also General Documentation | Tool Docs Index Tool Documentation Index | Support Forum

GATK version 4.1.2.0 built at Tue, 23 Apr 2019 14:55:55 -0400.