QCommandLine

Genome STRiP uses the Queue workflow engine (developed by the Broad Institute GSA group) to implement executable analysis pipelines.

Queue is invoked using the QCommandLine program. This page documents the common arguments to the Queue workflow engine itself (or similar arguments affecting the workflow execution environment that are common to all Genome STRiP pipelines). Not all Queue options or arguments are documented here, just the most common ones used with Genome STRiP.

Running a Queue script

Queue manages processing steps and dependencies, but generally does not directly execute the processing steps itself. Instead the processing steps are scheduled to run on a compute cluster through a workflow manager, such as LSF (by Platform Computing) or SGE (Sun Grid Engine).

A Queue script invocation for Genome STRiP generaly has the following form:

 classpath="${SV_DIR}/lib/SVToolkit.jar:${SV_DIR}/lib/gatk/GenomeAnalysisTK.jar:${SV_DIR}/lib/gatk/Queue.jar"
 java -Xmx4g -cp ${classpath} \
     org.broadinstitute.gatk.queue.QCommandLine \
     -S ${SV_DIR}/qscript/SVQScript.q
     -S pipeline_script_to_run.q \
     -cp ${classpath} \
     -gatk ${SV_DIR}/lib/gatk/GenomeAnalysisTK.jar \
     -jobProject project_name \
     -jobQueue queue_name \
     -jobLogDir log_directory \
     -jobRunner workflow_manager \
     -gatkJobRunner workflow_manager \
     -jobNative workflow_manager_options \
     -run

Note that -cp needs to be specified twice. Once to the java runtime (before QCommandLine) and once as an argument to QCommandLine (the latter controls the classpath that will be used for the individual processing steps).

The -run flag causes the processing steps to be executed. If you omit the -run flag, then Queue performs a "dry run" where it prints information about the processing steps that would have been run, but does not actually execute them. This can be used for debugging and can also be used to tell if a pipeline successfully run to completion.

Supported workflow managers

Queue has built-in support for LSF 7. For other workflow managers, you should generally try to use the Drmaa interface (Drmaa is a portable interface for workflow managers in the style of LSF and SGE).

Queue (and the Genome STRiP Queue scripts) have some built-in options that understand how to pass certain arguments to LSF, but it is generally better practice to use the Drmaa job runner and use -jobNative to pass the necessary arguments to your workflow manager.

• LSF 7.0

To use LSF 7, specify -jobRunner Lsf706.

• LSF 9.x

To use LSF 9, specify -jobRunner Drmaa from an LSF execution host configured to use LSF 9.

• SGE

To use SGE, specify -jobRunner Drmaa from an SGE execution host.

QCommandLine specific arguments

Argument details