Latest posts

By Ruchi Munshi, Senior Software Product Manager, Data Sciences Platform

For a long time now, the Cromwell workflow engine has supported popular high-performance computing (HPC) platforms such as SGE, LSF and SLURM. But when it comes to clouds, Cromwell only had native support for the Google Cloud Platform (GCP), which is done via the Google Pipelines API.

Today, I’m thrilled to announce that over the span of this year, we’ve expanded Cromwell's native cloud support to a suite of several cloud backends to parallel our suite of supported HPC backends. The newly released Cromwell version 35 will support three cloud platforms natively: GCP with an upgraded backend to support the new Pipelines API v2; Amazon Web Services (AWS) through a brand new backend that connects to the Batch API (in beta status); and Alibaba Cloud, whose Cromwell backend was introduced in a previous version earlier this year.

Read the whole post
See comments (0)

One of my favorite features of the Cromwell workflow management system is that it was designed from the start to "support multiple computing platforms in order to maximize portability and reproducibility of analysis workflows". In my role as an advocate for a research community that struggles with standardization and interoperability, just reading those words in a sentence makes me perk up like a kid hearing the ice cream truck song. But then it immediately raises the question -- what does it mean for Cromwell to support a given platform? Let's take a moment today to unpack that, since "support" is kind of a loaded term in the world of workflow management systems.

Read the whole post
See comments (0)

By Jeff Gentry, Software Engineering Manager, Data Sciences Platform

In January I reported that Cromwell was expanding workflow language support beyond WDL via a concept we call the Workflow Object Model (WOM). In that post I announced that development towards supporting the Common Workflow Language (CWL) was already underway. As of today, that work is complete and we have released Cromwell version 32 which supports running CWL workflows.

Read the whole post
See comments (0)

By Jose Soto, Software Engineer in the Broad Institute's Data Sciences Platform

Over the past few months, I spent a lot of time optimizing our pipelines for cost (including for the $5 Genome Analysis Pipeline), so I thought I'd share a few of the tricks I find especially effective, starting with what we call dynamic sizing.

The idea is that on a pay-as-you-go cloud like Google Cloud Platform, the rate you pay for compute is based on the technical specs of the machines you use; the beefier the machine, the higher the rate. So you can save yourself a lot of money by making sure you don't use machines that are beefier than you need.

Here's how we do that in our pipelines.

Read the whole post
See comments (0)

Cromwell and WDL started out as a codependent couple, matched by design and tied together by habit. But I like to emphasize that Cromwell and WDL share more than just development history. They also share a fundamental philosophy that prioritizes user-friendliness, with the goal of making simple use cases easy to realize while ensuring that users with more complex needs have a reasonable path forward. I believe that this philosophy plays a key role in making Cromwell and WDL best in class tools for bioinformatics computing. But sometimes user-friendliness means a different thing for one compared to the other. And so it has become clear that in order to maximize their usefulness and satisfy the needs of a wide community, Cromwell and WDL need to be decoupled so that they can both evolve more freely.

Read the whole post
See comments (3)

New Cromwell Documentation

Posted by KateVoss on 31 Oct 2017 (0)

Previously all of the Cromwell "documentation" was in the ReadMe on the Cromwell Github repo. We knew it, you knew it. The documentation needed a lot of work.

But not anymore!

Visit the new and much-improved Cromwell Documentation Site at

Take a look around and check out the new section of Cromwell Tutorials.

If you spot any errors or want to make any additions, the source of the documentation is in Github and pull requests are always welcome :wink: Finally if you have trouble finding anything use the Search in the top left-hand corner.

Thank you for your patience!

See comments (0)

Github “speaks” WDL

Posted by KateVoss on 17 Oct 2017 (0)

WDL is now an official GitHub language! This will make developing WDL even easier as you can read, write, and review pipelines on Github with beautiful syntax highlighting.

Check it out in a Broad Production pipeline WDL:

We also have syntax highlighting for authoring your WDLs in Sublime and vim, and full IDE support in all JetBrains IDEs (such as IntelliJ, pyCharm, CLion, etc).

See comments (0)

TL;DR: We are handing over governance of WDL to an open community-based organization called OpenWDL that will ensure WDL development is driven by and for the research community.

When WDL was created a few years ago, our mission was ostensibly to produce a pipelining system for the Broad's internal needs. But we knew at the time it would have to be much more than just a Broad-specific tool, because we had already decided we would share our analysis pipelines as widely as possible. The most immediate consequence was that whatever pipelining solution we adopted needed to be portable across not only our method development and production environments, but also across a variety of systems used in the research community. The other was that the scripting language we used needed to be as user-friendly as possible, to be at the very least readable by e.g. biomedical scientists without formal computational training (which happens to be my own background, so I have an especially soft spot for that constituency). Those were huge factors that led to our decision to create a new engine (Cromwell) and domain language (WDL itself, obviously ;)) rather than adopt an existing system. It also motivated us to provide as much help (via the forum) and documentation as we could manage to anyone who expressed interest, whether they were Broadies or not.

Fast-forward to the present day, where we're starting to see individuals and organizations throughout the world adopt WDL (sometimes independently of Cromwell) for their own pipelining needs. This is a incredibly gratifying but also rather daunting, because that means we now bear some degree of responsibility in ensuring that the people who adopt WDL are empowered to be successful in their pipelining endeavors.

Read the whole post
See comments (0)

Heads up! We're participating in an event organized by DNAstack in the context of the GA4GH meetings, just prior to the ASHG meeting in Orlando, FL in a few weeks. It's a two-hour evening event focused on open reproducible workflows, and will feature hands-on demos of WDL and Cromwell, as well as Dockstore and DNAstack's own platform, which uses WDL. The WDL and Cromwell section will be presented by our very own Jeff Gentry, who leads the Cromwell development team, accompanied by Cromwell product owner Kate Voss and GATK's resident scientific writer, Soo Hee Lee. We'll be running some GATK4 pipelines and generally having a good time nerding out about easy breezy reproducible workflows.

So join us if you're in town; register here now, as space is limited.

See comments (0)

29 Release Notes

Breaking Changes

  • Command line
    In preparation for supporting CWL scripts (yes, you read that right!), we have extensively revised the Command Line in Cromwell 29. For more details about the usage changes please see the README. And stay tuned to the WDL/Cromwell blog over the next couple of months for more news about CWL.

  • Request timeouts
    Cromwell now returns more specific 503 Service Unavailable error codes on request timeouts, rather than the more generic 500 Internal Server Error. The response for a request timeout will now be plain text, rather than a JSON format.

  • Metadata endpoint
    The response from the metadata endpoint can be quite large depending on your workflow. You can now opt-in to have Cromwell gzip your metadata file, in order to reduce file size, by sending the Accept-Encoding: gzip header. The default behavior now does not gzip encode responses.

  • Engine endpoints
    Previously the engine endpoints were available under /api/engine but now the endpoints are under /engine so they don't require authentication. Workflow endpoints are still available under /api/workflows. We also deprecated the setting api.routeUnwrapped as a part of this internal consistency effort.

  • Call caching diff
    We updated the response format of the callcaching/diff endpoint.

Other changes

  • Cromwell server
    When running in server mode, Cromwell now attempts to gracefully shutdown after receiving a SIGINT (Ctrl-C) or SIGTERM (kill) signal. This means that Cromwell waits for all pending database writes before exiting, as long as you include application.conf at the top of your config file. You can find detailed information about how to configure this feature in the Cromwell Wiki.

  • Concurrent jobs
    You can now limit the number of concurrent jobs for any backend. Previously this was only possible in some backend implementations. Please see the README for details.


  • Optional WDL variables
    Empty optional WDL values are now rendered as the null JSON value instead of the JSON string "null" in the metadata and output endpoints. You do not need to migrate previous workflows. Workflows run on Cromwell 28 and prior will still render empty values as "null".

  • Empty WDL variables
    Cromwell now accepts null JSON values in the input file and coerces them as an empty WDL value. WDL variables must be declared optional in order to be supplied with a null JSON value.


    "null_input_values.maybeString": null,
    "null_input_values.arrayOfMaybeInts": [1, 2, null, 4]


workflow null_input_values {
    String? maybeString
    Array[Int?] arrayOfMaybeInts
See comments (0)

Latest posts

- Recent posts

- Upcoming events

See Events calendar for full list and dates

- Recent events

See Events calendar for full list and dates

- Follow us on Twitter

WDL Dev Team


RT @DNAmlin: If you maintain @WDL_dev workflows, "miniwdl check" can speed up your code/test/debug loop by detecting mistakes and oversight…
6 Feb 19
RT @micknudsen: On my way home after a fabulous @gatk_dev workshop in Copenhagen. Looking forward to get started implementing #GATK4 in our…
1 Feb 19
RT @BlayzeStefaniak: Have Cromwell running on AWS Batch, very easy to work with WDL and get things working. Cool stuff!
11 Nov 18
RT @geoffjentry: Cool to see Cromwell and @WDL_dev on display at #sc18
11 Nov 18
RT @michaelhoffman: CY: Took 2 years, but today would only take 6 months due to availability of CWL, WDL, Cromwell, Kubernetes, Dockstore,…
17 Oct 18

- Our favorite tweets from others

Last week I coded with... 1) a group of 6th graders (Scratch) 2) an Xoogler (Python) 3) a Stanford EE major (Julia)…
10 Feb 19
@jdidion @DNAmlin @WDL_dev Probably 🙂 it’s part of why I’m super excited for miniwdl. It’s fantastic to see the com…
7 Feb 19
@gladrandomgraph @gatk_dev Many things. Too many to list in a tweet. You’ll have to wait until you bump into me in person.
1 Feb 19
@micknudsen @gatk_dev So what's cool about WDL? I'm curious :)
1 Feb 19
Featured now: Cromwell + WDL, a pipelining solution that features a flexible workflow management system. Vote it up…
18 Jan 19

See more of our favorite tweets...