FireCloud is now powered by Terra! -- STARTING MAY 1st, 2019 THIS WEBPAGE WILL NO LONGER BE UPDATED.
From now on, please visit the Terra Help Center for documentation, tutorials, roadmap and feature announcements.
Want to talk to a human? Email the helpdesk, post feature requests or chat with peers in the community forum.
FIRECLOUD | Issue #2420

Error Message: insufficient_data_written
closed | Created 2017-08-15 | Last updated 2019-11-08| Posted by knoblett

fc-bug firecloud

This problem was discovered here.

This problem is due to size limitations. When you submit a job to FireCloud, Cromwell can batch your job with any other jobs that are ready to be created (from other users, other workflows, other billing projects). If the cumulative content of the batched requests is too large, it leads to batch creation failure and job failure, resulting in an insufficient_data_written error.

Currently there are no good workarounds for this error. You can stand up your own instance of Cromwell to avoid having other jobs batched with yours. This is unfortunately difficult, but you can read the Cromwell readme to learn how to set up configuration options and such. Alternatively, you can keep trying to re-run your job to try to not be batched with other large jobs.

It is possible that your own batch contribution could be too large. If you have too many or too long file names (for inputs or outputs), you can possibly run out of space even if you are batched with smaller jobs. If you think that applies to your workflow, try making the file names and/or variable names shorter.

Our Cromwell developers are working on a fix for this problem now, which will be incorporated into Cromwell version 29.1. When this version is available, we will be able to update FireCloud.

UPDATE: This error was fixed in an update that went out on 8/31/17. If you are still experiencing this problem, please let us know.

Return to top