We have updated some of the libraries and packages that are pre-loaded upon creation of a notebook cluster on FireCloud. These changes will affect creation of NEW clusters ONLY; existing, paused clusters will not be affected.
The aforementioned updates, listed below, are expected to be compatible with existing libraries and packages with the exception of the Hail 0.1 to 0.2 upgrade; this change will be a breaking change*.
Version changes for libraries and packages:
*Hail 0.2 has a new file format and cannot read Hail 0. You can learn more about the Hail 0.1 --> 0.2 changes here. If you need to create a new cluster with the Hail 0.1 version, please reach out to FireCloud support and we can help you to do so!
In the future, we plan to make it possible to enable access to prior versions of the notebook kernels when we roll out new upgrades to ensure long-term reproducibility.
sendReceive doesn’t support chunked responses, try sendTo instead. This has been updated to better explain what is happening with new error message
The response payload was over 20MB and cannot be processed
In the near future, we will update a number of the libraries and packages that are preloaded on notebook clusters on both FireCloud and Terra (beta). The updates are listed below. We expect all of these updates to be backwards compatible, with the exception of the upgrade from Hail 0.1 to 0.2 which is a breaking change. However it is possible that even the minor version updates could break dependencies in some of your notebooks, so we have developed a rollout plan to minimize disruption as follows.
The updates Version changes for libraries and packages:
Hail 0.2 has a new file format and cannot read Hail 0.1. You can learn more about Hail 0.1 -> 0.2 changes here.
If you have any concerns, please comment below as soon as possible. Our goal is to equip you with the tools you need to get your work done, so the more we know, the better we can serve you.
Thank you for your patience as we work to improve the platform!
We are planning to make a few changes to our APIs, which are outlined below. If you do not use the FireCloud APIs, or call only the Swagger UI (api.firecloud.org) your work will not be affected and you do not need to read any further. If you do use the APIs, please keep reading, as these changes may break some of your API calls if you do not modify them accordingly.
We are happy to schedule separate conversations with you if you would like further detail or assistance to determine what modifications make sense for your specific project. Please email firstname.lastname@example.org.
By Robert Title, Engineering Manager, Data Sciences Platform at the Broad Institute
We are excited to announce that we have substantially improved the way you interact with Jupyter Notebooks in FireCloud. We hope this will increase your productivity and empower you to collaborate more effectively. These changes are publicly available as of today, Sep 12, 2018, and can be accessed in the Notebooks tab of your FireCloud workspace. Read on for a more detailed description of what is changing, and why it is better.
By Moran Cabili, product manager, Data Sciences Platform at the Broad Institute
We heard from many of you --both new FireCloud users and experienced WDL pipeline developers-- that you need to be able to quickly test that a WDL workflow can be run successfully on FireCloud. Until now, you were required to reference an entity in the workspace data model, which took extra effort and tended to confuse newcomers. We are happy to announce that this speed bump has been eliminated; you can now bypass the data model and even upload a JSON of inputs to get your WDL up and running in record time.