Latest posts
 


In case you haven't met us yet, Beri and I (Sushma) are part of the Frontline Support Team here, aka the real humans behind the FireCloud and Terra forums and helpdesk. We're on deck every day answering questions and helping troubleshoot problems, so we have plenty of opportunity to get real friendly with both platforms - it comes with the territory. What you may not know is that we actually use FireCloud and Terra to create and maintain many of the Featured Workspaces, which on any given day might involve managing data, writing WDL workflows or hammering out some R or Python code in a Jupyter notebook. That's why we'll always be super sympathetic if you reach out to us; we've literally felt the same sink-back-into-your-chair frustrations as you at some point. On the flipside, we figure that when we feel joy or relief at some newly released feature that makes our work easier, that means you probably will too!

So we were talking about the upcoming transition from FireCloud to Terra, and we figured that we should share our unanimously favorite features, mighty or small, that didn’t exist in FireCloud but have really revolutionized our ability to get things done in Terra.


Read the whole post
See comments (0)



Summary

We're planning to upgrade one of the main servers at the heart of FireCloud in order to unlock some performance enhancements ahead of a training event that is taking place later this week. We plan to perform this operation between 6:30-7am EST on March 20, 2019 and expect it should take about 5 to 10 minutes to complete.

Impact

  • For the duration of the upgrade process, it won't be possible to log into the application, and it's very likely that any API calls will either timeout or fail.
  • Actively running workflows should not be impacted but querying of status will timeout or fail.

For more information

If you would like to be notified of all service incidents or upcoming scheduled maintenance, click Follow on this page

See comments (0)



Heads up if you use the TARGET and/or TCGA data workspaces: we're planning an update that will change the paths to the data files. This will disable access to the original files in any clones of the original workspaces that retain the old paths, and it will affect call caching in any new clones of the original workspaces that you create after the update rolls out in a few weeks.

Read on to understand what this change entails and why it's an important improvement that is worth making.


Read the whole post
See comments (5)


Release Notes: March 2019

Posted by SChaluvadi on 13 Mar 2019 (0)


March 12, 2019

Notebooks

  1. We have upgraded Hail to 0.2.10. This requires a cluster rebuild to take effect.
  2. We now set a lifecycle policy on clusters' staging buckets after the cluster is deleted. This helps control storage costs in users' billing projects.
See comments (0)



Last month we were excited to announce that FireCloud is getting a big upgrade, including a new and improved user interface called Terra. As a reminder, going forward we'll use the name “Terra” to refer to the overall platform that you use to get your work done, while “FireCloud” will refer to the NCI-sponsored project page that features TCGA and TARGET data. You can already access Terra here if you have a FireCloud account, and you may have received an email invitation to check it out.

Today we're following up on that announcement with some important news: the "classic" FireCloud portal will be turned off on May 1st. At that point, all traffic to FireCloud will be automatically redirected to Terra.


Read the whole post
See comments (0)



We are excited to announce that we’re rolling out a new and improved user interface that addresses much of the feedback you’ve shared with us over the past three years. Using the new interface will not require you to change anything and should make doing research easier than before.


Read the whole post
See comments (0)



February 27, 2019

FireCloud

  1. The Terms of Service API has updated its library dependencies, to proactively address security vulnerability warnings.

Notebooks

  1. A bug fix to address an issue where the token refresh that runs as a Jupyter extension was not running Lab, which was causing user tokens to expire and resulting in 401 errors. The workaround for this was to have a Terra notebook (not lab) open in a different tab while using Jupyter Lab which will no longer be needed.

Cromwell

  1. A new version of Cromwell, version 36.1, has been released with details listed here. Cromwell version 36.1 is an interim to Cromwell 37 containing many of the same features while small bug fixes are being resolved in v37.

Read the whole post
See comments (0)


Release Notes: January 2019

Posted by KateN on 4 Jan 2019 (0)


January 17, 2019

Notebooks

We have updated some of the libraries and packages that are pre-loaded upon creation of a notebook cluster on FireCloud. These changes will affect creation of NEW clusters ONLY; existing, paused clusters will not be affected.

The aforementioned updates, listed below, are expected to be compatible with existing libraries and packages with the exception of the Hail 0.1 to 0.2 upgrade; this change will be a breaking change*.

Version changes for libraries and packages:

  1. hail 0.1 --> 0.2
  2. python 3.4 --> 3.6
  3. R 3.4 --> 3.5
  4. pandas 0.22.0 --> 0.23.4
  5. numpy 1.15.0 --> 1.15.2
  6. google-api-core 1.3.0 --> 1.5.0
  7. google-cloud-bigquery 1.4.0 --> 1.6.0
  8. bokeh 1.0.0 --> 1.0.1
  9. table of contents extension for Notebooks will be installed by default

*Hail 0.2 has a new file format and cannot read Hail 0. You can learn more about the Hail 0.1 --> 0.2 changes here. If you need to create a new cluster with the Hail 0.1 version, please reach out to FireCloud support and we can help you to do so!

In the future, we plan to make it possible to enable access to prior versions of the notebook kernels when we roll out new upgrades to ensure long-term reproducibility.


Read the whole post
See comments (0)



December 4, 2018

FireCloud UI

  1. Improved error message: Users would see a sendReceive doesn’t support chunked responses, try sendTo instead. This has been updated to better explain what is happening with new error message The response payload was over 20MB and cannot be processed
  2. The warning banner that says “Your offline credentials are missing or out-of-date…” is now red instead of yellow to draw attention. New users who accept the Free Credits offer will see browsers refresh after accepting so all the cached data can be reloaded.
  3. For a single workflow’s details, the FireCloud UI will deter cost calculation until it renders workflow details. Since cost calculation can be slow, this allows the UI to serve the workflow details faster.
  4. The FireCloud UI requests smaller, streamlined payload from the API when rendering workflow details. This allows the UI to serve workflow details faster and eliminates some – but not all – errors caused when the workflow details API response is too large.
  5. The FireCloud UI now only requests data to populate workflow’s timing diagram on demand once a user clicks to show the diagram whereas the UI used to load the data when the page was rendered. This allows the UI to serve the workflow details faster and eliminates some- but not all – errors caused when the workflow details API response is too large.
  6. The FireCloud UI will now show an error message in certain situations on the workspace list and the submission list within a workspace. Before, the error message was suppressed and, in the case of a backend error, the UI rendered an empty page.
See comments (0)



In the near future, we will update a number of the libraries and packages that are preloaded on notebook clusters on both FireCloud and Terra (beta). The updates are listed below. We expect all of these updates to be backwards compatible, with the exception of the upgrade from Hail 0.1 to 0.2 which is a breaking change. However it is possible that even the minor version updates could break dependencies in some of your notebooks, so we have developed a rollout plan to minimize disruption as follows.

  • For FireCloud, we are taking a conservative approach and soliciting feedback from all users before scheduling the update. Please comment below if you think any of the updates listed below will be disruptive for your work. These updates only affect the creation of new clusters; existing clusters that are paused will not be affected by this change.
  • For Terra (new beta UI for FireCloud), the updates listed below will be released next week (planned for Dec 4). Because these updates will affect all clusters on Terra, we recommend you save a version of your current notebook and results before that date. After the release, re-run your analysis results using the using the old vs new packages and compare results against your saved version. Let us know if you find any problematic discrepancies that you need help with.
  • In the future, we plan to make it possible to enable access to prior versions of the notebook kernels to ensure long-term reproducibility and avoid this becoming a recurring issue when we roll out kernel upgrades.

The updates Version changes for libraries and packages:

  • hail 0.1 -> 0.2
  • python 3.4 -> 3.6
  • R 3.4 -> 3.5
  • pandas 0.22.0 -> 0.23.4
  • numpy 1.15.0 -> 1.15.2
  • google-api-core 1.3.0 -> 1.5.0
  • google-cloud-bigquery 1.4.0 -> 1.6.0
  • bokeh 1.0.0 -> 1.0.1

Hail 0.2 has a new file format and cannot read Hail 0.1. You can learn more about Hail 0.1 -> 0.2 changes here.

If you have any concerns, please comment below as soon as possible. Our goal is to equip you with the tools you need to get your work done, so the more we know, the better we can serve you.

Thank you for your patience as we work to improve the platform!

See comments (0)



Latest posts
 

- Recent posts



- Follow us on Twitter

FireCloud

@BroadFireCloud

@jamesaeddy Going to phone a friend on that one — @geoffjentry care to explain?
19 Mar 19
Heads up if you use the TARGET and/or TCGA data workspaces: the file paths to TARGET and TCGA data are going to cha… https://t.co/rq2pfpWlcz
19 Mar 19
RT @lynnlangit: really love the sizing notebooks feature in https://t.co/5NCSbM0mYz @ProjectJupyter #Bioinformatics https://t.co/iR1UuPIy7Y
15 Mar 19
To @harvardmed researchers and friends: Come meet https://t.co/Fn1HB3gWD7, the platform that powers FireCloud and m… https://t.co/kIJBPwwlff
13 Mar 19

- Our favorite tweets from others

The macaque genome isn't finished so it has over 200K contigs. I call them the Rhesus pieces.
15 Mar 19
It's been years in the making, but 500k structural variants (SVs) from 15k genomes in #gnomAD are out 🎉today🎉! Pre… https://t.co/5mYCvan5ou
15 Mar 19
@fdmts @tanyacash21 and R. Munshi @broadinstitute are beginning their session on hype vs. reality in cloud capabili… https://t.co/26Pdlh7IMw
12 Mar 19
watching "Genomics TV" https://t.co/M6NuEyKGuz #bioinformatics #GATK @broadinstitute https://t.co/jHteePNKcp
5 Mar 19
“Running Containers — for Biologists” #Bioinformatics #Docker https://t.co/lxeiFoiJlx
4 Mar 19

See more of our favorite tweets...