September 2019

These features and Databricks platform improvements were released in September 2019.

Note

Releases are staged. Your Databricks account may not be updated until up to a week after the initial release date.

Databricks Runtime 5.2 support ends

September 30, 2019

Support for Databricks Runtime 5.2 ended on September 30. See Databricks support lifecycles.

Launch pool-backed automated clusters that use Databricks Light (Public Preview)

September 26 - October 1, 2019: Version 3.3

When we introduced Pool configuration reference in July, you couldn’t select Databricks Light as your runtime version when you configured a pool-backed cluster for an automated job. Now you can have both quick cluster start times and cost-efficient clusters!

Beta support for m5a and r5a instances

September 26 - October 1, 2019: Version 3.3

Databricks now provides beta support for the m5a and r5a series of EC2 instances, for use cases when you need to be economical.

pandas DataFrames now render in notebooks without scaling

September 12-17, 2019: Version 3.2

In Databricks notebooks, displayHTML was scaling some framed HTML content to fit the available width of the rendered notebook. While this behavior is desirable for images, it rendered wide pandas DataFrames poorly. But not anymore!

Python version selector display now dynamic

September 12-17, 2019: Version 3.2

When you select a Databricks runtime that doesn’t support Python 2 (like Databricks 6.0), the cluster creation page hides the Python version selector.

Databricks Runtime 6.0 Beta

September 12, 2019

Databricks Runtime 6.0 Beta brings many library upgrades and new features, including:

  • New Scala and Java APIs for Delta Lake DML commands, as well as the vacuum and history utility commands.

  • Enhanced DBFS FUSE v2 client for faster and more reliable reads and writes during model training.

  • Support for multiple matplotlib plots per notebook cell.

  • Update to Python 3.7, as well as updated numpy, pandas, matplotlib, and other libraries.

  • Sunset of Python 2 support.

For more information, see the complete Databricks Runtime 6.0 (EoS) release notes.