April 2022

These features and Databricks platform improvements were released in April 2022.

Note

Releases are staged. Your Databricks account may not be updated until a week or more after the initial release date.

Use tags to better manage your Databricks jobs

April 21-26, 2022

You can now add tags to your Databricks jobs, allowing you to easily track and search for jobs based on one or more custom keys or key-value pairs. Because tags also propagate to job clusters, you can use the tags with your cluster monitoring.

Databricks Runtime 10.0 series support ends

April 20, 2022

Support for Databricks Runtime 10.0 and Databricks Runtime 10.0 for Machine Learning ended on April 20. See Databricks support lifecycles.

Get a visual overview of your job runs with the new jobs matrix view

April 19-27, 2022

You can now easily visualize runs of a Databricks job with the new matrix view in the jobs user interface. The matrix view supplements the existing table view, and provides an overview of job and task run details, including the start time, duration, and status of each run. See View runs for a job.

Save time and resources when your Databricks job runs are unsuccessful

April 19-25, 2022

When a job run fails, the new jobs repair and re-run feature allows you to re-run only the subset of unsuccessful tasks and any dependent tasks. Because successful tasks are not re-run, this feature reduces the time and resources required to recover from unsuccessful job runs. See Re-run failed and skipped tasks.

View the run history for job tasks

April 19-25, 2022

You can now view the run history for each task that is part of a job that orchestrates multiple tasks. See View task run history.

Assign a new cluster in the jobs UI when the Single User access no longer exists

April 18-25, 2022: Version 3.70

This release fixes an issue that removed the Swap cluster button from the Databricks jobs user interface when the Single User access is unavailable. You can now assign a new cluster to a job in the UI when the configured cluster is unavailable, for example, because of a network change.

Databricks Runtime 10.5 (Beta)

April 15, 2022

Databricks Runtime 10.5, 10.5 Photon, and 10.5 ML are now available as Beta releases.

See the full release notes at Databricks Runtime 10.5 (EoS) and Databricks Runtime 10.5 for Machine Learning (EoS).

Feature Store now supports publishing features to AWS DynamoDB

April 7, 2022

You can now publish offline feature tables to Amazon DynamoDB for low-latency online lookup. See Publish features to an online store and Publish time series features to an online store.

The Delta Live Tables UI is enhanced to disable unauthorized actions

April 4-11, 2022: Version 3.69

The Delta Live Tables user interface is updated to disable pipeline actions you are not authorized to use. For example, if you have only CAN_VIEW permission, the start, stop, and delete buttons are disabled. Previously, selecting an unauthorized action resulted in the action failing.

Mosaic AutoML is generally available

April 4-11, 2022: Version 3.69

Mosaic AutoML is generally available in Databricks Runtime 10.4 LTS ML and above.

Use datasets from Unity Catalog with AutoML

April 4-11, 2022: Version 3.69

You can now select datasets from Unity Catalog in the AutoML UI. You must be the designated single user of a cluster in Single User access mode.

Delta Live Tables is GA on AWS and Azure, and in Public Preview on GCP

April 5, 2022

Databricks is pleased to announce the general availability of Delta Live Tables on AWS and Microsoft Azure, and Public Preview on Google Cloud. Delta Live Tables is the first ETL framework that uses a simple, declarative approach to building reliable data pipelines. Delta Live Tables automatically manages your infrastructure at scale so data analysts and engineers can spend less time on tooling and focus on getting value from data. See What is Delta Live Tables?.

Delta Live Tables SQL interface: non-breaking change to table names

April 5, 2022

This release introduces a non-breaking change to the syntax used to name tables in the Delta Live Tables SQL interface. These changes align the semantics and syntax of the SQL interface with upcoming enhancements to the Delta Live Tables platform. To support these future enhancements, we are making the following changes:

  • Incremental live tables are renamed to streaming live tables. Streaming live tables inherit the semantics of incremental live tables. Delta Live Tables will continue to support INCREMENTAL as a deprecated keyword for backward compatibility.

  • Complete live tables will be referred to as simply live tables. Live tables will guarantee that at the end of a pipeline update, the contents of the resulting table is exactly equal to the result of the specified transformation on the current inputs.