April 2022
These features and Databricks platform improvements were released in April 2022.
Note
Releases are staged. Your Databricks account may not be updated until a week or more after the initial release date.
Use tags to better manage your Databricks jobs
April 21-26, 2022
You can now add tags to your Databricks jobs, allowing you to easily track and search for jobs based on one or more custom keys or key-value pairs. Because tags also propagate to job clusters, you can use the tags with your cluster monitoring.
Databricks Runtime 10.0 series support ends
April 20, 2022
Support for Databricks Runtime 10.0 and Databricks Runtime 10.0 for Machine Learning ended on April 20. See Databricks support lifecycles.
Get a visual overview of your job runs with the new jobs matrix view
April 19-27, 2022
You can now easily visualize runs of a Databricks job with the new matrix view in the jobs user interface. The matrix view supplements the existing table view, and provides an overview of job and task run details, including the start time, duration, and status of each run. See View runs for a job.
Save time and resources when your Databricks job runs are unsuccessful
April 19-25, 2022
When a job run fails, the new jobs repair and re-run feature allows you to re-run only the subset of unsuccessful tasks and any dependent tasks. Because successful tasks are not re-run, this feature reduces the time and resources required to recover from unsuccessful job runs. See Re-run failed and skipped tasks.
View the run history for job tasks
April 19-25, 2022
You can now view the run history for each task that is part of a job that orchestrates multiple tasks. See View task run history.
Assign a new cluster in the jobs UI when the Single User access no longer exists
April 18-25, 2022: Version 3.70
This release fixes an issue that removed the Swap cluster button from the Databricks jobs user interface when the Single User access is unavailable. You can now assign a new cluster to a job in the UI when the configured cluster is unavailable, for example, because of a network change.
Databricks Runtime 10.5 (Beta)
April 15, 2022
Databricks Runtime 10.5, 10.5 Photon, and 10.5 ML are now available as Beta releases.
See the full release notes at Databricks Runtime 10.5 (EoS) and Databricks Runtime 10.5 for Machine Learning (EoS).
Feature Store now supports publishing features to AWS DynamoDB
April 7, 2022
You can now publish offline feature tables to Amazon DynamoDB for low-latency online lookup. See Publish features to an online store and Publish time series features to an online store.
The Delta Live Tables UI is enhanced to disable unauthorized actions
April 4-11, 2022: Version 3.69
The Delta Live Tables user interface is updated to disable pipeline actions you are not authorized to use. For example, if you have only CAN_VIEW
permission, the start, stop, and delete buttons are disabled. Previously, selecting an unauthorized action resulted in the action failing.
Databricks AutoML is generally available
April 4-11, 2022: Version 3.69
Databricks AutoML is generally available in Databricks Runtime 10.4 LTS ML and above.
Use datasets from Unity Catalog with AutoML
April 4-11, 2022: Version 3.69
You can now select datasets from Unity Catalog in the AutoML UI. You must be the designated single user of a cluster in Single User access mode.
Delta Live Tables is GA on AWS and Azure, and in Public Preview on GCP
April 5, 2022
Databricks is pleased to announce the general availability of Delta Live Tables on AWS and Microsoft Azure, and Public Preview on Google Cloud. Delta Live Tables is the first ETL framework that uses a simple, declarative approach to building reliable data pipelines. Delta Live Tables automatically manages your infrastructure at scale so data analysts and engineers can spend less time on tooling and focus on getting value from data. See What is Delta Live Tables?.
Delta Live Tables SQL interface: non-breaking change to table names
April 5, 2022
This release introduces a non-breaking change to the syntax used to name tables in the Delta Live Tables SQL interface. These changes align the semantics and syntax of the SQL interface with upcoming enhancements to the Delta Live Tables platform. To support these future enhancements, we are making the following changes:
Incremental live tables are renamed to streaming live tables. Streaming live tables inherit the semantics of incremental live tables. Delta Live Tables will continue to support
INCREMENTAL
as a deprecated keyword for backward compatibility.Complete live tables will be referred to as simply live tables. Live tables will guarantee that at the end of a pipeline update, the contents of the resulting table is exactly equal to the result of the specified transformation on the current inputs.