March 2022
These features and Databricks platform improvements were released in March 2022.
Note
Releases are staged. Your Databricks account may not be updated until a week or more after the initial release date.
Better performance and cost for your Delta Live Tables pipelines with Databricks enhanced autoscaling
March 31, 2022
Databricks enhanced autoscaling improves the existing cluster autoscaling functionality by adding streaming workload optimizations and improved support for batch workloads. Enhanced autoscaling improvements provide more efficient cluster utilization, reduced resource usage, and lower cost. See Optimize the cluster utilization of Delta Live Tables pipelines with enhanced autoscaling.
Files in Repos enabled by default in new workspaces
March 30, 2022
The Files in Repos feature is now enabled by default in newly created workspaces.
Databricks Feature Store is generally available
March 22, 2022
Databricks Feature Store is generally available. Databricks Feature Store enables feature sharing and discovery across your organization and also ensures consistent feature computation code for model training and inference.
RStudio Workbench bug fix
March 21-28, 2022: Version 3.68
The latest RStudio Workbench depends on cookies when you use a license file. These cookies were not allowlisted for proxying. This issue has been fixed.
New workspace language options
March 21, 2022
Workspaces are now available in the following languages:
Español
Português (Brasil)
Databricks Runtime 10.4 LTS and 10.4 LTS ML are GA; 10.4 Photon is Public Preview
March 18, 2022
Databricks Runtime 10.4 LTS and Databricks Runtime 10.4 LTS ML are now generally available. Databricks Runtime 10.4 Photon is in Public Preview.
Note
LTS means this version is under long-term support. See Databricks Runtime LTS version lifecycle.
See Databricks Runtime 10.4 LTS and Databricks Runtime 10.4 LTS for Machine Learning.
Unity Catalog is available in Public Preview
March 10, 2022
Unity Catalog (Preview) is a fine-grained governance solution for data and AI on the Databricks lakehouse.
Unity Catalog helps simplify security and governance of your data with the following key features:
Define once, secure everywhere: Unity Catalog offers a single place to administer data access policies that apply across all workspaces and personas.
Standards-compliant security model: Unity Catalog’s security model is based on standard ANSI SQL, and allows administrators to grant permissions at the level of catalogs, databases (also called schemas), tables, and views in their existing data lake using familiar syntax.
Built-in auditing: Unity Catalog automatically captures user-level audit logs that record access to your data.
You can access data in Unity Catalog in the same ways you access data that is local to a workspace: from a Data Science & Engineering or Databricks Mosaic AI cluster, a SQL warehouse, the Databricks SQL editor, or using the Unity Catalog REST API or the Unity Catalog command-line interface (CLI).
For more information, see What is Unity Catalog?. To participate in the Unity Catalog public preview, contact your Databricks account team.
Delta Sharing is available in Public Preview
March 10, 2022
Delta Sharing (Preview) is an open protocol developed by Databricks for secure data sharing with other organizations regardless of which computing platforms they use. A Databricks user, called a “data provider”, can use Delta Sharing to share data in a Unity Catalog metastore with a person or group outside of their organization, called a “data recipient”. Data recipients can immediately begin working with the latest version of the shared data. For a full list of connectors and information about how to use them, see the Delta Sharing project documentation. When Delta Sharing is enabled on a Unity Catalog metastore, Unity Catalog runs a Delta Sharing server.
Delta Sharing must be enabled by an account admin to use the feature.
For more information, see What is Delta Sharing?. To participate in the Delta Sharing public preview, contact your Databricks account team.
Enhanced access control for Delta Live Tables pipelines
March 9, 2022
You now have more control over access to your Delta Live Tables pipelines, including the owner of a pipeline, who can view or manage the pipeline, and who can run the pipeline. See Delta Live Tables pipeline ACLs.
Test Delta Live Tables preview functionality with the new channel
setting (Public Preview)
March 7-March 14, 2022: Version 3.67
You can test upcoming features in your Delta Live Tables pipelines with the new channel
setting. To test your pipeline with upcoming features, set channel
to preview
. To switch back to the current release, set channel
to current
.
Improved error handling for Delta Live Tables Python functions (Public Preview)
March 7-March 14, 2022: Version 3.67
This release adds type checking to validate that Delta Live Tables Python functions use the string
data type when setting names and values for table properties. Previously, setting table properties using non-string values could cause your pipeline to fail without an error message indicating the underlying cause.
Improvements to Databricks Repos
March 7-14, 2022: Version 3.67
The following improvements have been made to Databricks Repos.
Text search and replace in files in repos. The file editor now has File and Edit menus that provide search, replace, and replace all functionality.
Previously, Databricks recommended that the total number of notebooks and files in a repo not exceed 2000. That limit has been increased to 5000.
Audit logging for cluster policy changes
March 7-14, 2022: Version 3.67
When audit logging is enabled, an audit event is now logged when you create, update, or delete a cluster policy, or when you update user permissions for a cluster policy. For more information, see Audit log reference.
Databricks Runtime 10.4 (Beta)
March 4, 2022
Databricks Runtime 10.4, 10.4 Photon, and 10.4 ML are now available as Beta releases.
See the full release notes at Databricks Runtime 10.4 LTS and Databricks Runtime 10.4 LTS for Machine Learning.