September 2022
These features and Databricks platform improvements were released in September 2022.
Note
Releases are staged. Your Databricks account might not be updated until a week or more after the initial release date.
New reference solution for natural language processing
September 29, 2022
The documentation now includes a new reference solution for natural language processing. See Natural language processing.
More regions for Unity Catalog
September 27, 2022
Unity Catalog is now generally available in the us-west-1 region.
Protect and control access to some types of encrypted data with customer-managed keys (GA)
September 23, 2022
Customer-managed keys for encryption, which includes both managed services and workspace storage, is now generally available. See Customer-managed keys for encryption.
Compliance security profile workspaces support i4i
instance types
September 22, 2022
Workspaces with the compliance security profile enabled now support i4i
instances.
Audit logs now include events for web terminal
September 22, 2022
Audit logs now include events related to web terminal. There are log entries for the start of a session and the close of a session. See Audit log reference.
Audit logs now include events for managing credentials for Git repos
September 22, 2022
Audit logs now include events related to management of credentials for Git repos. See Audit log reference.
Select cluster policies directly in the Delta Live Tables UI
September 12-19, 2022
You can now select a cluster policy in the Delta Live Tables UI when you create or edit a pipeline. Previously, setting the cluster policy for a pipeline required editing the pipeline’s JSON settings.
New data trasformation card on workspace landing pages
September 8, 2022
The new Transform data card on the Data Science & Engineering and SQL landing pages displays data transformation options such as Delta Live Tables and dbt Core.
Orchestrate Databricks SQL tasks in your Databricks workflows (Public Preview)
September 12, 2022
You can include Databricks SQL tasks in your Databricks workflows, including running SQL queries, refreshing SQL dashboards, and refreshing SQL alerts. The SQL task requires a Databricks SQL enabled workspace and a serverless SQL warehouse. For more information on creating, running, and scheduling a workflow that includes the SQL task, see Schedule and orchestrate workflows.
Delta cache is now disk cache
September 8, 2022
Databricks caching functionality previously referred to as the “Delta cache” has been renamed “disk caching” with no changes to existing behavior. See Optimize performance with caching on Databricks.
Capture and view lineage data with Unity Catalog (Public Preview)
September 9, 2022
Unity Catalog now supports capturing and viewing the lineage for your queries that run in a Databricks cluster or SQL warehouse. Lingeage data is captured across all languages and for all notebooks, jobs and dashboards related to the query. Lineage can be visualized in Catalog Explorer in near real-time or retrieved with the Databricks REST API. For more information about enabling and using lineage with Unity Catalog, see Capture and view data lineage using Unity Catalog.
Search for tables using Catalog Explorer (Public Preview)
September 8, 2022
You can now use Catalog Explorer to search for tables that are registered in Unity Catalog. You can search on table names, table comments, column names, and column comments. Only tables that you have permission to see appear in search results. For details, see Search for workspace objects.
View and organize assets in the workspace browser across personas
September 7-21, 2022
You can now view and organize workspace assets such as notebooks, libraries, experiments, queries, and dashboards in the workspace browser across the Data Science & Engineering, Databricks Mosaic AI, and Databricks SQL personas.
New Databricks SQL queries, dashboards, and alerts are visible in the workspace browser. To view and organize existing queries, dashboards, and articles in the workspace browser, users (or admins) must migrate them into the workspace browser.
Databricks Runtime 11.2 and 11.2 ML are GA
September 7, 2022
Databricks Runtime 11.2 and 11.2 ML are now generally available.
See Databricks Runtime 11.2 (EoS) and Databricks Runtime 11.2 for Machine Learning (EoS).
Support for AWS Graviton instances is GA
September 1, 2022
Announcing the general availability of Databricks support for AWS Graviton2-based Amazon Elastic Compute Cloud (Amazon EC2) instances.
The Graviton processors are custom designed and optimized by AWS to deliver the best price-performance for cloud workloads running in Amazon EC2. When used with Photon, the high performance Databricks query engine, Graviton2-based Amazon EC2 instances can deliver up to 3x better price-performance than comparable Amazon EC2 instances for your data lakehouse workloads.