September 2022

These features and Databricks platform improvements were released in September 2022.

Note

Releases are staged. Your Databricks account might not be updated until a week or more after the initial release date.

New reference solution for natural language processing

September 29, 2022

The documentation now includes a new reference solution for natural language processing. See Natural language processing.

More regions for Unity Catalog

September 27, 2022

Unity Catalog is now generally available in the us-west-1 region.

Protect and control access to some types of encrypted data with customer-managed keys (GA)

September 23, 2022

Customer-managed keys for encryption, which includes both managed services and workspace storage, is now generally available. See Customer-managed keys for encryption.

Compliance security profile workspaces support i4i instance types

September 22, 2022

Workspaces with the compliance security profile enabled now support i4i instances.

Audit logs now include events for web terminal

September 22, 2022

Audit logs now include events related to web terminal. There are log entries for the start of a session and the close of a session. See Audit log reference.

Audit logs now include events for managing credentials for Git repos

September 22, 2022

Audit logs now include events related to management of credentials for Git repos. See Audit log reference.

Select cluster policies directly in the Delta Live Tables UI

September 12-19, 2022

You can now select a cluster policy in the Delta Live Tables UI when you create or edit a pipeline. Previously, setting the cluster policy for a pipeline required editing the pipeline’s JSON settings.

New data trasformation card on workspace landing pages

September 8, 2022

The new Transform data card on the Data Science & Engineering and SQL landing pages displays data transformation options such as Delta Live Tables and dbt Core.

Orchestrate Databricks SQL tasks in your Databricks workflows (Public Preview)

September 12, 2022

You can include Databricks SQL tasks in your Databricks workflows, including running SQL queries, refreshing SQL dashboards, and refreshing SQL alerts. The SQL task requires a Databricks SQL enabled workspace and a serverless SQL warehouse. For more information on creating, running, and scheduling a workflow that includes the SQL task, see Schedule and orchestrate workflows.

Delta cache is now disk cache

September 8, 2022

Databricks caching functionality previously referred to as the “Delta cache” has been renamed “disk caching” with no changes to existing behavior. See Optimize performance with caching on Databricks.

Capture and view lineage data with Unity Catalog (Public Preview)

September 9, 2022

Unity Catalog now supports capturing and viewing the lineage for your queries that run in a Databricks cluster or SQL warehouse. Lingeage data is captured across all languages and for all notebooks, jobs and dashboards related to the query. Lineage can be visualized in Catalog Explorer in near real-time or retrieved with the Databricks REST API. For more information about enabling and using lineage with Unity Catalog, see Capture and view data lineage using Unity Catalog.

Search for tables using Catalog Explorer (Public Preview)

September 8, 2022

You can now use Catalog Explorer to search for tables that are registered in Unity Catalog. You can search on table names, table comments, column names, and column comments. Only tables that you have permission to see appear in search results. For details, see Search for workspace objects.

View and organize assets in the workspace browser across personas

September 7-21, 2022

You can now view and organize workspace assets such as notebooks, libraries, experiments, queries, and dashboards in the workspace browser across the Data Science & Engineering, Databricks Mosaic AI, and Databricks SQL personas.

New Databricks SQL queries, dashboards, and alerts are visible in the workspace browser. To view and organize existing queries, dashboards, and articles in the workspace browser, users (or admins) must migrate them into the workspace browser.

Databricks Runtime 11.2 and 11.2 ML are GA

September 7, 2022

Databricks Runtime 11.2 and 11.2 ML are now generally available.

See Databricks Runtime 11.2 (EoS) and Databricks Runtime 11.2 for Machine Learning (EoS).

Support for AWS Graviton instances is GA

September 1, 2022

Announcing the general availability of Databricks support for AWS Graviton2-based Amazon Elastic Compute Cloud (Amazon EC2) instances.

The Graviton processors are custom designed and optimized by AWS to deliver the best price-performance for cloud workloads running in Amazon EC2. When used with Photon, the high performance Databricks query engine, Graviton2-based Amazon EC2 instances can deliver up to 3x better price-performance than comparable Amazon EC2 instances for your data lakehouse workloads.