March 2025
These features and Databricks platform improvements were released in March 2025.
Releases are staged. Your Databricks account might not be updated until a week or more after the initial release date.
Install custom dependencies in Unity Catalog Python UDFs (Public Preview)
March 20, 2025
Extend the functionality of Python UDFs beyond the Databricks Runtime environment by defining custom dependencies for external libraries. See Extend UDFs using custom dependencies.
Web terminal activity is now considered compute activity
March 20, 2025
Compute resources configured to automatically terminate after a period of inactivity will continue to run if activity is detected in a Databricks web terminal connected to that compute, including through a notebook. See Automatic termination.
Budget policies renamed to serverless budget policies
March 19, 2025
Serverless budget policies (Public Preview) have been renamed from budget policies. See Attribute usage with serverless budget policies.
Delta Sharing supports renaming tables
March 19, 2025
Delta Sharing now supports renaming shared tables. If a provider renames a shared table, the table on the recipient side automatically reconciles without erroring. See Update shares.
OIDC federation for Databricks-to-open Delta Sharing (Public Preview)
March 17, 2025
Data providers in Databricks can now enable recipients to use their own identity provider (IdP) to establish access to Delta Sharing shares created in Databricks. This authentication flow uses Open ID Connect (OIDC) token federation to exchange JSON web tokens (JWT) generated by the recipient's IdP for short-lived OAuth tokens generated by Databricks. This Databricks-to-open sharing authentication flow is for recipients who do not have access to a Unity Catalog-enabled Databricks workspace.
OIDC federation is an alternative to using long-lived Databricks-issued bearer tokens to connect non-Databricks recipients to providers. It enables fine-grained access control, supports MFA, and reduces security risks by eliminating the need for recipients to manage and secure shared credentials.
Databricks-to-open sharing with OIDC federation supports user-to-machine (U2M) authentication for recipients on Power BI and Tableau, and machine-to-machine (M2M) OAuth Client Credentials authentication from Python client apps registered in a recipient's IdP.
This feature is in Public Preview. Contact your Databricks representative to request access.
The compute system tables are enabled by default
March 17, 2025
The system.compute schema is now enabled by default in all Unity Catalog workspaces. See Compute system tables reference.
Hive metastore federation is generally available
March 14, 2025
Hive metastore federation enables you to use Unity Catalog to access and govern data that is registered in a Hive metastore. This includes both externally-managed Hive metastores and legacy internal Databricks Hive Hive metastores.
GA adds support for:
- Hive metastore 3.1 (in addition to versions 0.13 and 2.3, which were supported in the Public Preview).
- Postgres and SQL Server (in addition to MySQL, which was supported in the Public Preview).
- Applying row filters and column-level masks to Hive tables that are accessed using Hive metastore federation.
In addition, this release includes preview support for:
- Reading Delta shallow clones that are defined in the Hive metastore (Public Preview).
- Reading DBFS-mount-backed tables across workspaces (Private Preview: contact your Databricks account representative to sign up).
See Hive metastore federation: enable Unity Catalog to govern tables registered in a Hive metastore.
Compute logs can now be delivered to volumes (Public Preview)
March 13, 2025
You can now deliver a compute resource's driver, worker, and event logs to a re:[UC] volume. See Compute log delivery.
Download as Excel is now supported in notebooks connected to serverless compute
March 13, 2025
For notebooks connected to serverless compute, you can now download tabular cell results as an Excel file. Download as Excel is also supported for notebooks connected to SQL warehouses. See Download results.
Bind storage credentials, service credentials, and external locations to specific workspaces (GA)
March 12, 2025
The ability to bind storage credentials, service credentials, and external locations to specific workspaces is now generally available. Workspace binding prevents access to those objects from other workspaces. This feature is especially helpful if you use workspaces to isolate user data access, for example, if you have separate production and development workspaces or a dedicated workspace for handling sensitive data.
For more information, see (Optional) Assign an external location to specific workspaces, (Optional) Assign a storage credential to specific workspaces and (Optional) Assign a service credential to specific workspaces.
Databricks Runtime 16.3 (Beta)
March 12, 2025
Databricks Runtime 16.3 and Databricks Runtime 16.3 ML are now available as Beta releases.
See Databricks Runtime 16.3 (Beta) and Databricks Runtime 16.3 for Machine Learning (Beta).
Databricks JDBC Driver (OSS) is GA
March 11, 2025
The Databricks JDBC Driver (OSS) is now generally available (GA) and will soon be open source. Download the Databricks JDBC Driver (OSS) from the Maven Repository.
Monitor and revoke personal access tokens in your account (Public Preview)
March 11, 2025
Token report is now in Public Preview. Token report enables account admins to monitor and revoke personal access tokens (PATs) in the account console. Databricks recommends you use OAuth access tokens instead of PATs for greater security and convenience. See Monitor and revoke personal access tokens in the account.
Genie Conversation API is now in Public Preview
March 11, 2025
The Genie Conversations API is now in Public Preview. You can now ask questions, retrieve generated SQL and query results, or get the details associated with a Genie space using the Databricks REST API. To learn how to use the API, see the Genie REST API reference.
Perform batch inference using AI Functions (Public Preview)
March 10, 2025
Mosaic AI now supports batch inference using AI Functions. This functionality is in Public Preview. See the following guides for more details:
ServiceNow connector (Public Preview)
March 10, 2025
You can now ingest from ServiceNow using Databricks Lakeflow Connect. See ServiceNow connector overview. To participate in the preview, contact your Databricks account team.
External models now supports OpenAI-compatible LLMs
March 10, 2025
External models in Mosaic AI Model Serving now supports a custom provider option. You can use this option to specify alternate providers that are not explicitly listed as a model provider or models behind custom proxies that are OpenAI API compatible. See Model providers.
Delta Sharing support for query optimizer statistics is GA
March 4, 2025
Data recipients accessing Delta tables shared via Delta Sharing can now benefit from optimizer statistics to improve query performance. The data provider must either enable predictive optimization or manually run the ANALYZE command on the shared Delta table.
Performance improvements include the following:
-
Same-region Databricks-to-Databricks queries are 11% faster.
-
Cross-region Databricks-to-Databricks queries are 28% faster.
See Predictive optimization for Unity Catalog managed tables and ANALYZE TABLE.
AI Gateway now supports fallbacks for external models
March 4, 2025
You can now enable fallbacks on your model serving endpoints using AI Gateway. If your request to an external model fails, you can specify that the request be redirected to other external models served on that endpoint. See Configure AI Gateway using the UI.
Control the location of Databricks Assistant with the new docking feature
March 4, 2025
You now have more control over the placement of the Databricks Assistant pane and the ability to minimize it with the new docking feature.