April 2025
These features and Databricks platform improvements were released in April 2025.
Releases are staged. Your Databricks account might not be updated until a week or more after the initial release date.
DLT maintenance tasks managed by predictive optimization
April 28, 2025
All streaming tables and materialized views are moving to use predictive optimization for maintenance tasks. This will roll out over the next several months, depending on region and account. For information about predictive optimization, see Predictive optimization for Unity Catalog managed tables.
Claude 3.7 Sonnet now available in EU regions
April 25, 2025
Claude 3.7 Sonnet is now available in EU regions where Foundation Model APIs pay-per-token or AI Functions workloads are supported. See Model serving feature availability.
Google Analytics raw data connector (Public Preview)
April 25, 2025
You can now ingest raw, event-level data from Google Analytics using Databricks Lakeflow Connect and Google BigQuery. See Set up Google Analytics 4 and Google BigQuery for Databricks ingestion.
Share streaming tables and materialized views to all recipients using Delta Sharing (Public Preview)
April 23, 2025
You can now use Delta Sharing to share streaming tables and materialized views to all recipients.
See Create and manage shares for Delta Sharing and Read shared streaming tables and materialized views.
Use vector search indexes to prototype tool-calling generative AI agents in AI Playground
April 23, 2025
Rather than creating custom Unity Catalog functions with a vector search index, you can now add the vector search index to your gen AI agents directly in AI Playground. The agent will be able to use the vector search index as a tool to help it respond to queries. See Prototype tool-calling agents in AI Playground.
AI Gateway now supports custom model serving endpoints
April 21, 2025
Mosaic AI Gateway now supports custom models endpoints.
You can now enable the following governance and monitoring features on your custom model serving endpoints:
- Permission and rate limiting to control who has access and how much access.
- Payload logging to monitor and audit data being sent to model APIs using inference tables.
- Usage tracking to monitor operational usage on endpoints and associated costs using system tables.
Strict enforcement of row-level security and column masking policies in Delta Sharing
April 21, 2025
Delta Sharing now consistently enforces row-level security and column masking policies applied to tables a shared data asset is dependent on, of whether those policies were applied before or after the data asset was shared. Recipients may experience differences in query behavior when accessing shared data that is depending on tables with row-level security or column masking policies. This ensures that data access aligns with the provider’s intended security controls at all times.
See Filter sensitive table data using row filters and column masks.
Multi-file editor for ETL pipeline development in DLT
April 21, 2025
You can now use the new multi-file editor in DLT to develop and debug ETL pipelines. The multi-file editor shows a pipeline as a set of files in the pipeline assets browser. You can edit the files and control the configuration of the pipeline and which files to include in one location. See Develop and debug ETL pipelines with the multi-file editor in DLT.
Run a subset of tasks within a job
April 21, 2025
You can now run a subset of the tasks when manually triggering a job. See Run a job with different settings.
AWS PrivateLink now supported in us-west-1 region
April 18, 2025
AWS PrivateLink is now supported in the us-west-1 region for Databricks workspaces. See Enable private connectivity using AWS PrivateLink.
Use Llama 4 Maverick on serving endpoints optimized for AI Functions
April 18, 2025
Llama 4 Maverick is now available as a Databricks-hosted foundation model. You can use it for batch inference endpoints optimized for AI Functions.
Run code in Databricks Assistant on serverless compute
April 18, 2025
Databricks Assistant can now run any code it generates in the right panel on serverless compute. See Run code in Assistant on serverless compute.
AI Builder: Model specialization for custom gen AI text agents is in Beta
April 17, 2025
AI Builder provides a simple, no-code approach to build and optimize domain-specific, high-quality AI agent systems for common AI use cases. In Beta, AI Builder supports model specialization for custom text-based tasks, such as summarization, classification, text transformation, and content generation.
See Use AI Builder: Model specialization to create a gen AI agent for text.
Python type error highlighting
April 14, 2025
Python code in notebooks and file editors can highlight type errors for non-existent attributes, missing arguments, and mismatched arguments. See Python type error highlighting.
Reference SQL output in downstream tasks of a job
April 14, 2025
You can now use dynamic values to reference the output of a SQL task in downstream tasks in the same job. For each tasks can iterate over the rows of data in the output.
See What is a dynamic value reference?.
Batch Unity Catalog Python UDFs (Public Preview)
April 14, 2025
Unity Catalog Batch Python UDFs extend the capabilities of Unity Catalog UDFs by allowing you to write Python code to operate on batches of data, significantly improving efficiency by reducing the overhead associated with row-by-row UDFs. Batch Python UDFs support service credentials to access external cloud services. See Batch Python User-defined functions (UDFs) in Unity Catalog.
Access UDF context information using TaskContext
April 14, 2025
The TaskContext PySpark API now allows you to retrieve context information—such as user identity and cluster tags—while running Batch Unity Catalog Python UDFs or PySpark UDFs. This feature lets you pass user-specific details, like identity, to authenticate external services within UDFs. See Get task context in a UDF.
Meta Llama 4 Maverick is available using Foundation Model APIs pay-per-token
April 5, 2025
Mosaic AI Model Serving now supports Meta Llama 4 Maverick. You can access and use Llama 4 Maverick on Foundation Model API pay-per-token endpoints.
Databricks Assistant provides doc citations and follow-up suggestions
April 3, 2025
In answer to prompts and queries, Assistant now provides suggestions for next steps and citations from documentation. See Get follow-up suggestions or answers from documentation.
Salesforce ingestion and Workday Reports connectors are GA
April 2, 2025
The managed Salesforce ingestion and Workday Reports connectors in Lakeflow Connect are now generally available.
The compute system tables are now generally available (GA)
April 1, 2025
The compute system tables clusters, node_types, and node_timeline are now generally available. These tables provide insights into the all-purpose compute, jobs compute, and DLT pipelines in your account.
For a reference of these tables, see Compute system tables reference.
Databricks multi-factor authentication (MFA) is now available (Public Preview)
April 1, 2025
Account admins can now enable multi-factor authentication (MFA) directly in Databricks, allowing them to either recommend or require MFA for users at login.
This feature is available only for accounts that do not use single sign-on (SSO). If your account uses SSO, configure MFA through your organization’s identity provider.
For more information on Databricks MFA, see Configure multi-factor authentication.
The BROWSE privilege is GA
April 1, 2025
The BROWSE privilege is now generally available. The BROWSE privilege allows you to grant users, service principals, and account groups permission to view a Unity Catalog object’s metadata. This enables users to discover data without having read access to the data. A user can view an object’s metadata using Catalog Explorer, the schema browser, search results, the lineage graph, information_schema, and the REST API.
See BROWSE.