March 2024

These features and Databricks platform improvements were released in March 2024.

Note

Releases are staged. Your Databricks account might not be updated until a week or more after the initial release date.

DBRX Base and DBRX Instruct are now available in Model Serving

March 27, 2024

Mosaic AI Model Serving now supports DBRX Base and DBRX Instruct, state-of-the-art mixture of experts (MoE) language models trained by Databricks. Both models are part of Foundation Model APIs, where DBRX Instruct is a fine-tuned model available in pay-per-token serving endpoint regions and DBRX Base is a pretrained model available in limited provisioned throughput serving endpoint regions. See Use Foundation Model APIs.

Model Serving is HIPAA compliant in all regions

March 27, 2024

Mosaic AI Model Serving is HIPAA compliant in all regions where the service is generally available. See Region availability for Model Serving supported regions.

Provisioned throughput in Foundation Model APIs is GA and HIPAA compliant

March 27, 2024

Databricks Foundation Model APIs now offers provisioned throughput model serving in general availability. As part of general availability, Foundation Model APIs provisioned throughput workloads are HIPAA compliant. See Provisioned throughput Foundation Model APIs.

MLflow now enforces quota limits for experiments and runs

March 27, 2024

MLflow now imposes a quota limit on the number of total parameters, tags, and metric steps for all existing and new runs, and the number of total runs for all existing and new experiments. See Resource limits.

The Jobs UI is updated to better manage jobs deployed by Databricks Asset Bundles

March 26, 2024

Because modifications to Databricks jobs deployed by Databricks Asset Bundles should be applied only by updating the bundle configuration, these jobs are, by default, read-only when viewed in the Jobs UI. Previously, by default, these jobs could be modified in the UI and cause unintentional gaps between the configuration in the UI and the bundle configuration. However, an option is provided for cases where you must make emergency changes to a job. See View and run a job created with a Databricks Asset Bundle.

Google Cloud Vertex AI supported as model provider for external models

March 25, 2024

External models in Mosaic AI Model Serving now supports models provided by Google Cloud Vertex AI. See Model providers for external models.

Accessing resources from serving endpoints using instance profiles is GA

March 25, 2024

Add an instance profile to your model serving endpoint to access any AWS resources from the model that are permitted by the instance profile. See Add an instance profile to a model serving endpoint.

Interactive notebook debugging

March 22, 2024

Databricks now supports interactive Python debugging directly in the notebook for clusters in Single user or No isolation shared access mode. With interactive debugging, you can step through code line by line and view variable values to discover and fix errors in code. For more information, see Debug notebooks.

Self-service sign-up for private exchange providers in Marketplace

March 22, 2024

If you want to publish only private exchange listings in Databricks Marketplace, you can now sign up using a self-service workflow. To publish public listings, you still need to apply through the Databricks partner portal. See Sign up to be a Databricks Marketplace provider.

Databricks Runtime 15.0 is GA

March 22, 2024

Databricks Runtime 15.0 and Databricks Runtime 15.0 ML are now generally available.

See Databricks Runtime 15.0 (EoS) and Databricks Runtime 15.0 for Machine Learning (EoS).

Databricks Repos changed to Git folders

March 21, 2024

The former Databricks Repos feature is now called “Git folders”. If you have existing Repos, they are preserved under the same file system paths. See What happened to Databricks Repos?.

Databricks Runtime 14.1 and 14.2 series support extended

March 20, 2024

Support for Databricks Runtime 14.1 and Databricks Runtime 14.1 for Machine Learning has been extended from April 11, 2024 to October 1, 2024.

Support for Databricks Runtime 14.2 and Databricks Runtime 14.2 for Machine Learning has been extended from May 22, 2024 to October 1, 2024.

See All supported Databricks Runtime releases.

Databricks ODBC driver 2.8.0

March 19, 2024

We have released version 2.8.0 of the Databricks ODBC driver (download). This release adds the following new features and enhancements:

  • Support for JWT assertion as the client credentials for OAuth.

  • Token renew support. For token passthrough authentication, you can now renew your token.

  • Update third party libraries: Arrow 15.0.0 (previously 9.0.0)(on Windows), libcURL 8.6.0 (previously 8.4.0), Zlib 1.3.1 (previously 1.2.13).

  • Accept Undetermined Revocation support.

This release also resolves the following issue in 2.8.0:

  • When UseNativeQuery is set to 1 for a cluster later than DBR 11, the connector returns an incorrect column number after SQLPrepare.

For more information, see the release notes or the Installation and Configuration Guide in the installation package.

SQL warehouses for notebooks is GA

March 15, 2024

SQL warehouses for notebooks, now generally available, allow you to take advantage of fully managed, instant, and scalable compute for your SQL workloads within the rich, collaborative authoring environment of a notebook. For details, see Use a notebook with a SQL warehouse.

Delegate the ability to view an object’s metadata in Unity Catalog (Public Preview)

March 15, 2024

You can now grant users, service principals, and account groups permission to view a Unity Catalog object’s metadata using the new BROWSE privilege. This enables users to discover data without having read access to the data. A user can view an object’s metadata using Catalog Explorer, the schema browser, search results, the lineage graph, information_schema, and the REST API.

The BROWSE privilege can be granted on a catalog or on an external location. Granting BROWSE on a catalog automatically grants BROWSE to all current and future objects within the catalog. A user with the BROWSE privilege does not require USE CATALOG on the parent catalog or USE SCHEMA on the parent schema to view an object’s metadata.

See BROWSE.

Databricks Runtime 15.0 (Beta)

March 11, 2024

Databricks Runtime 15.0 and Databricks Runtime 15.0 ML are now available as Beta releases.

Databricks Runtime 14.0 series support ends

March 11, 2024

Support for Databricks Runtime 14.0 and Databricks Runtime 14.0 for Machine Learning ended on March 11. See Databricks support lifecycles.

New computation for sys.path and CWD in Repos

March 8, 2024

We’ve updated how sys.path and the current working directory (CWD) are computed for Python notebooks and files in Repos. There are no functional changes. For sys.path information see the sys.path spec.

Feature Serving is GA

March 7, 2024

With Databricks Feature Serving, data in the Databricks platform can be made available to models or applications deployed outside of Databricks. Like Mosaic AI Model Serving endpoints, Feature Serving endpoints automatically scale to adjust to real-time traffic and provide a high-availability, low-latency service at any scale. For details, see What is Databricks Feature Serving?.

You can use Databricks Feature Serving to serve structured data for retrieval augmented generation (RAG) applications. For an example notebook, see online tables with RAG applications.

Predictive optimization available in more regions

March 5, 2024

Predictive optimization is now available in the following regions, in addition to those it was already available in:

  • ap-southeast-2

  • eu-central-1

  • eu-west-1

  • us-east-2

For a complete list of supported regions, see Features with limited regional availability. For more information, see Predictive optimization for Unity Catalog managed tables.