October 2022

These features and Databricks platform improvements were released in October 2022.

Note

Releases are staged. Your Databricks account might not be updated until a week or more after the initial release date.

GA: Repos support for non-notebook files

October 31, 2022

Databricks Repos support for non-notebook files is GA. See Work with notebooks and other files in Databricks Repos.

Deploy models for streaming inference with Delta Live Tables notebooks

October 27, 2022

You can now automatically generate a notebook to use the model for streaming inference via Delta Live Tables. For details, see Configure streaming inference using Delta Live Tables.

Connect to Fivetran from the add data UI

October 27 - November 3, 2022

You can now connect to Fivetran from the add data UI by following the same steps you follow to connect to Fivetran from the Partner Connect portal. For more information, see Add data with Partner Connect.

Databricks SQL Driver for Node.js is Generally Available

October 27, 2022

Databricks now provides full support for the Databricks SQL Driver for Node.js. The new version number is 1.0.0.

Changes include:

  • DBSQLClient.openSession now uses the latest protocol version by default.

  • The direct results feature is now available for all IOperation methods that support it. It is on by default.

  • Direct results are now enabled by default. If maxRows is omitted, it defaults to 100,000. To disable direct results, set maxRows to null.

  • The runAsync option is now available for all operations that support it.

  • Logging functionality is added for client-side logging, and a new optional logger param is added to the DBSQLClient constructor.

See the full changelog.

Partner Connect supports connecting to erwin Data Modeler by Quest

October 25-28, 2022

You can now easily create a connection between erwin Data Modeler by Quest and your Databricks workspace using Partner Connect. For more information, see Connect to erwin Data Modeler using Partner Connect.

Enforce user isolation cluster types on a workspace

October 24, 2022

There is a new workspace setting called Enforce User Isolation. Admins can enable this setting to prevent creating or starting a cluster with a “No isolation shared” cluster access type or equivalent legacy cluster type.

Databricks Runtime 11.3 LTS and 11.3 LTS ML are GA

October 19, 2022

Databricks Runtime 11.3 LTS and 11.3 LTS ML are now generally available.

See Databricks Runtime 11.3 LTS and Databricks Runtime 11.3 LTS for Machine Learning.

Format Python code in notebooks (Public Preview)

Oct 17-21, 2022

You can now format Python cells in a notebook, making it easier to read and maintain code and enforce coding standards across notebooks. For details, see Format code cells.

Update a failed workspace with Databricks-managed VPC to use a customer-managed VPC

October 17, 2022

When errors happen during workspace deployment, the workspace’s state becomes failed instead of running. For a failed workspace, you can now convert a workspace with Databricks-managed VPC to use a customer-managed VPC. You cannot change the VPC type of a Running workspace.

If you make this change, Databricks recommends changing your credentials configuration to use an access policy specific to Databricks workspaces with customer-managed VPCs, since it’s a smaller set of required permissions.

Personal Compute cluster policy is available by default to all users

October 11, 2022

The Personal Compute policy is available on all workspaces by default. The policy allows users to easily create single-machine compute resources for their individual use so they can start running workloads immediately, minimizing compute management overhead.

Admins can manage the Personal Compute policy for their workspaces. See Manage the Personal Compute cluster policy

Add data UI provides a central UI for loading data to Databricks

October 11, 2022

The add data UI provides access to common data sources configurations and file upload UIs for all Databricks personas. See Load data using the add data UI.

Upload data UI unifies experience for small file upload to Delta Lake

October 11, 2022

Databricks now supports uploading small files to Delta Lake using a UI from all personas. See Upload data to Databricks.

Partner Connect supports connecting to Hevo Data

October 11, 2022

You can now easily create a connection between Hevo Data and your Databricks workspace using Partner Connect. For more information, see Connect to Hevo Data using Partner Connect.

Enable admin protection for No Isolation Shared clusters

October 7, 2022

Account admins can prevent internal credentials from being automatically generated for Databricks workspace admins on No Isolation Shared clusters.

SQL persona integrated with new search experience

October 6, 2022

Global search is now available in the SQL persona. You can now search for queries, dashboards, and alerts with global search in any persona by name using the new search experience. You can also search for content within a query and see a preview of matching content.

Databricks is a FedRAMP® Authorized Cloud Service Offering (CSO) at the moderate impact Level

October 4, 2022

Databricks is a FedRAMP® Authorized Cloud Service Offering (CSO) at the moderate impact Level in the AWS US East-1 and US West-2 (commercial) regions. See FedRAMP Moderate compliance controls.

Serverless SQL warehouses are available in regions eu-central-1 and us-east-2

October 4, 2022

Serverless SQL warehouses are now available in regions eu-central-1 and us-east-2. To create serverless SQL warehouses, an admin must first enable them for your workspace.

Privilege inheritance in now supported in Unity Catalog

October 3-6, 2022

Unity Catalog now supports privilege inheritance. This means that granting a privilege on a catalog or schema automatically grants the privilege to all current and future objects within the catalog or schema. For more information about the Unity Catalog privilege model see Unity Catalog privileges and securable objects.

If you created your Unity Catalog metastore during the public preview (before August 25, 2022), you can upgrade to Privilege Model version 1.0. to take advantage of privilege inheritance. Privileges have been renamed in Privilege Model version 1.0, however Databricks will continue to support grants expressed using the old privilege model and automatically map them to the equivalent grant. Databricks recommends that you upgrade existing code that performs grants to reference the updated privilege model.

Top navigation bar in the UI

October 3-6, 2022

The following items have moved from the side bar to the top bar in the UI:

  • User Settings

  • Admin Console

  • Manage Account

  • Help

  • Workspace switcher

  • Search

Databricks Runtime 11.3 (Beta)

October 3, 2022

Databricks Runtime 11.3 is now available as a Beta release.

See Databricks Runtime 11.3 LTS and Databricks Runtime 11.3 LTS for Machine Learning.