November 2023
These features and Databricks platform improvements were released in November 2023.
Note
Releases are staged. Your Databricks workspace might not be updated until a week or more after the initial release date.
Access controls lists can no longer be disabled
November 29, 2023
The following settings are enabled on all Premium plan or above workspaces and can no longer be disabled from the workspace admin settings page or the Workspace Conf API:
Workspace access control: enables users to configure permissions to workspace objects.
Cluster, pool, jobs access control: enables users to configure permissions to clusters, pools, and jobs.
Cluster visibility control: prevents clusters from being visible to users that have no permissions on them.
Jobs visibility control: prevents jobs being from visible to users that have no permissions on them.
Workspace visibility control setting: prevents notebooks and folders from being from visible to users that have no permissions on them.
Access control settings have been enabled on new workspaces by default since March 2018 and visibility control settings have been enabled by default since December 2020.
For more information, see Access control overview.
AI assistive features are enabled by default
November 28, 2023
AI assistive features are now enabled by default for workspaces without the compliance security profile enabled. These include:
Admins can opt out of AI assistive features for all workspaces in their account or a subset of workspaces. See Q: How do I enable or disable Databricks Assistant?.
Databricks Runtime 14.2 is GA
November 22, 2023
Databricks Runtime 14.2 and Databricks Runtime 14.2 ML are now generally available.
See Databricks Runtime 14.2 and Databricks Runtime 14.2 for Machine Learning.
Deprecation of workspace libraries
November 15, 2023
Workspace libraries are now deprecated and creating new workspace libraries has been disabled. Existing workspace libraries continue to function. Databricks recommends adding libraries directly to clusters or jobs. See Libraries.
Delegate the ability to create a storage credential in Unity Catalog
November 15, 2023
You can now grant users permission to create a storage credential in a Unity Catalog metastore using the new CREATE STORAGE CREDENTIAL
privilege. See CREATE STORAGE CREDENTIAL.
Search for Databricks Marketplace listings using global workspace search
November 13, 2023
You can now find data sets, AI models, and Databricks Solution Accelerators in Databricks Marketplace using the global search bar in the Databricks workspace UI. See What is Databricks Marketplace? and Search for workspace objects.
Consume data products in Databricks Marketplace using external platforms
November 10, 2023
Consumers without a Unity Catalog-enabled Databricks workspace can now access data products in Databricks Marketplace. You can use Delta Sharing open sharing connectors to access Marketplace data using a number of common platforms, including Microsoft Power BI, Microsoft Excel, pandas, Apache Spark, and non-Unity Catalog Databricks workspaces. Only tabular data sets are available on external platforms (not Databricks notebooks or models, for example). See Request and access data products in Databricks Marketplace using external platforms.
Automatic enablement of Unity Catalog for new workspaces
November 8, 2023
Databricks has begun to enable Unity Catalog automatically for new workspaces. This removes the need for account admins to configure Unity Catalog after a workspace is created. Rollout will proceed gradually across accounts. See Automatic enablement of Unity Catalog.
Authentication using OAuth is GA
November 8, 2023
Both OAuth machine-to-machine (M2M) and user-to-machine (U2M) authentication are now GA.
OAuth M2M authentication uses Databricks service principals for authentication. See OAuth machine-to-machine (M2M) authentication. OAuth U2M authentication uses Databricks users for authentication. See See OAuth user-to-machine (U2M) authentication
Databricks recommends using OAuth M2M with Databricks client unified authentication to create and manage OAuth access tokens.
Databricks Runtime 14.2 (beta)
November 8, 2023
Databricks Runtime 14.2 and Databricks Runtime 14.2 ML are now available as beta releases.
See Databricks Runtime 14.2 and Databricks Runtime 14.2 for Machine Learning.
Databricks Marketplace includes Databricks Solution Accelerators
November 7, 2023
Databricks Marketplace now gives you access to Databricks Solution Accelerators, purpose-built guides that provide fully functional notebooks, best practices guidance, and sample data to get you set up quickly with many popular use cases on Databricks. See Get access to Databricks Solution Accelerators.
Lakehouse Federation adds support for Google BigQuery
November 3, 2023
You can now configure Lakehouse Federation to run federated queries with the BigQuery
connector type. The new BigQuery connector is supported on Pro and Serverless SQL warehouses and Databricks Runtime 13.3 LTS and above. See Lakehouse Federation for Google BigQuery.