Amazon
Web Services
Microsoft Azure
Google Cloud PlatformOctober 2022
These features and Databricks platform improvements were released in October 2022.
Note
Releases are staged. Your Databricks account might not be updated until a week or more after the initial release date.
GA: Repos support for non-notebook files
October 31, 2022
Databricks Repos support for non-notebook files is GA. See What are workspace files?.
Deploy models for streaming inference with Delta Live Tables notebooks
October 27, 2022
You can now automatically generate a notebook to use the model for streaming inference via Delta Live Tables. For details, see Configure streaming inference using Delta Live Tables.
Connect to Fivetran from the add data UI
October 27 - November 3, 2022
You can now connect to Fivetran from the add data UI by following the same steps you follow to connect to Fivetran from the Partner Connect portal. For more information, see Add data with Partner Connect.
Databricks SQL Driver for Node.js is Generally Available
October 27, 2022
Databricks now provides full support for the Databricks SQL Driver for Node.js. The new version number is 1.0.0.
Changes include:
DBSQLClient.openSessionnow uses the latest protocol version by default.The direct results feature is now available for all
IOperationmethods that support it. It is on by default.Direct results are now enabled by default. If
maxRowsis omitted, it defaults to 100,000. To disable direct results, setmaxRowstonull.The
runAsyncoption is now available for all operations that support it.Logging functionality is added for client-side logging, and a new optional
loggerparam is added to theDBSQLClientconstructor.
See the full changelog.
Partner Connect supports connecting to erwin Data Modeler by Quest
October 25-28, 2022
You can now easily create a connection between erwin Data Modeler by Quest and your Databricks workspace using Partner Connect. For more information, see Connect to erwin Data Modeler using Partner Connect.
Enforce user isolation cluster types on a workspace
October 24, 2022
There is a new workspace setting called Enforce User Isolation. Admins can enable this setting to prevent creating or starting a cluster with a “No isolation shared” cluster access type or equivalent legacy cluster type.
Databricks Runtime 11.3 LTS and 11.3 LTS ML are GA
October 19, 2022
Databricks Runtime 11.3 LTS and 11.3 LTS ML are now generally available.
Note
LTS means this version is under long-term support. See Databricks Runtime LTS version lifecycle.
See Databricks Runtime 11.3 LTS and Databricks Runtime 11.3 LTS for Machine Learning.
Format Python code in notebooks (Public Preview)
Oct 17-21, 2022
You can now format Python cells in a notebook, making it easier to read and maintain code and enforce coding standards across notebooks. For details, see Format code cells.
IP access lists no longer block PrivateLink traffic
October 17, 2022
Starting October 17, 2022, the IP access lists feature applies only to requests over the internet (public IP addresses). This change may affect some workspaces with both AWS PrivateLink private connectivity and also use IP access lists.
There is no change to the security posture for requests from public IP addresses. Private IP addresses that were allowed by the previous rules continue to be allowed. However, private IP addresses that were blocked by the previous rules are now allowed if allowed by your workspace’s PrivateLink configuration.
Review your IP access lists for workspaces with PrivateLink enabled. Look for private IPs of VPC endpoints that are in the block lists, or are omitted from allow lists. You can continue to restrict private access over AWS PrivateLink using Private Endpoint access control. For private IP addresses that you want to block, use AWS Network Firewall. If you rely on IP access lists to restrict the PrivateLink connection to a set of VPC endpoints, Databricks recommends changing your private access settings object to use the ENDPOINT access level.
AWS PrivateLink support is now generally available
October 17, 2022
AWS PrivateLink provides private connectivity from AWS VPCs and on-premises networks to AWS services without exposing the traffic to the public network. AWS PrivateLink support is now generally available. See Enable AWS PrivateLink.
Improvements to AWS PrivateLink support for updating workspaces
October 17, 2022
You can now make some AWS PrivateLink workspace updates by using the admin console or Account API instead of contacting Databricks to make the update:
Upgrade a workspace’s PrivateLink support to add support for front-end, back-end, or both types of connectivity. Add a private access settings object (UI or API). However, you cannot remove (downgrade) any existing front-end or back-end PrivateLink support on a workspace.
Add or update a workspace’s registered VPC endpoints by creating a new network configuration object with registered VPC endpoints and then update the workspace’s network configuration (UI or API).
Update a failed workspace with Databricks-managed VPC to use a customer-managed VPC
October 17, 2022
When errors happen during workspace deployment, the workspace’s state becomes failed instead of running. For a failed workspace, you can now convert a workspace with Databricks-managed VPC to use a customer-managed VPC. You cannot change the VPC type of a Running workspace.
If you make this change, Databricks recommends changing your credentials configuration to use an access policy specific to Databricks workspaces with customer-managed VPCs, since it’s a smaller set of required permissions.
Personal Compute cluster policy is available by default to all users
October 11, 2022
The Personal Compute policy is available on all workspaces by default. The policy allows users to easily create single-machine compute resources for their individual use so they can start running workloads immediately, minimizing compute management overhead.
Admins can manage the Personal Compute policy for their workspaces. See Manage the Personal Compute policy
Add data UI provides a central UI for loading data to Databricks
October 11, 2022
The add data UI provides access to common data sources configurations and file upload UIs for all Databricks personas. See Load data using the add data UI.
Create or modify table from file upload page unifies experience for small file upload to Delta Lake
October 11, 2022
Databricks now supports uploading small files to Delta Lake using a UI from all personas. See Create or modify a table using file upload.
Partner Connect supports connecting to Hevo Data
October 11, 2022
You can now easily create a connection between Hevo Data and your Databricks workspace using Partner Connect. For more information, see Connect to Hevo Data using Partner Connect.
SQL persona integrated with new search experience
October 6, 2022
Global search is now available in the SQL persona. You can now search for queries, dashboards, and alerts with global search in any persona by name using the new search experience. You can also search for content within a query and see a preview of matching content.
Databricks is a FedRAMP® Authorized Cloud Service Offering (CSO) at the moderate impact Level
October 4, 2022
Databricks is a FedRAMP® Authorized Cloud Service Offering (CSO) at the moderate impact Level in the AWS US East-1 and US West-2 (commercial) regions. See FedRAMP Moderate compliance controls.
Serverless SQL warehouses are available in regions eu-central-1 and us-east-2
October 4, 2022
Serverless SQL warehouses are now available in regions eu-central-1 and us-east-2. To create serverless SQL warehouses, an admin must first enable them for your workspace.
Privilege inheritance in now supported in Unity Catalog
October 3-6, 2022
Unity Catalog now supports privilege inheritance. This means that granting a privilege on a catalog or schema automatically grants the privilege to all current and future objects within the catalog or schema. For more information about the Unity Catalog privilege model see Unity Catalog privileges and securable objects.
If you created your Unity Catalog metastore during the public preview (before August 25, 2022), you can upgrade to Privilege Model version 1.0. to take advantage of privilege inheritance. Privileges have been renamed in Privilege Model version 1.0, however Databricks will continue to support grants expressed using the old privilege model and automatically map them to the equivalent grant. Databricks recommends that you upgrade existing code that performs grants to reference the updated privilege model.
Databricks Runtime 11.3 (Beta)
October 3, 2022
Databricks Runtime 11.3 is now available as a Beta release.
See Databricks Runtime 11.3 LTS and Databricks Runtime 11.3 LTS for Machine Learning.
The account console is available in multiple languages
October 1, 2022
The account console is now available in the following languages:
English
Deutsch
Español
Français
Italiano
日本語
한국어
Português (Brasil)
Português (Portugal)
To change the account console language, click your username in the console’s top navigation bar, select User Settings, and go to the Language settings tab.
