Databricks Asset Bundles feature release notes

This article contains details on releases of major new and upcoming Databricks Asset Bundles features and changes. Bundle features are released with the Databricks CLI. For a complete list of updates, see the Databricks CLI GitHub repository release notes.

Behavioral changes

The following Databricks Asset Bundles behavioral changes were released in a Databricks CLI release.

Warning for bundles deployed to Shared with conflicting permissions

October 23, 2024

This change was released with Databricks CLI 0.231.0.

A warning now occurs when a bundle is deployed to the /Workspace/Shared folder, which is accessible by all users, but the top-level permissions defined for the bundle conflicts with this. Either modify the permissions defined for the bundle or move the bundle out of the /Workspace/Shared folder.

To modify the permissions for a bundle, see permissions.

To move the bundle out of the /Workspace/Shared folder, change the workspace.root_path in the bundle configuration to point to a new location, then re-deploy the bundle. Remove the old bundle folder using the UI or by running databricks fs rm.

Workspace paths are now automatically prefixed

October 9, 2024

This change was released with Databricks CLI 0.230.0.

Workspace paths are now automatically prefixed with /Workspace. In addition, all usage of path strings such as /Workspace/${workspace.root_path}/... in bundle configuration are automatically replaced with ${workspace.root_path}/... and will generate a warning as part of bundle validate.

If you have specified a custom workspace.root_path, workspace.artifact_path, or workspace.file_path, Databricks Asset Bundles will automatically prefix it with /Workspace, but if you use any of these as variables (for example, my_config_path: /Workspace/${workspace.file_path}/config), you will need to update those entries to remove the /Workspace prefix to avoid the warning.

If you pass one of these as variables and prefix them in your code, you’ll need to update your code to not do this.

This change is required because originally when the workspace file system was rooted at / and home directories were under /Users, to access workspace paths through the Databricks REST API you would use these paths directly. To access workspace paths from your code, you could use the /Workspace file path and home directories were also available under /Workspace/Users. To avoid this duality of workspace paths, as well as the ambiguity between workspace paths and Unity Catalog /Volumes paths, all workspace paths are prefixed with /Workspace.

New features

The following Databricks Asset Bundles features and updates were released in a Databricks CLI release.

Volume support

December 5, 2024

This feature was released with Databricks CLI 0.236.0.

You can now create Unity Catalog volumes using Databricks Asset Bundles by defining a volume resource in your bundle configuration. See volume (Unity Catalog).

Variable lookup for notification destinations

December 5, 2024

This feature was released with Databricks CLI 0.236.0.

Variable lookups now support notification destinations. See Retrieve an object’s ID value.

Dashboard support

October 30, 2024

This feature was released with Databricks CLI 0.232.0.

AI/BI dashboards can now be managed in bundle configuration using the new dashboard resource type. You can generate dashboard configuration using bundle generate dashboard and keep your local and remote version of a dashboard in sync with the --force and --watch options. See dashboard and Generate a bundle configuration file.

For information about AI/BI dashboards, see Dashboards.

New bundle summary command

October 23, 2024

This feature was released with Databricks CLI 0.231.0.

A new bundle summary command has been added. This command outputs a summary of a bundle’s identity and resources, including deep links for resources so that you can easily navigate to the resource in the Databricks workspace. See Output a bundle summary.

Support for creating all-purpose clusters

October 1, 2024

This feature was released with Databricks CLI 0.229.0.

You can now create all-purpose clusters in your bundle configuration using the new cluster resource type. See cluster.

New Delta Live Tables pipeline recreation prompt and –auto-approve flag

September 5, 2024

This feature was released with Databricks CLI 0.228.0.

A prompt will now occur if there are bundle configuration changes that would lead to a recreation of a Delta Live Tables pipeline. You can skip the prompt by specifying the --auto-approve flag.

Added support for Unity Catalog schema resource

July 31, 2024

This feature was released with Databricks CLI 0.225.0.

Unity Catalog schemas can now be defined in your bundle configuration. See schema (Unity Catalog).

Bundle commands in web terminal

July 4, 2024

This feature was released with Databricks CLI 0.223.0.

You can now interact with bundles using the Databricks CLI from the web terminal. For information about using the web terminal, see Run shell commands in Databricks web terminal.

Added support for complex variables

June 26, 2024

This feature was released with Databricks CLI v0.222.0.

Databricks Asset Bundles now support complex variables. See Define a complex variable.

Databricks Asset Bundles is generally available

April 23, 2024

With the Databricks CLI 0.218.0 release, Databricks Asset Bundles are now generally available.