Databricks Asset Bundles feature release notes
This article contains details on releases of major new and upcoming Databricks Asset Bundles features and changes. Bundle features are released with the Databricks CLI. For a complete list of updates, see the Databricks CLI GitHub repository release notes.
Behavioral changes
The following Databricks Asset Bundles behavioral changes were released in a Databricks CLI release.
Workspace paths are now automatically prefixed
October 9, 2024
This change was released with Databricks CLI 0.230.0.
Workspace paths are now automatically prefixed with /Workspace
. In addition, all usage of path strings such as /Workspace/${workspace.root_path}/...
in bundle configuration are automatically replaced with ${workspace.root_path}/...
and will generate a warning as part of bundle validate
.
If you have specified a custom workspace.root_path
, workspace.artifact_path
, or workspace.file_path
, Databricks Asset Bundles will automatically prefix it with /Workspace
, but if you use any of these as variables (for example, my_config_path: /Workspace/${workspace.file_path}/config
), you will need to update those entries to remove the /Workspace
prefix to avoid the warning.
If you pass one of these as variables and prefix them in your code, you’ll need to update your code to not do this.
This change is required because originally when the workspace file system was rooted at /
and home directories were under /Users
, to access workspace paths through the Databricks REST API you would use these paths directly. To access workspace paths from your code, you could use the /Workspace
file path and home directories were also available under /Workspace/Users
. To avoid this duality of workspace paths, as well as the ambiguity between workspace paths and Unity Catalog /Volumes
paths, all workspace paths are prefixed with /Workspace
.
New features
The following Databricks Asset Bundles features and updates were released in a Databricks CLI release.
Support for creating all-purpose clusters
October 1, 2024
This feature was released with Databricks CLI 0.229.0.
You can now create all-purpose clusters in your bundle configuration using the new clusters
resource type. See clusters.
New Delta Live Tables pipeline recreation prompt and –auto-approve flag
September 5, 2024
This feature was released with Databricks CLI 0.228.0.
A prompt will now occur if there are bundle configuration changes that would lead to a recreation of a Delta Live Tables pipeline. You can skip the prompt by specifying the --auto-approve
flag.
Added support for Unity Catalog schema resource
July 31, 2024
This feature was released with Databricks CLI 0.225.0.
Unity Catalog schemas can now be defined in your bundle configuration. See schema.
Bundle commands in web terminal
July 4, 2024
This feature was released with Databricks CLI 0.223.0.
You can now interact with bundles using the Databricks CLI from the web terminal. For information about using the web terminal, see Run shell commands in Databricks web terminal.
Added support for complex variables
June 26, 2024
This feature was released with Databricks CLI v0.222.0.
Databricks Asset Bundles now support complex variables. See Define a complex variable.
Databricks Asset Bundles is generally available
April 23, 2024
With the Databricks CLI 0.218.0 release, Databricks Asset Bundles are now generally available.