Databricks asset bundle deployment modes

Preview

This article describes the syntax for Databricks asset bundle deployment modes. Bundles enable programmatic management of Databricks workflows. See What are Databricks asset bundles?

In CI/CD workflows, developers typically code, test, deploy, and run solutions in various phases, or modes. For example, the simplest set of modes includes a development mode for pre-production validation, followed by a production mode for validated deliverables. Databricks asset bundles provides an optional collection of default behaviors that correspond to each of these modes. Your bundle deployments can use these behaviors with a one-line mode declaration, instead of manually specifying these default behaviors in your bundle configuration file–and perhaps forgetting to add or configure one of more intended behaviors along the way.

Development mode

To deploy your bundle in development mode, you must first add the mode mapping, set to development, to the intended target. For example, this target named dev is treated as a development target:

targets:
  dev:
    mode: development

Deploying a development target by running the databricks bundle deploy -t <target-name> command implements the following behaviors:

Prepends all resources that are not deployed as files or notebooks with the prefix [dev ${workspace.current_user.userName}] and tags each deployed job and pipeline with a dev Databricks tag.
Marks all related deployed Delta Live Tables pipelines as development: true. See Use development mode to run pipeline updates.
Enables the use of --compute-id <cluster-id> in related calls to the bundle deploy command, which overrides any and all existing cluster definitions that are already specified in the related bundle configuration file. Instead of using --compute-id <cluster-id> in related calls to the bundle deploy command, you can set the compute_id mapping here, or as a child mapping of the bundle mapping, to the ID of the cluster to use.
Pauses all schedules and triggers on related deployed jobs.
Enables concurrent runs on all related deployed jobs.

Production mode

To deploy your bundle in production mode, you must first add the mode mapping, set to production, to the intended target. For example, this target named prod is treated as a production target:

targets:
  prod:
    mode: production

Deploying a production target by running the databricks bundle deploy -t <target-name> command implements the following behaviors:

Validates that all related deployed Delta Live Tables pipelines are marked as development: false.
Validates that the current Git branch is equal to the Git branch that is specified in the target. Specifying a Git branch in the target is optional and can be done with an additional git property as follows:
```
git:
  branch: main
```
This validation can be overridden by specifying --force while deploying.
Databricks recommends that you use service principals for production deployments. See Use service principals with Databricks. If you do not use service principals, then note the following additional behaviors:
- Validates that artifact_path, file_path, root_path, or state_path mappings are not overridden to a specific user.
- Validates that the run_as and permissions mappings are specified to clarify which identities have specific permissions for deployments.
Unlike the preceding behavior for setting the mode mapping to development, setting the mode mapping to production does not allow overriding any and all existing cluster definitions that are already specified in the related bundle configuration file, for instance by using the --compute-id <cluster-id> option or the compute_id mapping.