Databricks asset bundle deployment modes
Preview
This feature is in Public Preview.
This article describes the syntax for Databricks asset bundle deployment modes. Bundles enable programmatic management of Databricks workflows. See What are Databricks asset bundles?
In CI/CD workflows, developers typically code, test, deploy, and run solutions in various phases, or modes. For example, the simplest set of modes includes a development mode for pre-production validation, followed by a production mode for validated deliverables. Databricks asset bundles provides an optional collection of default behaviors that correspond to each of these modes. Your bundle deployments can use these behaviors with a one-line mode declaration, instead of manually specifying these default behaviors in your bundle configuration file–and perhaps forgetting to add or configure one of more intended behaviors along the way.
Development mode
To deploy your bundle in development mode, you must first add the mode
mapping, set to development
, to the intended target. For example, this target named dev
is treated as a development target:
targets:
dev:
mode: development
Deploying a development target by running the databricks bundle deploy -t <target-name>
command implements the following behaviors:
Prepends all resources that are not deployed as files or notebooks with the prefix
[dev ${workspace.current_user.userName}]
and tags each deployed job and pipeline with adev
Databricks tag.Marks all related deployed Delta Live Tables pipelines as
development: true
. See Use development mode to run pipeline updates.Enables the use of
--compute-id <cluster-id>
in related calls to thebundle deploy
command, which overrides any and all existing cluster definitions that are already specified in the related bundle configuration file. Instead of using--compute-id <cluster-id>
in related calls to thebundle deploy
command, you can set thecompute_id
mapping here, or as a child mapping of thebundle
mapping, to the ID of the cluster to use.Pauses all schedules and triggers on related deployed jobs.
Enables concurrent runs on all related deployed jobs.
Production mode
To deploy your bundle in production mode, you must first add the mode
mapping, set to production
, to the intended target. For example, this target named prod
is treated as a production target:
targets:
prod:
mode: production
Deploying a production target by running the databricks bundle deploy -t <target-name>
command implements the following behaviors:
Validates that all related deployed Delta Live Tables pipelines are marked as
development: false
.Validates that the current Git branch is equal to the Git branch that is specified in the target. Specifying a Git branch in the target is optional and can be done with an additional
git
property as follows:git: branch: main
This validation can be overridden by specifying
--force
while deploying.Databricks recommends that you use service principals for production deployments. See Use service principals with Databricks. If you do not use service principals, then note the following additional behaviors:
Validates that
artifact_path
,file_path
,root_path
, orstate_path
mappings are not overridden to a specific user.Validates that the
run_as
andpermissions
mappings are specified to clarify which identities have specific permissions for deployments.
Unlike the preceding behavior for setting the
mode
mapping todevelopment
, setting themode
mapping toproduction
does not allow overriding any and all existing cluster definitions that are already specified in the related bundle configuration file, for instance by using the--compute-id <cluster-id>
option or thecompute_id
mapping.