Skip to main content

Attribute usage with serverless budget policies

Preview

This feature is in Public Preview.

This article explains how to use serverless budget policies to enforce cost attribution tags on serverless compute workloads.

Serverless budget policies consist of tags that are applied to any serverless compute activity incurred by a user assigned to the policy. The tags are logged in your billing records, allowing you to attribute serverless usage to specific budgets. For more on creating budgets, see Create and monitor budgets.

Serverless budget policy permissions​

You must be a workspace admin to create serverless budget policies. Non-admins can manage policies if they are assigned Serverless budget policy: Manager permissions.

Workspace admins can manage and view serverless budget policies they created or the ones they have explicit permissions on. To view and manage all policies for a given account, the workspace admin must additionally have the Billing admin account-level role. This role is assigned in the account console.

By default, new policies can only be accessed by the policy creator and users with the billing admin role.

Because serverless budget policies are account-level resource objects, managers can manage policies from any workspace they have access to.

Create a serverless budget policy​

Serverless budget policies are managed in your workspace admin settings. To create a new serverless budget policy:

  1. Click your username in the top bar of the Databricks workspace and select Settings.
  2. Click Compute.
  3. Next to Serverless budget policies, click Manage.
  4. Click Create.
  5. Add a name and your tags then click Create.

Create serverless budget policy UI

Manage serverless budget policy permissions​

There are two types of permissions you can grant on your serverless budget policies:

  • User: A serverless budget policy user can select the policy when they create a notebook, job, pipeline, or serving endpoint.
  • Manager: Managers can use the serverless budget policy, but can also edit the policy’s definitions and permissions. Any user in your workspace can be given the manager permissions.

If a user is only assigned a single policy, then that policy is automatically applied to the user's newly created resources. If a user is assigned multiple policies, they must select the appropriate policy when they create a new serverless notebook, job, pipeline, or serving endpoint. If a user doesn’t select a policy, the setting defaults to whichever policy comes first alphabetically.

Assign permissions on a policy​

To assign permissions on a serverless budget policy:

  1. On the policy’s page, click the Permissions tab.
  2. Click Grant access.
  3. Select the user, group, or service principal you want to assign the policy to.
  4. Under Role, select the role(s) you want to give the user.
  5. To continue adding identities to the policy, click Add another.
  6. Click Save.

Update a serverless budget policy​

To update an existing serverless budget policy’s name or tags:

  1. Click your username in the top bar of the Databricks workspace and select Settings.
  2. Click Compute.
  3. Next to Serverless budget policies, click Manage.
  4. Select the serverless budget policy you want to update, then click Edit.
  5. Make the updates then click Save.

Policy changes are only applied to usage initiated after the policy update. They are not applied to currently running serverless usage.

Analyze serverless budget policy tags in the billing records​

After a policy is applied to a notebook, job, or DLT pipeline, any tags contained in the policy propagate to your system.billing.usage system table in the custom_tags column.

If a notebook is run as part of a job, only the job’s serverless budget policy is applied to the usage record.

Where to select the serverless budget policy​

Users assigned to multiple serverless budget policies must select the appropriate policy whenever they create a new notebook, job, pipeline or model serving endpoint.

Known limitations​

Serverless budget policies have the following limitations:

  • Existing notebooks, jobs, and DLT pipelines are not automatically assigned policies after their owners are granted access to a policy. To add a serverless budget policy to an existing asset, you must manually update the asset’s serverless budget policy setting in the UI.
  • When creating a scheduled notebook job from the notebook page, the notebook’s serverless budget policy defaults back to the first available policy by alphabetical order.
  • Policy IDs stored with an asset remain even if the policy is deleted. These policies do not apply any tags.
  • Updates to serverless budget policy tags are reflected on any serverless usage initiated after the change to the policy. For example, if a job is running while a serverless budget policy is updated, the update does not take effect for the existing job run. The next job run uses the updated tags.
  • Git repo integration does not have repo-configurable serverless budget policies, the serverless budget policy defaults to the user’s last chosen policy.
  • Serverless budget policies do not apply tags to classic compute resources.
  • Tags inherited from serverless budget policies do not appear on the Jobs list UI.
  • Updates to tags won’t be reflected in new pipeline updates if the pipeline is in Development mode. The changes take 24 hours to propagate.
  • Pipelines triggered by jobs do not inherit the job’s serverless budget policy. Users must set the pipeline’s policy.
Last updated on