Skip to main content

Enable workload identity federation for Terraform Cloud, Bitbucket Pipelines, or Jenkins

Preview

Databricks OAuth token federation is in Public Preview.

Databricks OAuth token federation, also known as OpenID Connect (OIDC), allows your automated workloads running outside of Databricks to securely access Databricks without the need for Databricks secrets. See Authenticate access to Databricks using OAuth token federation.

To enable workload identity federation for Terraform Cloud, Atlassian Bitbucket Pipelines, or Jenkins:

  1. Create a federation policy
  2. Configure the identity provider

After you enable workload identity federation, the Databricks SDKs and the Databricks CLI automatically fetch workload identity tokens from the identity provider and exchange them for Databricks OAuth tokens.

Create a federation policy​

First, use the Databricks CLI to create a workload identity federation policy. You will need to set the following values:

  • issuer: typically the provider's token URL
  • audiences: typically an organization identifier
  • subject: typically a value that specifies the job or project context

For example, given a GitLab URL https://gitlab.com/example-group with a Databricks service principal numeric ID of 5581763342009999, create a federation policy using the following Databricks CLI command:

Bash
databricks account service-principal-federation-policy create 5581763342009999 --json '{
  "oidc_policy": {
	"issuer": "https://gitlab.com/example-group",
	"audiences": [
  	  "https://gitlab.com/example-group"
	],
	"subject": "project_path:my-group/my-project:..."
  }
}'

Configure the identity provider​

Next, in the identity provider configuration, set the Databricks OIDC token environment variables to the provider's default identity OIDC token environment variables.

Jenkins​

To enable workload identity federation for Jenkins, set the token in DATABRICKS_OIDC_TOKEN. Alternatively, set the token in a file and set DATABRICKS_OIDC_TOKEN_FILEPATH to point to that file.

Terraform Cloud​

To enable workload identity federation for Terraform Cloud, set the DATABRICKS_OIDC_TOKEN_ENV environment variable to instruct Databricks SDK to search for the token in TFC_WORKLOAD_IDENTITY_TOKEN.

DATABRICKS_OIDC_TOKEN_ENV = TFC_WORKLOAD_IDENTITY_TOKEN

Atlassian Bitbucket Pipelines​

To enable workload identity federation for Bitbucket Pipelines, set the DATABRICKS_OIDC_TOKEN_ENV environment variable to instruct the Databricks SDK to search for the token in BITBUCKET_STEP_OIDC_TOKEN.

YAML
image: atlassian/default-image

pipelines:
  default:
    - step:
        oidc: true
        script:
          - export DATABRICKS_CLIENT_ID=a1b2c3d4-ee42-1eet-1337-f00b44r
          - export DATABRICKS_HOST=https://my-workspace.cloud.databricks.com/
          - export DATABRICKS_OIDC_TOKEN_ENV=BITBUCKET_STEP_OIDC_TOKEN
          - export DATABRICKS_AUTH_TYPE=env-oidc
          - curl -fsSL https://raw.githubusercontent.com/databricks/setup-cli/main/install.sh | sh
          - databricks --version
          - databricks current-user me
Last updated on