Skip to main content

Configure a federation policy

Preview

Databricks OAuth token federation is in Public Preview.

Databricks OAuth token federation enables you to securely access Databricks APIs using tokens from your identity provider (IdP). To enable OAuth token federation, you must configure a federation policy, either as Databricks account-wide or for workloads.

This article describes how to create and configure an OAuth token federation policy.

note

You cannot create a federation policy using the Databricks CLI in the Databricks workspace web terminal.

Account-wide token federation​

Account admins can configure OAuth token federation in the Databricks account using an account federation policy. An account federation policy enables all users and service principals in your Databricks account to access Databricks APIs using tokens from your identity provider. An account federation policy specifies:

  • The identity provider or issuer from which Databricks will accept tokens.
  • The criteria for mapping a token to the corresponding Databricks user or service principal.

For example, given the following account federation policy:

issuer: "https://idp.mycompany.com/oidc"
audiences: ["databricks"]
subject_claim: "sub"

This JWT body can be used to authenticate to Databricks as username@mycompany.com:

JSON
{
  "iss": "https://idp.mycompany.com/oidc",
  "aud": "databricks",
  "sub": "username@mycompany.com"
}

Configure an account federation policy​

To configure an account federation policy, you must specify:

  • The required token issuer, specified in the iss claim of your tokens. The issuer is an HTTPS URL that identifies your identity provider.

  • The allowed token audiences, specified in the aud claim of your tokens. This identifier represents the recipient of the token. As long as the audience in the token matches at least one audience in the policy, the token is considered a match. If unspecified, the default value is your Databricks account ID.

  • The subject claim. This indicates which token claim contains the Databricks username of the user the token was issued for. If unspecified, the default value is sub.

  • Optionally, the public keys (or the URL of the public keys) used to validate the signature of your tokens, in JSON Web Key Sets (JWKS) format. If unspecified (recommended), Databricks automatically fetches the public keys from your issuer's well known endpoint. Databricks strongly recommends relying on your issuer's well known endpoint for discovering public keys.

    note

    If you do not specify a JWKS or a jwks_uri in your federation policy, your identity provider must serve OpenID Provider Metadata at {issuer-url}/.well-known/openid-configuration. The OpenID Provider Metadata must include a jwks_uri that specifies the location of the public keys used to verify token signatures.

Account admins can configure an account federation policy using the Databricks CLI (version 0.239.0 and above) or the Databricks API. You can specify up to five account federation policies in your Databricks account.

  1. Install or update to the newest version of the Databricks CLI.

  2. As an account admin, authenticate to your Databricks account using the CLI. Specifying the ACCOUNT_CONSOLE_URL (e.g.https://accounts.cloud.databricks.com) and your Databricks ACCOUNT_ID:

    Bash
    databricks auth login --host ${ACCOUNT_CONSOLE_URL} --account-id ${ACCOUNT_ID}

  3. Create the account federation policy. For example:

    Bash
    databricks account federation-policy create --json \
    '{
      "oidc_policy": {
        "issuer": "https://idp.mycompany.com/oidc",
        "audiences": [
          "databricks"
        ],
        "subject_claim": "sub"
      }
    }'

Example account federation policies​

The following table provides example account federation policies and the matching JWT body.

Federation policy

Example matching token

issuer: "https://idp.mycompany.com/oidc"

audiences: ["2ff814a6-3304-4ab8-85cb-cd0e6f879c1d"]

{

"iss": "https://idp.mycompany.com/oidc",

"aud": "2ff814a6-3304-4ab8-85cb-cd0e6f879c1d",

"sub": "username@mycompany.com"

}

issuer: "https://idp.mycompany.com/oidc"

audiences: ["2ff814a6-3304-4ab8-85cb-cd0e6f879c1d"]

subject_claim: "preferred_username"

{

"iss": "https://idp.mycompany.com/oidc",

"aud": ["2ff814a6-3304-4ab8-85cb-cd0e6f879c1d",

"other-audience"],

"preferred_username": "username@mycompany.com",

"sub": "some-other-ignored-value"

}

issuer: "https://idp.mycompany.com/oidc"

audiences: ["2ff814a6-3304-4ab8-85cb-cd0e6f879c1d"]

jwks_json: {"keys":[{"kty":"RSA","e":"AQAB","use":"sig",

"kid":"<key-id>","alg":"RS256", "n":"uPUViFv..."}]}

{

"iss": "https://idp.mycompany.com/oidc",

"aud": "2ff814a6-3304-4ab8-85cb-cd0e6f879c1d",

"sub": "username@mycompany.com"

}

(signature verified using public key in policy)

issuer: "https://idp.mycompany.com/oidc"

audiences: ["2ff814a6-3304-4ab8-85cb-cd0e6f879c1d"]

jwks_uri: "https://idp.mycompany.com/jwks.json"

{

"iss": "https://idp.mycompany.com/oidc",

"aud": "2ff814a6-3304-4ab8-85cb-cd0e6f879c1d",

"sub": "username@mycompany.com"

}

(signature verified using public key fetched from jwks_uri)

Workload identity federation​

Workload identity federation allows your automated workloads running outside of Databricks to access Databricks APIs without the need for Databricks secrets. Account admins can configure workload identity federation using a service principal federation policy.

A service principal federation policy is associated with a service principal in your Databricks account, and specifies:

  • The identity provider (or issuer) from which the service principal can authenticate.
  • The workload identity (or subject) that is permitted to authenticate as the Databricks service principal.

For example, given the following service principal federation policy for a Github Actions workload:

issuer: "https://token.actions.githubusercontent.com"
audiences: ["https://github.com/my-github-org"]
subject: "repo:my-github-org/my-repo:environment:prod"

This JWT body can be used to authenticate to Databricks:

JSON
{
  "iss": "https://token.actions.githubusercontent.com",
  "aud": "https://github.com/my-github-org",
  "sub": "repo:my-github-org/my-repo:environment:prod"
}

Configure a service principal federation policy​

To configure a service principal federation policy, you must specify:

  • The required token issuer, specified in the iss claim of workload identity tokens. The issuer is an HTTPS URL that identifies the workload identity provider.

  • The required token subject, specified in the sub claim of workload identity tokens. The subject uniquely identifies the workload in the workload runtime environment.

  • The allowed token audiences, specified in the aud claim of workload identity tokens. The audience represents the recipient of the token. As long as the audience in the token matches at least one audience in the policy, the token is considered a match. If unspecified, the default value is your Databricks account ID.

  • Optionally, the public keys (or the URL of the public keys) used to validate the signature of the workload identity tokens, in JSON Web Key Sets (JWKS) format. If unspecified (recommended), Databricks automatically fetches the public keys from the issuer's well known endpoint. Databricks strongly recommends relying on the issuer's well known endpoint for discovering public keys.

    note

    If you do not specify a JWKS or a jwks_uri in your federation policy, your identity provider must serve OpenID Provider Metadata at {issuer-url}/.well-known/openid-configuration. The OpenID Provider Metadata must include a jwks_uri that specifies the location of the public keys used to verify token signatures.

  • Optionally, the subject claim. This indicates which token claim contains the workload identity (or subject) of the token. If it is unspecified, the default value is sub. Databricks strongly recommends using the default sub claim for workload identity federation. A claim other than sub should only be used in cases where the sub claim is not an appropriate or stable subject identifier, which is uncommon. See Example service principal federation policies below for details.

Account admins can configure a service principal federation policy using the Databricks CLI (version 0.239.0 and above) or the Databricks API. You can create up to five service principal federation policies per Databricks service principal.

  1. Install or update to the newest version of the Databricks CLI.

  2. As an account admin, authenticate to your Databricks account using the CLI. Specifying the ACCOUNT_CONSOLE_URL (for example, https://accounts.cloud.databricks.com) and your Databricks ACCOUNT_ID:

    Bash
    databricks auth login --host ${ACCOUNT_CONSOLE_URL} --account-id ${ACCOUNT_ID}

  3. Get the numeric ID of the service principal that will have the federation policy applied to it. (For example, 3659993829438643.)

    If you know the service principal application ID (typically a GUID value, such as bc3cfe6c-469e-4130-b425-5384c4aa30bb) in advance, you can then determine the service principal numeric ID using the Databricks CLI:

    Bash
    databricks account service-principals list --filter 'applicationId eq "<service-principal-application-id>"'

  4. Create the service principal federation policy. Here is an example of creating a federation policy for a GitHub Action:

    Bash
    databricks account service-principal-federation-policy create ${SERVICE_PRINCIPAL_NUMERIC_ID} --json \
    '{
      "oidc_policy": {
        "issuer": "https://token.actions.githubusercontent.com",
        "audiences": [
          "https://github.com/my-github-org"
        ],
        "subject": "repo:my-github-org/my-repo:environment:prod"
      }
    }'

Example Databricks service principal federation policies​

The following table provides example service principal federation policies and the matching JWT body.

For complete configuration steps for enabling workload identity federation for some of these common identity providers, see Enable workload identity federation in CI/CD.

Tool

Federation policy

Example matching token

GitHub Actions

issuer: "https://token.actions.githubusercontent.com"

audiences: ["https://github.com/<github-org>"]

subject: "repo:<github-org>/<repo>:environment:prod"

{

"iss": "https://token.actions.githubusercontent.com",

"aud": "https://github.com/<github-org>",

"sub": "repo:<github-org>/<repo>:environment:prod"

}

Kubernetes

issuer: "https://kubernetes.default.svc"

audiences: ["https://kubernetes.default.svc"]

subject: "system:serviceaccount:namespace:podname"

jwks json: {"keys":[{"kty":"rsa","e":"AQAB","use":"sig",

"kid":"<key-id>","alg":"RS256","n":"uPUViFv..."}]}

{

"iss": "https://kubernetes.default.svc",

"aud": ["https://kubernetes.default.svc"],

"sub": "system:serviceaccount:namespace:podname"

}

Azure DevOps

issuer: "https://vstoken.dev.azure.com/<org_id>"

audiences: ["api://AzureADTokenExchange"]

subject: "sc://my-org/my-project/my-connection"

{

"iss": "https://vstoken.dev.azure.com/<org_id>",

"aud": "api://AzureADTokenExchange",

"sub": "sc://my-org/my-project/my-connection"

}

GitLab

issuer: "https://gitlab.example.com"

audiences: ["https://gitlab.example.com"]

subject: "project_path:my-group/my-project:..."

{

"iss": "https://gitlab.example.com",

"aud": "https://gitlab.example.com",

"sub": "project_path:my-group/my-project:..."

}

CircleCI

issuer: "https://oidc.circleci.com/org/<org_id>"

audiences: ["<org_id>"]

subject: "7cc1d11b-46c8-4eb2-9482-4c56a910c7ce"

subject_claim: "oidc.circleci.com/project-id"

{

"iss": "https://oidc.circleci.com/org/<org_id>",

"aud": "<org_id>",

"oidc.circleci.com/project-id": "7cc1d11b-46c8-4eb2-9482-4c56a910c7ce"

}

Next steps​

After you have configured a federation policy for your account:

  • You might need to configure your identity provider to generate tokens for your users to exchange with Databricks. See the documentation for your identity provider for instructions. For complete configuration steps for enabling workload identity federation for some common identity providers, see Enable workload identity federation in CI/CD.
  • You can use a JWT from your identity provider to access the Databricks API. First exchange a JWT token from your identity provider for a Databricks OAuth token, and then use the Databricks OAuth token in the Bearer: field of the API call to gain access and complete the call. Tokens must be valid JWTs that are signed using the RS256 or ES256 algorithms. For details about implementing the token exchange, see Use an identity provider token to authenticate Databricks REST APIs.
Last updated on