Skip to main content

Databricks personal access token authentication (legacy)

Databricks personal access tokens (PATs) let you authenticate to resources and APIs at the workspace level. You can store them in environment variables or Databricks configuration profiles. Each PAT is valid for only one workspace, and a user can create up to 600 PATs per workspace. Databricks automatically revokes PATs that haven’t been used for 90 days.

important

Username and password authentication (without tokens) reached end of life on July 10, 2024.

Databricks strongly recommends using OAuth instead of PATs for user account authentication because OAuth provides stronger security. To learn how to authenticate with a Databricks user account using OAuth, see Authorize user access to Databricks with OAuth.

note

To learn whether Google ID tokens are supported by your tools, SDKs, scripts, and apps, see your provider's documentation.

To automate Databricks account-level functionality, you can't use personal access tokens. Instead, use Google ID tokens from Databricks account-level admins, which are Google service accounts with admin rights. For details, see Authentication with Google ID tokens and the Account API. See also:

Databricks personal access tokens for workspace users​

To create a Databricks personal access token for your Databricks workspace user, do the following:

  1. In your Databricks workspace, click your Databricks username in the top bar, and then select Settings from the drop down.

  2. Click Developer.

  3. Next to Access tokens, click Manage.

  4. Click Generate new token.

  5. Enter a comment that helps you to identify this token in the future.

  6. Set the token's lifetime in days.

    If you leave the Lifetime (days) box empty, the token lifetime is set to the maximum lifetime for your workspace. By default, the maximum token lifetime for a workspace is 730 days. See Set maximum lifetime of new personal access tokens.

  7. Click Generate.

  8. Copy the displayed token to a secure location, and then click Done.

note

Save your token in a secure location and never share it. If you lose it, you can't recover the same token and must create a new one. If a token is lost or compromised, immediately revoke it from the Access tokens page by clicking the trash can (Revoke) icon.

If you can't create or use tokens, your workspace administrator might have disabled them or not granted you permission. Contact your administrator or see the following topics:

Databricks personal access tokens for service principals​

Step 1: As a Databricks admin, create a PAT for your Databricks service principal from the CLI​

A workspace admin can create a Databricks personal access tokens on behalf of a service principal using the CLI, as follows:

  1. Set up authentication for the Databricks CLI, if you have not done so already.

  2. Get the application ID for the Databricks service principal, if you do not already have it available:

    1. If the admin console for your workspace is not already open, click your username in the workspace's top bar and click Settings.
    2. Under Workspace admin, click Identity and access.
    3. Next to Service principals, click Manage.
    4. Click the name of the Databricks service principal to open its settings page. If the name is not visible, use Filter service principals to find it.
    5. On the Configurations tab, note the Application Id value.

  3. Use the Databricks CLI to run the following command, which generates the access token for the Databricks service principal.

    Run the following command:

    Bash
    databricks token-management create-obo-token <application-id> --lifetime-seconds <lifetime-seconds> --comment <comment> -p <profile-name>
    • Replace <application-id> with the application ID of the Databricks service principal.
    • --lifetime-seconds: Replace <lifetime-seconds> with the number of seconds that the access token is valid for. For example, 1 day is 86400 seconds. If the --lifetime-seconds option is not specified, the access token is set to the maximum lifetime for your workspace. By default, the maximum token lifetime for a workspace is 730 days.
    • --comment: Replace <comment> with a meaningful comment about the access token's purpose. If the --comment option is not specified, then no comment is generated.
    • --profile-name: Replace <profile-name> with the name of a Databricks configuration profile that contains authentication information for the Databricks service principal and the target workspace. If the -p option is not specified, the Databricks CLI will attempt to find and use a configuration profile named DEFAULT.

  4. In the response, copy the value of token_value, which is the access token for your Databricks service principal.

    Be sure to save the copied token in a secure location. Do not share your copied token with others. If you lose the copied token, you cannot regenerate that exact same token. Instead, you must repeat this procedure to create a new token.

    If you are not able to create or use tokens in your workspace, this might be because your workspace administrator has disabled tokens or has not given you permission to create or use tokens. See your workspace administrator or the following:

Step 2: Create additional PATs for your service principal​

Once a PAT has been created for your Databricks service principal, you can use that PAT to create additional Databricks personal access tokens for the service principal, as follows:

This procedure assumes that you have already generated the first Databricks personal access token for the Databricks service principal. You use this access token to set up the Databricks CLI to authenticate the Databricks service principal so that it can then generate additional access tokens for itself. See Personal access token authentication (deprecated).

  1. Use the Databricks CLI to run the following command, which generates another access token for the Databricks service principal.

    Run the following command:

    Bash
    databricks tokens create --comment <comment> --lifetime-seconds <lifetime-seconds> -p <profile-name>
    • --comment: Replace <comment> with a meaningful comment about the access token's purpose. If the --comment option is not specified, then no comment is generated.
    • --lifetime-seconds: Replace <lifetime-seconds> with the number of seconds that the access token is valid for. For example, 1 day is 86400 seconds. If the --lifetime-seconds option is not specified, the access token is set to the maximum lifetime for your workspace. By default, the maximum token lifetime for a workspace is 730 days.
    • --profile-name: Replace <profile-name> with the name of a Databricks configuration profile that contains authentication information for the Databricks service principal and the target workspace. If the -p option is not specified, the Databricks CLI will attempt to find and use a configuration profile named DEFAULT.

  2. In the response, copy the value of token_value, which is the access token for the Databricks service principal.

    Be sure to save the copied token in a secure location. Do not share your copied token with others. If you lose the copied token, you cannot regenerate that exact same token. Instead, you must repeat this procedure to create a new token.

    If you are not able to create or use tokens in your workspace, this might be because your workspace administrator has disabled tokens or has not given you permission to create or use tokens. See your workspace administrator or the following:

Perform Databricks personal access token authentication​

To configure Databricks personal access token authentication, you must set the following associated environment variables, .databrickscfg fields, Terraform fields, or Config fields:

  • The Databricks host, specified as the target Databricks workspace URL, for example https://1234567890123456.7.gcp.databricks.com.
  • The Databricks personal access token for the Databricks user account.

To perform Databricks personal access token authentication, integrate the following within your code, based on the participating tool or SDK:

To use environment variables for a specific Databricks authentication type with a tool or SDK, see Authorizing access to Databricks resources or the tool's or SDK's documentation. See also Environment variables and fields for unified authentication and the Authentication method priority.

Set the following environment variables:

  • DATABRICKS_HOST, set to the Databricks workspace URL, for example https://1234567890123456.7.gcp.databricks.com.
  • DATABRICKS_TOKEN, set to the token string.

Use the Databricks REST API to issue personal access tokens​

Databricks provides a REST endpoint /api/2.0/token/create to issue PATs. See Create a user token for API details.

You must provide specific values to the REST API. In the following example, set these values:

  • Replace <databricks-instance> with your Databricks workspace URL. For example, dbc-abcd1234-5678.cloud.databricks.com.
  • Replace <your-existing-access-token> with an existing valid PAT (string) that has permissions to create new tokens.

Provide the values for these parameters:

  • comment: A description for the new token.
  • lifetime_seconds: The token's lifetime in seconds.
Bash
curl -X POST https://<databricks-instance>/api/2.0/token/create \
-H "Authorization: Bearer <your-existing-access-token>" \
-H "Content-Type: application/json" \
-d '{
  "comment": "New PAT using DB API",
  "lifetime_seconds": <lifetime-of-pat-in-seconds>
}'

The -d flag provides the JSON payload for the request.

If successful, this results in a response payload similar to:

{
  "access_token": "<your-newly-issued-pat>",
  "token_type": "Bearer",
  "expires_in": <the-duration-of-the-new-pat>
}

Provide the new token from the response in the Authorization header of subsequent calls to Databricks REST APIs. For example:

Bash
# This example uses a simple GET. For POST or other REST verbs, you may need to provide additional parameters.
curl -X GET "https://<databricks-instance>/api/2.0/<path-to-endpoint>" \
     -H "Authorization: Bearer <your-new-pat>"
Python
import requests

headers = {
    'Authorization': 'Bearer <your-new-pat>'
}
# This example is for an HTTP GET operation.
response = requests.get('https://<databricks-instance>/api/2.0/<path-to-endpoint>', headers=headers)
Last updated on