Skip to main content
Last updated on

Authenticate with Databricks personal access tokens (legacy)

Databricks personal access tokens (PATs) let you authenticate to resources and APIs at the workspace level. You can store them in environment variables or Databricks configuration profiles. Each PAT is valid for only one workspace, and a user can create up to 600 PATs per workspace. Databricks automatically revokes PATs that haven’t been used for 90 days.

important

Username and password authentication (without tokens) reached end of life on July 10, 2024. Databricks strongly recommends using OAuth instead of PATs for user account authentication because OAuth provides stronger security. To learn how to authenticate with a Databricks user account using OAuth, see Authorize user access to Databricks with OAuth.

You can't use personal access tokens to automate Databricks account-level functionality. Instead, use either OAuth tokens for Databricks account admin users or service principals. For more information, see:

Create personal access tokens for workspace users​

To create a personal access token for your Databricks workspace user, do the following:

  1. In your Databricks workspace, click your username in the top bar and select Settings.
  2. Click Developer.
  3. Next to Access tokens, click Manage.
  4. Click Generate new token.
  5. Enter a comment that helps you to identify this token in the future.
  6. Set the token's lifetime in days. See Set the maximum lifetime of new personal access tokens.
  7. To limit the token's permissions, select a token type and add API scopes. See Scoped personal access tokens.
  8. Click Generate.
  9. Copy the displayed token to a secure location, then click Done. Save the token securely and don't share it. If you lose it, you must create a new token.

If you can't create or use tokens, your workspace administrator might have disabled tokens or not granted you permission. See your workspace administrator or the following:

Scoped personal access tokens​

Scoped personal access tokens restrict a token's permissions to specific API operations. Instead of granting full workspace access, you assign one or more API scopes, such as sql, unity-catalog, or scim, that limit which REST API operations the token can call.

warning

Tokens with the authentication scope can create new tokens with any scope. Grant this scope only to tokens that need to manage other tokens.

To create a scoped token in the workspace UI, select a token type and add API scopes when you generate a new token. If you don't assign any scopes, the token retains the full permissions of the creating identity.

For a complete list of scopes and their associated API operations, see API scopes.

Create personal access tokens for service principals​

A workspace admin creates the initial PAT for a service principal, then the service principal can create additional tokens for itself.

Step 1: Create a PAT for your service principal​

As a workspace admin, create a Databricks personal access token on behalf of a service principal:

  1. Set up authentication for the Databricks CLI if not already configured.

  2. Get the application ID for the Databricks service principal:

    1. Click your username in the top bar, then click Settings.
    2. Under Workspace admin, click Identity and access > Manage (next to Service principals).
    3. Click the Databricks service principal name to open its settings page.
    4. On the Configurations tab, note the Application Id value.

  3. Run the following command to generate the access token:

    Bash
    databricks token-management create-obo-token \
      <application-id> \
      --lifetime-seconds <lifetime-seconds> \
      -p <profile-name>

    Replace the following values:

    • <application-id>: The Databricks service principal application ID.
    • <lifetime-seconds>: Token lifetime in seconds, such as 86400 for 1 day. Defaults to the workspace maximum (typically 730 days).
    • <profile-name>: Configuration profile with authentication information. Defaults to DEFAULT.

  4. In the response, copy the token_value, which is the access token for your Databricks service principal. Save the token securely and don't share it. If you lose it, you must create a new token.

If you can't create or use tokens, your workspace administrator might have disabled tokens or not granted you permission. See your workspace administrator or the following:

Step 2: Create additional PATs for your service principal​

Use the existing PAT to create additional tokens for the service principal.

  1. Run the following command to generate an access token:

    Bash
    databricks tokens create \
      --lifetime-seconds <lifetime-seconds> \
      -p <profile-name>

    Replace the following values:

    • <lifetime-seconds>: Token lifetime in seconds, such as 86400 for 1 day. Defaults to the workspace maximum (typically 730 days).
    • <profile-name>: Configuration profile with authentication information. Defaults to DEFAULT.

  2. Copy the token_value from the response, which is the access token for your Databricks service principal. Save the token securely and don't share it. If you lose it, you must create a new token.

If you can't create or use tokens, your workspace administrator might have disabled tokens or not granted you permission. See your workspace administrator or the following:

Perform personal access token authentication​

To configure Databricks personal access token authentication, set the following associated environment variables, .databrickscfg fields, Terraform fields, or Config fields:

  • The Databricks host, specified as the target Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.
  • The Databricks personal access token, for the Databricks user account or Databricks service principal.

To perform Databricks personal access token authentication, integrate the following within your code, based on the participating tool or SDK:

To use environment variables for a specific Databricks authentication type with a tool or SDK, see Authorize access to Databricks resources or the tool's or SDK's documentation. See also Environment variables and fields for unified authentication and the Authentication method priority.

Set the following environment variables:

  • DATABRICKS_HOST, set to the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.
  • DATABRICKS_TOKEN, set to the token string.

Use the Databricks REST API to issue personal access tokens​

Databricks provides a REST endpoint /api/2.0/token/create to issue PATs. See Create a user token for API details.

In the following example, set these values:

  • <databricks-instance>: Your Databricks workspace URL. For example, dbc-abcd1234-5678.cloud.databricks.com.
  • <your-existing-access-token>: An existing valid PAT (string) that has permissions to create new tokens.
  • <lifetime-seconds>: The token's lifetime in seconds.
  • <scopes>: A list of scopes to assign to the token. See Scoped personal access tokens.
Bash
curl -X POST https://<databricks-instance>/api/2.0/token/create \
-H "Authorization: Bearer <your-existing-access-token>" \
-H "Content-Type: application/json" \
-d '{
  "lifetime_seconds": <lifetime-seconds>,
  "scopes": [
    "sql",
    "authentication"
  ]
}'

If successful, this results in a response payload similar to:

JSON
{
  "token_value": "<your-newly-issued-pat>",
  "token_info": {
    "token_id": "<token-id>",
    "creation_time": <creation-timestamp>,
    "expiry_time": <expiry-timestamp>,
    "comment": "<comment>",
    "scopes": ["authentication", "sql"],
    "last_accessed_time": 0
  }
}

Provide the new token from the response in the Authorization header of subsequent calls to Databricks REST APIs. For example:

Bash
# This example uses a simple GET. For POST or other REST verbs, you may need to provide additional parameters.
curl -X GET "https://<databricks-instance>/api/2.0/<path-to-endpoint>" \
     -H "Authorization: Bearer <your-new-pat>"
Python
import requests

headers = {
    'Authorization': 'Bearer <your-new-pat>'
}
# This example is for an HTTP GET operation.
response = requests.get('https://<databricks-instance>/api/2.0/<path-to-endpoint>', headers=headers)

Update the scopes of a personal access token​

If a scoped token is missing the required scope for an API call, the request fails with an error indicating the missing scope. To update a token's scopes, use the REST endpoint /api/2.0/token/<token_id>. The calling token must have the authentication scope, which permits managing other tokens. Use the update_mask field to specify which token fields to update.

Bash
curl -X PATCH https://<databricks-instance>/api/2.0/token/<token_id> \
-H "Authorization: Bearer <your-existing-access-token>" \
-H "Content-Type: application/json" \
-d '{
  "token": {
    "scopes": ["sql", "unity-catalog"]
  },
  "update_mask": "scopes"
}'

Scope changes can take up to ten minutes to propagate.

To view all available scopes, use GET /api/2.0/token-scopes.