Manage personal access tokens

To authenticate to the Databricks REST API, a user can create a personal access token and use it in their REST API request. Tokens have an optional expiration date and can be revoked. See Authentication using Databricks personal access tokens.

Important

Databricks strongly recommends token-based REST authentication instead of username and password (native authentication).

The ability to use personal access tokens is enabled by default for all Databricks workspaces that were created in 2018 or later. Workspace administrators can enable or disable personal token access for all workspaces, regardless of creation date.

Workspace administrators can also monitor tokens, control which non-admin users can create tokens, and set maximum lifetimes for new tokens.

When the ability to generate personal access tokens is enabled for your workspace, by default all users in your Databricks workspace can generate personal access tokens to access Databricks REST APIs, and they can generate these tokens with any expiration date they like, including an indefinite lifetime.

As a Databricks admin, you can use the Token Management API and Permissions API to control token usage at a more fine-grained level. The APIs are published on each workspace instance. To learn how to access and authenticate to the API, see Authentication using Databricks personal access tokens. You must access the API as a Databricks admin.

For some tasks, you can also use the Admin Console.

The following table indicates the tasks you can perform using the web application and those you can perform using the REST APIs. For the cells that say Yes, click the word to view the related documentation.

Task Description Admin Console REST API
Enable/disable Enable or disable all tokens for this workspace Yes Yes
Control who can use tokens Limit personal access token creation and usage to specified users and groups in this workspace. If you revoke a user’s permission to create and use tokens, that user’s existing tokens are also revoked. Yes Yes
Max lifetime for new tokens. Set the maximum lifetime of new tokens in this workspace No Yes
Manage existing tokens For existing tokens, get the token creator, expiration date, and user-provided token description. Revoke tokens for users who should no longer have access to Databricks APIs. By monitoring and controlling token creation, you reduce the risk of lost tokens or long-lasting tokens that could lead to data exfiltration from the workspace. No Yes

See the following diagram for the typical flow of token management using the REST API for administration and token usage. See the table in this topic for the set of tasks that you could alternatively use the Admin Console.

Token management

Enable or disable token-based authentication for the workspace

Token-based authentication is enabled by default for all Databricks workspaces that were created in 2018 or later. You can change this setting in the Admin Console. To specify which users are allowed to use tokens, see Control who can use or create tokens.

To enable or disable personal access tokens for the workspace:

  1. Go the Admin Console.
  2. Select the Access Control tab.
  3. To enable access, click the Enable button next to Personal Access Tokens. To disable access, click the Disable button.
  4. Click Confirm. This change may take a few seconds to take effect.

To use token-based authentication for a REST API request, see Authentication using Databricks personal access tokens.

No tokens are deleted when you disable token-based authentication for a workspace. If tokens are re-enabled later, any non-expired tokens are immediately available for use.

If you want to disable token access for a subset of users, keep token-based authentication enabled for the workspace and set fine-grained permissions for users and groups. See Control who can use or create tokens.

You can also use use the REST API to make this change. To enable or disable the token management feature for a workspace, call the workspace configuration for tokens API (PATCH /workspace-conf). In a JSON request body, specify enableTokensConfig as true (enabled) or false (disabled).

For example, to enable the feature:

curl -X PATCH -n \
  https://<databricks-instance>/api/2.0/workspace-conf \
  -d '{
    "enableTokensConfig": "true",
    }'

Control who can use or create tokens

A user can have one of the following token permissions:

  • No permissions

  • Can Use – For workspaces created after the release of Databricks platform version 3.28 (Sept 9-15, 2020) the default is for no users to have the Can Use permission. Admins must explicitly grant those permissions, whether to the entire users group or on a user-by-user or group-by-group basis.

    Important

    Workspaces created before 3.28 was released will maintain the permissions that were already in place. The default was for all users to have the Can Use permission. Admins can revoke that group permission and grant it to other groups or to individual non-admin users. See Remove permissions.

  • Can Manage – Users in the admins group have this permission by default and you cannot revoke it. No other groups can be granted this permission. The API enforces these rules.

This table lists the permissions required for each token-related task:

Task No permission Can Use Can Manage
Create a token Yes Yes
Use a token for authentication Yes Yes
Revoke your own token Yes Yes
Revoke any user’s token ** Yes
List all tokens ** Yes
Modify token permissions *** Yes

Actions that are marked with ** require the Token Management API.

Actions that are marked with *** can be performed in the Admin Console or with the Permissions API. See Control who can use or create tokens.

Manage token permissions using the Admin Console

To manage token permissions for the workspace using the Admin Console:

  1. Go the Admin Console.

  2. Select the Access Control tab.

  3. If token-based authentication is disabled, click the Enable button next to Personal Access Tokens. Click Confirm to confirm the change. This change may take a few seconds to take effect.

    Token enablement
  4. Click the Permissions Settings button to open the token permission editor.

    Token permissions editor
  5. Add or remove permissions

    Important

    For workspaces created after the release of Databricks platform version 3.28 (Sept 9-15, 2020) the default is for no users to have the Can Use permission. Admins must explicitly grant those permissions, whether to the entire users group or on a user-by-user or group-by-group basis. Workspaces created before 3.28 was released maintain the permissions that were already in place. The default was for all users to have the Can Use permission. Admins can revoke that group permission assignment and grant it to other groups or to individual non-admin users.

    If the users group has the Can Use permission and you want to apply more fine-grained access for non-admin users, remove the Can Use permissions from the users group by clicking the X next to the permission drop-down in the users row.

    To grant the permission to other entities, select each user or group to whom you want to grant access. Select a user or group from the Select User or Group… drop-down, select Can Use, and click the + Add button. In the following example, the admin has removed access for the users group and is granting access to the Data Science B2 group.

    Token permissions editor

    The admins group has Can Manage permissions, which you cannot change, nor can you assign Can Manage to any entity other than the admins group.

  6. Click Save to save your changes.

    Warning

    After saving your changes, any users who previously had either Can Use or Can Manage permission but no longer have either permission are denied access to token-based authentication and their active tokens are immediately deleted (revoked). Deleted tokens cannot be retrieved.

Manage token permissions using the Permissions API

Get all token permissions for the workspace

To get token permissions for all Databricks users and Databricks groups in the workspace, call the get all token permissions for the workspace API (GET /permissions/authorization/tokens) as part of the Permissions API.

The response includes an access_control_list array. Each element is a user object or a group object. They each have an identity field appropriate to the type: users have a user_name field and groups have a group_name field. All elements have an all_permissions field that specifies which permission levels (CAN_USE or CAN_MANAGE) are granted.

For example:

curl -n -X GET "https://<databricks-instance>/api/2.0/preview/permissions/authorization/tokens"

Example response:

{
  "object_id": "authorization/tokens",
  "object_type": "tokens",
  "access_control_list": [
    {
      "user_name": "jsmith@example.com",
      "all_permissions": [
        {
          "permission_level": "CAN_USE",
          "inherited": false
        }
      ]
    }
  ]
}

Set token permissions

To set token permissions, call the set token permissions API (PATCH /permissions/authorization/tokens).

You can set permissions on one or more users and groups. For each user, you need to know the email address, which is specified in the user_name request property. For each group, specify the group name in the group_name property.

Entities (such as users or groups) not mentioned explicitly are not directly affected by this request, although changes to group membership can indirectly affect user access.

You can only grant, not revoke, permissions with this API.

For example, the following example grants access to user jsmith@example.com and the group mygroup.

curl -n -X PATCH "https://<databricks-instance>/api/2.0/preview/permissions/authorization/tokens"
  -d '{
    "access_control_list": [
      {
        "user_name": "jsmith@example.com",
        "permission_level": "CAN_USE",
      },
      {
        "group_name": "mygroup",
        "permission_level": "CAN_USE",
      }
    ]
  }'

Example response:

{
  "access_control_list": [
    {
      "user_name": "jsmith@example.com",
      "all_permissions": [
        {
          "permission_level": "CAN_USE",
          "inherited": false
        }
      ]
    },
    {
      "group_name": "mygroup",
      "all_permissions": [
        {
          "permission_level": "CAN_USE",
          "inherited": false
        }
      ]
    }
  ]
}

If you want to set token permissions for all entities in the workspace in one request, use the update all permissions API (PUT /permissions/authorization/tokens). See Remove permissions

Remove permissions

Note

For workspaces created after the release of Databricks platform version 3.28 (Sept 9-15, 2020) the default is for no users to have the Can Use permission. Admins must explicitly grant those permissions, whether to the entire users group or on a user-by-user or group-by-group basis. Workspaces created before 3.28 was released maintain the permissions that were already in place. The default was for all users to have the Can Use permission. Admins can revoke that group permission assignment and grant it to other groups or to individual non-admin users.

To revoke permissions from all or some non-admin users, use the update all permissions API (PUT /permissions/authorization/tokens), which requires that you specify the complete set of permissions for all objects that are granted permissions for the entire workspace.

If you want to authorize a subset of non-admin users to create and use tokens, do all three of the following:

  • Grant the users and groups the CAN_USE permission.
  • Do not grant the CAN_USE permission to the built-in users group if you want to only authorize some non-admin users. You can optionally choose to assign the permission to this group, in which case all non-admin users can create and use tokens.
  • Grant the built-in admins group the CAN_MANAGE permission. This is a requirement of the API.

Warning

After a successful request, if a user does not have token permissions directly or indirectly through a group, that user’s tokens are immediately deleted. Deleted tokens cannot be retrieved.

The following example grants Can Use tokens permission to group field-automation-group, omits permissions for the users (all users) group, and grants CAN_MANAGE permission to the admins group as required by the API. Any non-admin users that are not in the group field-support-engineers will lose access to token creation and their existing tokens are immediately deleted (revoked).

curl -n -X PUT "https://<databricks-instance>/api/2.0/preview/permissions/authorization/tokens"
  -d '{
    "access_control_list": [
      {
        "group_name": "field-automation-group",
        "permission_level": "CAN_USE",
      },
      {
        "group_name": "admins",
        "permission_level": "CAN_MANAGE",
      },
    ]
  }'

Set maximum lifetime of new tokens (REST API only)

Use the workspace configuration for tokens APIs to manage the maximum lifetime of new tokens in this workspace.

To set the maximum lifetime for new tokens, call the Set the maximum token lifetime for new tokens API (PATCH /workspace-conf). Set maxTokenLifetimeDays to the maximum token lifetime of new tokens in days, as an integer. If you set it to zero, new tokens are permitted to have no lifetime limit.

For example:

curl -n -X PATCH "https://<databricks-instance>/api/2.0/workspace-conf"
  -d '{
  "maxTokenLifetimeDays": "90"
  }'

Warning

This limit applies only to new tokens. To review existing tokens, see the get tokens API.

To get the workspace’s maximum lifetime for new tokens, call the workspace configuration for tokens API (GET /workspace-conf) and pass keys=maxTokenLifetimeDays as a query parameter. The response includes an maxTokenLifetimeDays property that is the maximum token lifetime of new tokens in days, as an integer. If it is zero, new tokens are permitted to have no lifetime limit.

For example:

curl -n -X GET "https://<databricks-instance>/api/2.0/workspace-conf?keys=maxTokenLifetimeDays"

Example response:

{
    "maxTokenLifetimeDays": "90"
}

Monitor and revoke tokens (REST API only)

Use the token management APIs to manage existing tokens in the workspace.

Get tokens for the workspace

To get the workspace’s tokens, call the get all tokens API (GET /token-management/tokens). The response includes a token_infos array. Each element represents a token and includes fields for ID (token_id), creation time (creation_time), expiry time (expiry_time), description (comment), and the user that created it (the ID created_by_id or username created_by_username). You can learn more about the user using the SCIM get user API (GET /scim/v2/Users/{id}).

To filter results by a user, set the request body property created_by_id (for the ID) or created_by_username (for the username). You can get a user ID from a display name using the SCIM get users API (GET /scim/v2/Users)

For example:

curl -n -X GET "https://<databricks-instance>/api/2.0/token-management/tokens"
  -d '{
  "created_by_id": "1234567890"
  }'

Example response:

{
  "token_infos": [
    {
      "token_id": "<token-id>",
      "creation_time": 1580265020299,
      "expiry_time": 1580265020299,
      "comment": "This is for ABC division's automation scripts.",
      "created_by_id": 1234567890,
      "created_by_username": "jsmith@example.com"
    }
  ]
}

Alternatively, get a specific token using the get a token API (GET /token-management/tokens/{token_id})

Delete (revoke) a token

  1. Find the token ID. See Get tokens for the workspace.
  2. Call the delete a token API (DELETE /token-management/tokens). Pass the token ID in the path.

For example:

curl -n -X DELETE "https://<databricks-instance>/api/2.0/token-management/tokens/<token-id>"