Monitor and manage access to personal access tokens
To authenticate to the Databricks REST API, a user can create a personal access token and use it in their REST API request. A user can also create a service principal and use it with a personal acccess token to call Databricks REST APIs in their CI/CD tools and automation. This article explains how Databricks workspace admins can manage these access tokens in their workspace.
To create an OAuth access token (instead of a PAT) to use with a service principal in automation, see Authenticate access to Databricks with a service principal using OAuth (OAuth M2M).
Using personal access tokens (PATs) instead of OAuth for access to Databricks
Databricks recommends you use OAuth access tokens instead of PATs for greater security and convenience. Databricks continues to support PATs but due to their greater security risk suggests that you audit your accounts current PAT usage and migrate your users and service principals to OAuth access tokens.
Overview of personal access token management
Note
The following documentation covers the use of PATs for customers who have not yet migrated their code to use OAuth instead. To assess your own organization’s usage of PATs and plan a migration from PATs to OAuth access tokens, see Assess personal access token usage in your Databricks account.
Personal access tokens are enabled by default for all Databricks workspaces that were created in 2018 or later.
When personal access tokens are enabled on a workspace, users with the CAN USE permission can generate personal access tokens to access Databricks REST APIs, and they can generate these tokens with any expiration date they like, including an indefinite lifetime. By default, no non-admin workspace users have the CAN USE permission, meaning that they cannot create or use personal access tokens.
As a Databricks workspace admin, you can disable personal access tokens for a workspace, monitor and revoke tokens, control which non-admin users can create tokens and use tokens, and set a maximum lifetime for new tokens.
Managing personal access tokens in your workspace requires the Premium plan or above. To create a personal access token, see Databricks personal access token authentication.
Enable or disable personal access token authentication for the workspace
Personal access token authentication is enabled by default for all Databricks workspaces that were created in 2018 or later. You can change this setting in the workspace settings page.
When personal access tokens are disabled for a workspace, personal access tokens cannot be used to authenticate to Databricks and workspace users and service principals cannot create new tokens. No tokens are deleted when you disable personal access token authentication for a workspace. If tokens are re-enabled later, any non-expired tokens are available for use.
If you want to disable token access for a subset of users, you can keep personal access token authentication enabled for the workspace and set fine-grained permissions for users and groups. See Control who can create and use tokens.
Warning
Partner Connect, partner integrations, and service principals require personal access tokens to be enabled on a workspace.
To disable the ability to create and use personal access tokens for the workspace:
Go to the settings page.
Click the Advanced tab.
Click the Personal Access Tokens toggle.
Click Confirm.
This change may take a few seconds to take effect.
You can also use the Workspace configuration API to disable personal access tokens for the workspace.
Control who can create and use tokens
Workspace admins can set permissions on personal access tokens to control which users, service principals, and groups can create and use tokens. For details on how to configure personal access token permissions, see Manage personal access token permissions.
Set maximum lifetime of new tokens
You can manage the maximum lifetime of new tokens in your workspace using the Databricks CLI or the Workspace configuration API. This limit applies only to new tokens.
Databricks automatically revokes access tokens that have been unused for 90 or more days. Because of this policy, set the token lifetime to 90 days or less.
Set maxTokenLifetimeDays
to the maximum token lifetime of new tokens in days, as an integer. If you set it to zero, new tokens are permitted to have no lifetime limit. For example:
databricks workspace-conf set-status --json '{
"maxTokenLifetimeDays": "90"
}'
curl -n -X PATCH "https://<databricks-instance>/api/2.0/workspace-conf" \
-d '{
"maxTokenLifetimeDays": "90"
}'
To use the Databricks Terraform provider to manage the maximum lifetime for new tokens in a workspace, see databricks_workspace_conf Resource.
Monitor and revoke tokens
This section describes how to use the Databricks CLI to manage existing tokens in the workspace. You can also use the Token Management API.
Get tokens for the workspace
To get the workspace’s tokens:
from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
spark.createDataFrame([token.as_dict() for token in w.token_management.list()]).createOrReplaceTempView('tokens')
display(spark.sql('select * from tokens order by creation_time'))
# Filter results by a user by using the `created-by-id` (to filter by the user ID) or `created-by-username` flags.
databricks token-management list
Automatic revocation of old access tokens
Databricks will automatically revoke PATs for your Databricks workspaces when the token hasn’t been used in 90 or more days. As a best practice, regularly audit the PATs in your Databricks account and remove any unused ones before they are automatically revoked. Import and run the notebook provided in the Assess personal access token usage in your Databricks account to determine the number of unexpired PATs in your organization’s Databricks account.
Databricks recommends that you configure your organization’s Databricks account to use OAuth tokens for access authentication instead of PATs for greater security and ease-of-use.