Authorize user access to Databricks with OAuth
This article explains how to authorize user access to Databricks resources when using the Databricks CLI or Databricks REST APIs.
Databricks uses OAuth 2.0 as the preferred protocol for user authorization and authentication outside of the UI. Unified client authentication automates token generation and refresh. After a user signs in and grants consent, OAuth issues an access token for the CLI, SDK, or other tool to use on the user’s behalf. Each access token is valid for one hour, after which a new token is automatically requested.
In this article, authorization refers to using OAuth to grant access to Databricks resources, while authentication refers to validating credentials through access tokens.
For more high-level details, see Authorizing access to Databricks resources.
Ways to authorize access to Databricks resources
Databricks supports two ways to authorize user accounts with OAuth:
-
Automatic (recommended): Use unified client authentication if you're working with supported tools and SDKs, such as the Databricks Terraform SDK. This approach handles token generation and refresh automatically.
-
Manual: Generate a code verifier and challenge, then exchange them for an OAuth token. Use this method if your tool doesn't support unified client authentication. For details, see Manually generate OAuth U2M access tokens.
Automatic authorization with unified client authentication
Before you configure authorization, review the ACL permissions for the type of workspace operations you plan to perform and confirm that your account has the required access level. For details, see Access control lists.
To perform OAuth authorization with Databricks SDKs and tools that support unified client authentication, integrate the following within your code:
- Environment
- Profile
- CLI
- VS Code
- Connect
- Terraform
- Python
- Java
- Go
To use environment variables for a specific Databricks authentication type with a tool or SDK, see Authorizing access to Databricks resources or the tool's or SDK's documentation. See also Environment variables and fields for unified client authentication and the Default methods for client unified authentication.
For account-level operations, set the following environment variables:
DATABRICKS_HOST
, set to the value of your Databricks account console URL,https://accounts.gcp.databricks.com
.DATABRICKS_ACCOUNT_ID
For workspace-level operations, set the following environment variables:
DATABRICKS_HOST
, set to the value of your Databricks workspace URL, for examplehttps://1234567890123456.7.gcp.databricks.com
.
Create or identify a Databricks configuration profile with the following fields in your .databrickscfg
file. If you create the profile, replace the placeholders with the appropriate values. To use the profile with a tool or SDK, see Authorizing access to Databricks resources or the tool's or SDK's documentation. See also Environment variables and fields for unified client authentication and the Default methods for client unified authentication.
For account-level operations, set the following values in your .databrickscfg
file. In this case, the Databricks account console URL is https://accounts.gcp.databricks.com
:
[<some-unique-configuration-profile-name>]
host = <account-console-url>
account_id = <account-id>
For workspace-level operations, set the following values in your .databrickscfg
file. In this case, the host is the Databricks workspace URL, for example https://1234567890123456.7.gcp.databricks.com
:
[<some-unique-configuration-profile-name>]
host = <workspace-url>
For the Databricks CLI, run the databricks auth login
command with the following options:
- For account-level operations,
--host https://accounts.cloud.databricks.com --account-id <account-id>
. - For workspace-level operations,
--host <workspace-url>
.
Then follow the instructions in your web browser to log in to your Databricks account or workspace.
For more details, see OAuth authorization with the Databricks CLI.
For the Databricks extension for Visual Studio Code, follow the steps in Set up authorization for the Databricks extension for Visual Studio Code.
OAuth U2M authentication is supported in Databricks Connect for Python starting with Databricks Runtime 13.1 and for Scala starting with Databricks Runtime 13.3 LTS.
For Databricks Connect, you can either:
- Use a config profile: Set workspace-level values in your
.databrickscfg
file as described on the Profile tab. Also set thecluster_id
to your workspace instance URL. - Use environment variables: Set the same values as shown on the Environment tab. Also set the
DATABRICKS_CLUSTER_ID
to your workspace instance URL.
Values in .databrickscfg
take precedence over environment variables.
To initialize Databricks Connect with these settings, see Compute configuration for Databricks Connect.
Before you apply your Terraform configuration, you must run one of the databricks auth login
commands on the CLI tab depending on whether your configuration uses workspace or account operations. These commands generate and cache the required OAuth token at .databricks/token-cache.json
in your user's home folder.
Account-level operations
For default authentication:
provider "databricks" {
alias = "account"
}
For direct configuration:
provider "databricks" {
alias = "account"
host = <retrieve-account-console-url>
account_id = <retrieve-account-id>
}
Replace the retrieve-
placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider. In this example, you can set account_id
to the Databricks account console URL.
Workspace-level operations
For default authentication:
provider "databricks" {
alias = "workspace"
}
For direct configuration:
provider "databricks" {
alias = "workspace"
host = <retrieve-workspace-url>
}
Before you run your code, you must run the databricks auth login
command on the CLI tab with either workspace or account operations options. These commands generate and cache the required OAuth token at .databricks/token-cache.json
in your user's home folder.
Account-level operations
For default authentication:
from databricks.sdk import AccountClient
a = AccountClient()
# ...
For direct configuration:
from databricks.sdk import AccountClient
a = AccountClient(
host = retrieveAccountConsoleUrl(),
account_id = retrieveAccountId()
)
# ...
Replace the retrieve
placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as Google Cloud Secret Manager.
Workspace-level operations
For default authentication:
from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
# ...
For direct configuration:
from databricks.sdk import WorkspaceClient
w = WorkspaceClient(host = retrieveWorkspaceUrl())
# ...
For more information about authenticating with Databricks tools and SDKs that use Python and that implement Databricks client unified authentication, see:
Before you run your code, you must run the databricks auth login
command on the CLI tab with either workspace or account operations options. These commands generate and cache the required OAuth token at .databricks/token-cache.json
in your user's home folder.
Account-level operations
For default authentication:
import com.databricks.sdk.AccountClient;
// ...
AccountClient a = new AccountClient();
// ...
For direct configuration:
import com.databricks.sdk.AccountClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
.setHost(retrieveAccountConsoleUrl())
.setAccountId(retrieveAccountId());
AccountClient a = new AccountClient(cfg);
// ...
Replace the retrieve
placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as Google Cloud Secret Manager.
Workspace-level operations
For default authentication:
import com.databricks.sdk.WorkspaceClient;
// ...
WorkspaceClient w = new WorkspaceClient();
// ...
For direct configuration:
import com.databricks.sdk.WorkspaceClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
.setHost(retrieveWorkspaceUrl())
WorkspaceClient w = new WorkspaceClient(cfg);
// ...
For more information about authorizing and authenticating with Databricks tools and SDKs that use Java and that implement Databricks client unified authentication, see:
- Set up the Databricks Connect client for Scala (uses the Databricks SDK for Java for authentication)
- Authenticate the Databricks SDK for Java with your Databricks account or workspace
Before you run your code, you must run the databricks auth login
command on the CLI tab with either workspace or account operations options. These commands generate and cache the required OAuth token at .databricks/token-cache.json
in your user's home folder.
Account-level operations
For default authentication:
import (
"github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient())
// ...
For direct configuration:
import (
"github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient(&databricks.Config{
Host: retrieveAccountConsoleUrl(),
AccountId: retrieveAccountId(),
}))
// ...
Replace the retrieve
placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as Google Cloud Secret Manager.
Workspace-level operations
For default authentication:
import (
"github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient())
// ...
For direct configuration:
import (
"github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient(&databricks.Config{
Host: retrieveWorkspaceUrl(),
}))
// ...
For more information about authenticating with Databricks tools and SDKs that use Go and that implement Databricks client unified authentication, see Authenticate the Databricks SDK for Go with your Databricks account or workspace.
Manually generate OAuth U2M access tokens
This section is for users working with third-party tools or services that don't support the Databricks unified client authentication standard. If you need to manually generate, refresh, or use Databricks OAuth tokens for OAuth U2M authentication, follow the steps in this section.
Step 1: Generate a code verifier and challenge
To generate OAuth U2M access tokens manually, start by creating a code verifier and a matching code challenge. You'll use the challenge in Step 2 to get an authorization code, and the verifier in Step 3 to exchange that code for an access token.
Follow the OAuth PKCE standard:
- The code verifier is a cryptographically random string (43–128 characters) using characters
A–Z
,a–z
,0–9
,-._~
. - The code challenge is a Base64 URL-encoded SHA256 hash of the verifier.
For more information, see Authorization Request.
The following Python script generates a verifier and challenge. Although you can use them multiple times, Databricks recommends generating a new pair each time you manually generate access tokens.
import hashlib, base64, secrets, string
# Allowed characters for the code verifier, per PKCE spec
allowed_chars = string.ascii_letters + string.digits + "-._~"
# Generate a secure code verifier (43–128 characters)
code_verifier = ''.join(secrets.choice(allowed_chars) for _ in range(64))
# Create the SHA256 hash of the code verifier
sha256_hash = hashlib.sha256(code_verifier.encode()).digest()
# Base64-url-encode the hash and strip any trailing '=' padding
code_challenge = base64.urlsafe_b64encode(sha256_hash).decode().rstrip("=")
# Output values
print(f"code_verifier: {code_verifier}")
print(f"code_challenge: {code_challenge}")
Step 2: Generate an authorization code
To get a Databricks OAuth access token, you must first generate an OAuth authorization code. This code expires immediately after use. You can generate the code at either the account or workspace level:
- Account level: Use to call both account-level and workspace-level REST APIs across all workspaces that your user can access.
- Workspace level: Use to call REST APIs within a single workspace.
These examples use databricks-cli
as the client ID. If you're not using a built-in Databricks tool such as the CLI or SDKs, you must enable a custom OAuth application and use its client_id
in your requests. See Enable or disable partner OAuth applications.
Generate an account-level authorization code
-
In your browser, navigate to the URL with the following replacements:
<account-id>
: Your Databricks account ID<redirect-url>
: A local redirect URI (for example,http://localhost:8020
)<state>
: Any plain-text string to validate the response<code-challenge>
: The code challenge from Step 1
https://accounts.gcp.databricks.com/oidc/accounts/<account-id>/v1/authorize
?client_id=databricks-cli
&redirect_uri=<redirect-url>
&response_type=code
&state=<state>
&code_challenge=<code-challenge>
&code_challenge_method=S256
&scope=all-apis+offline_access -
Log in when prompted to access your Databricks account.
-
Copy the authorization code from your browser’s address bar. It’s the value after
code=
and before&
in the redirect URL:http://localhost:8020/?code=dcod...7fe6&state=<state>
Verify that the
state
value matches what you originally provided. If it doesn’t, discard the code. -
Continue to Generate an account-level access token.
Generate a workspace-level authorization code
-
In your browser, navigate to the URL with the following replacements:
<databricks-instance>
: Your<databricks-instance>
with the Databricks workspace instance name, for example1234567890123456.7.gcp.databricks.com
<redirect-url>
: A local redirect (e.g.,http://localhost:8020
)<state>
: Any plain-text value for response validation<code-challenge>
: The challenge string from Step 1
https://<databricks-instance>/oidc/v1/authorize
?client_id=databricks-cli
&redirect_uri=<redirect-url>
&response_type=code
&state=<state>
&code_challenge=<code-challenge>
&code_challenge_method=S256
&scope=all-apis+offline_access -
Log in when prompted to access your Databricks account.
-
Copy the authorization code from your browser’s address bar. It’s the value after
code=
and before&
in the redirect URL:http://localhost:8020/?code=dcod...7fe6&state=<state>
Verify that the
state
value matches what you originally provided. If it doesn’t, discard the code. -
Continue to Generate a workspace-level access token.
Step 3: Exchange the authorization code for an access token
To exchange the authorization code for a Databricks OAuth access token, choose the appropriate level:
- Account level: Use to call both account-level and workspace-level REST APIs across all workspaces your user can access.
- Workspace level: Use to call REST APIs within a single workspace.
Generate an account-level access token
-
Use
curl
to exchange the account-level authorization code for an OAuth access token.Replace the following in the request:
<account-id>
: Your Databricks account ID<redirect-url>
: The redirect URL from the previous step<code-verifier>
: The verifier you generated earlier<authorization-code>
: The authorization code from the previous step
Bashcurl --request POST \
https://accounts.gcp.databricks.com/oidc/accounts/<account-id>/v1/token \
--data "client_id=databricks-cli" \
--data "grant_type=authorization_code" \
--data "scope=all-apis offline_access" \
--data "redirect_uri=<redirect-url>" \
--data "code_verifier=<code-verifier>" \
--data "code=<authorization-code>" -
Copy the
access_token
value from the response. For example:JSON{
"access_token": "eyJr...Dkag",
"refresh_token": "doau...f26e",
"scope": "all-apis offline_access",
"token_type": "Bearer",
"expires_in": 3600
}The token is valid for one hour.
-
Continue to Step 4: Call a Databricks REST API.
Generate a workspace-level access token
-
Use
curl
to exchange the workspace-level authorization code for an OAuth access token.Replace the following in the request:
<databricks-instance>
: Your<databricks-instance>
with the Databricks workspace instance name, for example1234567890123456.7.gcp.databricks.com
<redirect-url>
: The redirect URL from the previous step<code-verifier>
: The verifier you generated earlier<authorization-code>
: The workspace-level authorization code
Bashcurl --request POST \
https://<databricks-instance>/oidc/v1/token \
--data "client_id=databricks-cli" \
--data "grant_type=authorization_code" \
--data "scope=all-apis offline_access" \
--data "redirect_uri=<redirect-url>" \
--data "code_verifier=<code-verifier>" \
--data "code=<authorization-code>" -
Copy the
access_token
value from the response. For example:JSON{
"access_token": "eyJr...Dkag",
"refresh_token": "doau...f26e",
"scope": "all-apis offline_access",
"token_type": "Bearer",
"expires_in": 3600
}The token is valid for one hour.
Step 4: Call a Databricks REST API
Use the access token to call account-level or workspace-level REST APIs, depending on its scope. To call account-level APIs, your Databricks user must be an account admin.
Example account-level REST API request
This example uses curl
along with Bearer
authentication to get a list of all workspaces associated with an account.
- Replace
<oauth-access-token>
with the account-level OAuth access token. - Replace
<account-id>
with your account ID.
export OAUTH_TOKEN=<oauth-access-token>
curl --request GET --header "Authorization: Bearer $OAUTH_TOKEN" \
"https://accounts.gcp.databricks.com/api/2.0/accounts/<account-id>/workspaces"
Example workspace-level REST API request
This example uses curl
along with Bearer
authentication to list all available clusters in the specified workspace.
- Replace
<oauth-access-token>
with the account-level or workspace-level OAuth access token. - Replace
<databricks-instance>
with the Databricks workspace instance name, for example1234567890123456.7.gcp.databricks.com
.
export OAUTH_TOKEN=<oauth-access-token>
curl --request GET --header "Authorization: Bearer $OAUTH_TOKEN" \
"https://<databricks-instance>/api/2.0/clusters/list"