Authentication for Databricks automation

In Databricks, authentication refers to verifying a Databricks identity (such as a user, service principal, or group). Databricks uses credentials (such as an access token or a username and password) to verify the identity.

After Databricks verifies the caller’s identity, Databricks then uses a process called authorization to determine whether the verified identity has sufficient access permissions to perform the specified action on the resource at the given location. This article includes details only about authentication. It does not include details about authorization or access permissions; see Authentication and access control.

When a tool makes an automation or API request, it includes credentials that authenticate an identity with Databricks. This article describes typical ways to create, store, and pass credentials and related information that Databricks needs to authenticate and authorize requests. To learn which credential types, related information, and storage mechanism are supported by your tools, SDKs, scripts, and apps, see your provider’s documentation.

Databricks account and workspace REST APIs

Databricks organizes its Databricks REST API into two categories of APIs: account APIs and workspace APIs. Each of these categories requires different sets of information to authenticate the target Databricks identity. Also, each supported Databricks authentication type requires additional information that uniquely identifies the target Databricks identity.

For instance, to authenticate a Databricks identity for calling Databricks account-level API operations, you must provide:

• The target Databricks account console URL, which is typically https://accounts.cloud.databricks.com.

• The target Databricks account ID. See Locate your account ID.

• Information that uniquely identifies the target Databricks identity for the target Databricks authentication type. For the specific information to provide, see the section later in this article for that authentication type.

To authenticate a Databricks identity for calling Databricks workspace-level API operations, you must provide:

• The target Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.

• Information that uniquely identifies the target Databricks identity for the target Databricks authentication type. For the specific information to provide, see the section later in this article for that authentication type.

Databricks client unified authentication

Databricks provides a consolidated and consistent architectural and programmatic approach to authentication, known as Databricks client unified authentication. This approach helps make setting up and automating authentication with Databricks more centralized and predictable. It enables you to configure Databricks authentication once and then use that configuration across multiple Databricks tools and SDKs without further authentication configuration changes.

Participating Databricks tools and SDKs include:

• The Databricks CLI

• The Databricks Terraform provider

• Databricks Connect

• The Databricks extension for Visual Studio Code

• The Databricks SDK for Python

• The Databricks SDK for Java

• The Databricks SDK for Go

All participating tools and SDKs accept special environment variables as well as Databricks configuration profiles for authentication. The Databricks Terraform provider and the Databricks SDKs for Python, Java, and Go also accept direct configuration of authentication settings within code. For details, see the following sections and the tool’s or SDK’s documentation.

The following sections contain examples of how to configure your machine for authentication by using special environment variables, Databricks configuration profiles, Databricks Terraform provider code, and code for the Databricks SDKs for Python, Java, and Go. For other participating tools and SDKs:

• The Databricks CLI supports special environment variables as well as Databricks configuration profiles for many authentication types. See the Environment and Profile examples in the following sections. See also Authentication for the Databricks CLI.

• Databricks Connect supports multiple authentication configuration options that are unique to Databricks Connect. See Set up the client in the Databricks Connect documentation.

• The Databricks extension for Visual Studio Code provides a unique user interface for configuring some authentication types and relies on its integration with Databricks Connect for some other authentication types. See Authentication for the Databricks extension for Visual Studio Code.

Databricks personal access token authentication

Databricks personal access tokens are one of the most well-supported types of credentials for resources and operations at the Databricks workspace level. Many storage mechanisms for credentials and related information, such as environment variables and Databricks configuration profiles, provide support for Databricks personal access tokens. Although users can have multiple personal access tokens in a Databricks workspace, each personal access token works for only a single Databricks workspace. The number of personal access tokens per user is limited to 600 per workspace.

Note

To automate Databricks account-level functionality, you cannot use Databricks personal access tokens. Instead, you must use either OAuth tokens for Databricks account admin users or service principals, or the username and password of Databricks account-level admins. For more information, see:

• Basic authentication

• OAuth machine-to-machine (M2M) authentication

• OAuth user-to-machine (U2M) authentication

Databricks personal access tokens for workspace users

To create a Databricks personal access token for your Databricks workspace user, do the following:

1. In your Databricks workspace, click your Databricks username in the top bar, and then select User Settings from the dropdown.

2. Click Developer.

3. Next to Access tokens, click Manage.

4. Click Generate new token.

5. (Optional) Enter a comment that helps you to identify this token in the future, and change the token’s default lifetime of 90 days. To create a token with no lifetime (not recommended), leave the Lifetime (days) box empty (blank).

6. Click Generate.

7. Copy the displayed token to a secure location, and then click Done.

   Be sure to save the copied token in a secure location. Do not share your copied token with others. If you lose the copied token, you cannot regenerate that exact same token. Instead, you must repeat this procedure to create a new token. If you lose the copied token, or you believe that the token has been compromised, Databricks strongly recommends that you immediately delete that token from your workspace by clicking the X next to the token on the Access tokens page.

   Note

   If you are not able to create or use tokens in your workspace, this might be because your workspace administrator has disabled tokens or has not given you permission to create or use tokens. See your workspace administrator or the following:

   • Enable or disable personal access token authentication for the workspace

   • Personal access token permissions

Databricks personal access tokens for service principals

To create a Databricks personal access token for a Databricks service principal instead of a Databricks workspace user, see Manage tokens for a service principal.

Note

If you are not able to create or use tokens in your workspace, this might be because your workspace administrator has disabled tokens or has not given you permission to create or use tokens. See your workspace administrator or the following:

• Enable or disable personal access token authentication for the workspace

• Personal access token permissions

Perform Databricks personal access token authentication

To configure Databricks personal access token authentication, you must set the following associated environment variables, .databrickscfg fields, Terraform fields, or Config fields:

• The Databricks host, specified as the target Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.

• The Databricks personal access token, for the Databricks user account or Databricks service principal.

To perform Databricks personal access token authentication, integrate the following within your code, based on the participating tool or SDK:

To use environment variables with a tool or SDK, see the tool’s or SDK’s documentation. See also Environment variables and fields for client unified authentication and the Default order of evaluation for client unified authentication methods and credentials.

Set the following environment variables:

• DATABRICKS_HOST, set to the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.

• DATABRICKS_TOKEN

Create or identify a Databricks configuration profile with the following fields in your .databrickscfg file. If you create the profile, replace the placeholders with the appropriate values. To use the profile with a tool or SDK, see the tool’s or SDK’s documentation. See also Environment variables and fields for client unified authentication and the Default order of evaluation for client unified authentication methods and credentials.

Set the following values in your .databrickscfg file. In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

[<some-unique-configuration-profile-name>]
host  = <workspace-url>
token = <token>

For default authentication:

provider "databricks" {
  alias = "workspace"
}

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

provider "databricks" {
  alias = "workspace"
  host  = <retrieve-workspace-url>
  token = <retrieve-token>
}

For default authentication:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient()
# ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient(
   host  = retrieve_workspace_url(),
   token = retrieve_token()
)
# ...

For default authentication:

import com.databricks.sdk.WorkspaceClient;
// ...
WorkspaceClient w = new WorkspaceClient();
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

import com.databricks.sdk.WorkspaceClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
   .setHost(retrieveWorkspaceUrl())
   .setToken(retrieveToken());
WorkspaceClient w = new WorkspaceClient(cfg);
// ...

For default authentication:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient())
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient(&databricks.Config{
  Host:  retrieveWorkspaceUrl(),
  Token: retrieveToken(),
}))
// ...

Basic authentication

Basic authentication uses a Databricks username and password to authenticate the target Databricks user account.

To configure basic authentication, you must set the following associated environment variables, .databrickscfg fields, Terraform fields, or Config fields:

• The Databricks host.

  • For account operations, specify https://accounts.cloud.databricks.com.

  • For workspace operations, specify the workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.

• The username of the Databricks user account.

• The password of the Databricks user account.

• For Databricks account operations, the Databricks account ID.

To perform basic authentication, integrate the following within your code, based on the participating tool or SDK:

To use environment variables with a tool or SDK, see the tool’s or SDK’s documentation. See also Environment variables and fields for client unified authentication and the Default order of evaluation for client unified authentication methods and credentials.

For account-level operations, set the following environment variables:

• DATABRICKS_HOST, set to the Databricks account console URL, https://accounts.cloud.databricks.com.

• DATABRICKS_ACCOUNT_ID

• DATABRICKS_USERNAME

• DATABRICKS_PASSWORD

For workspace-level operations, set the following environment variables:

• DATABRICKS_HOST, set to the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.

• DATABRICKS_USERNAME

• DATABRICKS_PASSWORD

Create or identify a Databricks configuration profile with the following fields in your .databrickscfg file. If you create the profile, replace the placeholders with the appropriate values. To use the profile with a tool or SDK, see the tool’s or SDK’s documentation. See also Environment variables and fields for client unified authentication and the Default order of evaluation for client unified authentication methods and credentials.

For account-level operations, set the following values in your .databrickscfg file. In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

[<some-unique-configuration-profile-name>]
host       = <account-console-url>
account_id = <account-id>
username   = <username>
password   = <password>

For workspace-level operations, set the following values in your .databrickscfg file. In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

[<some-unique-configuration-profile-name>]
host     = <workspace-url>
username = <username>
password = <password>

For account-level operations, for default authentication:

provider "databricks" {
  alias = "accounts"
}

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider). In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

provider "databricks" {
  alias      = "accounts"
  host       = <retrieve-account-console-url>
  account_id = <retrieve-account-id>
  username   = <retrieve-username>
  password   = <retrieve-password>
}

For workspace-level operations, for default authentication:

provider "databricks" {
  alias = "workspace"
}

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

provider "databricks" {
  alias    = "workspace"
  host     = <retrieve-workspace-url>
  username = <retrieve-username>
  password = <retrieve-password>
}

For account-level operations, for default authentication:

from databricks.sdk import AccountClient

a = AcccountClient()
# ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

from databricks.sdk import AccountClient

a = AcccountClient(
  host       = retrieve_account_console_url(),
  account_id = retrieve_account_id(),
  username   = retrieve_username(),
  password   = retrieve_password()
)
# ...

For workspace-level operations, for default authentication:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient()
# ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient(
  host     = retrieve_workspace_url(),
  username = retrieve_username(),
  password = retrieve_password()
)
# ...

For account-level operations, for default authentication:

import com.databricks.sdk.AccountClient;
// ...
AccountClient a = new AccountClient();
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

import com.databricks.sdk.AccountClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
  .setHost(retrieveAccountConsoleUrl())
  .setAccountId(retrieveAccountId())
  .setUsername(retrieveUsername())
  .setPassword(retrievePassword());
AccountClient a = new AccountClient(cfg);
// ...

For workspace-level operations: for default authentication:

import com.databricks.sdk.WorkspaceClient;
// ...
WorkspaceClient w = new WorkspaceClient();
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

import com.databricks.sdk.WorkspaceClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
  .setHost(retrieveWorkspaceUrl())
  .setUsername(retrieveUsername())
  .setPassword(retrievePassword());
WorkspaceClient w = new WorkspaceClient(cfg);
// ...

For account-level operations, for default authentication:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient())
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient(&databricks.Config{
  Host:      retrieveAccountConsoleUrl(),
  AccountId: retrieveAccountId(),
  UserName:  retrieveUsername(),
  Password:  retreivePassword(),
}))
// ...

For workspace-level operations, for default authentication:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient())
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient(&databricks.Config{
  Host:     retrieveWorkspaceUrl(),
  UserName: retrieveUsername(),
  Password: retreivePassword(),
}))
// ...

OAuth machine-to-machine (M2M) authentication

Preview

This feature is in Public Preview.

OAuth machine-to-machine (M2M) authentication uses the credentials of an automated entity (in this case, a Databricks service principal) to authenticate the target entity. After Databricks successfully authenticates the target service principal through the OAuth M2M authentication request, an OAuth token is given to the participating tool or SDK to perform token-based authentication from that time forward on the service principal’s behalf. The OAuth token has a lifespan of one hour, following which the tool or SDK involved will make an automatic background attempt to obtain a new token that is also valid for one hour.

To begin configuring OAuth M2M authentication, complete the OAuth M2M authentication setup instructions. See Steps 1–3 in Authentication using OAuth for service principals.

Important

You only need to complete Steps 1–3 in the preceding article’s instructions.

Step 4 in that article covers manually creating OAuth access tokens; however, tools and SDKs that implement the Databricks client unified authentication standard automatically create and manage OAuth access tokens for your target Databricks service principal on your behalf. Step 5 in that article covers using curl to call the Databricks REST API, instead of using tools and SDKs that implement the Databricks client unified authentication standard.

To finish configuring OAuth M2M authentication, you must set the following associated environment variables, .databrickscfg fields, Terraform fields, or Config fields:

• The Databricks host, specified as https://accounts.cloud.databricks.com for account operations or the target workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com for workspace operations.

• The Databricks account ID, for Databricks account operations.

• The service principal client ID.

• The service principal secret.

To perform OAuth M2M authentication, integrate the following within your code, based on the participating tool or SDK:

To use environment variables with a tool or SDK, see the tool’s or SDK’s documentation. See also Environment variables and fields for client unified authentication and the Default order of evaluation for client unified authentication methods and credentials.

For account-level operations, set the following environment variables:

• DATABRICKS_HOST, set to the Databricks account console URL, https://accounts.cloud.databricks.com.

• DATABRICKS_ACCOUNT_ID

• DATABRICKS_CLIENT_ID

• DATABRICKS_CLIENT_SECRET

For workspace-level operations, set the following environment variables:

• DATABRICKS_HOST, set to the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.

• DATABRICKS_CLIENT_ID

• DATABRICKS_CLIENT_SECRET

Create or identify a Databricks configuration profile with the following fields in your .databrickscfg file. If you create the profile, replace the placeholders with the appropriate values. To use the profile with a tool or SDK, see the tool’s or SDK’s documentation. See also Environment variables and fields for client unified authentication and the Default order of evaluation for client unified authentication methods and credentials.

For account-level operations, set the following values in your .databrickscfg file. In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

[<some-unique-configuration-profile-name>]
host          = <account-console-url>
account_id    = <account-id>
client_id     = <service-principal-client-id>
client_secret = <service-principal-secret>

For workspace-level operations, set the following values in your .databrickscfg file. In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

[<some-unique-configuration-profile-name>]
host          = <workspace-url>
client_id     = <service-principal-client-id>
client_secret = <service-principal-secret>

For account-level operations, for default authentication:

provider "databricks" {
  alias = "accounts"
}

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider). In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

provider "databricks" {
  alias         = "accounts"
  host          = <retrieve-account-console-url>
  account_id    = <retrieve-account-id>
  client_id     = <retrieve-client-id>
  client_secret = <retrieve-client-secret>
}

For workspace-level operations, for default authentication:

provider "databricks" {
  alias = "workspace"
}

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

provider "databricks" {
  alias         = "workspace"
  host          = <retrieve-workspace-url>
  client_id     = <retrieve-client-id>
  client_secret = <retrieve-client-secret>
}

For account-level operations, for default authentication:

from databricks.sdk import AccountClient

a = AcccountClient()
# ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

from databricks.sdk import AccountClient

a = AcccountClient(
  host          = retrieve_account_console_url(),
  account_id    = retrieve_account_id(),
  client_id     = retrieve_client_id(),
  client_secret = retrieve_client_secret()
)
# ...

For workspace-level operations, for default authentication:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient()
# ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient(
  host          = retrieve_workspace_url(),
  client_id     = retrieve_client_id(),
  client_secret = retrieve_client_secret()
)
# ...

For account-level operations, for default authentication:

import com.databricks.sdk.AccountClient;
// ...
AccountClient a = new AccountClient();
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

import com.databricks.sdk.AccountClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
  .setHost(retrieveAccountConsoleUrl())
  .setAccountId(retrieveAccountId())
  .setClientId(retrieveClientId())
  .setClientSecret(retrieveClientSecret());
AccountClient a = new AccountClient(cfg);
// ...

For workspace-level operations, for default authentication:

import com.databricks.sdk.WorkspaceClient;
// ...
WorkspaceClient w = new WorkspaceClient();
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

import com.databricks.sdk.WorkspaceClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
  .setHost(retrieveWorkspaceUrl())
  .setClientId(retrieveClientId())
  .setClientSecret(retrieveClientSecret());
WorkspaceClient w = new WorkspaceClient(cfg);
// ...

For account-level operations, for default authentication:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient())
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient(&databricks.Config{
  Host:         retrieveAccountConsoleUrl(),
  AccountId:    retrieveAccountId(),
  ClientId:     retrieveClientId(),
  ClientSecret: retrieveClientSecret(),
}))
// ...

For workspace-level operations, for default authentication:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient())
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient(&databricks.Config{
  Host:         retrieveWorkspaceUrl(),
  ClientId:     retrieveClientId(),
  ClientSecret: retrieveClientSecret(),
}))
// ...

OAuth user-to-machine (U2M) authentication

Note

OAuth user-to-machine (U2M) authentication is in Public Preview.

OAuth user-to-machine (U2M) authentication uses real-time human sign in and consent to authenticate the target Databricks user account. After the user successfully signs in and consents to the OAuth authentication request, an OAuth token is given to the participating tool or SDK to perform token-based authentication from that time forward on the user’s behalf. The OAuth token has a lifespan of one hour, following which the tool or SDK involved will make an automatic background attempt to obtain a new token that is also valid for one hour.

Requirements for OAuth U2M authentication setup

• You must be an administrator for the Databricks account that corresponds to your Databricks workspaces. See also Assign account admin roles to a user.

• You must have Databricks CLI version 0.205 or above installed locally. See Install or update the Databricks CLI.

• You must add Databricks client unified authentication as an OAuth app integration to your Databricks account. To do this by using Databricks CLI version 0.205 or above or the Databricks SDKs for Python, Java, or Go, create or identify a Databricks configuration profile with the following fields in your .databrickscfg file.

  [<some-unique-configuration-profile-name>]
  host       = <account-console-url>
  account_id = <account-id>
  username   = <username>
  password   = <password>
  

  In the preceding fields, replace the following values:

  • Replace <some-unique-configuration-profile-name> with a name for this configuration profile. This name must be unique within your .databrickscfg file.

  • Replace <account-console-url> with https://accounts.cloud.databricks.com. (Do not set this to the value of your Databricks workspace URL.)

  • Replace <account-id> with the value of your Databricks account. See Locate your account ID.

  • Replace <username> with the value of the username of your Databricks account administrator user. (Do not set this to the value of the username of your Databricks workspace user.)

  • Replace <password> the value of the password for your Databricks account administrator user. (Do not set this to the value of the password of your Databricks workspace user.)

• To add Databricks client unified authentication as an OAuth app integration to your Databricks account by using curl instead, set the following environment variables:

  • DATABRICKS_HOST, set to the value https://accounts.cloud.databricks.com. (Do not set this to the value of your Databricks workspace URL.)

  • DATABRICKS_ACCOUNT_ID, set to the value of your Databricks account. See Locate your account ID.

  • DATABRICKS_USERNAME, set to the value of the username of your Databricks account administrator user. (Do not set this to the value of the username of your Databricks workspace user.)

  • DATABRICKS_PASSWORD, set to the value of the password for your Databricks account administrator user. (Do not set this to the value of the password of your Databricks workspace user.)

  To set environment variables, see your operating system’s documentation.

To add Databricks client unified authentication as an OAuth app integration to your Databricks account, run the following command. If the integration succeeds, the response is: {"integration_id":"<integration-id>"} (or similar).

You need to run this command only once for your account. If you are not sure whether Databricks client unified authentication is already integrated as an available OAuth application within your Databricks account, you can safely run this command again. If Databricks client unified authentication is already integrated, the response contains the phrase “Published app integration for appId databricks-cli already exists.”

databricks account published-app-integration create --app-id databricks-cli -p <profile-name>

from databricks.sdk import AccountClient

a = AccountClient(profile = '<profile-name>')
create_result = a.published_app_integration.create(app_id = 'databricks-cli')
print(f"integration_id: {create_result.integration_id}")

import com.databricks.sdk.DatabricksAccount;
import com.databricks.sdk.core.DatabricksConfig;
import com.databricks.sdk.service.oauth2.CreatePublishedAppIntegration;
import com.databricks.sdk.service.oauth2.GetPublishedAppIntegrationOutput;

public class Main {
  public static void main(String[] args) {
    DatabricksAccount a = new DatabricksAccount(new DatabricksConfig().setProfile("<profile-name>"));
    CreatePublishedAppIntegrationOutput result = a.publishedAppIntegration().create(
      new CreatePublishedAppIntegration().setAppId("databricks-cli")
    );
    System.out.println("integration_id: " + result.getIntegrationId());
  }
}

import (
  "context"
  "fmt"

  "github.com/databricks/databricks-sdk-go"
  "github.com/databricks/databricks-sdk-go/service/oauth2"
)

func main() {
  a := databricks.Must(databricks.NewAccountClient(&databricks.Config{
    Profile: "<profile-name>",
  }))

  pai, err := a.PublishedAppIntegration.Create(
    context.Background(),
    oauth2.CreatePublishedAppIntegration{
      AppId: "databricks-cli",
    },
  )

  if err != nil {
    panic(err)
  }

  fmt.Printf("integration_id: %s", pai.IntegrationId)
}

For Linux or macOS:

curl --request POST \
${DATABRICKS_HOST}/api/2.0/accounts/${DATABRICKS_ACCOUNT_ID}/oauth2/published-app-integrations \
--header 'Content-Type: application/json' \
--data '{ "app_id": "databricks-cli" }' \
--user "${DATABRICKS_USERNAME}:${DATABRICKS_PASSWORD}"

For Windows:

curl --request POST ^
%DATABRICKS_HOST%/api/2.0/accounts/%DATABRICKS_ACCOUNT_ID%/oauth2/published-app-integrations ^
--header "Content-Type: application/json" ^
--data "{ \"app_id\": \"databricks-cli\" }" ^
--user "%DATABRICKS_USERNAME%:%DATABRICKS_PASSWORD%"

Configure Databricks client authentication

To finish configuring OAuth U2M authentication with Databricks, you must set the following associated environment variables, .databrickscfg fields, Terraform fields, or Config fields:

• The Databricks host, specified as https://accounts.cloud.databricks.com for account operations or the target workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com for workspace operations.

• The Databricks account ID, for Databricks account operations.

To perform OAuth U2M authentication authentication with Databricks, integrate the following within your code, based on the participating tool or SDK. Note that depending on the Databricks operations that your code calls, you do not necessarily need to be an administrator for the Databricks account:

To use environment variables with a tool or SDK, see the tool’s or SDK’s documentation. See also Environment variables and fields for client unified authentication and the Default order of evaluation for client unified authentication methods and credentials.

For account-level operations, set the following environment variables:

• DATABRICKS_HOST, set to the value of your Databricks account console URL, https://accounts.cloud.databricks.com.

• DATABRICKS_ACCOUNT_ID

For workspace-level operations, set the following environment variables:

• DATABRICKS_HOST, set to the value of your Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.

Create or identify a Databricks configuration profile with the following fields in your .databrickscfg file. If you create the profile, replace the placeholders with the appropriate values. To use the profile with a tool or SDK, see the tool’s or SDK’s documentation. See also Environment variables and fields for client unified authentication and the Default order of evaluation for client unified authentication methods and credentials.

For account-level operations, set the following values in your .databrickscfg file. In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

[<some-unique-configuration-profile-name>]
host       = <account-console-url>
account_id = <account-id>

For workspace-level operations, set the following values in your .databrickscfg file. In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

[<some-unique-configuration-profile-name>]
host = <workspace-url>

For account-level operations, you should first use the Databricks CLI to run the following command, before you apply your Terraform configuration. This command instructs the Databricks CLI to generate and cache the necessary OAuth token in the path .databricks/token-cache.json within your user’s home folder on your machine:

databricks auth login --host <account-console-url> --account-id <account-id>

Replace the following placeholders:

• Replace <account-console-url> with the value https://accounts.cloud.databricks.com. (Do not set this to the value of your Databricks workspace URL.)

• Replace <account-id> with the value of your Databricks account. See Locate your account ID.

Note

If you have an existing Databricks configuration profile with the host and account_id fields already set, you can substitute --host <account-console-url> --account-id <account-id> with --profile <profile-name>.

After you run the auth login command, you are prompted to save the account login URL and account ID as a Databricks configuration profile. When prompted, enter the name of a new or existing profile in your .databrickscfg file. Any existing profile with the same name in your .databrickscfg file is overwritten.

If prompted, complete your web browser’s on-screen instructions to complete the login. Then use Terraform code similar to one of the following snippets:

For default authentication:

provider "databricks" {
  alias = "account"
}

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider). In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

provider "databricks" {
  alias      = "account"
  host       = <retrieve-account-console-url>
  account_id = <retrieve-account-id>
}

For workspace-level operations, you should first use the Databricks CLI to run the following command, before you apply your Terraform configuration. This command instructs the Databricks CLI to generate and cache the necessary OAuth token in the path .databricks/token-cache.json within your user’s home folder on your machine:

databricks auth login --host <workspace-url>

Replace the placeholder <workspace-url> with the target Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.

Note

If you have an existing Databricks configuration profile with the host field already set, you can substitute --host <workspace-url> with --profile <profile-name>.

After you run the auth login command, you are prompted to save the workspace URL as a Databricks configuration profile. When prompted, enter the name of a new or existing profile in your .databrickscfg file. Any existing profile with the same name in your .databrickscfg file is overwritten.

If prompted, complete your web browser’s on-screen instructions to complete the login. Then use Terraform code similar to one of the following snippets:

For default authentication:

provider "databricks" {
  alias = "workspace"
}

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

provider "databricks" {
  alias = "workspace"
  host  = <retrieve-workspace-url>
}

For account-level operations, you should first use the Databricks CLI to run the following command, before you run your Python code. This command instructs the Databricks CLI to generate and cache the necessary OAuth token in the path .databricks/token-cache.json within your user’s home folder on your machine:

databricks auth login --host <account-console-url> --account-id <account-id>

Replace the following placeholders:

• Replace <account-console-url> with the value https://accounts.cloud.databricks.com. (Do not set this to the value of your Databricks workspace URL.)

• Replace <account-id> with the value of your Databricks account. See Locate your account ID.

Note

If you have an existing Databricks configuration profile with the host and account_id fields already set, you can substitute --host <account-console-url> --account-id <account-id> with --profile <profile-name>.

After you run the auth login command, you are prompted to save the account login URL and account ID as a Databricks configuration profile. When prompted, enter the name of a new or existing profile in your .databrickscfg file. Any existing profile with the same name in your .databrickscfg file is overwritten.

If prompted, complete your web browser’s on-screen instructions to complete the login. Then use Python code similar to one of the following snippets:

For default authentication:

from databricks.sdk import AccountClient

a = AccountClient()
# ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

from databricks.sdk import AccountClient

a = AccountClient(
  host       = retrieveAccountConsoleUrl(),
  account_id = retrieveAccountId()
)
# ...

For workspace-level operations, you should first use the Databricks CLI to run the following command, before you run your Python code. This command instructs the Databricks CLI to generate and cache the necessary OAuth token in the path .databricks/token-cache.json within your user’s home folder on your machine:

databricks auth login --host <worskpace-url>

Replace the placeholder <workspace-url> with the target Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.

Note

If you have an existing Databricks configuration profile with the host field already set, you can substitute --host <workspace-url> with --profile <profile-name>.

Ater you run the auth login command, you are prompted to save the workspace URL as a Databricks configuration profile. When prompted, enter the name of a new or existing profile in your .databrickscfg file. Any existing profile with the same name in your .databrickscfg file is overwritten.

If prompted, complete your web browser’s on-screen instructions to complete the login. Then use Python code similar to one of the following snippets:

For default authentication:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient()
# ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient(host = retrieveWorkspaceUrl())
# ...

For account-level operations, you should first use the Databricks CLI to run the following command, before you run your Java code. This command instructs the Databricks CLI to generate and cache the necessary OAuth token in the path .databricks/token-cache.json within your user’s home folder on your machine:

databricks auth login --host <account-console-url> --account-id <account-id>

Replace the following placeholders:

• Replace <account-console-url> with the value https://accounts.cloud.databricks.com. (Do not set this to the value of your Databricks workspace URL.)

• Replace <account-id> with the value of your Databricks account. See Locate your account ID.

Note

If you have an existing Databricks configuration profile with the host and account_id fields already set, you can substitute --host <account-console-url> --account-id <account-id> with --profile <profile-name>.

After you run the auth login command, you are prompted to save the account login URL and account ID as a Databricks configuration profile. When prompted, enter the name of a new or existing profile in your .databrickscfg file. Any existing profile with the same name in your .databrickscfg file is overwritten.

If prompted, complete your web browser’s on-screen instructions to complete the login. Then use Java code similar to one of the following snippets:

For default authentication:

import com.databricks.sdk.AccountClient;
// ...
AccountClient a = new AccountClient();
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

import com.databricks.sdk.AccountClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
  .setHost(retrieveAccountConsoleUrl())
  .setAccountId(retrieveAccountId());
AccountClient a = new AccountClient(cfg);
// ...

For workspace-level operations, you should first use the Databricks CLI to run the following command, before you run your Java code. This command instructs the Databricks CLI to generate and cache the necessary OAuth token in the path .databricks/token-cache.json within your user’s home folder on your machine:

databricks auth login --host <worskpace-url>

Replace the placeholder <workspace-url> with the target Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.

Note

If you have an existing Databricks configuration profile with the host field already set, you can substitute --host <workspace-url> with --profile <profile-name>.

After you run the auth login command, you are prompted to save the workspace URL as a Databricks configuration profile. When prompted, enter the name of a new or existing profile in your .databrickscfg file. Any existing profile with the same name in your .databrickscfg file is overwritten.

If prompted, complete your web browser’s on-screen instructions to complete the login. Then use Java code similar to one of the following snippets:

For default authentication:

import com.databricks.sdk.WorkspaceClient;
// ...
WorkspaceClient w = new WorkspaceClient();
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

import com.databricks.sdk.WorkspaceClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
  .setHost(retrieveWorkspaceUrl())
WorkspaceClient w = new WorkspaceClient(cfg);
// ...

For account-level operations, you should first use the Databricks CLI to run the following command, before you run your Go code. This command instructs the Databricks CLI to generate and cache the necessary OAuth token in the path .databricks/token-cache.json within your user’s home folder on your machine:

databricks auth login --host <account-login-url> --account-id <account-id>

Replace the following placeholders:

• Replace <account-console-url> with the value https://accounts.cloud.databricks.com. (Do not set this to the value of your Databricks workspace URL.)

• Replace <account-id> with the value of your Databricks account. See Locate your account ID.

Note

If you have an existing Databricks configuration profile with the host and account_id fields already set, you can substitute --host <account-console-url> --account-id <account-id> with --profile <profile-name>.

After you run the auth login command, you are prompted to save the account login URL and account ID as a Databricks configuration profile. When prompted, enter the name of a new or existing profile in your .databrickscfg file. Any existing profile with the same name in your .databrickscfg file is overwritten.

If prompted, complete your web browser’s on-screen instructions to complete the login. Then use Go code similar to one of the following snippets:

For default authentication:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient())
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the Databricks account console URL is https://accounts.cloud.databricks.com:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient(&databricks.Config{
  Host:      retrieveAccountConsoleUrl(),
  AccountId: retrieveAccountId(),
}))
// ...

For workspace-level operations, you should first use the Databricks CLI to run the following command, before you run your Go code. This command instructs the Databricks CLI to generate and cache the necessary OAuth token in the path .databricks/token-cache.json within your user’s home folder on your machine:

databricks auth login --host <worskpace-url>

Replace the placeholder <workspace-url> with the target Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.

Note

If you have an existing Databricks configuration profile with the host field already set, you can substitute --host <workspace-url> with --profile <profile-name>.

After you run the auth login command, you are prompted to save the workspace URL as a Databricks configuration profile. When prompted, enter the name of a new or existing profile in your .databrickscfg file. Any existing profile with the same name in your .databrickscfg file is overwritten.

If prompted, complete your web browser’s on-screen instructions to complete the login. Then use Go code similar to one of the following snippets:

For default authentication:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient())
// ...

For direct configuration (replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as AWS Systems Manager Parameter Store). In this case, the host is the Databricks workspace URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient(&databricks.Config{
  Host: retrieveWorkspaceUrl(),
}))
// ...

Default order of evaluation for client unified authentication methods and credentials

Whenever a participating tool or SDK needs to authenticate with Databricks, the tool or SDK tries the following types of authentication in the following order by default. When the tool or SDK succeeds with the type of authentication that it tries, the tool or SDK stops trying to authenticate with the remaining authentication types. To force an SDK to authenticate with a specific authentication type, set the Config API’s Databricks authentication type field.

1. Databricks personal access token authentication

2. Basic authentication

3. OAuth machine-to-machine (M2M) authentication

4. OAuth user-to-machine (U2M) authentication

For each authentication type that the participating tool or SDK tries, the tool or SDK tries to find authentication credentials in the following locations, in the following order. When the tool or SDK succeeds in finding authentication credentials that can be used, the tool or SDK stops trying to find authentication credentials in the remaining locations.

1. Credential-related Config API fields (for SDKs). To set Config fields, see the SDK’s reference documentation.

2. Credential-related environment variables. To set environment variables, see your operating system’s documentation.

3. Credential-related fields in the DEFAULT configuration profile within the .databrickscfg file. To set configuration profile fields, see Databricks configuration profiles.

To provide maximum portability for your code, Databricks recommends that you create a custom configuration profile within the .databrickscfg file, add the required fields for your target Databricks authentication type to the custom configuration profile, and then set the DATABRICKS_CONFIG_PROFILE environment variable to the name of the custom configuration profile.

Environment variables and fields for client unified authentication

The following tables list the names and descriptions of the supported environment variables and fields for Databricks client unified authentication. In the following tables:

• Environment variable, where applicable, is the name of the environment variable. To set environment variables, see your operating system’s documentation.

• .databrickscfg field, where applicable, is the name of the field within a Databricks configuration profiles file or Databricks Terraform configuration. To set .databrickscfg fields, see Databricks configuration profiles.

• Terraform field, where applicable, is the name of the field within a Databricks Terraform configuration. To set Databricks Terraform fields, see Authentication in the Databricks Terraform provider documentation.

• Config field is the name of the field within the Config API for the specified SDK. To use the Config API, see the SDK’s reference documentation.

General host, token, and account ID environment variables and fields

Common name	Description	Environment variable	.databrickscfg field, Terraform field	Config field
Databricks host	(String) The Databricks host URL for either the Databricks workspace endpoint or the Databricks accounts endpoint.	DATABRICKS_HOST	host	host (Python), setHost (Java), Host (Go)
Databricks token	(String) The Databricks personal access token.	DATABRICKS_TOKEN	token	token (Python), setToken (Java), Token (Go)
Databricks account ID	(String) The Databricks account ID for the Databricks account endpoint. Only has effect when the Databricks host is also set to https://accounts.cloud.databricks.com.	DATABRICKS_ACCOUNT_ID	account_id	account_id (Python), setAccountID (Java), AccountID (Go)

AWS-specific environment variables and fields

Common name	Description	Environment variable	.databrickscfg field, Terraform field	Config field
Databricks username	(String) The Databricks user’s username.	DATABRICKS_USERNAME	username	username (Python), setUsername (Java), Username (Go)
Databricks password	(String) The Databricks user’s password.	DATABRICKS_PASSWORD	password	password (Python), setPassword (Java), Password (Go)
Service principal client ID	(String) The Databricks service principal’s client ID.	DATABRICKS_CLIENT_ID	client_id	client_id (Python), setClientId (Java), ClientId (Go)
Service principal secret	(String) The Databricks service principal’s secret.	DATABRICKS_CLIENT_SECRET	client_secret	client_secret (Python), setClientSecret (Java), ClientSecret (Go)

.databrickscfg-specific environment variables and fields

Use these environment variables or fields to specify non-default settings for .databrickscfg. See also Databricks configuration profiles.

Common name	Description	Environment variable	Terraform field	Config field
.databrickscfg file path	(String) A non-default path to the .databrickscfg file.	DATABRICKS_CONFIG_FILE	config_file	config_file (Python), setConfigFile (Java), ConfigFile (Go)
.databrickscfg default profile	(String) The default named profile to use, other than DEFAULT.	DATABRICKS_CONFIG_PROFILE	profile	profile (Python), setProfile (Java), Profile (Go)

Authentication type field

Use this environment variable or field to force an SDK to use a specific type of Databricks authentication.

Common name	Description	Terraform field	Config field
Databricks authentication type	(String) When multiple authentication attributes are available in the environment, use the authentication type specified by this argument.	auth_type	auth_type (Python), setAuthType (Java), AuthType (Go)

Supported Databricks authentication type field values include:

• pat: Databricks personal access token authentication

• basic: Basic authentication

• oauth-m2m: OAuth machine-to-machine (M2M) authentication

• databricks-cli: OAuth user-to-machine (U2M) authentication

Databricks configuration profiles

A Databricks configuration profile (sometimes refered to as a configuration profile, a config profile, or simply a profile) contains settings and other information that Databricks needs to authenticate. Databricks configuration profiles are stored in Databricks configuration profiles files for your tools, SDKs, scripts, and apps to use. To learn whether Databricks configuration profiles are supported by your tools, SDKs, scripts, and apps, see your provider’s documentation. All participating tools and SDKs that implement Databricks client unified authentication support Databricks configuration profiles.

To create a Databricks configuration profiles file:

1. Use your favorite text editor to create a file named .databrickscfg in your ~ (your user home) folder on Unix, Linux, or macOS, or your %USERPROFILE% (your user home) folder on Windows, if you do not already have one. Do not forget the dot (.) at the beginning of the file name. Add the following contents to this file:

   [<some-unique-name-for-this-configuration-profile>]
   <field-name> = <field-value>
   

2. In the preceding contents, replace the following values, and then save the file:

   • <some-unique-name-for-this-configuration-profile> with a unique name for the configuration profile, such as DEFAULT, DEVELOPMENT, PRODUCTION, or similar. You can have multiple configuration profiles in the same .databrickscfg file, but each configuration profile must have a unique name within this file.

   • <field-name> and <field-value> with the name and a value for one of the required fields for the target Databricks authentication type. For the specific information to provide, see the section earlier in this article for that authentication type.

   • Add a <field-name> and <field-value> pair for each of the additional required fields for the target Databricks authentication type.

For example, for Databricks personal access token authentication, the .databrickscfg file might look like this:

[DEFAULT]
host  = https://dbc-a1b2345c-d6e7.cloud.databricks.com
token = dapi123...

To create additional configuration profiles, specify different profile names within the same .databrickscfg file. For example, to specify separate Databricks workspaces, each with their own Databricks personal access token:

[DEFAULT]
host  = https://dbc-a1b2345c-d6e7.cloud.databricks.com
token = dapi123...

[DEVELOPMENT]
host  = https://dbc-b2c3456d-e7f8.cloud.databricks.com
token = dapi234...

You can also specify different profile names within the .databrickscfg file for Databricks accounts and different Databricks authentication types, for example:

[DEFAULT]
host  = https://dbc-a1b2345c-d6e7.cloud.databricks.com
token = dapi123...

[ACCOUNT]
host       = https://accounts.cloud.databricks.com
username   = someone@example.com
password   = MyP25...
account_id = ab0cd1...

ODBC DSNs

In ODBC, a data source name (DSN) is a symbolic name that tools, SDKs, scripts, and apps use to request a connection to an ODBC data source. A DSN stores connection details such as the path to an ODBC driver, networking details, authentication credentials, and database details. To learn whether ODBC DSNs are supported by your tools, scripts, and apps, see your provider’s documentation.

To install and configure the Databricks ODBC Driver and create an ODBC DSN for Databricks, see ODBC driver.

JDBC connection URLs

In JDBC, a connection URL is a symbolic URL that tools, SDKs, scripts, and apps use to request a connection to a JDBC data source. A connection URL stores connection details such as networking details, authentication credentials, database details, and JDBC driver capabilities. To learn whether JDBC connection URLs are supported by your tools, SDKs, scripts, and apps, see your provider’s documentation.

To install and configure the Databricks JDBC Driver and create a JDBC connection URL for Databricks, see JDBC driver.

OAuth for service principals

Preview

This feature is in Public Preview.

OAuth is supported for Databricks service principals at both the Databricks account and workspace levels. You can use the same OAuth token for both the account and workspaces in the account, as long as the service principal has the correct access. Account admins can create a client secret for a service principal. You can then use the client secret with the client ID, also known as the service principal’s application ID, to request an OAuth token for the service principal. The OAuth tokens last for one hour. See Authentication using OAuth for service principals.

Note

Databricks does not recommend that you create OAuth tokens for Databricks service principals manually. This is because each OAuth token is short-lived, typically expiring within one hour. After this time, you must manually generate a replacement OAuth token. Instead, use one of the participating tools or SDKs that implement the Databricks client unified authentication standard. These tools and SDKs automatically generate and replace expired OAuth tokens for Databricks service principals for you, leveraging OAuth machine-to-machine (M2M) authentication.

An account admin’s username and password can also be used to authenticate to Databricks account-level APIs. However, Databricks strongly recommends that you use OAuth for service principals. For an example of using a username and password to authenticate, see How to use the Account API.