Skip to main content
Last updated on

Authentication for the Databricks CLI

note

Databricks CLI use is subject to the Databricks License and Databricks Privacy Notice, including any Usage Data provisions.

This article explains how to set up authentication between the Databricks CLI and your Databricks accounts and workspaces. It assumes you already installed the Databricks CLI. See Install or update the Databricks CLI.

Before you run Databricks CLI commands, you must configure authentication for the accounts or workspaces that you plan to use. The required setup depends on whether you want to run workspace-level commands, account-level commands, or both.

To view available CLI command groups, run databricks -h. For the list of corresponding REST API operations, see Databricks REST API.

OAuth machine-to-machine (M2M) authentication​

Machine-to-machine (M2M) authentication with OAuth allows services, scripts, or applications to access Databricks resources without interactive user sign-in. Instead of relying on personal access tokens (PATs) or user credentials, M2M authentication uses a service principal and an OAuth client credential flow to request and manage tokens.

To configure and use OAuth M2M authentication:

  1. Complete the OAuth M2M authentication setup steps. See Authorize service principal access to Databricks with OAuth.

  2. Create a Databricks configuration profile with the following fields in your .databrickscfg file.

    For account-level commands

    [<some-unique-configuration-profile-name>]
    host          = <account-console-url>
    account_id    = <account-id>
    client_id     = <service-principal-client-id>
    client_secret = <service-principal-oauth-secret>

    For workspace-level commands

    [<some-unique-configuration-profile-name>]
    host          = <workspace-url>
    client_id     = <service-principal-client-id>
    client_secret = <service-principal-oauth-secret>

To use the profile, pass it with the --profile or -p flag in CLI commands. For example:

Bash
databricks account groups list -p <profile-name>

Press Tab after --profile or -p to show a list of available profiles.

OAuth user-to-machine (U2M) authentication​

With OAuth user-to-machine (U2M) authentication, you log in interactively and the CLI manages short-lived tokens on your behalf. OAuth tokens expire in under an hour, which reduces risk if a token is accidentally exposed. See Authorize user access to Databricks with OAuth.

To log in:

For account-level commands

Bash
databricks auth login --host <account-console-url> --account-id <account-id>

For workspace-level commands

Bash
databricks auth login --host <workspace-url>

The CLI guides you through a browser-based login flow. When you finish, the CLI saves a configuration profile with your workspace host and profile name. You can accept the suggested profile name or enter your own.

To use the profile, pass it with the --profile or -p flag in CLI commands. For example:

Bash
databricks clusters list -p <profile-name>

Press Tab after --profile or -p to show a list of available profiles.

Token storage​

By default starting in Databricks CLI version 1.0.0, U2M tokens are stored in OS-native secure storage: Keychain on macOS, Credential Manager on Windows, and D-Bus Secret Service on Linux. Your .databrickscfg file holds only non-secret configuration such as host and profile name. The token itself is never written there.

Each profile has its own cached token. If you have multiple profiles pointing at the same workspace, logging out of one does not log out of the others.

If your environment does not support OS-native secure storage (for example, a headless server or CI environment), you can fall back to plaintext token storage.

For troubleshooting secure storage errors, see Troubleshoot the Databricks CLI.

Use the plaintext fallback​

If your environment does not support secure storage, or if you depend on file-based token storage (for example, syncing the credentials file over SSH), you can configure the CLI to use the plain JSON file with either of the following options. After setting either option, run databricks auth login again.

  • Configure the DATABRICKS_AUTH_STORAGE environment variable.

    Bash
    export DATABRICKS_AUTH_STORAGE=plaintext

  • Set the auth_storage setting in the Databricks configuration profile file ~/.databrickscfg.

    ini
    [__settings__]
    auth_storage = plaintext

The environment variable takes precedence over the configuration profile setting.

Authentication order of evaluation​

Whenever the Databricks CLI authenticates to a Databricks workspace or account, it looks for required settings in the following order:

  1. Bundle settings files, for commands run from a bundle working directory. Bundle settings files can't contain credential values directly.
  2. Environment variables, as listed in this article and in Environment variables and fields for unified authentication.
  3. Configuration profiles in the .databrickscfg file.

As soon as the CLI finds the required setting, it stops searching other locations.

Examples:

  • If a DATABRICKS_TOKEN environment variable is set, the CLI uses it, even if multiple tokens exist in .databrickscfg.
  • If no DATABRICKS_TOKEN is set, and a bundle environment references a profile name such as dev → profile DEV, the CLI uses the credentials from that profile in .databrickscfg.
  • If no DATABRICKS_TOKEN is set, and a bundle environment specifies a host value, the CLI searches for a profile in .databrickscfg with a matching host and uses its token.

Personal access token authentication (legacy)​

important

Where possible, Databricks recommends using OAuth instead of PATs for user account authentication because OAuth provides stronger security. Consider the following authentication methods:

Databricks personal access token authentication uses a Databricks personal access token to authenticate the target Databricks entity, such as a Databricks user account or a Databricks service principal. See Authenticate with Databricks personal access tokens (legacy).

To create a personal access token, follow the steps in Create personal access tokens for workspace users.

On this page