Authentication setup for the Databricks extension for VS Code

This article describes how to set up authentication between the Databricks extension for Visual Studio Code and your Databricks workspace. See What is the Databricks extension for Visual Studio Code?

The Databricks extension for Visual Studio Code implements portions of the Databricks client unified authentication standard, a consolidated and consistent architectural and programmatic approach to authentication. This approach helps make setting up and automating authentication with Databricks more centralized and predictable. It enables you to configure Databricks authentication once and then use that configuration across multiple Databricks tools and SDKs without further authentication configuration changes.

The following information assumes that you have already installed the Databricks extension for Visual Studio Code. See Install the Databricks extension for Visual Studio Code.

Note

If you have already completed the VSCode extension for Databricks tutorial: Run Python on a cluster and as a job, and want to keep using the Databricks personal access token authentication that you set up in that tutorial, skip ahead to Select a cluster for the Databricks extension for Visual Studio Code.

Depending on the type of authentication that you want to use, complete the instructions for your target Databricks authentication type.

For the following authentication types, skip ahead to Set up authentication with a configuration profile:
For OAuth user-to-machine (U2M) authentication, skip ahead to Set up OAuth U2M authentication.

Set up authentication with a configuration profile

Note

The following procedure uses the Databricks extension for Visual Studio Code, version 1, which is generally available. To complete this procedure for the Databricks extension for Visual Studio Code, version 2, currently in Private Preview, skip ahead to Set up authentication with a configuration profile and version 2 of the extension.

The following instructions assume that you have already set up a Databricks configuration profile with the required fields for your Databricks authentication type. See Databricks configuration profiles and the Profile tab or section for your authentication type in Authentication for Databricks automation - overview.

For example, if you want to authenticate by using Databricks personal access token authentication, create your personal access token by doing the following:

In your Databricks workspace, click your Databricks username in the top bar, and then select Settings from the drop down.
Click Developer.
Next to Access tokens, click Manage.
Click Generate new token.
(Optional) Enter a comment that helps you to identify this token in the future, and change the token’s default lifetime of 90 days. To create a token with no lifetime (not recommended), leave the Lifetime (days) box empty (blank).
Click Generate.
Copy the displayed token to a secure location, and then click Done.

Note

Be sure to save the copied token in a secure location. Do not share your copied token with others. If you lose the copied token, you cannot regenerate that exact same token. Instead, you must repeat this procedure to create a new token. If you lose the copied token, or you believe that the token has been compromised, Databricks strongly recommends that you immediately delete that token from your workspace by clicking the trash can (Revoke) icon next to the token on the Access tokens page.

If you are not able to create or use tokens in your workspace, this might be because your workspace administrator has disabled tokens or has not given you permission to create or use tokens. See your workspace administrator or the following:

And, for example, to create a configuration profile named DEFAULT for your personal access token, do the following:

Note

The following procedure uses the Databricks CLI to create a Databricks configuration profile with the name DEFAULT. If you already have a DEFAULT configuration profile, this procedure overwrites your existing DEFAULT configuration profile.

To check whether you already have a DEFAULT configuration profile, and to view this profile’s settings if it exists, use the Databricks CLI to run the command databricks auth env --profile DEFAULT.

To create a configuration profile with a name other than DEFAULT, replace the DEFAULT part of --profile DEFAULT in the following databricks configure command with a different name for the configuration profile.

Use the Databricks CLI to create a Databricks configuration profile named DEFAULT that uses Databricks personal access token authentication. To do this, run the following command:
```
databricks configure --profile DEFAULT
```
For the prompt Databricks Host, enter your Databricks workspace instance URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.
For the prompt Personal Access Token, enter the Databricks personal access token for your workspace.

After you have your Databricks configuration profile ready, with your project and the extension opened, do the following:

In the Configuration pane, click Configure Databricks.

Note

If Configure Databricks is not visible, click the gear (Configure workspace) icon next to Configuration instead.
In the Command Palette, for Databricks Host, enter your workspace instance URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com. Then press Enter.
Select your target Databricks configuration profile in the list for the URL.

The extension creates a hidden folder in your project named .databricks if it does not already exist. The extension also creates in this folder a file named project.json if it does not already exist. This file contains the URL that you entered, along with some Databricks authentication details that the Databricks extension for Visual Studio Code needs to operate.

The extension also adds a hidden .gitignore file to the project if the file does not exist or if an existing .gitignore cannot be found in any parent folders. If a new .gitignore file is created, the extension adds a .databricks/ entry to this new file. If the extension finds an existing .gitignore file, it adds a .databricks/ entry to the existing file.

Continue with Select a cluster for the Databricks extension for Visual Studio Code.

Set up OAuth U2M authentication

Note

The following procedure uses the Databricks extension for Visual Studio Code, version 1, which is generally available. To complete this procedure for the Databricks extension for Visual Studio Code, version 2, currently in Private Preview, skip ahead to Set up OAuth U2M authentication with version 2 of the extension.

Databricks supports OAuth user to machine (U2M) authentication. This enables you to generate short-lived (one hour) OAuth access tokens, which eliminates the risk of accidentally exposing longer-lived tokens such as Databricks personal access tokens through version control checkins or other means. This also enables better server-side session invalidation and scoping.

To complete the setup for OAuth U2M authentication, with your project and the extension opened, do the following:

In the Configuration pane, click Configure Databricks.

Note

If Configure Databricks is not visible, click the gear (Configure workspace) icon next to Configuration instead.
In the Command Palette, for Databricks Host, enter your workspace instance URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com. Then press Enter.
Select OAuth (user to machine).
Complete the on-screen instructions in your web browser to finish authenticating with your Databricks account. If prompted, allow all-apis access.

The extension creates a hidden folder in your project named .databricks if it does not already exist. The extension also creates in this folder a file named project.json if it does not already exist. This file contains the URL that you entered, along with some Databricks authentication details that the Databricks extension for Visual Studio Code needs to operate.

The extension also adds a hidden .gitignore file to the project if the file does not exist or if an existing .gitignore cannot be found in any parent folders. If a new .gitignore file is created, the extension adds a .databricks/ entry to this new file. If the extension finds an existing .gitignore file, it adds a .databricks/ entry to the existing file.

Continue with Select a cluster for the Databricks extension for Visual Studio Code.

Set up authentication with a configuration profile and version 2 of the extension

Note

The Databricks extension for Visual Studio Code, version 2 is in Private Preview.

The following instructions assume that you have already set up a Databricks configuration profile with the required fields for your Databricks authentication type. To create a profile, see Databricks configuration profiles. To set the correct settings for the profile, see the Profile tab or section for your authentication type in Authentication for Databricks automation - overview. The extension’s user interface sometimes refers to these configuration profiles as Databricks CLI profiles or just profiles.

To select the configuration profile that you want the extension to use, with your project and the extension open, do the following:

In the Configuration pane, click Auth Type, and then click the gear (Sign in to Databricks workspace) icon.
In the Command Palette, select your target Databricks configuration profile in the list.
Continue with Select a cluster for the Databricks extension for Visual Studio Code.

If you do not have a configuration profile, you can create one that authenticates by using a Databricks personal access token for example, as follows:

With your project and the extension open, in the Configuration pane, click Auth Type or Login to Databricks, and then click the gear (Sign in to Databricks workspace) icon.
In the Command Palette, select Personal Access Token.
Complete the on-screen prompts to do the following:
1. Enter some name for the associated Databricks authentication profile.
2. Generate the personal access token in your Databricks workspace and then copy the token’s value.
3. Return to the extension and enter the copied token’s value.
Continue with Select a cluster for the Databricks extension for Visual Studio Code.

Set up OAuth U2M authentication with version 2 of the extension

Note

The Databricks extension for Visual Studio Code, version 2 is in Private Preview.

Databricks supports OAuth user to machine (U2M) authentication. This enables you to generate short-lived (one hour) OAuth access tokens, which eliminates the risk of accidentally exposing longer-lived tokens such as Databricks personal access tokens through version control checkins or other means. This also enables better server-side session invalidation and scoping.

To complete the setup for OAuth U2M authentication, with your project and the extension opened, do the following:

In the Configuration pane, click Auth Type, and then click the gear (Sign in to Databricks workspace) icon.
If you already have an authentication configuration profile in this list that has the Authenticate using OAuth (User to Machine) label and that you know corresponds to the target Databricks host, select it from the list, and then do the following:
1. If prompted, complete any on-screen instructions in your web browser to finish authenticating with Databricks.
2. If also prompted, allow all-apis access.
3. Skip ahead to Select a cluster for the Databricks extension for Visual Studio Code.
In the Command Palette, for Select authentication method, click OAuth (user to machine).
For Select authentication method, select OAuth (user to machine).
Enter some name for the associated Databricks authentication profile.
In the Configuration pane, click Login to Databricks.
In the Command Palette, for Select authentication method, select the name of the authentication configuration profile that you just created.
If prompted, complete any on-screen instructions in your web browser to finish authenticating with Databricks. If also prompted, allow all-apis access.
After you have successfully logged in, return to Visual Studio Code.
Continue with Select a cluster for the Databricks extension for Visual Studio Code.