Configure OAuth for dbt
Preview
This feature is in Public Preview.
This article describes how to enable your Databricks account for OAuth and add dbt as an OAuth application. After you complete this one-time configuration as a Databricks account admin, users can connect Databricks to dbt Core using OAuth.
Before you begin
Before you configure OAuth for dbt:
You must be a Databricks account administrator.
(Optional) To use a custom IdP for dbt login, see Set up SSO in your Databricks account console.
Authenticate to the Account API
Preview
OAuth is in Public Preview.
To authenticate to the Account API, you can use Databricks OAuth for service principals, Databricks OAuth for users, or a Databricks account admin’s username and password. Databricks strongly recommends that you use Databricks OAuth for users or service principals. A service principal is an identity that you create in Databricks for use with automated tools, jobs, and applications. To create an OAuth token, see Authentication using OAuth for service principals.
Use the following examples to authenticate to a Databricks account. You can use OAuth for service principals, OAuth for users, or a user’s username and password. For background, see:
For OAuth for service principals, see OAuth machine-to-machine (M2M) authentication.
For OAuth for users, see OAuth user-to-machine (U2M) authentication.
For a user’s username and password, see Basic authentication.
For authentication examples, choose from the following:
Install Databricks CLI version 0.205 or above. See Install or update the Databricks CLI.
Complete the steps to configure OAuth M2M authentication for service principals in the account. See OAuth machine-to-machine (M2M) authentication.
Identify or manually create a Databricks configuration profile in your
.databrickscfg
file, with the profile’s fields set correctly for the relatedhost
,account_id
, andclient_id
andclient_secret
mapping to the service principal. See OAuth machine-to-machine (M2M) authentication.Run your target Databricks CLI command, where
<profile-name>
represents the name of the configuration profile in your.databrickscfg
file:databricks account <command-name> <subcommand-name> -p <profile-name>
For example, to list all users in the account:
databricks account users list -p MY-AWS-ACCOUNT
For a list of available account commands, run the command
databricks account -h
.For a list of available subcommands for an account command, run the command
databricks account <command-name> -h
.
Install Databricks CLI version 0.205 or above. See Install or update the Databricks CLI.
Complete the steps to configure OAuth U2M authentication for users in the account. See OAuth user-to-machine (U2M) authentication.
Start the user authentication process by running the following Databricks CLI command:
databricks auth login --host <account-console-url> --account-id <account-id>
For example:
databricks auth login --host https://accounts.cloud.databricks.com --account-id 00000000-0000-0000-0000-000000000000
Note
If you have an existing Databricks configuration profile with the
host
andaccount_id
fields already set, you can substitute--host <account-console-url> --account-id <account-id>
with--profile <profile-name>
.Follow the on-screen instructions to have the Databricks CLI automatically create the related Databricks configuration profile in your
.databrickscfg
file.Continue following the on-screen instructions to sign in to your Databricks account through your web browser.
Run your target Databricks CLI command, where
<profile-name>
represents the name of the configuration profile in your.databrickscfg
file:databricks account <command-name> <subcommand-name> -p <profile-name>
For example, to list all users in the account:
databricks account users list -p ACCOUNT-00000000-0000-0000-0000-000000000000
For a list of available account commands, run the command
databricks account -h
.For a list of available subcommands for an account command, run the command
databricks account <command-name> -h
.
Install Databricks CLI version 0.205 or above. See Install or update the Databricks CLI.
Identify or manually create a Databricks configuration profile in your
.databrickscfg
file, with the profile’s fields set correctly for the relatedhost
,account_id
, andusername
andpassword
mapping to your Databricks user account. See Basic authentication.Run your target Databricks CLI command, where
<profile-name>
represents the name of the configuration profile in your.databrickscfg
file:databricks account <command-name> <subcommand-name> -p <profile-name>
For example, to list all users in the account:
databricks account users list -p MY-AWS-ACCOUNT
For a list of available account commands, run the command
databricks account -h
.For a list of available subcommands for an account command, run the command
databricks account <command-name> -h
.
Add dbt as an OAuth application
To add dbt as an OAuth application to your Databricks account, run the following curl
command:
curl -X POST https://accounts.cloud.databricks.com/api/2.0/accounts/<Account ID>/oauth2/published-app-integrations -d '{ "app_id" : "databricks-dbt-adapter" }' \
--header "Authorization: Bearer $OAUTH_TOKEN"
Important
(Windows) You must use double quotes around the data argument and escape all quotes inside this. For example:
curl -X POST https://accounts.cloud.databricks.com/api/2.0/accounts/<Account ID>/oauth2/published-app-integrations -d "{ \"app_id\" : \"databricks-dbt-adapter\" }" \
--header "Authorization: Bearer $OAUTH_TOKEN"
The following output returns:
{"integration_id":"<Integration ID>"}
Next steps
You can now use OAuth to connect Databricks to dbt Core. See Connect to dbt Core.