Manage connections in Partner Connect

You can perform administrative tasks with Databricks workspace connections to partner solutions, such as:

  • Managing users of partner accounts.

  • Managing the Databricks service principal and related Databricks personal access token that a connection uses.

  • Disconnecting a workspace from a partner.

To administer Partner Connect, you must sign in to your workspace as a workspace admin. For more information, see Manage users.

Manage partner account users

For partners that allow users to use Partner Connect to sign in to that partner’s account or website (such as Fivetran and Rivery), when someone in your organization connects from one of your Databricks workspaces to a partner for the first time, that person becomes the partner account administrator for that partner across all of your organization’s workspaces. To enable other users within your organization to sign in to that partner, your partner account administrator must first add those users to your organization’s partner account. Some partners allow the partner account administrator to delegate this permission as well. For details, see the documentation on the partner’s website.

If no one can add users to your organization’s partner account (for example, your partner account administrator is no longer available), contact the partner for assistance. For support links, see the list of Databricks Partner Connect partners.

Connect data managed by Unity Catalog to partner solutions

If your workspace is Unity Catalog-enabled, you can connect select partner solutions to data managed by Unity Catalog. When you create the connection using Partner Connect, you can choose whether the partner uses the legacy Hive metastore (hive_metastore) or another catalog that you own. Metastore admins can select any catalog in the metastore that’s assigned to your workspace.

Note

If a partner solution doesn’t support Unity Catalog with Partner Connect, you can only use the workspace default catalog. If the default catalog isn’t hive_metastore and you don’t own the default catalog, you’ll receive an error.

For a list of partners that support Unity Catalog with Partner Connect, see the Databricks Partner Connect partners list.

For information about troubleshooting connections, see Troubleshoot Partner Connect.

Manage service principals and personal access tokens

For partners that require Databricks service principals, when someone in your Databricks workspace connects to a specific partner for the first time, Partner Connect creates a Databricks service principal in your workspace for use with that partner. Partner Connect generates service principal display names by using the format <PARTNER-NAME>_USER. For example, for the partner Fivetran, the service principal’s display name is FIVETRAN_USER.

Partner Connect also creates a Databricks personal access token and associates it with that Databricks service principal. Partner Connect provides this token’s value to the partner behind the scenes to complete the connection to that partner. You cannot view or get this token’s value. This token does not expire until you or someone else deletes it. See also Disconnect from a partner.

Partner Connect grants the following access permissions to Databricks service principals in your workspace:

Partners

Permissions

Fivetran, Matillion, Power BI, Tableau, erwin Data Modeler, Alteryx Designer Cloud, Precisely Data Integrity Suite

These solutions do not require service principals.

Hevo Data, Hunters, Rivery, RudderStack, Snowplow

  • The CAN USE token permission to create a personal access token.

  • SQL warehouse creation permission.

  • Access to your workspace.

  • Access to Databricks SQL.

  • (Unity Catalog) The USE CATALOG privilege on the selected catalog.

  • (Unity Catalog) The CREATE SCHEMA privilege on the selected catalog.

  • (Legacy Hive metastore) The USAGE privilege on the selected catalog.

  • (Legacy Hive metastore) The CREATE privilege on the hive_metastore catalog so Partner Connect can create objects in the legacy Hive metastore on your behalf.

  • Ownership of the tables that it creates. The service principal cannot query any tables that it does not create.

Prophecy

  • The CAN USE token permission to create a personal access token.

  • Access your workspace.

  • Cluster creation permission. The service principal cannot access any clusters that it does not create.

  • Job creation permission. The service principal cannot access any jobs that it does not create.

John Snow Labs, Labelbox

  • The CAN USE token permission to create a personal access token.

  • Access to your workspace.

Alation, Anomalo, AtScale, Census, dbt Cloud, Hex, Hightouch, Lightup, Monte Carlo, Preset, Precisely, Privacera, Qlik Sense, Sigma, Stardog, ThoughtSpot

  • The CAN USE token permission to create a personal access token.

  • The CAN USE privilege on the selected Databricks SQL warehouse.

  • The SELECT privilege on the selected schemas.

  • The IS OWNER permission on jobs.

  • (Unity Catalog) The USE CATALOG privilege on the selected catalog.

  • (Unity Catalog) The USE SCHEMA privilege on the selected schema.

  • (Legacy Hive metastore) The USAGE privilege on the selected schema.

  • (Legacy Hive metastore) The READ METADATA privilege for the selected schemas.

Dataiku

  • The CAN USE token permission to create a personal access token.

  • SQL warehouse creation permission.

  • (Unity Catalog) The USE CATALOG privilege on the selected catalog.

  • (Unity Catalog) The USE SCHEMA privilege on the selected schemas.

  • (Unity Catalog) The CREATE SCHEMA privilege on the selected catalog.

  • (Legacy Hive metastore) The USAGE privilege on the hive_metastore catalog and on the selected schemas.

  • (Legacy Hive metastore) The CREATE privilege on the hive_metastore catalog so Partner Connect can create objects in the legacy Hive metastore on your behalf.

  • (Legacy Hive metastore) The SELECT privilege on the selected schemas.

Alteryx Designer Cloud

  • The CAN USE token permission to create a personal access token.

  • SQL warehouse creation permission.

  • (Unity Catalog) The USE CATALOG privilege on the selected catalog.

  • (Unity Catalog) The USE SCHEMA privilege on the selected schemas.

  • (Unity Catalog) The CREATE SCHEMA privilege on the selected catalog.

  • (Legacy Hive metastore) The USAGE privilege on the hive_metastore catalog and on the selected schemas.

  • (Legacy Hive metastore) The CREATE privilege on the hive_metastore catalog so Partner Connect can create objects in the legacy Hive metastore on your behalf.

  • (Legacy Hive metastore) The SELECT privilege on the selected schemas.

SuperAnnotate

  • The CAN USE token permission to create a personal access token.

  • The CAN USE privilege on the selected Databricks SQL warehouse.

  • The SELECT privilege on the selected schemas.

  • (Unity Catalog) The`USE CATALOG` privilege on the selected catalog.

  • (Unity Catalog) The USE SCHEMA privilege on the selected schemas.

  • (Legacy Hive metastore) The USAGE privilege on the hive_metastore catalog and on the selected schemas.

  • (Legacy Hive metastore) The SELECT privilege on the selected schemas.

Informatica Cloud Data Integration

  • The CAN USE token permission to create a personal access token.

  • The CAN MANAGE privilege on the selected Databricks SQL warehouse.

  • The CREATE and USAGE privilege on data objects.

  • (Unity Catalog) The`USE CATALOG` privilege on the selected catalog.

  • (Unity Catalog) The USE SCHEMA privilege on the selected schemas.

  • (Legacy Hive metastore) The USAGE and CREATE privilege on the hive_metastore catalog and on the selected schemas.

  • (Legacy Hive metastore) The SELECT privilege on the selected schemas.

You might need to generate a new Databricks personal access token if the existing token has been compromised, is lost or deleted, or your organization has a periodic token rotation policy. To generate a new token, use the Databricks REST API:

  1. Get the Databricks service principal’s application ID by calling the GET /preview/scim/v2/ServicePrincipals operation in the Workspace Service Principals API for your workspace. Make a note of the service principal’s applicationId in the response.

  2. Use the service principal’s applicationId to call the POST /token-management/on-behalf-of/tokens operation in the Databricks Token Management REST API for your workspace.

  3. Make a note of the token_value in the response and store it in a safe location, as there is no other way to access it again if you ever need to retrieve it.

For example, to get the list of available Databricks service principal display names and application IDs for a workspace, you can call curl as follows:

curl --netrc --request GET \
https://<databricks-instance>/api/2.0/preview/scim/v2/ServicePrincipals \
| jq '[ .Resources[] | { displayName: .displayName, applicationId: .applicationId } ]'

Replace <databricks-instance> with the Databricks workspace instance name, for example dbc-a1b2345c-d6e7.cloud.databricks.com for the workspace where the service principal exists.

The service principal’s application ID value is in the response’s applicationId field, for example 123456a7-8901-2b3c-45de-f678a901b2c.

To create the new token value for the service principal, you can call curl as follows:

curl --netrc --request POST \
https://<databricks-instance>/api/2.0/token-management/on-behalf-of/tokens \
--data @create-token.json \
| jq '[ . | { token_value: .token_value } ]'

create-token.json:

{
  "application_id": "<application-id>",
  "comment": "Partner Connect",
  "lifetime_seconds": 1209600
}

Replace:

  • <databricks-instance> with the workspace instance name, for example dbc-a1b2345c-d6e7.cloud.databricks.com for the workspace where the service principal exists.

  • <application-id> with the service principal’s application ID value.

  • 1209600 with the number of seconds until this token expires. For example, 1209600 is the number of seconds in 14 days.

The new token’s value is in the response’s token_value field, for example dapi12345678901234567890123456789012. Make a note of the new token’s value in the response and store it in a safe location, as there is no other way to access it again if you ever need to retrieve it.

The preceding examples use a .netrc file and jq. Note that in this case, the .netrc file uses your personal access token value–not the one for the service principal.

After you create the new token, you must update your partner account with the new token’s value. To do this, see the documentation on the partner’s website. For documentation links, see appropriate partner connection guide.

To delete an existing token:

Warning

Deleting an existing Databricks personal access token is permanent and cannot be undone.

  1. Get the list of existing tokens by calling the GET /token-management/tokens GET operation in the Databricks Token Management REST API for your workspace.

  2. In the response, make a note of the token_id value for the token that you want to delete.

  3. Use this token_id value to delete the token by calling the DELETE /token-management/tokens/{token_id} operation in the Databricks Token Management REST API for your workspace.

Disconnect from a partner

If the tile for a partner has a check mark icon, this means that someone in your Databricks workspace has already created a connection to that partner. To disconnect from that partner, you reset that partner’s tile in Partner Connect. Resetting a partner’s tile does the following:

  • Clears the check mark icon from the partner’s tile.

  • Deletes the associated SQL warehouse or cluster if the partner requires one.

  • Deletes the associated Databricks service principal, if the partner requires one. Deleting a service principal also deletes that service principal’s related Databricks personal access token. This token’s value is what completes the connection between your workspace and the partner. For more information, see Manage service principals and personal access tokens.

Warning

Deleting a SQL warehouse, a cluster, a Databricks service principal, or a Databricks service principal’s personal access token is permanent and cannot be undone.

Resetting a partner’s tile does not delete your organization’s related partner account or change related connection settings with the partner. However, resetting a partner’s tile does break the connection between the workspace and the related partner account. To reconnect, you must create a new connection from the workspace to the partner, and then you must manually edit the original connection settings in the related partner account to match the new connection settings.

To reset a partner’s tile, click the tile, click Delete Connection, and then follow the on-screen directions.

Alternatively, you can manually disconnect a Databricks workspace from a partner by deleting the related Databricks service principal in your workspace that is associated with that partner. You might want to do this if you want to disconnect your workspace from a partner but still keep other associated resources and still keep the check mark icon displayed on the tile. Deleting a service principal also deletes that service principal’s related personal access token. This token’s value is what completes the connection between your workspace and the partner. For more information, see Manage service principals and personal access tokens.

To delete a Databricks service principal, you use the Databricks REST API as follows:

  1. Get the Databricks service principal’s application ID by calling the GET /preview/scim/v2/ServicePrincipals operation in the Workspace Service Principals API for your workspace. Make a note of the service principal’s applicationId in the response.

  2. Use the service principal’s applicationId to call the DELETE /preview/scim/v2/ServicePrincipals operation in the Workspace Service Principals API for your workspace.

For example, to get the list of available service principal display names and application IDs for a workspace, you can call curl as follows:

curl --netrc --request GET \
https://<databricks-instance>/api/2.0/preview/scim/v2/ServicePrincipals \
| jq '[ .Resources[] | { displayName: .displayName, applicationId: .applicationId } ]'

Replace <databricks-instance> with the Databricks workspace instance name, for example dbc-a1b2345c-d6e7.cloud.databricks.com for your workspace.

The service principal’s display name is in the output’s displayName field. Partner Connect generates service principal display names using the format <PARTNER-NAME>_USER. For example, for the partner Fivetran, the service principal’s display name is FIVETRAN_USER.

The service principal’s application ID value is in the output’s applicationId field, for example 123456a7-8901-2b3c-45de-f678a901b2c.

To delete the service principal, you can call curl as follows:

curl --netrc --request DELETE \
https://<databricks-instance>/api/2.0/preview/scim/v2/ServicePrincipals/<application-id>

Replace:

  • <databricks-instance> with the workspace instance name, for example dbc-a1b2345c-d6e7.cloud.databricks.com for your workspace.

  • <application-id> with the service principal’s application ID value.

The preceding examples use a .netrc file and jq. Note that in this case, the .netrc file uses your personal access token value–not the one for the service principal.

After you disconnect your workspace from a partner, you might want to clean up any related resources that the partner creates in the workspace. This could include a SQL warehouse or cluster and any related data storage locations. For more information, see Connect to a SQL warehouse or Delete a compute.

If you’re sure that there are no other workspaces across your organization that are connected to the partner, you might also want to delete your organization’s account with that partner. To do this, contact the partner for assistance. For support links, see the appropriate partner connection guide.