Databricks Partner Connect guide

Jump directly to the list of Databricks Partner Connect partners.

Partner Connect easily connects Databricks SQL endpoints and Databricks clusters in your Databricks workspaces to Databricks Partner Connect partners.

Note

Some partner solutions allow you to integrate with only Databricks SQL endpoints or Databricks clusters, but not both. For details, see the partner documentation.

These solutions allow you to:

  • Load data from external data sources into local Delta Lake tables.
  • Create and manage extract-transform-load (ETL) and extract-load-transform (ELT) data pipelines through simplified user interfaces.
  • Perform business intelligence operations on data.
  • Work with machine learning (ML) on data.

For a list of participating partners, see Databricks Partner Connect partners.

Requirements

  • Your Databricks account must be on the Databricks Premium or Enterprise Plan. This is because Partner Connect uses Databricks SQL, which is available only on these plans. To view your Databricks account details, use the account console.
  • Your Databricks workspace must be on the E2 version of the Databricks platform. This is because Partner Connect uses Databricks service principals, which are available only on this version of the platform. To determine whether your workspace is on the E2 version of the Databricks platform, contact your Databricks representative.
  • To create new connections to partner solutions, you must first sign in to your workspace as a Databricks admin. For information about Databricks admins, see Manage users.
  • For all other Partner Connect tasks, you must first sign in to your workspace as a Databricks admin; or a Databricks user who has at least the Workspace access entitlement and, if you are working with SQL endpoints, the Databricks SQL access entitlement. For more information, see Manage users.

Quickstart: Connect to a partner solution

  1. Make sure your Databricks account, workspace, and the signed-in user all meet the requirements for Partner Connect.

  2. In the sidebar, click Partner Connect button Partner Connect.

  3. Click the tile for the partner that you want to connect your workspace to. If the tile has a check mark icon, stop here, as your workspace is already connected. Otherwise, follow the on-screen directions to finish creating the connection.

    Note

    Partner solutions that use a locally-installed application instead of a web-based one (such as Power BI Desktop and Tableau Desktop) do not display a check mark icon, even after you connect your workspace to them.

  4. To work with your new connection, see the corresponding Details in the list of Databricks Partner Connect partners.

Example walkthrough: Connect to Fivetran

In this walkthrough, you use Partner Connect to connect a Databricks SQL endpoint in your workspace to Fivetran and then use Fivetran to ingest sample data from Google Sheets into your workspace.

  1. Make sure your Databricks account, workspace, and the signed-in user all meet the requirements for Partner Connect.

  2. In the sidebar, click Partner Connect button Partner Connect.

  3. Click the Fivetran tile.

    Note

    If the Fivetran tile has a check mark icon inside of it, this means one of your administrators has already used Partner Connect to connect Fivetran to your workspace. Contact that admin, who can add you to the Fivetran account that they created by using Partner Connect. After they add you, click the Fivetran tile.

  4. If the Connect to partner dialog displays a Next button, click it.

  5. For Email, enter the email address that you want Fivetran to use to create a 14-day trial Fivetran account for you, or enter the email address for your existing Fivetran account.

  6. Click the button with the label Connect to Fivetran or Sign in.

    Important

    If an error displays stating that someone from your organization has already created an account with Fivetran, do one of the following:

    • Enter an email address that is not associated with your organization, and then click Connect to Fivetran or Sign in again.
    • Contact one of your organization’s administrators and have them add you to your organization’s Fivetran account. After they add you, click Connect to Fivetran or Sign in again.
  7. A new tab opens in your web browser, which displays the Fivetran website.

  8. Complete the on-screen instructions in Fivetran to create your 14-day trial Fivetran account, or to sign in to your existing Fivetran account.

  9. Do one of the following:

Use a new 14-day trial Fivetran account

  1. On the Fivetran is modern ELT page, click Set up a connector.
  2. On the Select your data source page, click Google Sheets, and then click Continue Setup.
  3. Follow the on-screen instructions in the Setup Guide in Fivetran to finish setting up the connector.
  4. Click Save & Test.
  5. After the test succeeds, click Continue.
  6. Do one of the following:

Use an existing Fivetran account

To complete this series of steps, you get the connection details for an existing SQL endpoint in your workspace and then add those details to your Fivetran account.

  • To get the connection details for an existing SQL endpoint, see Get connection details for a SQL endpoint. Specifically, you will need the SQL endpoint’s Server Hostname and HTTP Path field values.

    Note

    By default, the name of the SQL endpoint is FIVETRAN_ENDPOINT.

  • To view the available SQL endpoints in your workspace, see View SQL endpoints.

  • To create a SQL endpoint in your workspace, see Create a SQL endpoint.

You must also generate a Databricks personal access token.

Tip

If the Fivetran tile in Partner Connect has a check mark icon inside of it, you can get the connection details for the connected SQL endpoint by clicking the tile and then expanding Connection details. Note however that the Personal access token here is hidden; you must create a replacement personal access token and enter that new token instead when Fivetran asks you for it.

  1. In your Dashboard page in Fivetran, click the Destinations tab. (If the Dashboard page is not displayed, go to https://fivetran.com/account.

    Important

    If you sign in to your organization’s Fivetran account, a Choose Destination page may display, listing one or more existing destination entries with the Databricks logo. These entries might contain connection details for SQL endpoints in workspaces that are separate from yours. If you still want to reuse one of these connections, and you trust the SQL endpoint and have access to it, choose that destination, click Add Connector, and then skip ahead to step 5. Otherwise, choose any available destination to get past this page, and then go to https://fivetran.com/account.

  2. Click Add Destination.

  3. Enter a Destination name and click Add.

  4. On the Fivetran is modern ELT page, click Set up a connector.

  5. Click Google Sheets, and then click Next.

  6. Follow the on-screen instructions in the Setup Guide in Fivetran to finish setting up the connector.

  7. Click Save & Test.

  8. After the test succeeds, click Continue.

  9. On the Select your data’s destination page, click Databricks on AWS.

  10. Click Continue Setup.

  11. Complete the on-screen instructions in Fivetran to enter the connection details for your existing SQL endpoint.

    To get the connection details for an existing SQL endpoint, see Get connection details for a SQL endpoint. Specifically, you will need the SQL endpoint’s Server Hostname and HTTP Path field values.

    Note

    By default, the name of the SQL endpoint is FIVETRAN_ENDPOINT.

    You will also need a Databricks personal access token.

  12. Click Save & Test.

  13. After the test succeeeds, click Continue.

  14. Continue with Ingest sample data.

Ingest sample data

  1. Click Start Initial Sync.

  2. View the sample data in your workspace: after the sync succeeds, go to your Databricks workspace.

  3. In Databricks SQL, click Queries.

  4. Click Create Query.

  5. Choose the name of the SQL endpoint.

  6. Enter a query, for example SELECT * FROM google_sheets.my_sheet.

    Note

    Your database and table name here will be different. For the correct database and table name, see the details for the connector in Fivetran that you just created.

  7. Click Run. The ingested data displays.

Clean up

After you complete this walkthrough, you should clean up any related resources that you no longer plan to use.

Delete the table

  1. In Databricks SQL, click Queries on the sidebar.

  2. Click Create Query.

  3. Choose the name of the related SQL endpoint.

  4. Enter a query, for example DROP TABLE google_sheets.my_sheet.

    Note

    Your database and table name here will be different. For the correct database and table name, see the details for the related connector in Fivetran.

    This query only deletes the table. It does not delete your Google Sheet. You can manually delete your Google Sheet if you no longer plan to use it.

  5. Click Run.

Delete the SQL endpoint

Do one of the following:

  • If you used Partner Connect to create the SQL endpoint:
    1. In Partner Connect, click the Fivetran tile with the check mark icon inside of it.
    2. Click Delete connection.
    3. Click Delete. The SQL endpoint and the related Databricks service principal are deleted.
  • If you used Databricks SQL to create the SQL endpoint:
    1. In Databricks SQL, click SQL Endpoints in the sidebar.
    2. Next to the endpoint, for Actions, click the ellipsis button.
    3. Click Delete.
    4. Confirm the deletion by clicking Delete.

Delete the connection details

  1. In your Dashboard page in Fivetran, click the Destinations tab. (If the Dashboard page is not displayed, go to https://fivetran.com/account.)
  2. Next to the related destination entry with the Databricks logo, click the X icon.
  3. Click Remove Destination.

Congratulations! You have completed this walkthrough.

Key tasks

This section describes how to complete key tasks that are associated with Partner Connect.

Allow users to access partner-generated databases and tables

Partner solutions in the Data Ingestion category in Partner Connect can create databases and tables in your workspace. These databases and tables are owned by the partner solution’s associated Databricks service principal in your workspace.

By default, these databases and tables can be accessed only by the service principal and by Databricks admins in your workspace. To allow other users in your workspace to access these databases and tables, use the SQL GRANT statement. To get access details for an existing database or table, use the SQL SHOW GRANT statement.

Create an access token

Databricks partner solutions require you to provide the partner with a Databricks personal access token. The partner uses this token to authenticate with your Databricks workspace.

For cloud-based partner solutions in Partner Connect (such as Fivetran, Labelbox, Prophecy, and Rivery), Partner Connect automatically creates the token (along with a Databricks service principal that is associated with that token) and then shares the token’s value with the partner. You cannot access the token’s value. If for any reason the token expires or the token’s value is no longer shared with the partner, you must create a replacement token for the service principal; to do this, see Manage service principals and personal access tokens. To share the replacement token with the partner, see the partner’s documentation.

Only Databricks workspace administrators can generate replacement tokens for Databricks service principals. If you cannot generate a replacement token, contact your administrator. See also Manage service principals.

For desktop-based partner solutions in Partner Connect (such as Power BI and Tableau), you must create the token and then share the token’s value with the partner. To create the token, see Generate a personal access token. To set up the partner solution so that it shares the new token with the partner, follow the on-screen instructions in Partner Connect or see the partner’s documentation.

Important

When you create the token and share the token’s value with the partner, the partner can take whatever actions that the related entity (such as your Databricks user or a Databricks service principal) can normally take within your Databricks workspace. Do not share token values with partners whom you do not trust.

Databricks workspace administrators can disable token generation. If you cannot generate a token, contact your administrator. See also Manage personal access tokens.

Allow a SQL endpoint to access external data

To allow a Databricks SQL endpoint to access data outside of Databricks, see Configure access to cloud storage.

Troubleshooting

This section provides information to help address common issues with Partner Connect.

When trying to connect to a partner, an error message is displayed

Issue: When you try to connect your Databricks workspace to a partner solution by using Partner Connect, an error displays, and you cannot create the connection.

Causes:

There are multiple reasons why this issue may occur:

  • If multiple individuals within an organization try to create an account with a partner, only the first individual will succeed. This is because the partner may offer some accounts at only the organizational level, and the first individual who creates such an account also establishes the account for the organization.
  • If you already have an account with the partner, the connection may still fail. This is because the partner may mistakenly try to create a duplicate account.

Solutions:

Do one of the following:

  • Ask the first individual who created the organizational account with the partner to add your email address to that account. Then bypass Partner Connect and sign in to the partner account directly to begin using the partner’s solution.
  • Try making the connection again through Partner Connect, but this time, specify an email address that is not associated with your organization’s domain, such as a personal email address. This may require you to also add that email address as a user to your Databricks workspace.
  • Bypass Partner Connect and sign in to the partner directly with your existing account, and begin using the partner solution.

If your workspace is not already connected after you sign in to the partner, complete the connection by following the instructions in the Details section in the list of Databricks Partner Connect partners.

When trying to sign in to a partner’s account or website, a pop-up blocker is displayed

Issue: For a partner solution that uses Partner Connect to sign in to the partner’s account or website, when you click Sign In, Partner Connect opens a new tab in your web browser and a pop-up blocker is displayed. This pop-up blocker prevents you from signing in to the partner’s account or website.

Cause: Your web browser is blocking pop-ups.

Solution: Allow pop-ups for the partner’s website in your web browser. Specific instructions vary by web browser. For example, for Google Chrome, see Block or allow popups in Chrome on the Google Chrome Help website. For other web browsers, search the Internet with a phrase such as “how do I allow pop-ups for a specific website?”

Additional resources

Explore these resources to learn more about Partner Connect.