Dataiku is an end-to-end AI platform for data preparation, AutoML, and MLOps. You can integrate your Databricks SQL warehouses and Databricks clusters with Dataiku.
Partner Connect only supports SQL warehouses for Dataiku. To connect a cluster to Dataiku, connect to Dataiku manually.
To connect your Databricks workspace to Dataiku using Partner Connect, do the following:
In the sidebar, click Partner Connect.
Click the partner tile.
If the Dataiku tile has a check mark icon inside it, an administrator has already used Partner Connect to connect Dataiku to your workspace. Skip to step 5. The partner uses the email address for your Databricks account to prompt you to sign in to your existing Dataiku account.
Select a catalog from the drop-down list, and then click Next.
If your workspace is Unity Catalog-enabled, but the partner doesn’t support Unity Catalog with Partner Connect, the workspace default catalog is used. If your workspace isn’t Unity Catalog-enabled,
Select a schema from the drop-down list, and then click Add. You can repeat this step to add multiple schemas.
Partner Connect creates the following resources in your workspace:
A Databricks service principal named
A Databricks personal access token that is associated with the
Partner Connect also grants the following privileges to the
(Unity Catalog) USE CATALOG: Required to interact with objects within the selected catalog.
(Unity Catalog) USE SCHEMA: Grants the ability to read the schemas you selected.
(Unity Catalog) CREATE SCHEMA: Grants the ability to create schemas in the selected catalog.
(Legacy Hive metastore) USAGE: Required to interact with objects within
hive_metastoreand the selected schemas.
(Legacy Hive metastore) CREATE: Grants the ability to create a schema in
(Legacy Hive metastore) READ_METADATA: Grants the ability to read metadata for the schemas you selected.
(Legacy Hive metastore) SELECT: Grants the ability to read the schemas you selected.
The Email box displays the email address for your Databricks account. Dataiku uses this email address to prompt you to either create a new partner account or sign in to your existing partner account.
Click Connect to Dataiku or Sign in.
A new tab opens in your web browser, which displays the Dataiku website.
Complete the on-screen instructions on the Dataiku website to create your trial Dataiku account or sign in to your existing Dataiku account.
This section describes how to connect an existing SQL warehouse or cluster in your Databricks workspace to Dataiku manually.
For Databricks SQL warehouses, you can connect to Dataiku using Partner Connect to simplify the experience.
Before you connect to Dataiku manually, you need the following:
A cluster or SQL warehouse in your Databricks workspace.
The connection details for your cluster or SQL warehouse, specifically the Server Hostname, Port, and HTTP Path values.
A Databricks personal access token.. To create a personal access token, do the following:
In your Databricks workspace, click your Databricks username in the top bar, and then select User Settings from the drop down.
Next to Access tokens, click Manage.
Click Generate new token.
(Optional) Enter a comment that helps you to identify this token in the future, and change the token’s default lifetime of 90 days. To create a token with no lifetime (not recommended), leave the Lifetime (days) box empty (blank).
Copy the displayed token to a secure location, and then click Done.
Be sure to save the copied token in a secure location. Do not share your copied token with others. If you lose the copied token, you cannot regenerate that exact same token. Instead, you must repeat this procedure to create a new token. If you lose the copied token, or you believe that the token has been compromised, Databricks strongly recommends that you immediately delete that token from your workspace by clicking the trash can (Revoke) icon next to the token on the Access tokens page.
If you are not able to create or use tokens in your workspace, this might be because your workspace administrator has disabled tokens or has not given you permission to create or use tokens. See your workspace administrator or the following:
As a security best practice when you authenticate with automated tools, systems, scripts, and apps, Databricks recommends that you use OAuth tokens.
If you use personal access token authentication, Databricks recommends using personal access tokens belonging to service principals instead of workspace users. To create tokens for service principals, see Manage tokens for a service principal.
To connect to Dataiku manually, follow Databricks in the Dataiku documentation.