Manage Delta Sharing providers (for data recipients)

This article describes how to use Unity Catalog to get information about data providers who are sharing data with you using Delta Sharing. It also describes what a provider object is and when you might need to create a provider object in your Unity Catalog metastore, a task that most recipients should never need to do.

Important

Data recipients must have access to a Databricks workspace that is enabled for Unity Catalog to use the functionality described in this article.

Create a provider

In Delta Sharing, the term “provider” can mean both the organization that is sharing data with you and a securable object in a recipient’s Unity Catalog metastore that represents that organization. The existence of that securable object in a recipient’s Unity Catalog metastore enables recipients to manage their team’s access to shared data using Unity Catalog.

Do I need to create a provider?

As a recipient, you can create a provider object in your Unity Catalog metastore, but typically you do not need to. Whether or not you do depends on how data has been shared with you using Delta Sharing:

  • Databricks-to-Databricks sharing: Provider objects are created automatically in the recipient’s metastore. There is no need for you, the recipient, to create providers, and you can skip this section. You can proceed to the sections that describe how to view providers, view provider details, list shares that a provider has shared with you, and delete providers.

  • Open sharing: Provider objects are not created automatically in the recipient’s metastore. Ideally, if you, the recipient, use a Databricks workspace that is attached to a Unity Catalog metastore, the data provider would have set up Databricks-to-Databricks sharing with you. But if they did not, you can create a provider object in your Unity Catalog metastore and then use Unity Catalog to manage your team’s access to the shared data, just as a recipient of Databricks-to-Databricks sharing would do.

Instructions

To create a provider object in your Unity Catalog metastore, you must use the Databricks Unity Catalog CLI, and you must have access to the token shared by the provider. In this example, we use the credential file that was shared by a Databricks provider. See Get access in the open sharing model.

Note

Creating a provider using a Databricks credential file does not elevate the sharing protocol from open sharing to Databricks-to-Databricks sharing.

Permissions required: Metastore admin or user with the CREATE_PROVIDER privilege for the metastore.

Run the following command using the Databricks CLI, replacing <provider_name> with the name you want to give to the provider and config.share with the path to your downloaded credential file.

databricks unity-catalog providers create --name <provider_name>  \
                  --recipient-profile-json-file config.share

View providers

To view a list of available data providers, you can use Data Explorer, the Databricks Unity Catalog CLI, or the SHOW PROVIDERS SQL command in a Databricks notebook or the Databricks SQL query editor.

Permissions required: You must be a metastore admin or have the USE PROVIDER privilege to view all providers in the metastore. Other users have access only to the providers that they own.

  1. In your Databricks workspace, click Data Icon Data.

  2. In the left pane, expand the Delta Sharing menu and select Shared with me.

  3. On the Providers tab, view all available providers.

Run the following command in a notebook or the Databricks SQL query editor. Optionally, replace <pattern> with a `LIKE` predicate.

SHOW PROVIDERS [LIKE <pattern>];

Run the following command using the Databricks CLI.

databricks unity-catalog providers list

View provider details

To view details about a provider, you can use Data Explorer, the Databricks Unity Catalog CLI, or the DESCRIBE PROVIDER SQL command in a Databricks notebook or the Databricks SQL query editor.

Permissions required: Metastore admin, user with the USE PROVIDER privilege, or the provider object owner.

Details include:

  • Shares shared by the provider (see View shares that a provider has shared with you).

  • The provider’s creator, creation timestamp, comments, and authentication type (TOKEN or DATABRICKS). TOKEN represents providers who have shared data with you using the Delta Sharing open sharing protocol. DATABRICKS represents providers who have shared data with you using the Databricks-to-Databricks sharing protocol.

  • If the provider uses Databricks-to-Databricks sharing: the cloud, region, and metastore ID of the provider’s Unity Catalog metastore.

  • If the provider uses open sharing: your recipient profile endpoint, which is the where the Delta Sharing sharing server is hosted.

  1. In your Databricks workspace, click Data Icon Data.

  2. In the left pane, expand the Delta Sharing menu and select Shared with me.

  3. On the Providers tab, find and select the provider.

  4. View provider details on the Details tab.

Run the following command in a notebook or the Databricks SQL query editor.

DESC PROVIDER <provider_name>;

Run the following command using the Databricks CLI.

databricks unity-catalog providers get --name <provider_name>

View shares that a provider has shared with you

To view the shares that a provider has shared with you, you can use Data Explorer, the Databricks Unity Catalog CLI, or the SHOW SHARES IN PROVIDER SQL command in a Databricks notebook or the Databricks SQL query editor.

Permissions required: Metastore admin, user with the USE PROVIDER privilege, or the provider object owner.

  1. In your Databricks workspace, click Data Icon Data.

  2. In the left pane, expand the Delta Sharing menu and select Shared with me.

  3. On the Providers tab, select the provider whose shares you want to view.

Run the following command in a notebook or the Databricks SQL query editor. Optionally, replace <pattern> with a `LIKE` predicate.

SHOW SHARES IN PROVIDER [LIKE <pattern>];

Run the following command using the Databricks CLI.

 databricks unity-catalog providers list-shares --name <provider_name>

Update a provider (rename, change owner, comment)

You can use Data Explorer, the Databricks Unity Catalog CLI, or the ALTER PROVIDER SQL command in a Databricks notebook or the Databricks SQL query editor to modify the provider object in your Unity Catalog metastore:

  • Rename the provider to modify the way users see the provider object in their Databricks interfaces.

  • Change the owner of the provider object.

  • Add or modify comments.

Permissions required: You must be a metastore admin or owner of the provider object to update the owner. You must be a metastore admin (or user with the CREATE_PROVIDER privilege) and provider owner to update the provider name. You must be the owner to update the comment. The initial owner is the metastore admin.

  1. In your Databricks workspace, click Data Icon Data.

  2. In the left pane, expand the Delta Sharing menu and select Shared with you.

  3. On the Providers tab, find and select the provider.

  4. On the details page, update the owner, comment, or provider name.

Run the following command in a notebook or the Databricks SQL query editor.

ALTER PROVIDER <provider_name> RENAME TO <new_provider_name>
OWNER TO <new_owner>
COMMENT "<comment>";

Run the following command using the Databricks CLI. Replace <provider_name> with the current provider name and <new_provider_name> with the new name.

databricks unity-catalog providers update --name <provider_name> /
                                    --new-name <new_provider_name> /
                                    --comment "<new comment>" /
                                    --owner <new_owner_name>

Delete a provider

To delete a provider, you can use Data Explorer, the Databricks Unity Catalog CLI, or the DROP PROVIDER SQL command in a Databricks notebook or the Databricks SQL query editor. You must be the provider object owner to delete the provider.

When you delete a provider, you and the users in your organization (the recipient) can no longer access the data shared by the provider.

Permissions required: Provider object owner.

  1. In your Databricks workspace, click Data Icon Data.

  2. In the left pane, expand the Delta Sharing menu and select Shared with you.

  3. On the Providers tab, find and select the provider.

  4. Click the Kebab menu kebab menu (also known as the three-dot menu) and select Delete.

  5. On the confirmation dialog, click Delete.

Run the following command in a notebook or the Databricks SQL query editor.

DROP PROVIDER [IF EXISTS] <provider_name>;

Run the following command using the Databricks CLI.

databricks unity-catalog providers delete --name <provider_name>

If the operation is successful, no results are returned.