Delta Sharing

Preview

Delta Sharing is in Public Preview. To participate in the preview, you must enable the External Data Sharing feature group in the Databricks Account Console. See Enable the External Data Sharing feature group for your account.

Delta Sharing is subject to applicable terms. Enabling the External Data Sharing feature group represents acceptance of those terms.

Delta Sharing is an open protocol for secure data sharing with other organizations regardless of which computing platforms they use. It can share collections of tables in a Unity Catalog metastore in real time without copying them, so that data recipients can immediately begin working with the latest version of the shared data.

Since: Databricks Runtime 10.3

There are two components to Delta Sharing:

  • Shares

    A share provides a logical grouping for the tables you intend to share.

  • Recipients

    A recipient identifies an organization with which you want to share any number of shares.

Shares

A share is a container instantiated with the CREATE SHARE command. Once created you can iteratively register a collection of existing tables defined within the metastore using the ALTER SHARE command. You can register tables under their original name, qualified by their original schema, or provide alternate exposed names.

You must be a metastore admin or account admin to create, alter, and drop shares.

Examples 

-- Create share `customer_share` only if share with same name doesn't exist, with a comment.
> CREATE SHARE IF NOT EXISTS customer_share COMMENT 'This is customer share';

-- Add 2 tables to the share.
-- Expose my_schema.tab1 a different name.
-- Expose only two partitions of other_schema.tab2
> ALTER SHARE customer_share ADD TABLE my_schema.tab1 AS their_schema.tab1;
> ALTER SHARE customer_share ADD TABLE other_schema.tab2 PARTITION (c1 = 5), (c1 = 7);

-- List the content of the share
> SHOW ALL IN SHARE customer_share;
  name              type  shared_object           added_at                     added_by                   comment partitions
  ----------------- ----  ---------------------- ---------------------------- -------------------------- ------- -----------------
  other_schema.tab2 TABLE main.other_schema.tab2 2022-01-01T00:00:01.000+0000 alwaysworks@databricks.com NULL
  their_schema.tab1 TABLE main.myschema.tab2     2022-01-01T00:00:00.000+0000 alwaysworks@databricks.com NULL   (c1 = 5), (c1 = 7)

Recipients

A recipient is an object you create using CREATE RECIPIENT to represent an organization which you want to allow access shares. When you create a recipient Databricks Runtime generates an activation link you can send to the organization. To retrieve the activation link after creation you use DESCRIBE RECIPIENT.

Once a recipient has been created you can give it SELECT privileges on shares of your choice using GRANT ON SHARE.

You must be a metastore administrator to create recipients, drop recipients, and grant access to shares.

Examples 

-- Create a recipient.
> CREATE RECIPIENT IF NOT EXISTS other_org COMMENT 'other.org';

-- Retrieve the activation link to send to other.org
> DESCRIBE RECIPIENT other_org;
  name      created_at                   created_by                 comment   activation_link active_token_id                      active_token_expiration_time rotated_token_id rotated_token_expiration_time
  --------- ---------------------------- -------------------------- --------- --------------- ------------------------------------ ---------------------------- ---------------- -----------------------------
  other_org 2022-01-01T00:00:00.000+0000 alwaysworks@databricks.com other.org https://....    0160c81f-5262-40bb-9b03-3ee12e6d98d7 9999-12-31T23:59:59.999+0000 NULL              NULL

-- Choose shares that other.org has access to
> GRANT SELECT ON SHARE customer_share TO RECIPIENT other_org;