Use Open ID Connect (OIDC) federation to enable authentication to Delta Sharing shares (open sharing)

Preview

This feature is in Public Preview. Contact your Databricks account team to request access.

This page explains how data providers in Databricks can federate authentication to an identity provider (IdP) to govern access to Delta Sharing shares created in Databricks. This authentication flow uses OIDC federation, allowing JSON Web Tokens (JWTs) issued by the recipient’s IdP as short-lived OAuth tokens authenticated by Databricks. This Databricks-to-open sharing authentication method is designed for recipients who do not have access to a Unity Catalog-enabled Databricks workspace.

In OIDC Federation, the recipient’s IdP is responsible for issuing JWT tokens and enforcing security policies, such as Multi-Factor Authentication (MFA). Likewise, the lifetime of the JWT token is governed by the recipient’s IdP. Databricks does not generate or manage these tokens. It only federates authentication to the recipient’s IdP and validates the JWT against the recipient’s configured federation policy. Data providers can also choose to federate authentication to their own IdP when sharing data internally with other users or departments within their organization.

OIDC federation is an alternative to using long-lived Databricks-issued bearer tokens to connect non-Databricks recipients to providers. It enables fine-grained access control, supports MFA, and reduces security risks by eliminating the need for recipients to manage and secure shared credentials. For information about using bearer tokens to manage authentication to shares instead, see Create a recipient object for non-Databricks users using bearer tokens (open sharing).

How does OIDC federation work in Delta Sharing?

1. When the data provider creates the recipient in Delta Sharing on Databricks, they configure an OIDC token federation policy that specifies the issuer URL of the recipient IdP, such as Microsoft Entra ID or Okta, and defines the recipient user, service principal, group, or client application that should have access to the share.

2. Databricks generates an OIDC profile web portal URL, based on the policy, and the provider shares that URL with the recipient.

   The end user copies the endpoint URL or downloads the profile file, depending on their preferred platform, and provides the URL or profile file to the platform on which they will query the shared data. This shared profile file downloaded from Databricks OIDC portal web does not contain any sensitive information.

   • For user-to-machine (U2M) authentication, the recipient inputs the recipient endpoint from the OIDC profile web portal into their U2M application.
   • For machine-to-machine (M2M) authentication, the recipient application developer downloads the profile file and references it in the recipient client app.

3. When the recipient attempts to access shared data using their preferred platform, the authentication is federated to their IdP.

   Databricks does not generate or manage any tokens or credentials. Instead, the recipient's IdP generates a JWT containing identity claims. The lifetime of this short-lived token is enforced by the recipient's IdP. The Delta Sharing service then validates the JWT against the recipient's policy to ensure it matches the expected claims, including issuer, audience, and subject. If the validation is successful, the request is authenticated, and access is granted based on Unity Catalog permissions.

Before you begin

To create a recipient, you must meet the following requirements:

• You must be a metastore admin or have the CREATE_RECIPIENT privilege for the Unity Catalog metastore where the data you want to share is registered.
• You must create the recipient using a Databricks workspace that has that Unity Catalog metastore attached.
• If you use a Databricks notebook to create the recipient, your compute must use Databricks Runtime 11.3 LTS or above and either standard or dedicated access mode (formerly shared and single user access modes).

Which Identity Provider to Use?

You can use OIDC federation with either an internal or external identity provider, depending on your sharing scenario:

• Internal Identity Provider (Provider-Managed)

  • This is useful for sharing data within large organizations where different departments do not have direct Databricks access but share the same IdP.
  • This approach allows the provider to manage access on behalf of the recipient.
  • Security policies, such as MFA and role-based access control, are enforced by the provider’s IdP.

• External Identity Provider (Recipient-Managed)

  • The provider sets up the sharing policy to trust the recipient's IdP.
  • The recipient organization retains full control over who can access the shared data.
  • Security policies, such as MFA and role-based access control, are enforced by the recipient’s IdP.

Authentication Scenario U2M or M2M

Secure Open Sharing with OIDC Token Federation supports both User-to-Machine (U2M) and Machine-to-Machine (M2M) authentication flows, enabling a wide range of secure data-sharing scenarios.

User-to-Machine (U2M) Authentication

A user from the recipient organization authenticates using their IdP. If MFA is configured, it is enforced during login.

Once authenticated, users can access shared data using tools like Power BI or Tableau. The data provider can define access policies that restrict data access to specific users or groups within the recipient organization, ensuring precise control over who can access shared resources. The U2M client application (e.g., Power BI) uses the OAuth Authorization Code Grant flow to obtain access tokens from the IdP.

Machine-to-Machine (M2M) Authentication

M2M is ideal for automated workloads, such as nightly jobs or background services, that require access without user interaction. The recipient organization registers a Service Principal in its IdP. This service identity enables applications or scripts to securely access resources programmatically. No secrets or credentials are exchanged between Databricks, the provider, or the recipient. All secret management remains internal to each organization. M2M clients, such as the Python Delta Sharing Client or Spark Delta Sharing Client, use the OAuth Client Credentials Grant flow to retrieve access tokens from the IdP.

Create a recipient that uses an OIDC federation policy

Step 1. Create an Open OIDC Federation Recipient

To create a recipient that authenticates using OIDC:

1. In your Databricks workspace, click Catalog.

2. At the top of the Catalog pane, click the gear icon and select Delta Sharing.

   Alternatively, from the Quick access page, click the Delta Sharing > button.

3. On the Shared by me tab, click New recipient.

4. Enter the Recipient name.

5. For Recipient type, select Open.

6. Choose OIDC Federation as the Open Authentication method.

7. Click Create.

8. (Optional) Create custom Recipient properties. On the recipient Details tab, click Edit properties > +Add property. Then add a property name (Key) and Value. For details, see Manage recipient properties.

Step 2. Create OIDC Federation Policy

Before creating the policy, gather the necessary information from the recipient about their IdP, including the users, groups, or service principals that should have access to the share. If you’re using your own (internal) IdP for internal sharing, retrieve this information from your own identity system.

You must first request information from the recipient about their IdP and the users, groups, and service principals that should have access to the share. You then provide that information in Databricks when you create the recipient.

On the recipient edit page, under OIDC federation policies, click Add policy.

Enter the following:

• Policy name: Human-readable name for the policy.
• Issuer URL: The HTTPS URL of the IdP issuing the JWT token.
• Subject claim: The claim in the JWT that identifies the authenticating identity (sub, groups, or oid).
• Subject: The specific user, group, or service principal allowed to access the share.
• Audiences: One or more resource identifiers the JWT must match. A token is considered valid if its aud claim matches any of the listed audiences.

If you’re unsure about the values to use (issuer, subject claim, subject, audience), refer to the following example. You need to determine the details of the OIDC Federation Policy before creating it.

If you are using an external recipient managed IdP request the following information from the recipient shared using a secure channel. If you are using your internal provider managed IdP, this information is from your own IdP based on the identities that you are sharing with.

Example for U2M when IdP is Entra ID:

These are example configuration for sharing with a specific user with Object ID 11111111-2222-3333-4444-555555555555 in Entra ID tenant aaaaaaaa-bbbb-4ccc-dddd-eeeeeeeeeeee

• Issuer: https://login.microsoftonline.com/aaaaaaaa-bbbb-4ccc-dddd-eeeeeeeeeeee/v2.0
• subject claim: oid (Object ID)
• Subject: 11111111-2222-3333-4444-555555555555
• Audiences: 64978f70-f6a6-4204-a29e-87d74bfea138 (This is the client ID of the multi-tenant app registered by Databricks in Entra ID)

These are example configuration for sharing with a specific group with Object ID 66666666-2222-3333-4444-555555555555 in Entra ID tenant aaaaaaaa-bbbb-4ccc-dddd-eeeeeeeeeeee

• Issuer: https://login.microsoftonline.com/aaaaaaaa-bbbb-4ccc-dddd-eeeeeeeeeeee/v2.0
• subject claim: groups
• Subject: 66666666-2222-3333-4444-555555555555
• Audiences: 64978f70-f6a6-4204-a29e-87d74bfea138 (This is the client ID of the multi-tenant app registered by Databricks in Entra ID)

note

For U2M applications like Power BI and Tableau, the audience should be the multi-tenant app ID registered by Databricks in Entra ID, which is 64978f70-f6a6-4204-a29e-87d74bfea138.

For more information about U2M applications and their OIDC Federation policies see Receive Delta Sharing shares using Open ID Connect (OIDC) federation in a user-to-machine flow (open sharing).

Example for M2M when IdP is Entra ID:

For an M2M OAuth application with Service Principal Object ID 11111111-2222-3333-4444-555555555555 in Entra ID tenant aaaaaaaa-bbbb-4ccc-dddd-eeeeeeeeeeee:

• Issuer: https://login.microsoftonline.com/aaaaaaaa-bbbb-4ccc-dddd-eeeeeeeeeeee/v2.0
• Subject claim: oid
• subject: 11111111-2222-3333-4444-555555555555 (This is the service principal Object ID, which can be found under Enterprise Applications in the recipient’s Entra ID portal.)
• Audiences: 66666666-2222-3333-4444-555555555555 (This can be any valid audience identifier defined by the recipient, such as the client ID of the registered OAuth application.) For more information about M2M applications and their OIDC Federation policies see Receive Delta Sharing shares using a Python client and Open ID Connect (OIDC) federation in a machine-to-machine flow (open sharing).

Step 3. Grant the recipient access to a share

Once you’ve created the recipient and created shares, you can grant the recipient access to those shares.

To grant share access to recipients, you can use Catalog Explorer, the Databricks Unity Catalog CLI, or the GRANT ON SHARE SQL command in a Databricks notebook or the Databricks SQL query editor.

Permissions required: One of the following:

• Metastore admin.
• Delegated permissions or ownership on both the share and the recipient objects ((USE SHARE + SET SHARE PERMISSION) or share owner) AND (USE RECIPIENT or recipient owner).

For instructions, see Manage access to Delta Sharing data shares (for providers).

Recipient workflow

To learn about how recipients authenticate and access shares using OIDC token federation, see:

• Receive Delta Sharing shares using Open ID Connect (OIDC) federation in a user-to-machine flow (open sharing)
• Receive Delta Sharing shares using a Python client and Open ID Connect (OIDC) federation in a machine-to-machine flow (open sharing)