Sync users and groups from your identity provider

This article describes how to configure your identity provider (IdP) and Databricks to provision users and groups to Databricks using SCIM, or System for Cross-domain Identity Management, an open standard that allows you to automate user provisioning.

About SCIM provisioning in Databricks

SCIM lets you use an IdP to create users in Databricks, give them the proper level of access, and remove access (deprovision them) when they leave your organization or no longer need access to Databricks.

You can use a SCIM provisioning connector in your IdP or invoke the Identity and Access Management SCIM APIs to manage provisioning. You can also use these APIs to manage identities in Databricks directly, without an IdP.

Account-level and workspace-level SCIM provisioning

Databricks recommends that you use account-level SCIM provisioning to create, update, and delete all users from the account. You manage the assignment of users and groups to workspaces within Databricks. Your workspaces must be enabled for identity federation to manage users’ workspace assignments.

Workspace-level SCIM provisioning is a legacy configuration that is in Public Preview. If you already have workspace-level SCIM provisioning set up for a workspace, Databricks recommends that you enable the workspace for identity federation, set up account-level SCIM provisioning, and turn off the workspace-level SCIM provisioner. See Migrate workspace-level SCIM provisioning to the account level. For more information on workspace-level SCIM provisioning, see Provision identities to a Databricks workspace (legacy).

Requirements

To provision users and groups to Databricks using SCIM:

Your Databricks account must have the Premium plan or above.
You must be a Databricks account admin.

Databricks recommends that you configure single sign-on for users to log in to Databricks using your IdP. See Configure SSO in Databricks.

You can have a maximum of 10,000 combined users and service principals and 5000 groups in an account. Each workspace can have a maximum of 10,000 combined users and service principals and 5000 groups.

Note

When you use SCIM provisioning, user and group attributes stored in your identity provider can override changes you make using the Databricks admin settings page, account console, or SCIM (Groups) API.

For example, if a user is assigned the Allow Cluster Creation entitlement in your identity provider and you remove that entitlement using the Databricks admin settings, the user is re-granted that entitlement the next time the IdP syncs with Databricks, if the IdP is configured to provision that entitlement. The same behavior applies to groups.

Sync users and groups to your Databricks account

You can sync users and groups from your IdP to your Databricks account using a SCIM provisioning connector.

Important

If you already have SCIM connectors that sync identities directly to your workspaces, you must disable those SCIM connectors when the account-level SCIM connector is enabled. See Migrate workspace-level SCIM provisioning to the account level.

To configure a SCIM connector to provision users and groups to your account:

As an account admin, log in to the Databricks account console.
In the sidebar, click Settings.
Click User Provisioning.
Click Enable user provisioning.

Copy the SCIM token and the Account SCIM URL. You will use these to configure your IdP.
Log in to your IdP as a user who can configure a SCIM connector to provision users.
Enter the following values in your IdP’s SCIM connector:
- For the SAML provisioning URL, enter the SCIM URL you copied from Databricks.
- For the provisioning API token, enter the SCIM token you copied from Databricks.

After you configure account-level SCIM provisioning, Databricks recommends that you allow all users in your identity provider to access the Databricks account. See Enable all identity provider users to access Databricks.

You can also follow these IdP-specific instructions for your IdP:

Note

When you remove a user from the account-level SCIM connector, that user is deactivated from the account and all of their workspaces, regardless of whether or not identity federation has been enabled. When you remove a group from the account-level SCIM connector, all users in that group are deactivated from the account and from any workspaces they had access to, (unless they are members of another group or have been directly granted access to the account-level SCIM connector).

Rotate the account-level SCIM token

If the account-level SCIM token is compromised or if you have business requirements to rotate authentication tokens periodically, you can rotate the SCIM token.

As a Databricks account admin, log in to the account console.
In the sidebar, click Settings.
Click User Provisioning.
Click Regenerate token. Make a note of the new token. The previous token will continue to work for 24 hours.
Within 24 hours, update your SCIM application to use the new SCIM token.

Migrate workspace-level SCIM provisioning to the account level

If you are enabling account-level SCIM provisioning and you already have workspace-level SCIM provisioning set up for some workspaces, Databricks recommends that you turn off the workspace-level SCIM provisioner and instead sync users and group to the account level.

Create a group in your identity provider that includes all of the users and groups that you are currently provisioning to Databricks using your workspace-level SCIM connectors.

Databricks recommends that this group include all users in all workspaces in your account.
Configure a new SCIM provisioning connector to provision users and groups to your account, using the instructions in Sync users and groups to your Databricks account.

Use the group or groups that you created in step 1. If you add a user that shares a username (email address) with an existing account user, those users are merged. Existing groups in the account are not affected.
Confirm that the new SCIM provisioning connector is successfully provisioning users and groups to your account.
Shut down the old workspace-level SCIM connectors that were provisioning users and groups to your workspaces.

Do not remove users and groups from the workspace-level SCIM connectors before shutting them down. Revoking access from a SCIM connector deactivates the user in the Databricks workspace. For more information, see Deactivate a user in your Databricks workspace.
Migrate workspace-local groups to account groups.

If you have legacy groups in your workspaces, they are known as workspace-local groups. You cannot manage workspace-local groups using account-level interfaces. Databricks recommends that you convert them to account groups. See Migrate workspace-local groups to account groups