Configure SCIM provisioning for OneLogin

This article describes how to set up Databricks provisioning using Onelogin.

You can set set up provisioning at the Databricks account level or at the Databricks workspace level.

Databricks recommends that you provision users, service principals, and groups to the account level and assign users and groups to workspaces using identity federation. If you have any workspaces that are not enabled for identity federation, you must continue to provision users, service principals, and groups directly to those workspaces.

To learn more about SCIM provisioning in Databricks, including an explanation of the impact of identity federation on provisioning and advice about when to use account-level and workspace-level provisioning, see Sync users and groups from your identity provider.

To configure single sign-on with OneLogin see, Set up SSO for your workspace.

Requirements

  • Your Databricks account must have the Premium plan or above.

  • To set up provisioning for your Databricks account, you must be Databricks account admin.

  • To set up provisioning for a Databricks workspace, you must be Databricks workspace admin.

  • Your OneLogin account must support provisioning.

  • You must be a Super User or Account Owner for your OneLogin account.

  • Databricks recommends that you read the OneLogin article, What is User Provisioning and Deprovisioning?.

Set up account-level SCIM provisioning using OneLogin

This section describes how to configure an OneLogin SCIM connector to provision users and groups to your account.

Get the SCIM token and account SCIM URL in Databricks

  1. As an account admin, log in to the Databricks account console.

    1. Click User Settings Icon Settings.

    2. Click User Provisioning.

    3. Click Enable user provisioning.

      Copy the SCIM token and the Account SCIM URL. You will use these to configure your connector in OneLogin.

Note

The SCIM token is restricted to the Account SCIM API /api/2.0/accounts/{account_id}/scim/v2/ and cannot be used to authenticate to other Databricks REST APIs.

Configure the OneLogin SCIM provisioning app

  1. Log in to OneLogin as a Super User or Account Owner, and launch the OneLogin admin console.

  2. Go to Applications and click Add App.

  3. Search for and select SCIM Provisioner with SAML (SCIM v2 Core).

  4. Click Save. New configuration tabs appear at the left.

  5. Click Configuration.

  6. In Databricks subdomain, enter the Account SCIM URL.

  7. In the SCIM Bearer Token field, enter the Databricks personal access token.

  8. Under API Connection, click Enable. The application authenticates to Databricks.

  9. Go to Provisioning to enable and configure provisioning.

    1. Under Workflow, select Enable provisioning.

    2. Configure whether to require admin approval to create, delete, or update a user.

      Note

      Databricks recommends that you enable admin approval for all operations as an initial safeguard, so that you don’t trigger automatic provisioning for your users before setup and testing have been completed. After you have tested and verified that provisioning is working as expected, you can configure these settings to override admin approval.

    3. Configure the behavior in Databricks when a user is deleted from OneLogin:

      • Do nothing does not modify the user in Databricks.

      • Suspend disables the user in Databricks. The user can’t log in, but the user’s resources are not modified. This is reversible.

      • Delete deletes the user in Databricks and archives the user’s resources. This is not reversible.

    4. Configure the behavior in Databricks when a user is suspended in OneLogin.

      • Do nothing does not modify the user in Databricks.

      • Suspend disables the user in Databricks. The user can’t log in, but the user’s resources are not modified. This is reversible.

    1. Under Entitlements, click Refresh. In OneLogin, groups are called entitlements. This imports groups from Databricks into OneLogin. Importing OneLogin entitlements into Databricks is not supported.

  10. Click Save.

Continue to Use OneLogin to manage users and groups in Databricks to provision users and groups in your Databricks account.

Set up workspace-level SCIM provisioning using OneLogin (legacy)

Preview

This feature is in Public Preview.

When you follow these steps, log into the Databricks admin settings page in one browser tab and log into the OneLogin admin console in another.

Generate a Databricks personal access token

As a Databricks workspace administrator, generate a personal access token. See Token management. Store the personal access token in a secure location. OneLogin will use this personal access token to authenticate to Databricks.

Important

The user who owns this personal access token must not be managed within OneLogin. Otherwise, removing the user from OneLogin would disrupt the SCIM integration.

Configure the OneLogin SCIM provisioning app

  1. Log in to OneLogin as a Super User or Account Owner, and launch the OneLogin admin console.

  2. Go to Applications and click Add App.

  3. Search for and select SCIM Provisioner with SAML (SCIM v2 Core).

  4. Click Save. New configuration tabs appear at the left.

  5. Click Configuration.

  6. In Databricks subdomain, enter https://<databricks-instance>/api/2.0/preview/scim/v2. Replace <databricks-instance> with the workspace URL of your Databricks deployment. See Get identifiers for workspace objects.

  7. In the SCIM Bearer Token field, enter the Databricks personal access token.

  8. Under API Connection, click Enable. The application authenticates to Databricks.

  9. Go to Provisioning to enable and configure provisioning.

    1. Under Workflow, select Enable provisioning.

    2. Configure whether to require admin approval to create, delete, or update a user.

      Note

      Databricks recommends that you enable admin approval for all operations as an initial safeguard, so that you don’t trigger automatic provisioning for your users before setup and testing have been completed. After you have tested and verified that provisioning is working as expected, you can configure these settings to override admin approval.

    3. Configure the behavior in Databricks when a user is deleted from OneLogin:

      • Do nothing does not modify the user in Databricks.

      • Suspend disables the user in Databricks. The user can’t log in, but the user’s resources are not modified. This is reversible.

      • Delete deletes the user in Databricks and archives the user’s resources. This is not reversible.

    4. Configure the behavior in Databricks when a user is suspended in OneLogin.

      • Do nothing does not modify the user in Databricks.

      • Suspend disables the user in Databricks. The user can’t log in, but the user’s resources are not modified. This is reversible.

    1. Under Entitlements, click Refresh. In OneLogin, groups are called entitlements. This imports groups from Databricks into OneLogin. Importing OneLogin entitlements into Databricks is not supported.

  10. Click Save.

Continue to Use OneLogin to manage users and groups in Databricks to provision users and groups in your Databricks workspace.

Use OneLogin to manage users and groups in Databricks

This section describes how to use OneLogin to manage users and groups the Databricks account or workspace.

Assign groups to Databricks workspace users

You must create Databricks groups in Databricks and create mappings to keep them in sync with OneLogin fields. You cannot add groups to Databricks using OneLogin.

  1. In OneLogin, go to the Parameters tab.

  2. Under Optional Parameters, click on Groups.

  3. Verify that all of the group names were successfully imported from Databricks to the Values field when you clicked Refresh on the Provisioning tab (above), and select the Include in User Provisioning flag.

  4. Click Save.

Once you have configured your attribute mapping, you can assign groups to Databricks users when you provision them. To assign Group values, you can manually select them on the user login record for the OneLogin SCIM provisioning app. On the Users tab of the OneLogin SCIM provisioning app, select the user to edit.

You can also use OneLogin rules (mappings) to assign users to Databricks groups automatically, based on another OneLogin attribute, such as OneLogin Role. For example, to place all users who are in the OneLogin Role “Finance” in the Databricks “finance” group, you can go to the Rules tab in your OneLogin SCIM provisioning app and create a New Rule with the condition Roles – include – Finance and the action Set Groups in Databricks to – finance, as in this screenshot:

Rules tab

Now, whenever you add a user to the OneLogin “Finance” role and the OneLogin SCIM provisioning app, the user will be assigned the “finance” group in Databricks when you reapply entitlement mappings.

Users added to the admins group in a OneLogin workspace-level SCIM provisioning app become Databricks workspace administrators.

Remove or update group assignments

To remove or update group assignments go to the Users tab in the OneLogin SCIM provisioning app and select the user to edit. Remove or override the current selection in the group field, custom IAM role field, or custom entitlement field.

If you have set up rules to assign groups to the user based on a OneLogin attribute, such as OneLogin Role, remove that attribute from the user (for example, remove the user from the OneLogin Role). You can also change the rule that assigns the group, IAM role, or entitlement to users in that OneLogin role.

When you remove a user from the admins group in OneLogin and the change is synced to Databricks, the user is no longer a Databricks workspace administrator.

Important

Do not remove the administrator who configured the OneLogin SCIM provisioning app, and do not remove them from the admins group. Otherwise, the SCIM integration cannot authenticate to Databricks.

Trigger a sync

You can manually trigger a sync of OneLogin users with Databricks users by going to the OneLogin SCIM provisioning app and selecting MORE ACTIONS -> Sync logins. If a user is assigned to the app, the user will be added to your Databricks account or workspace. However, the reverse is not true: a user created in the Databricks account or workspace will not be added to the OneLogin SCIM provisioning app.

To manually sync users from the Databricks account or workspace to OneLogin, create a user in OneLogin with the same username and email address as the user in the Databricks account or workspace, and then assign the user to the app in OneLogin.

Delete users

You can delete users in OneLogin and OneLogin deactivates them from the Databricks account or workspace.

Important

Do not remove the administrator who configured the OneLogin SCIM provisioning app from Databricks or from the admins group. Otherwise, the SCIM integration cannot authenticate to Databricks.

You can deactivated a user in multiple ways:

  • Delete or suspend the user from OneLogin.

  • Remove the user from the app manually by going to the Users tab in the OneLogin SCIM provisioning app, selecting the user, and clicking the Delete button.

  • If you have set up rules to assign the user to the app based on a OneLogin attribute, such as OneLogin Role, remove that attribute from the user (for example, remove the user from the OneLogin Role). You can also remove then Databricks app from a OneLogin Role that the user is assigned to (this deprovisions Databricks for all users in the role).

Note

If you remove a user from the account-level SCIM application, that user is deactivated from the account and from their workspaces, regardless of whether or not identity federation has been enabled.

If you delete a OneLogin-managed user directly in the Databricks workspace, the user will remain active in the OneLogin SCIM provisioning app. When you try to delete the user from the OneLogin SCIM provisioning app, the attempt will fail, because the user is already deleted in the workspace.

Use OneLogin to manage entitlements and IAM roles

Databricks supports the assignment of IAM roles and workspace entitlements from workspace-level Databricks applications in OneLogin. The assignment of roles and entitlements is not supported from the account-level Databricks application in OneLogin. If you want to assign IAM roles and workspace entitlements from OneLogin, you must create a workspace-level Databricks application in OneLogin to that workspace.

Databricks recommends that you instead use an account-level Databricks application in OneLogin to provision users and groups to the account level. You assign users and groups to workspaces using identity federation and manage their entitlements and IAM roles within Databricks.

Map Databricks attributes to OneLogin attributes

In order to manage IAM roles and entitlements from OneLogin, you must first create mappings to keep Databricks IAM roles and entitlements in sync with OneLogin fields for provisioned users.

  1. In OneLogin, go to Users > Custom User Fields and create two custom fields:

    • One to hold a user’s allow-cluster-create permission. In our example, we name this databricksEntitlements.

    • One to hold the user’s Databricks IAM role. In our example, we name this IAM Role.

    Once you have created these fields, you can specify the IAM role and allow-cluster-create entitlement for any user in the Custom Fields section of the OneLogin user record.

  2. Return to your OneLogin SCIM provisioning app and go to the Parameters tab to map the Entitlement, Groups, and Role attributes (all optional).

    Click the field name to open the edit dialog, where you can set the OneLogin field (Value) to map to the Databricks field.

    • Entitlement: set the Value to the custom user field you created in Step 1.

    • Role: set the Value to the custom user field you created in Step 1.

  3. Click Save.

Repeat this procedure to assign additional IAM roles or entitlements.

Assign IAM roles and entitlements to Databricks workspace users

Once you have created your custom user fields and configured your attribute mapping, you can assign IAM roles and entitlements to Databricks users when you provision them.

When you enter a value for the entitlement and IAM role custom fields on the user record (Users > All Users > <username>), the entitlement and IAM role are automatically included when you provision the user to Databricks. You can also use OneLogin rules (mappings) to assign IAM roles and entitlements automatically, based on another OneLogin attribute, such as OneLogin Role.

You can remove or update IAM role or entitlement assignments by going to the Users tab in the OneLogin SCIM provisioning app and select the user to edit. Remove or override the current selection in the custom IAM role field, or custom entitlement field. If you have set up rules to assign IAM roles or entitlements to the user based on a OneLogin attribute, remove that attribute from the user. You can also change the rule that assigns the IAM role or entitlement to users in that OneLogin role.

Troubleshooting and tips

  • Users who existed in Databricks prior to provisioning setup:

    • Are automatically linked to a OneLogin user if they already exist in OneLogin and are matched based on email address (username).

    • Can be manually linked to an existing user or created as a new user in OneLogin if they are not automatically matched.

  • User permissions that are that are assigned individually and duplicated through membership in a group remain after the group membership is removed for the user.

  • Users removed from a Databricks workspace lose access to that workspace but may still have access to other Databricks workspaces.

  • You must create Databricks groups in Databricks; you cannot add groups using OneLogin.

  • You cannot update Databricks usernames and email addresses.