Microsoft Windows Active Directory

Microsoft Windows Active Directory (AD) provides identity based access controls for a wide range of Microsoft products. Windows AD also provides support for authenticating third party extranet applications including Databricks by using their Federated Single-Sign On product Windows Active Directory Federation Services (ADFS) which allows authentication using the SAML 2.0 standard.

Note

  • The following guide is for configuring ADFS integration using Windows Server 2012 R2 Active Directory Federation Services version 6.3.0.0. If you need support for other versions of ADFS or Azure Directory Services and you are an existing customer contact help@databricks.com. If you are a new customer, reach out to sales@databricks.com.
  • Windows AD typically uses a short employee ID or employee username as the authentication principal rather than an email address (for example DatabricksJDOE or 12345@ad.corp.com). When users log in to Databricks they will enter this username on the single sign-on page, but the user’s workspace directory and username in Databricks will still be their full email address (john.doe@corp.com).
  • Occasionally, there are time synchronization issues that invalidate all ADFS logins. Please make sure to update the clock on your ADFS server if you run into issues with logging in.
  • Some organizations use specific certificates for token signing and token encryption. If you are having trouble enabling ADFS SSO and your organization uses multiple certificates, contact Databricks at help@databricks.com (for existing customers) or sales@databricks.com (for new customers).

Step 1: Confirm requirements

You should have the following Windows Services installed:

  • Windows Domain Server
  • Windows DNS Service
  • Microsoft Internet Information Services IIS
  • Windows AD
  • Windows AD FS

For more details on how to install and configure these services please refer to the Microsoft Knowledge base at: https://technet.microsoft.com/library/c66c7f4b-6b8f-4e44-8331-63fa85f858c2

Other configuration requirements:

  • Install a signed SSL certificate for your ADFS login page and the fingerprint for that certificate.
  • Ensure that user objects in your active directory HAVE AN EMAIL ADDRESS ATTRIBUTE to map AD users to Databricks users

Step 2:

Open the AD FS Management console.

ActiveDirectory Federation Services Management console

Click the link to Edit Federation Service Properties on the right side Action bar. The federation service name and identifier should match the DNS entry for the environment. In AWS if you don’t use a custom DNS entry then the URL will look as follows:

Edit Federation Service Properties

Databricks uses SAML 2.0 as the standard authentication mechanism. To confirm that your ADFS services supports SAML 2.0 go to ADFS > Service > Endpoints and confirm that the URL Path /adfs/ls exists.

Confirm /adls/fs URL path

If this is the first time using ADFS to authenticate a client outside of your corporate intranet, you will need to enable intranet Forms Authentication. To do so, go to the ADFS Management application > ADFS > Authentication Policies > Edit Global Authentication Policy

Then check the box for Intranet Forms authentication.

Intranet Forms authentication

Then go to https://<your-sso-domain>.com/adfs/ls/idpinitiatedsignon to view the single sign-on page for your organization.

Step 3:

To add Databricks as a Relying Party trust, go to AD FS > Trust Relationships > Relying Party Trusts. Right click and select Add Relying Party Trust.

Add Relying Party Trust

On the next screen, select Data Sources, select the last option, enter data about the relying party manually.

Select data source

Under display name type Databricks and under Notes put any notes or additional information you want.

On the next screen select the first radio button AD FS profile.

Select AD FS profile

Optionally configure a certificate for AD FS to Databricks encryption (highly recommended)

Configure certificate

Under Configure URL, select the second option Enable support for the SAML 2.0 WebSSO protocol and enter the URL: https://<databricks-instance>.cloud.databricks.com/saml/consume.

Configure URL

On the Configure Identifiers step, enter https://<databricks-instance>.cloud.databricks.com/saml/consume.

Configure Identifiers

Click the Add button to add https://<databricks-instance>.cloud.databricks.com.

On the next screen, Configure Multi-factor Authentication …, choose the default I do not want to configure ….

Configure Multi-factor Authentication

Under Choose Issuance Authorization Rules select Permit all users to access this relying party. This setting is for testing purposes only and you will likely want to use Active Directory Groups or Membership rules to give a user in your organization access Databricks.

Choose Issuance Authorization Rules

Review the configuration on the next screen and leave the Open the Edit Claim Rules … checkbox checked on the screen following. Click the Close button to continue.

Review configuration
Complete add Relying party trust

Step 4: Transform Claim Rules

Claim rules in ADFS map user objects in Windows AD with users in Databricks. We will be creating a Claim Rule that maps users based on their e-mail address.

The Add Transform Claim Rule wizard should already be open if you finished step Step 3 above. If it is not, you can click on the Relying Party Trusts, select Databricks and then on the right Actions sidebar select Edit Claim Rules … then click the Add Rule button.

On the first screen choose Send LDAP Attributes as Claims.

Send LDAP Attributes as Claims

On the next screen, type the Claim Rule Name: Outgoing Databricks LDAP Email, set the Attribute Store to Active Directory and select the LDAP Attribute your company uses to store your corporate email addresses (the default is E-Mail Addresses) and map that to Name ID and E-Mail Address like so:

Configure claim rule name

Click Finish.

On the next screen click the Add Rule button. This time set the Claim Rule Template to Transform an Incoming Claim.

Add rule

On the Configure Claim Rule screen, type the Claim Rule Name: Incoming Databricks LDAP Email.

Set the following values:

  • Incoming claim type: E-Mail Address
  • Incoming name ID format: Unspecified
  • Outgoing claim type: Name ID
  • Outgoing name ID format: Email

Select Pass through all claim values.

Pass through all claim values

Then click Finish.

Finally click Apply then Ok to return to the main screen.

Step 5: Change Signature to SHA-256

On the ADFS Relying Party Trusts screen, select Databricks and then click Properties on the Action bar.

Configure Databricks properties

Select the Advanced tab and change the Secure hash algorithm to SHA-256.

Set secure hash algorithm

Step 6: Configure Databricks for Single Sign-On

  1. Login to your account as an Administrator.

  2. Go to the Admin Console.

  3. Select the Single Sign On tab.

    SSO tab
  4. Leave (1) as the default value.

  5. Set (2)

    • Single Sign-On URL: https://<your-sso-domain>.com/adfs/ls/
    • Identity Provider Entity ID: https://<your-sso-domain>.com/adfs/services/trust
  6. From the ADFS Management Console go to ADFS > Service > Certificates.

    1. Find the Token-signing certificate then click View Certificate from the Action sidebar.

      View certificate
    2. Click the Details tab.

    3. Click Copy to File.

    4. Choose Base-64 encoded X.509 (.CER) when prompted.

    5. Open the file with Notepad or another text editor.

    6. Copy the text between -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----.

  7. Paste into the X.509 Certificate field.

  8. Click Enable SSO. Then logout and try to login using your corporate single sign-on page.