Skip to main content
Last updated on

Configure OAuth U2M: Custom-managed for Google Drive ingestion

Custom-managed OAuth U2M authenticates Google Drive ingestion into Databricks using an OAuth app that you create and own in your own Google Cloud project. Use this method when your organization requires control over the OAuth application, such as for managing API rate limits or owning the app registration.

Prerequisites​

  • A Databricks workspace with Unity Catalog enabled.
  • The CREATE CONNECTION privilege on the metastore.
  • A Google account with the necessary permissions to create a Google Cloud project.

Step 1: Set up a Google Cloud project and enable the Google Drive API​

  1. Go to the Google Cloud console.
  2. Create a new project. You might be prompted to set up two-factor authentication.
  3. Go to APIs & Services > Library.
  4. Search for "Google Drive API".
  5. Select Google Drive API.
  6. Select Enable.

  1. On the Google Cloud console home screen, go to APIs & Services > OAuth Consent Screen. You will see a message that says "Google Auth Platform not configured yet".

  2. Select Get Started.

  3. Fill in the App Information section. Enter any name for the app name (for example, Databricks connection). The support email can be any email in your organization.

  4. Select Next.

  5. In the Audience section, select an audience, then select Next:

    • Internal: Select this option if the custom OAuth app is used only by people in your Google Workspace organization and your Google Workspace policy allows internal apps.
    • External: Select this option if the custom OAuth app is used by people outside your Google Workspace organization.

  6. Fill in the Contact Information section, then select Next.

  7. Review the Google API Services User Data Policy, then select Create.

  8. Go back to Google Auth Platform > Data Access.

  9. Select Add or remove scopes.

  10. Add the following scope in the Manually add scopes section, select Add to table, then select Update:

    https://www.googleapis.com/auth/drive.readonly

  11. Select Save.

Step 3: Create OAuth 2.0 client credentials​

  1. On the Google Cloud console home screen, go to APIs & Services > Credentials.
  2. Select Create Credentials > OAuth Client ID.
  3. Choose Web Application and set a custom name.
  4. In Authorized Redirect URIs, select Add URI.
  5. Add a redirect URI to <databricks-instance-url>/login/oauth/google.html, replacing <databricks-instance-url> with your Databricks instance URL. For example: https://instance-name.databricks.com/login/oauth/google.html
  6. Select Create. A dialog with your credentials appears.
  7. Record the following values. Alternatively, you can download the OAuth Client JSON file, which contains this information:
    • Client ID (format: 0123******-********************************.apps.googleusercontent.com)
    • Client secret (format: ABCD**-****************************)

Step 4: Add test users for an External app​

If you selected External in Step 2, add the Google account that you use to create the connection as a test user. If you selected Internal, skip this step.

  1. Go to Google Auth Platform > Audience.

  2. Under Test users, select Add users.

  3. Add the email address of the Google account you will use to create the connection.

    The connector ingests only the files that account can access.

warning

Sharing a personal account connection grants other users access to your personal account's data.

Next steps​

  1. Create a connection to store the authentication details that you've obtained.
  2. Create an ingestion pipeline.
On this page