Manage access to external cloud services using service credentials

This article describes how to create a service credential object in Unity Catalog that lets you govern access from Databricks to external cloud services. A service credential in Unity Catalog encapsulates a long-term cloud credential that grants access to such services.

Service credentials are not intended for governing access to cloud storage that is used as a Unity Catalog managed storage location or external storage location. For those use cases, use a storage credential. See Manage access to cloud storage using Unity Catalog.

To create a service credential for access to Google Cloud services:

1. You create the service credential object in Databricks.
2. Databricks generates a Google Cloud service account.
3. In Google Cloud, you grant that service account the GCP roles required to access the service.

Before you begin

Before you create a service credential, you must meet the following requirements:

In Databricks:

• Databricks workspace enabled for Unity Catalog.

• CREATE SERVICE CREDENTIAL privilege on the Unity Catalog metastore attached to the workspace. Account admins and metastore admins have this privilege by default. If your workspace was enabled for Unity Catalog automatically, workspace admins also have this privilege.

In your Google Cloud account:

• A Google Cloud service in the same region as the workspaces you want to access the data from.
• Permission to modify the access policy for that service.

Generate a Google Cloud service account using Catalog Explorer

1. Log in to your Unity Catalog-enabled Databricks workspace as a user who has the CREATE SERVICE CREDENTIAL privilege on the metastore.

   The metastore admin and account admin roles both include this privilege.

2. In the sidebar, click Catalog.

3. On the Quick access page, click the External data > button, go to the Credentials tab, and select Create credential.

4. Select Service credential.

5. Enter a Credential name and an optional comment.

6. Click Create.

   Databricks creates the service credential and generates a Google Cloud service account.

7. On the Service credential created dialog, make note of the service account ID, which is in the form of an email address, and click Done.

8. (Optional) Bind the storage credential to specific workspaces.

   By default, any privileged user can use the storage credential on any workspace attached to the metastore. If you want to allow access only from specific workspaces, go to the Workspaces tab and assign workspaces. See (Optional) Assign a service credential to specific workspaces.

Grant the service account access to the Google Cloud service

1. Go to the Google Cloud console and open the service that you want to access from your Databricks workspace.

2. In the service’s permission settings, give the Databricks-generated service account the roles that are required to access the service.

   These roles depend on the service. For guidance, see the documentation for your Google Cloud service.

   Use the service account’s email address as the principal identifier.

(Optional) Assign a service credential to specific workspaces

By default, a service credential is accessible from all of the workspaces in the metastore. This means that if a user has been granted a privilege on that service credential, they can exercise that privilege from any workspace attached to the metastore. If you use workspaces to isolate user data access, you may want to allow access to a service credential only from specific workspaces. This feature is known as workspace binding or service credential isolation.

A typical use case for binding a service credential to specific workspaces is the scenario in which a cloud admin configures a service credential using a production cloud account credential, and you want to ensure that Databricks users use this credential to access an external cloud service only in the production workspace.

For more information about workspace binding, see (Optional) Assign a storage credential to specific workspaces and Limit catalog access to specific workspaces.

Bind a service credential to one or more workspaces

To assign a service credential to specific workspaces, use Catalog Explorer.

Permissions required: Metastore admin or service credential owner.

note

Metastore admins can see all service credentials in a metastore using Catalog Explorer—and service credential owners can see all service credentials that they own in a metastore—regardless of whether the service credential is assigned to the current workspace. Service credentials that are not assigned to the workspace appear grayed out.

1. Log in to a workspace that is linked to the metastore.

2. In the sidebar, click Catalog.

3. On the Quick access page, click the External data > button and go to the Credentials tab.

4. Select the service credential and go to the Workspaces tab.

5. On the Workspaces tab, clear the All workspaces have access checkbox.

   If your service credential is already bound to one or more workspaces, this checkbox is already cleared.

6. Click Assign to workspaces and enter or find the workspaces you want to assign.

To revoke access, go to the Workspaces tab, select the workspace, and click Revoke. To allow access from all workspaces, select the All workspaces have access checkbox.

Next steps

• Learn how to view, update, delete, and grant other users permission to use service credentials. See Manage service credentials.
• Learn how to use service credentials in code. See Use Unity Catalog service credentials to connect to external cloud services.

Limitations

The following limitations apply:

• Until serverless compute is updated with Databricks Runtime 16.2, it will support only Python.
• SQL warehouses are not supported.
• Some audit events for actions performed on service credentials will not appear in the system.access.audit table. Audit information about who created, deleted, updated, read, listed, or used a service credential will be available. See Audit log system table reference.
• During the service credentials preview, INFORMATION_SCHEMA.STORAGE_CREDENTIALS (deprecated) displayed both storage credentials and service credentials, and INFORMATION_SCHEMA.STORAGE_CREDENTIAL_PRIVILEGES (deprecated) displayed privileges that applied both to storage credentials and service credentials. This is no longer the case. You should instead use INFORMATION_SCHEMA.CREDENTIALS and INFORMATION_SCHEMA.CREDENTIAL_PRIVILEGES for both storage and service credentials.