Uc Credentials

View as Markdown

Connect Unity Catalog to external data and services

Generate Temporary Path CredentialPOST /api/2.0/unity-catalog/temporary-path-credentialsGenerate Temporary Service CredentialPOST /api/2.1/unity-catalog/temporary-service-credentialsGenerate Temporary Table CredentialPOST /api/2.0/unity-catalog/temporary-table-credentialsGenerate Temporary Volume CredentialPOST /api/2.0/unity-catalog/temporary-volume-credentialsList CredentialsGET /api/2.1/unity-catalog/credentialsValidate CredentialPOST /api/2.1/unity-catalog/validate-credentialsValidate Storage CredentialPOST /api/2.1/unity-catalog/validate-storage-credentials

Generate Temporary Path Credential GA

POST /api/2.0/unity-catalog/temporary-path-credentials

Get a short-lived credential for directly accessing cloud storage locations registered in <Databricks>. The Generate Temporary Path Credentials API is only supported for external storage paths, specifically external locations and external tables. Managed tables are not supported by this API. The metastore must have external_access_enabled flag set to true (default false). The caller must have the EXTERNAL_USE_LOCATION privilege on the external location; this privilege can only be granted by external location owners. For requests on existing external tables, the caller must also have the EXTERNAL_USE_SCHEMA privilege on the parent schema; this privilege can only be granted by catalog owners.

API scopes: unity-catalog

Request body

urlstring

URL for path-based access.

operationstring

The operation being performed on the path.

Values: PATH_READ, PATH_READ_WRITE, PATH_CREATE_TABLE

dry_runboolean

Optional. When set to true, the service will not validate that the generated credentials can perform write operations, therefore no new paths will be created and the response will not contain valid credentials. Defaults to false.

Response

aws_temp_credentialsobjectRequired

Show child attributesHide child attributes

access_key_idstring

The access key ID that identifies the temporary credentials.

secret_access_keystring

The secret access key that can be used to sign AWS API requests.

session_tokenstring

The token that users must pass to AWS API to use the temporary credentials.

access_pointstring

The Amazon Resource Name (ARN) of the S3 access point for temporary credentials related the external location.

azure_user_delegation_sasobjectRequired

Show child attributesHide child attributes

sas_tokenstring

The signed URI (SAS Token) used to access blob services for a given path

gcp_oauth_tokenobjectRequired

Show child attributesHide child attributes

oauth_tokenstring

azure_aadobjectRequired

Show child attributesHide child attributes

aad_tokenstring

Opaque token that contains claims that you can use in Azure Active Directory to access cloud services.

r2_temp_credentialsobjectRequired

Show child attributesHide child attributes

access_key_idstring

The access key ID that identifies the temporary credentials.

secret_access_keystring

The secret access key associated with the access key.

session_tokenstring

The generated JWT that users must pass to use the temporary credentials.

expiration_timeint64

Server time when the credential will expire, in epoch milliseconds. The API client is advised to cache the credential given this expiration time.

urlstring

The URL of the storage path accessible by the temporary credential.

View as Markdown

Request

{
  "url": "string",
  "operation": "string",
  "dry_run": true
}

Response

{
  "aws_temp_credentials": {
    "access_key_id": "string",
    "secret_access_key": "string",
    "session_token": "string",
    "access_point": "string"
  },
  "azure_user_delegation_sas": {
    "sas_token": "string"
  },
  "gcp_oauth_token": {
    "oauth_token": "string"
  },
  "azure_aad": {
    "aad_token": "string"
  },
  "r2_temp_credentials": {
    "access_key_id": "string",
    "secret_access_key": "string",
    "session_token": "string"
  },
  "expiration_time": 0,
  "url": "string"
}

Generate Temporary Service Credential GA

POST /api/2.1/unity-catalog/temporary-service-credentials

Returns a set of temporary credentials generated using the specified service credential. The caller must be a metastore admin or have the metastore privilege ACCESS on the service credential.

API scopes: unity-catalog

AWS

Returns a set of temporary credentials generated using the specified service credential. The caller must be a metastore admin or have the metastore privilege ACCESS on the service credential.

The temporary credentials consist of an access key ID, a secret access key, and a security token.

Azure

Returns a set of temporary credentials generated using the specified service credential. The caller must be a metastore admin or have the metastore privilege ACCESS on the service credential.

The temporary credentials consist of an AAD token.

Request body

credential_namestring

The name of the service credential used to generate a temporary credential

azure_optionsobjectRequired

Show child attributesHide child attributes

resourcesarray of string

The resources to which the temporary Azure credential should apply. These resources are the scopes that are passed to the token provider (see https://learn.microsoft.com/python/api/azure-core/azure.core.credentials.tokencredential?view=azure-python)

gcp_optionsobjectRequired

Show child attributesHide child attributes

scopesarray of string

The scopes to which the temporary GCP credential should apply. These resources are the scopes that are passed to the token provider (see https://google-auth.readthedocs.io/en/latest/reference/google.auth.html#google.auth.credentials.Credentials)

Response

aws_temp_credentialsobjectRequired

Show child attributesHide child attributes

access_key_idstring

The access key ID that identifies the temporary credentials.

secret_access_keystring

The secret access key that can be used to sign AWS API requests.

session_tokenstring

The token that users must pass to AWS API to use the temporary credentials.

access_pointstring

The Amazon Resource Name (ARN) of the S3 access point for temporary credentials related the external location.

azure_user_delegation_sasobjectRequired

Show child attributesHide child attributes

sas_tokenstring

The signed URI (SAS Token) used to access blob services for a given path

gcp_oauth_tokenobjectRequired

Show child attributesHide child attributes

oauth_tokenstring

azure_aadobjectRequired

Show child attributesHide child attributes

aad_tokenstring

Opaque token that contains claims that you can use in Azure Active Directory to access cloud services.

r2_temp_credentialsobjectRequired

Show child attributesHide child attributes

access_key_idstring

The access key ID that identifies the temporary credentials.

secret_access_keystring

The secret access key associated with the access key.

session_tokenstring

The generated JWT that users must pass to use the temporary credentials.

expiration_timeint64

Server time when the credential will expire, in epoch milliseconds. The API client is advised to cache the credential given this expiration time.

urlstring

The URL of the storage path accessible by the temporary credential.

View as Markdown

Request

{
  "credential_name": "string",
  "azure_options": {
    "resources": [
      "string"
    ]
  },
  "gcp_options": {
    "scopes": [
      "string"
    ]
  }
}

Response

{
  "aws_temp_credentials": {
    "access_key_id": "string",
    "secret_access_key": "string",
    "session_token": "string",
    "access_point": "string"
  },
  "azure_user_delegation_sas": {
    "sas_token": "string"
  },
  "gcp_oauth_token": {
    "oauth_token": "string"
  },
  "azure_aad": {
    "aad_token": "string"
  },
  "r2_temp_credentials": {
    "access_key_id": "string",
    "secret_access_key": "string",
    "session_token": "string"
  },
  "expiration_time": 0,
  "url": "string"
}

Generate Temporary Table Credential GA

POST /api/2.0/unity-catalog/temporary-table-credentials

Get a short-lived credential for directly accessing the table data on cloud storage. The metastore must have external_access_enabled flag set to true (default false). The caller must have the EXTERNAL_USE_SCHEMA privilege on the parent schema and this privilege can only be granted by catalog owners.

API scopes: unity-catalog

Request body

table_idstring

UUID of the table to read or write.

operationstring

The operation performed against the table data, either READ or READ_WRITE. If READ_WRITE is specified, the credentials returned will have write permissions, otherwise, it will be read only.

Values: READ, READ_WRITE

Response

aws_temp_credentialsobjectRequired

Show child attributesHide child attributes

access_key_idstring

The access key ID that identifies the temporary credentials.

secret_access_keystring

The secret access key that can be used to sign AWS API requests.

session_tokenstring

The token that users must pass to AWS API to use the temporary credentials.

access_pointstring

The Amazon Resource Name (ARN) of the S3 access point for temporary credentials related the external location.

azure_user_delegation_sasobjectRequired

Show child attributesHide child attributes

sas_tokenstring

The signed URI (SAS Token) used to access blob services for a given path

gcp_oauth_tokenobjectRequired

Show child attributesHide child attributes

oauth_tokenstring

azure_aadobjectRequired

Show child attributesHide child attributes

aad_tokenstring

Opaque token that contains claims that you can use in Azure Active Directory to access cloud services.

r2_temp_credentialsobjectRequired

Show child attributesHide child attributes

access_key_idstring

The access key ID that identifies the temporary credentials.

secret_access_keystring

The secret access key associated with the access key.

session_tokenstring

The generated JWT that users must pass to use the temporary credentials.

expiration_timeint64

Server time when the credential will expire, in epoch milliseconds. The API client is advised to cache the credential given this expiration time.

urlstring

The URL of the storage path accessible by the temporary credential.

View as Markdown

Request

{
  "table_id": "string",
  "operation": "string"
}

Response

{
  "aws_temp_credentials": {
    "access_key_id": "string",
    "secret_access_key": "string",
    "session_token": "string",
    "access_point": "string"
  },
  "azure_user_delegation_sas": {
    "sas_token": "string"
  },
  "gcp_oauth_token": {
    "oauth_token": "string"
  },
  "azure_aad": {
    "aad_token": "string"
  },
  "r2_temp_credentials": {
    "access_key_id": "string",
    "secret_access_key": "string",
    "session_token": "string"
  },
  "expiration_time": 0,
  "url": "string"
}

Generate Temporary Volume Credential Public Preview

POST /api/2.0/unity-catalog/temporary-volume-credentials

Get a short-lived credential for directly accessing the volume data on cloud storage. The metastore must have external_access_enabled flag set to true (default false). The caller must have the EXTERNAL_USE_SCHEMA privilege on the parent schema and this privilege can only be granted by catalog owners.

API scopes: unity-catalog

Request body

volume_idstring

Id of the volume to read or write.

operationstring

The operation performed against the volume data, either READ_VOLUME or WRITE_VOLUME. If WRITE_VOLUME is specified, the credentials returned will have write permissions, otherwise, it will be read only.

Values: READ_VOLUME, WRITE_VOLUME

Response

aws_temp_credentialsobjectRequired

Show child attributesHide child attributes

access_key_idstring

The access key ID that identifies the temporary credentials.

secret_access_keystring

The secret access key that can be used to sign AWS API requests.

session_tokenstring

The token that users must pass to AWS API to use the temporary credentials.

access_pointstring

The Amazon Resource Name (ARN) of the S3 access point for temporary credentials related the external location.

azure_user_delegation_sasobjectRequired

Show child attributesHide child attributes

sas_tokenstring

The signed URI (SAS Token) used to access blob services for a given path

gcp_oauth_tokenobjectRequired

Show child attributesHide child attributes

oauth_tokenstring

azure_aadobjectRequired

Show child attributesHide child attributes

aad_tokenstring

Opaque token that contains claims that you can use in Azure Active Directory to access cloud services.

r2_temp_credentialsobjectRequired

Show child attributesHide child attributes

access_key_idstring

The access key ID that identifies the temporary credentials.

secret_access_keystring

The secret access key associated with the access key.

session_tokenstring

The generated JWT that users must pass to use the temporary credentials.

expiration_timeint64

Server time when the credential will expire, in epoch milliseconds. The API client is advised to cache the credential given this expiration time.

urlstring

The URL of the storage path accessible by the temporary credential.

View as Markdown

Request

{
  "volume_id": "string",
  "operation": "string"
}

Response

{
  "aws_temp_credentials": {
    "access_key_id": "string",
    "secret_access_key": "string",
    "session_token": "string",
    "access_point": "string"
  },
  "azure_user_delegation_sas": {
    "sas_token": "string"
  },
  "gcp_oauth_token": {
    "oauth_token": "string"
  },
  "azure_aad": {
    "aad_token": "string"
  },
  "r2_temp_credentials": {
    "access_key_id": "string",
    "secret_access_key": "string",
    "session_token": "string"
  },
  "expiration_time": 0,
  "url": "string"
}

List Credentials GA

GET /api/2.1/unity-catalog/credentials

Gets an array of credentials (as CredentialInfo objects).

The array is limited to only the credentials that the caller has permission to access. If the caller is a metastore admin, retrieval of credentials is unrestricted. There is no guarantee of a specific ordering of the elements in the array.

PAGINATION BEHAVIOR: The API is by default paginated, a page may contain zero results while still providing a next_page_token. Clients must continue reading pages until next_page_token is absent, which is the only indication that the end of results has been reached.

API scopes: unity-catalog

Parameters

include_unboundbooleanquery

Whether to include credentials not bound to the workspace. Effective only if the user has permission to update the credential–workspace binding.

max_resultsint32query

Maximum number of credentials to return.

• If not set, the default max page size is used.
• When set to a value greater than 0, the page length is the minimum of this value and a server-configured value.
• When set to 0, the page length is set to a server-configured value (recommended).
• When set to a value less than 0, an invalid parameter error is returned.

page_tokenstringquery

Opaque token to retrieve the next page of results.

Response

credentialsarray of object

Show child attributesHide child attributes

namestring

The credential name. The name must be unique among storage and service credentials within the metastore.

aws_iam_roleobjectRequired

The AWS IAM role configuration.

Show child attributesHide child attributes

role_arnstring

The Amazon Resource Name (ARN) of the AWS IAM role used to vend temporary credentials.

unity_catalog_iam_arnstring

The Amazon Resource Name (ARN) of the AWS IAM user managed by <Databricks>. This is the identity that is going to assume the AWS IAM role.

external_idstring

The external ID used in role assumption to prevent the confused deputy problem.

azure_service_principalobjectRequired

The Azure service principal configuration.

Show child attributesHide child attributes

directory_idstring

The directory ID corresponding to the Azure Active Directory (AAD) tenant of the application.

application_idstring

The application ID of the application registration within the referenced AAD tenant.

client_secretstring

The client secret generated for the above app ID in AAD.

gcp_service_account_keyobjectRequired

Show child attributesHide child attributes

emailstring

The email of the service account.

private_key_idstring

The ID of the service account's private key.

private_keystring

The service account's RSA private key.

azure_managed_identityobjectRequired

The Azure managed identity configuration.

Show child attributesHide child attributes

access_connector_idstring

The Azure resource ID of the Azure Databricks Access Connector. Use the format /subscriptions/{guid}/resourceGroups/{rg-name}/providers/Microsoft.Databricks/accessConnectors/{connector-name}.

managed_identity_idstring

The Azure resource ID of the managed identity. Use the format, /subscriptions/{guid}/resourceGroups/{rg-name}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{identity-name} This is only available for user-assgined identities. For system-assigned identities, the access_connector_id is used to identify the identity. If this field is not provided, then we assume the AzureManagedIdentity is using the system-assigned identity.

credential_idstring

The <Databricks> internal ID that represents this managed identity.

databricks_gcp_service_accountobjectRequired

The <Databricks> managed GCP service account configuration.

Show child attributesHide child attributes

emailstring

The email of the service account.

private_key_idstring

The ID that represents the private key for this Service Account

credential_idstring

The <Databricks> internal ID that represents this managed identity.

commentstring

Comment associated with the credential.

read_onlyboolean

Whether the credential is usable only for read operations. Only applicable when purpose is STORAGE.

ownerstring

Username of current owner of credential.

idstring

The unique identifier of the credential.

metastore_idstring

Unique identifier of the parent metastore.

created_atint64

Time at which this credential was created, in epoch milliseconds.

created_bystring

Username of credential creator.

updated_atint64

Time at which this credential was last modified, in epoch milliseconds.

updated_bystring

Username of user who last modified the credential.

used_for_managed_storageboolean

Whether this credential is the current metastore's root storage credential. Only applicable when purpose is STORAGE.

full_namestring

The full name of the credential.

isolation_modestring

Whether the current securable is accessible from all workspaces or a specific set of workspaces.

Values: ISOLATION_MODE_UNSPECIFIED, ISOLATION_MODE_OPEN, ISOLATION_MODE_ISOLATED

next_page_tokenstring

Opaque token to retrieve the next page of results. Absent if there are no more pages. page_token should be set to this value for the next request (for the next page of results).

View as Markdown

Response

{
  "credentials": [
    {
      "name": "string",
      "aws_iam_role": {},
      "azure_service_principal": {},
      "gcp_service_account_key": {},
      "azure_managed_identity": {},
      "databricks_gcp_service_account": {},
      "comment": "string",
      "read_only": true,
      "owner": "string",
      "id": "string",
      "metastore_id": "string",
      "created_at": 0,
      "created_by": "string",
      "updated_at": 0,
      "updated_by": "string",
      "used_for_managed_storage": true,
      "full_name": "string",
      "isolation_mode": "string"
    }
  ],
  "next_page_token": "string"
}

Validate Credential GA

POST /api/2.1/unity-catalog/validate-credentials

Validates a credential.

For service credentials (purpose is SERVICE), either the credential_name or the cloud-specific credential must be provided.

For storage credentials (purpose is STORAGE), at least one of external_location_name and url need to be provided. If only one of them is provided, it will be used for validation. And if both are provided, the url will be used for validation, and external_location_name will be ignored when checking overlapping urls. Either the credential_name or the cloud-specific credential must be provided.

The caller must be a metastore admin or the credential owner or have the required permission on the metastore and the credential (e.g., CREATE_EXTERNAL_LOCATION when purpose is STORAGE).

API scopes: unity-catalog

Request body

credential_namestringRequired

Required. The name of an existing credential or long-lived cloud credential to validate.

aws_iam_roleobjectRequired

Show child attributesHide child attributes

role_arnstring

The Amazon Resource Name (ARN) of the AWS IAM role used to vend temporary credentials.

unity_catalog_iam_arnstring

The Amazon Resource Name (ARN) of the AWS IAM user managed by <Databricks>. This is the identity that is going to assume the AWS IAM role.

external_idstring

The external ID used in role assumption to prevent the confused deputy problem.

azure_managed_identityobjectRequired

Show child attributesHide child attributes

access_connector_idstring

The Azure resource ID of the Azure Databricks Access Connector. Use the format /subscriptions/{guid}/resourceGroups/{rg-name}/providers/Microsoft.Databricks/accessConnectors/{connector-name}.

managed_identity_idstring

The Azure resource ID of the managed identity. Use the format, /subscriptions/{guid}/resourceGroups/{rg-name}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{identity-name} This is only available for user-assgined identities. For system-assigned identities, the access_connector_id is used to identify the identity. If this field is not provided, then we assume the AzureManagedIdentity is using the system-assigned identity.

credential_idstring

The <Databricks> internal ID that represents this managed identity.

databricks_gcp_service_accountobjectRequired

Show child attributesHide child attributes

emailstring

The email of the service account.

private_key_idstring

The ID that represents the private key for this Service Account

credential_idstring

The <Databricks> internal ID that represents this managed identity.

external_location_namestring

The name of an existing external location to validate. Only applicable for storage credentials (purpose is STORAGE.)

urlstring

The external location url to validate. Only applicable when purpose is STORAGE.

read_onlyboolean

Whether the credential is only usable for read operations. Only applicable for storage credentials (purpose is STORAGE.)

Response

resultsarray of object

The results of the validation check.

Show child attributesHide child attributes

resultstring

The results of the tested operation.

Values: PASS, FAIL, SKIP

messagestring

Error message would exist when the result does not equal to PASS.

isDirboolean

Whether the tested location is a directory in cloud storage. Only applicable for when purpose is STORAGE.

View as Markdown

Request

{
  "credential_name": "string",
  "aws_iam_role": {
    "role_arn": "string",
    "unity_catalog_iam_arn": "string",
    "external_id": "string"
  },
  "azure_managed_identity": {
    "access_connector_id": "string",
    "managed_identity_id": "string",
    "credential_id": "string"
  },
  "databricks_gcp_service_account": {
    "email": "string",
    "private_key_id": "string",
    "credential_id": "string"
  },
  "external_location_name": "string",
  "url": "string",
  "read_only": true
}

Response

{
  "results": [
    {
      "result": "string",
      "message": "string"
    }
  ],
  "isDir": true
}

Validate Storage Credential GA

POST /api/2.1/unity-catalog/validate-storage-credentials

Validates a storage credential. At least one of external_location_name and url need to be provided. If only one of them is provided, it will be used for validation. And if both are provided, the url will be used for validation, and external_location_name will be ignored when checking overlapping urls.

Either the storage_credential_name or the cloud-specific credential must be provided.

The caller must be a metastore admin or the storage credential owner or have the CREATE_EXTERNAL_LOCATION privilege on the metastore and the storage credential.

API scopes: unity-catalog

Request body

storage_credential_namestringRequired

Required. The name of an existing credential or long-lived cloud credential to validate.

aws_iam_roleobjectRequired

The AWS IAM role configuration.

Show child attributesHide child attributes

role_arnstring

The Amazon Resource Name (ARN) of the AWS IAM role used to vend temporary credentials.

unity_catalog_iam_arnstring

The Amazon Resource Name (ARN) of the AWS IAM user managed by <Databricks>. This is the identity that is going to assume the AWS IAM role.

external_idstring

The external ID used in role assumption to prevent the confused deputy problem.

azure_service_principalobjectRequired

The Azure service principal configuration.

Show child attributesHide child attributes

directory_idstring

The directory ID corresponding to the Azure Active Directory (AAD) tenant of the application.

application_idstring

The application ID of the application registration within the referenced AAD tenant.

client_secretstring

The client secret generated for the above app ID in AAD.

azure_managed_identityobjectRequired

The Azure managed identity configuration.

Show child attributesHide child attributes

access_connector_idstring

The Azure resource ID of the Azure Databricks Access Connector. Use the format /subscriptions/{guid}/resourceGroups/{rg-name}/providers/Microsoft.Databricks/accessConnectors/{connector-name}.

managed_identity_idstring

The Azure resource ID of the managed identity. Use the format, /subscriptions/{guid}/resourceGroups/{rg-name}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{identity-name} This is only available for user-assgined identities. For system-assigned identities, the access_connector_id is used to identify the identity. If this field is not provided, then we assume the AzureManagedIdentity is using the system-assigned identity.

credential_idstring

The <Databricks> internal ID that represents this managed identity.

databricks_gcp_service_accountobjectRequired

The <Databricks> created GCP service account configuration.

Show child attributesHide child attributes

emailstring

The email of the service account.

private_key_idstring

The ID that represents the private key for this Service Account

credential_idstring

The <Databricks> internal ID that represents this managed identity.

external_location_namestring

The name of an existing external location to validate.

urlstring

The external location url to validate.

read_onlyboolean

Whether the storage credential is only usable for read operations.

Response

isDirboolean

Whether the tested location is a directory in cloud storage.

resultsarray of object

The results of the validation check.

Show child attributesHide child attributes

operationstring

The operation tested.

Values: LIST, READ, WRITE, DELETE, PATH_EXISTS

resultstring

The results of the tested operation.

Values: PASS, FAIL, SKIP

messagestring

Error message would exist when the result does not equal to PASS.

View as Markdown

Request

{
  "storage_credential_name": "string",
  "aws_iam_role": {
    "role_arn": "string",
    "unity_catalog_iam_arn": "string",
    "external_id": "string"
  },
  "azure_service_principal": {
    "directory_id": "string",
    "application_id": "string",
    "client_secret": "string"
  },
  "azure_managed_identity": {
    "access_connector_id": "string",
    "managed_identity_id": "string",
    "credential_id": "string"
  },
  "databricks_gcp_service_account": {
    "email": "string",
    "private_key_id": "string",
    "credential_id": "string"
  },
  "external_location_name": "string",
  "url": "string",
  "read_only": true
}

Response

{
  "isDir": true,
  "results": [
    {
      "operation": "string",
      "result": "string",
      "message": "string"
    }
  ]
}