Uc Credentials

View as Markdown

Connect Unity Catalog to external data and services

Generate Temporary Path CredentialPOST /api/2.0/unity-catalog/temporary-path-credentialsGenerate Temporary Table CredentialPOST /api/2.0/unity-catalog/temporary-table-credentialsGenerate Temporary Volume CredentialPOST /api/2.0/unity-catalog/temporary-volume-credentialsValidate Storage CredentialPOST /api/2.1/unity-catalog/validate-storage-credentials

Generate Temporary Path Credential GA

POST /api/2.0/unity-catalog/temporary-path-credentials

Get a short-lived credential for directly accessing cloud storage locations registered in <Databricks>. The Generate Temporary Path Credentials API is only supported for external storage paths, specifically external locations and external tables. Managed tables are not supported by this API. The metastore must have external_access_enabled flag set to true (default false). The caller must have the EXTERNAL_USE_LOCATION privilege on the external location; this privilege can only be granted by external location owners. For requests on existing external tables, the caller must also have the EXTERNAL_USE_SCHEMA privilege on the parent schema; this privilege can only be granted by catalog owners.

API scopes: unity-catalog

Request body

urlstringRequired

URL for path-based access.

operationstringRequired

The operation being performed on the path.

Values: PATH_READ, PATH_READ_WRITE, PATH_CREATE_TABLE

dry_runboolean

Optional. When set to true, the service will not validate that the generated credentials can perform write operations, therefore no new paths will be created and the response will not contain valid credentials. Defaults to false.

Response

aws_temp_credentialsobject

Show child attributesHide child attributes

access_key_idstring

The access key ID that identifies the temporary credentials.

secret_access_keystring

The secret access key that can be used to sign AWS API requests.

session_tokenstring

The token that users must pass to AWS API to use the temporary credentials.

access_pointstring

The Amazon Resource Name (ARN) of the S3 access point for temporary credentials related the external location.

azure_user_delegation_sasobject

Show child attributesHide child attributes

sas_tokenstring

The signed URI (SAS Token) used to access blob services for a given path

gcp_oauth_tokenobject

Show child attributesHide child attributes

oauth_tokenstring

azure_aadobject

Show child attributesHide child attributes

aad_tokenstring

Opaque token that contains claims that you can use in Azure Active Directory to access cloud services.

r2_temp_credentialsobject

Show child attributesHide child attributes

access_key_idstring

The access key ID that identifies the temporary credentials.

secret_access_keystring

The secret access key associated with the access key.

session_tokenstring

The generated JWT that users must pass to use the temporary credentials.

expiration_timeint64

Server time when the credential will expire, in epoch milliseconds. The API client is advised to cache the credential given this expiration time.

urlstring

The URL of the storage path accessible by the temporary credential.

View as Markdown

Request

{
  "url": "string",
  "operation": "string",
  "dry_run": true
}

Response

{
  "aws_temp_credentials": {
    "access_key_id": "string",
    "secret_access_key": "string",
    "session_token": "string",
    "access_point": "string"
  },
  "azure_user_delegation_sas": {
    "sas_token": "string"
  },
  "gcp_oauth_token": {
    "oauth_token": "string"
  },
  "azure_aad": {
    "aad_token": "string"
  },
  "r2_temp_credentials": {
    "access_key_id": "string",
    "secret_access_key": "string",
    "session_token": "string"
  },
  "expiration_time": 0,
  "url": "string"
}

Generate Temporary Table Credential GA

POST /api/2.0/unity-catalog/temporary-table-credentials

Get a short-lived credential for directly accessing the table data on cloud storage. The metastore must have external_access_enabled flag set to true (default false). The caller must have the EXTERNAL_USE_SCHEMA privilege on the parent schema and this privilege can only be granted by catalog owners.

API scopes: unity-catalog

Request body

table_idstring

UUID of the table to read or write.

operationstring

The operation performed against the table data, either READ or READ_WRITE. If READ_WRITE is specified, the credentials returned will have write permissions, otherwise, it will be read only.

Values: READ, READ_WRITE

Response

aws_temp_credentialsobject

Show child attributesHide child attributes

access_key_idstring

The access key ID that identifies the temporary credentials.

secret_access_keystring

The secret access key that can be used to sign AWS API requests.

session_tokenstring

The token that users must pass to AWS API to use the temporary credentials.

access_pointstring

The Amazon Resource Name (ARN) of the S3 access point for temporary credentials related the external location.

azure_user_delegation_sasobject

Show child attributesHide child attributes

sas_tokenstring

The signed URI (SAS Token) used to access blob services for a given path

gcp_oauth_tokenobject

Show child attributesHide child attributes

oauth_tokenstring

azure_aadobject

Show child attributesHide child attributes

aad_tokenstring

Opaque token that contains claims that you can use in Azure Active Directory to access cloud services.

r2_temp_credentialsobject

Show child attributesHide child attributes

access_key_idstring

The access key ID that identifies the temporary credentials.

secret_access_keystring

The secret access key associated with the access key.

session_tokenstring

The generated JWT that users must pass to use the temporary credentials.

expiration_timeint64

Server time when the credential will expire, in epoch milliseconds. The API client is advised to cache the credential given this expiration time.

urlstring

The URL of the storage path accessible by the temporary credential.

View as Markdown

Request

{
  "table_id": "string",
  "operation": "string"
}

Response

{
  "aws_temp_credentials": {
    "access_key_id": "string",
    "secret_access_key": "string",
    "session_token": "string",
    "access_point": "string"
  },
  "azure_user_delegation_sas": {
    "sas_token": "string"
  },
  "gcp_oauth_token": {
    "oauth_token": "string"
  },
  "azure_aad": {
    "aad_token": "string"
  },
  "r2_temp_credentials": {
    "access_key_id": "string",
    "secret_access_key": "string",
    "session_token": "string"
  },
  "expiration_time": 0,
  "url": "string"
}

Generate Temporary Volume Credential Public Preview

POST /api/2.0/unity-catalog/temporary-volume-credentials

Get a short-lived credential for directly accessing the volume data on cloud storage. The metastore must have external_access_enabled flag set to true (default false). The caller must have the EXTERNAL_USE_SCHEMA privilege on the parent schema and this privilege can only be granted by catalog owners.

API scopes: unity-catalog

Request body

volume_idstring

Id of the volume to read or write.

operationstring

The operation performed against the volume data, either READ_VOLUME or WRITE_VOLUME. If WRITE_VOLUME is specified, the credentials returned will have write permissions, otherwise, it will be read only.

Values: READ_VOLUME, WRITE_VOLUME

Response

aws_temp_credentialsobject

Show child attributesHide child attributes

access_key_idstring

The access key ID that identifies the temporary credentials.

secret_access_keystring

The secret access key that can be used to sign AWS API requests.

session_tokenstring

The token that users must pass to AWS API to use the temporary credentials.

access_pointstring

The Amazon Resource Name (ARN) of the S3 access point for temporary credentials related the external location.

azure_user_delegation_sasobject

Show child attributesHide child attributes

sas_tokenstring

The signed URI (SAS Token) used to access blob services for a given path

gcp_oauth_tokenobject

Show child attributesHide child attributes

oauth_tokenstring

azure_aadobject

Show child attributesHide child attributes

aad_tokenstring

Opaque token that contains claims that you can use in Azure Active Directory to access cloud services.

r2_temp_credentialsobject

Show child attributesHide child attributes

access_key_idstring

The access key ID that identifies the temporary credentials.

secret_access_keystring

The secret access key associated with the access key.

session_tokenstring

The generated JWT that users must pass to use the temporary credentials.

expiration_timeint64

Server time when the credential will expire, in epoch milliseconds. The API client is advised to cache the credential given this expiration time.

urlstring

The URL of the storage path accessible by the temporary credential.

View as Markdown

Request

{
  "volume_id": "string",
  "operation": "string"
}

Response

{
  "aws_temp_credentials": {
    "access_key_id": "string",
    "secret_access_key": "string",
    "session_token": "string",
    "access_point": "string"
  },
  "azure_user_delegation_sas": {
    "sas_token": "string"
  },
  "gcp_oauth_token": {
    "oauth_token": "string"
  },
  "azure_aad": {
    "aad_token": "string"
  },
  "r2_temp_credentials": {
    "access_key_id": "string",
    "secret_access_key": "string",
    "session_token": "string"
  },
  "expiration_time": 0,
  "url": "string"
}

Validate Storage Credential GA

POST /api/2.1/unity-catalog/validate-storage-credentials

Validates a storage credential. At least one of external_location_name and url need to be provided. If only one of them is provided, it will be used for validation. And if both are provided, the url will be used for validation, and external_location_name will be ignored when checking overlapping urls.

Either the storage_credential_name or the cloud-specific credential must be provided.

The caller must be a metastore admin or the storage credential owner or have the CREATE_EXTERNAL_LOCATION privilege on the metastore and the storage credential.

API scopes: unity-catalog

Request body

storage_credential_namestring

Required. The name of an existing credential or long-lived cloud credential to validate.

aws_iam_roleobject

The AWS IAM role configuration.

Show child attributesHide child attributes

role_arnstring

The Amazon Resource Name (ARN) of the AWS IAM role used to vend temporary credentials.

unity_catalog_iam_arnstring

The Amazon Resource Name (ARN) of the AWS IAM user managed by <Databricks>. This is the identity that is going to assume the AWS IAM role.

external_idstring

The external ID used in role assumption to prevent the confused deputy problem.

azure_service_principalobject

The Azure service principal configuration.

Show child attributesHide child attributes

directory_idstringRequired

The directory ID corresponding to the Azure Active Directory (AAD) tenant of the application.

application_idstringRequired

The application ID of the application registration within the referenced AAD tenant.

client_secretstringRequired

The client secret generated for the above app ID in AAD.

azure_managed_identityobject

The Azure managed identity configuration.

Show child attributesHide child attributes

access_connector_idstringRequired

The Azure resource ID of the Azure Databricks Access Connector. Use the format /subscriptions/{guid}/resourceGroups/{rg-name}/providers/Microsoft.Databricks/accessConnectors/{connector-name}.

managed_identity_idstring

The Azure resource ID of the managed identity. Use the format, /subscriptions/{guid}/resourceGroups/{rg-name}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{identity-name} This is only available for user-assgined identities. For system-assigned identities, the access_connector_id is used to identify the identity. If this field is not provided, then we assume the AzureManagedIdentity is using the system-assigned identity.

credential_idstring

The <Databricks> internal ID that represents this managed identity.

databricks_gcp_service_accountobject

The <Databricks> created GCP service account configuration.

Show child attributesHide child attributes

emailstring

The email of the service account.

private_key_idstring

The ID that represents the private key for this Service Account

credential_idstring

The <Databricks> internal ID that represents this managed identity.

external_location_namestring

The name of an existing external location to validate.

urlstring

The external location url to validate.

read_onlyboolean

Whether the storage credential is only usable for read operations.

Response

isDirboolean

Whether the tested location is a directory in cloud storage.

resultsarray of object

The results of the validation check.

Show child attributesHide child attributes

operationstring

The operation tested.

Values: LIST, READ, WRITE, DELETE, PATH_EXISTS

resultstring

The results of the tested operation.

Values: PASS, FAIL, SKIP

messagestring

Error message would exist when the result does not equal to PASS.

View as Markdown

Request

{
  "storage_credential_name": "string",
  "aws_iam_role": {
    "role_arn": "string",
    "unity_catalog_iam_arn": "string",
    "external_id": "string"
  },
  "azure_service_principal": {
    "directory_id": "string",
    "application_id": "string",
    "client_secret": "string"
  },
  "azure_managed_identity": {
    "access_connector_id": "string",
    "managed_identity_id": "string",
    "credential_id": "string"
  },
  "databricks_gcp_service_account": {
    "email": "string",
    "private_key_id": "string",
    "credential_id": "string"
  },
  "external_location_name": "string",
  "url": "string",
  "read_only": true
}

Response

{
  "isDir": true,
  "results": [
    {
      "operation": "string",
      "result": "string",
      "message": "string"
    }
  ]
}