Metastore Data Access

View as Markdown

GetGET /api/2.1/unity-catalog/storage-credentials/{name_arg}CreatePOST /api/2.1/unity-catalog/storage-credentialsDeleteDELETE /api/2.1/unity-catalog/storage-credentials/{name_arg}

StorageCredentialInfo object

namestring

The credential name. The name must be unique among storage and service credentials within the metastore.

aws_iam_roleobject

The AWS IAM role configuration.

Show child attributesHide child attributes

role_arnstring

The Amazon Resource Name (ARN) of the AWS IAM role used to vend temporary credentials.

unity_catalog_iam_arnstring

The Amazon Resource Name (ARN) of the AWS IAM user managed by <Databricks>. This is the identity that is going to assume the AWS IAM role.

external_idstring

The external ID used in role assumption to prevent the confused deputy problem.

azure_service_principalobject

The Azure service principal configuration.

Show child attributesHide child attributes

directory_idstring

The directory ID corresponding to the Azure Active Directory (AAD) tenant of the application.

application_idstring

The application ID of the application registration within the referenced AAD tenant.

client_secretstring

The client secret generated for the above app ID in AAD.

gcp_service_account_keyobject

Show child attributesHide child attributes

emailstring

The email of the service account.

private_key_idstring

The ID of the service account's private key.

private_keystring

The service account's RSA private key.

azure_managed_identityobject

The Azure managed identity configuration.

Show child attributesHide child attributes

access_connector_idstring

The Azure resource ID of the Azure Databricks Access Connector. Use the format /subscriptions/{guid}/resourceGroups/{rg-name}/providers/Microsoft.Databricks/accessConnectors/{connector-name}.

managed_identity_idstring

The Azure resource ID of the managed identity. Use the format, /subscriptions/{guid}/resourceGroups/{rg-name}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{identity-name} This is only available for user-assgined identities. For system-assigned identities, the access_connector_id is used to identify the identity. If this field is not provided, then we assume the AzureManagedIdentity is using the system-assigned identity.

credential_idstring

The <Databricks> internal ID that represents this managed identity.

databricks_gcp_service_accountobject

The <Databricks> managed GCP service account configuration.

Show child attributesHide child attributes

emailstring

The email of the service account.

private_key_idstring

The ID that represents the private key for this Service Account

credential_idstring

The <Databricks> internal ID that represents this managed identity.

commentstring

Comment associated with the credential.

read_onlyboolean

Whether the credential is usable only for read operations. Only applicable when purpose is STORAGE.

ownerstring

Username of current owner of credential.

idstring

The unique identifier of the credential.

metastore_idstring

Unique identifier of the parent metastore.

created_atint64

Time at which this credential was created, in epoch milliseconds.

created_bystring

Username of credential creator.

updated_atint64

Time at which this credential was last modified, in epoch milliseconds.

updated_bystring

Username of user who last modified the credential.

used_for_managed_storageboolean

Whether this credential is the current metastore's root storage credential. Only applicable when purpose is STORAGE.

full_namestring

The full name of the credential.

isolation_modestring

Whether the current securable is accessible from all workspaces or a specific set of workspaces.

Values: ISOLATION_MODE_UNSPECIFIED, ISOLATION_MODE_OPEN, ISOLATION_MODE_ISOLATED

View as Markdown

The StorageCredentialInfo object

{
  "name": "string",
  "aws_iam_role": {
    "role_arn": "string",
    "unity_catalog_iam_arn": "string",
    "external_id": "string"
  },
  "azure_service_principal": {
    "directory_id": "string",
    "application_id": "string",
    "client_secret": "string"
  },
  "gcp_service_account_key": {
    "email": "string",
    "private_key_id": "string",
    "private_key": "string"
  },
  "azure_managed_identity": {
    "access_connector_id": "string",
    "managed_identity_id": "string",
    "credential_id": "string"
  },
  "databricks_gcp_service_account": {
    "email": "string",
    "private_key_id": "string",
    "credential_id": "string"
  },
  "comment": "string",
  "read_only": true,
  "owner": "string",
  "id": "string",
  "metastore_id": "string",
  "created_at": 0,
  "created_by": "string",
  "updated_at": 0,
  "updated_by": "string",
  "used_for_managed_storage": true,
  "full_name": "string",
  "isolation_mode": "string"
}

Get GA

GET /api/2.1/unity-catalog/storage-credentials/{name_arg}

Gets a storage credential from the metastore. The caller must be a metastore admin, the owner of the storage credential, or have some permission on the storage credential.

API scopes: unity-catalog

Parameters

name_argstringpath

Name of the storage credential.

Response

Returns the StorageCredentialInfo object.

View as Markdown

Response

{
  "name": "string",
  "aws_iam_role": {
    "role_arn": "string",
    "unity_catalog_iam_arn": "string",
    "external_id": "string"
  },
  "azure_service_principal": {
    "directory_id": "string",
    "application_id": "string",
    "client_secret": "string"
  },
  "gcp_service_account_key": {
    "email": "string",
    "private_key_id": "string",
    "private_key": "string"
  },
  "azure_managed_identity": {
    "access_connector_id": "string",
    "managed_identity_id": "string",
    "credential_id": "string"
  },
  "databricks_gcp_service_account": {
    "email": "string",
    "private_key_id": "string",
    "credential_id": "string"
  },
  "comment": "string",
  "read_only": true,
  "owner": "string",
  "id": "string",
  "metastore_id": "string",
  "created_at": 0,
  "created_by": "string",
  "updated_at": 0,
  "updated_by": "string",
  "used_for_managed_storage": true,
  "full_name": "string",
  "isolation_mode": "string"
}

Create GA

POST /api/2.1/unity-catalog/storage-credentials

Creates a new storage credential.

The caller must be a metastore admin or have the CREATE_STORAGE_CREDENTIAL privilege on the metastore.

API scopes: unity-catalog

AWS

Creates a new storage credential.

The caller must be a metastore admin or have the CREATE_STORAGE_CREDENTIAL privilege on the metastore.

The request object must contain an AwsIamRole detailing the credentials of an IAM role. To prevent the confused deputy problem, this role must specify an external ID in its trust policy.

To enable this credential, the external ID specified in the external_id field of the response object must be added to the IAM role's trust policy.

Azure

Creates a new storage credential.

The caller must be a metastore admin or have the CREATE_STORAGE_CREDENTIAL privilege on the metastore.

The request object must contain an AzureServicePrincipal or AzureManagedIdentity, depending on the credential type being registered.

Additionally, non-account admin users must have the contributor role over the access connector for any AzureManagedIdentity credential they create. Credentials of the type AzureServicePrincipal may only be created by an account admin.

GCP

Creates a new storage credential.

The caller must be a metastore admin or have the CREATE_STORAGE_CREDENTIAL privilege on the metastore.

The request object must contain a DatabricksGcpServiceAccount.

Request body

skip_validationboolean

Supplying true to this argument skips validation of the created credential.

Default: false

namestring

The credential name. The name must be unique among storage and service credentials within the metastore.

aws_iam_roleobjectRequired

The AWS IAM role configuration.

Show child attributesHide child attributes

role_arnstring

The Amazon Resource Name (ARN) of the AWS IAM role used to vend temporary credentials.

unity_catalog_iam_arnstring

The Amazon Resource Name (ARN) of the AWS IAM user managed by <Databricks>. This is the identity that is going to assume the AWS IAM role.

external_idstring

The external ID used in role assumption to prevent the confused deputy problem.

azure_service_principalobjectRequired

The Azure service principal configuration.

Show child attributesHide child attributes

directory_idstring

The directory ID corresponding to the Azure Active Directory (AAD) tenant of the application.

application_idstring

The application ID of the application registration within the referenced AAD tenant.

client_secretstring

The client secret generated for the above app ID in AAD.

gcp_service_account_keyobjectRequired

Show child attributesHide child attributes

emailstring

The email of the service account.

private_key_idstring

The ID of the service account's private key.

private_keystring

The service account's RSA private key.

azure_managed_identityobjectRequired

The Azure managed identity configuration.

Show child attributesHide child attributes

access_connector_idstring

The Azure resource ID of the Azure Databricks Access Connector. Use the format /subscriptions/{guid}/resourceGroups/{rg-name}/providers/Microsoft.Databricks/accessConnectors/{connector-name}.

managed_identity_idstring

The Azure resource ID of the managed identity. Use the format, /subscriptions/{guid}/resourceGroups/{rg-name}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{identity-name} This is only available for user-assgined identities. For system-assigned identities, the access_connector_id is used to identify the identity. If this field is not provided, then we assume the AzureManagedIdentity is using the system-assigned identity.

credential_idstring

The <Databricks> internal ID that represents this managed identity.

databricks_gcp_service_accountobjectRequired

The <Databricks> managed GCP service account configuration.

Show child attributesHide child attributes

emailstring

The email of the service account.

private_key_idstring

The ID that represents the private key for this Service Account

credential_idstring

The <Databricks> internal ID that represents this managed identity.

commentstring

Comment associated with the credential.

read_onlyboolean

Whether the credential is usable only for read operations. Only applicable when purpose is STORAGE.

ownerstring

Username of current owner of credential.

idstring

The unique identifier of the credential.

metastore_idstring

Unique identifier of the parent metastore.

created_atint64

Time at which this credential was created, in epoch milliseconds.

created_bystring

Username of credential creator.

updated_atint64

Time at which this credential was last modified, in epoch milliseconds.

updated_bystring

Username of user who last modified the credential.

used_for_managed_storageboolean

Whether this credential is the current metastore's root storage credential. Only applicable when purpose is STORAGE.

full_namestring

The full name of the credential.

isolation_modestring

Whether the current securable is accessible from all workspaces or a specific set of workspaces.

Values: ISOLATION_MODE_UNSPECIFIED, ISOLATION_MODE_OPEN, ISOLATION_MODE_ISOLATED

Response

Returns the StorageCredentialInfo object.

View as Markdown

Request

{
  "skip_validation": true,
  "name": "string",
  "aws_iam_role": {
    "role_arn": "string",
    "unity_catalog_iam_arn": "string",
    "external_id": "string"
  },
  "azure_service_principal": {
    "directory_id": "string",
    "application_id": "string",
    "client_secret": "string"
  },
  "gcp_service_account_key": {
    "email": "string",
    "private_key_id": "string",
    "private_key": "string"
  },
  "azure_managed_identity": {
    "access_connector_id": "string",
    "managed_identity_id": "string",
    "credential_id": "string"
  },
  "databricks_gcp_service_account": {
    "email": "string",
    "private_key_id": "string",
    "credential_id": "string"
  },
  "comment": "string",
  "read_only": true,
  "owner": "string",
  "id": "string",
  "metastore_id": "string",
  "created_at": 0,
  "created_by": "string",
  "updated_at": 0,
  "updated_by": "string",
  "used_for_managed_storage": true,
  "full_name": "string",
  "isolation_mode": "string"
}

Response

{
  "name": "string",
  "aws_iam_role": {
    "role_arn": "string",
    "unity_catalog_iam_arn": "string",
    "external_id": "string"
  },
  "azure_service_principal": {
    "directory_id": "string",
    "application_id": "string",
    "client_secret": "string"
  },
  "gcp_service_account_key": {
    "email": "string",
    "private_key_id": "string",
    "private_key": "string"
  },
  "azure_managed_identity": {
    "access_connector_id": "string",
    "managed_identity_id": "string",
    "credential_id": "string"
  },
  "databricks_gcp_service_account": {
    "email": "string",
    "private_key_id": "string",
    "credential_id": "string"
  },
  "comment": "string",
  "read_only": true,
  "owner": "string",
  "id": "string",
  "metastore_id": "string",
  "created_at": 0,
  "created_by": "string",
  "updated_at": 0,
  "updated_by": "string",
  "used_for_managed_storage": true,
  "full_name": "string",
  "isolation_mode": "string"
}

Delete GA

DELETE /api/2.1/unity-catalog/storage-credentials/{name_arg}

Deletes a storage credential from the metastore. The caller must be an owner of the storage credential.

API scopes: unity-catalog

Parameters

name_argstringpath

Name of the storage credential.

forcebooleanquery

Force an update even if there are dependent external locations or external tables (when purpose is STORAGE) or dependent services (when purpose is SERVICE).

View as Markdown

Response

{}