Log Delivery Configuration

View as Markdown

Log Delivery Configuration

Get log delivery configurationGET /api/2.0/accounts/{account_id}/log-delivery/{config_id}Get all log delivery configurationsGET /api/2.0/accounts/{account_id}/log-deliveryCreate a new log delivery configurationPOST /api/2.0/accounts/{account_id}/log-deliveryEnable or disable log delivery configurationPATCH /api/2.0/accounts/{account_id}/log-delivery/{config_id}

LogDeliveryConfiguration object

Log Delivery Configuration

config_idstringuuid

The unique UUID of log delivery configuration

config_namestring

The optional human-readable name of the log delivery configuration. Defaults to empty.

log_typestring

Log delivery type. Supported values are:

• BILLABLE_USAGE — Configure billable usage log delivery. For the CSV schema, see the View billable usage.
• AUDIT_LOGS — Configure audit log delivery. For the JSON schema, see Configure audit logging

Values:

• BILLABLE_USAGE
• AUDIT_LOGS

output_formatstring

The file type of log delivery.

• If log_type is BILLABLE_USAGE, this value must be CSV. Only the CSV (comma-separated values) format is supported. For the schema, see the View billable usage
• If log_type is AUDIT_LOGS, this value must be JSON. Only the JSON (JavaScript Object Notation) format is supported. For the schema, see the Configuring audit logs.

Values:

• CSV
• JSON

account_idstringuuid

<Databricks> account ID.

AWS

<Databricks> account ID of any type. For non-E2 account types, get your account ID from the Accounts Console.

credentials_idstringuuid

The ID for a method:credentials/create that represents the AWS IAM role with policy and trust relationship as described in the main billable usage documentation page. See Configure billable usage delivery.

storage_configuration_idstringuuid

The ID for a method:storage/create that represents the S3 bucket with bucket policy as described in the main billable usage documentation page. See Configure billable usage delivery.

workspace_ids_filterarray of int64

Optional filter that specifies workspace IDs to deliver logs for. By default the workspace filter is empty and log delivery applies at the account level, delivering workspace-level logs for all workspaces in your account, plus account level logs. You can optionally set this field to an array of workspace IDs (each one is an int64) to which log delivery should apply, in which case only workspace-level logs relating to the specified workspaces are delivered. If you plan to use different log delivery configurations for different workspaces, set this field explicitly. Be aware that delivery configurations mentioning specific workspaces won't apply to new workspaces created in the future, and delivery won't include account level logs. For some types of <Databricks> deployments there is only one workspace per account ID, so this field is unnecessary.

delivery_path_prefixstring

The optional delivery path prefix within Amazon S3 storage. Defaults to empty, which means that logs are delivered to the root of the bucket. This must be a valid S3 object key. This must not start or end with a slash character.

delivery_start_timestring

This field applies only if log_type is BILLABLE_USAGE. This is the optional start month and year for delivery, specified in YYYY-MM format. Defaults to current year and month. BILLABLE_USAGE logs are not available for usage before March 2019 (2019-03).

statusstring

Status of log delivery configuration. Set to ENABLED (enabled) or DISABLED (disabled). Defaults to ENABLED. You can enable or disable the configuration later. Deletion of a configuration is not supported, so disable a log delivery configuration that is no longer needed.

Values:

• ENABLED
• DISABLED

creation_timeint64

Time in epoch milliseconds when the log delivery configuration was created.

update_timeint64

Time in epoch milliseconds when the log delivery configuration was updated.

log_delivery_statusobject

The LogDeliveryStatus of this log delivery configuration

Show child attributesHide child attributes

statusstring

Enum that describes the status. Possible values are:

• CREATED: There were no log delivery attempts since the config was created.
• SUCCEEDED: The latest attempt of log delivery has succeeded completely.
• USER_FAILURE: The latest attempt of log delivery failed because of misconfiguration of customer provided permissions on role or storage.
• SYSTEM_FAILURE: The latest attempt of log delivery failed because of an <Databricks> internal error. Contact support if it doesn't go away soon.
• NOT_FOUND: The log delivery status as the configuration has been disabled since the release of this feature or there are no workspaces in the account.

Values:

• CREATED
• SUCCEEDED
• USER_FAILURE
• SYSTEM_FAILURE
• NOT_FOUND

last_attempt_timestring

The UTC time for the latest log delivery attempt.

last_successful_attempt_timestring

The UTC time for the latest successful log delivery.

messagestring

Informative message about the latest log delivery attempt. If the log delivery fails with USER_FAILURE, error details will be provided for fixing misconfigurations in cloud permissions.

View as Markdown

The LogDeliveryConfiguration object

{
  "config_id": "string",
  "config_name": "string",
  "log_type": "BILLABLE_USAGE",
  "output_format": "CSV",
  "account_id": "string",
  "credentials_id": "string",
  "storage_configuration_id": "string",
  "workspace_ids_filter": [
    0
  ],
  "delivery_path_prefix": "string",
  "delivery_start_time": "string",
  "status": "ENABLED",
  "creation_time": 0,
  "update_time": 0,
  "log_delivery_status": {
    "status": "CREATED",
    "last_attempt_time": "string",
    "last_successful_attempt_time": "string",
    "message": "string"
  }
}

Get log delivery configuration GA

GET /api/2.0/accounts/{account_id}/log-delivery/{config_id}

Gets a <Databricks> log delivery configuration object for an account, both specified by ID.

API scopes: billing

Parameters

account_idstringuuidRequiredpath

<Databricks> account ID.

AWS

<Databricks> account ID of any type. For non-E2 account types, get your account ID from the Accounts Console.

config_idstringuuidRequiredpath

The log delivery configuration id of customer

Response

log_delivery_configurationobject

The fetched log delivery configuration

Show child attributesHide child attributes

config_idstringuuid

The unique UUID of log delivery configuration

config_namestring

The optional human-readable name of the log delivery configuration. Defaults to empty.

log_typestringRequired

Log delivery type. Supported values are:

• BILLABLE_USAGE — Configure billable usage log delivery. For the CSV schema, see the View billable usage.
• AUDIT_LOGS — Configure audit log delivery. For the JSON schema, see Configure audit logging

Values:

• BILLABLE_USAGE
• AUDIT_LOGS

output_formatstringRequired

The file type of log delivery.

• If log_type is BILLABLE_USAGE, this value must be CSV. Only the CSV (comma-separated values) format is supported. For the schema, see the View billable usage
• If log_type is AUDIT_LOGS, this value must be JSON. Only the JSON (JavaScript Object Notation) format is supported. For the schema, see the Configuring audit logs.

Values:

• CSV
• JSON

account_idstringuuidRequired

<Databricks> account ID.

AWS

<Databricks> account ID of any type. For non-E2 account types, get your account ID from the Accounts Console.

credentials_idstringuuidRequired

The ID for a method:credentials/create that represents the AWS IAM role with policy and trust relationship as described in the main billable usage documentation page. See Configure billable usage delivery.

storage_configuration_idstringuuidRequired

The ID for a method:storage/create that represents the S3 bucket with bucket policy as described in the main billable usage documentation page. See Configure billable usage delivery.

workspace_ids_filterarray of int64

Optional filter that specifies workspace IDs to deliver logs for. By default the workspace filter is empty and log delivery applies at the account level, delivering workspace-level logs for all workspaces in your account, plus account level logs. You can optionally set this field to an array of workspace IDs (each one is an int64) to which log delivery should apply, in which case only workspace-level logs relating to the specified workspaces are delivered. If you plan to use different log delivery configurations for different workspaces, set this field explicitly. Be aware that delivery configurations mentioning specific workspaces won't apply to new workspaces created in the future, and delivery won't include account level logs. For some types of <Databricks> deployments there is only one workspace per account ID, so this field is unnecessary.

delivery_path_prefixstring

The optional delivery path prefix within Amazon S3 storage. Defaults to empty, which means that logs are delivered to the root of the bucket. This must be a valid S3 object key. This must not start or end with a slash character.

delivery_start_timestring

This field applies only if log_type is BILLABLE_USAGE. This is the optional start month and year for delivery, specified in YYYY-MM format. Defaults to current year and month. BILLABLE_USAGE logs are not available for usage before March 2019 (2019-03).

statusstring

Status of log delivery configuration. Set to ENABLED (enabled) or DISABLED (disabled). Defaults to ENABLED. You can enable or disable the configuration later. Deletion of a configuration is not supported, so disable a log delivery configuration that is no longer needed.

Values:

• ENABLED
• DISABLED

creation_timeint64

Time in epoch milliseconds when the log delivery configuration was created.

update_timeint64

Time in epoch milliseconds when the log delivery configuration was updated.

log_delivery_statusobject

The LogDeliveryStatus of this log delivery configuration

Show child attributesHide child attributes

statusstringRequired

Enum that describes the status. Possible values are:

• CREATED: There were no log delivery attempts since the config was created.
• SUCCEEDED: The latest attempt of log delivery has succeeded completely.
• USER_FAILURE: The latest attempt of log delivery failed because of misconfiguration of customer provided permissions on role or storage.
• SYSTEM_FAILURE: The latest attempt of log delivery failed because of an <Databricks> internal error. Contact support if it doesn't go away soon.
• NOT_FOUND: The log delivery status as the configuration has been disabled since the release of this feature or there are no workspaces in the account.

Values:

• CREATED
• SUCCEEDED
• USER_FAILURE
• SYSTEM_FAILURE
• NOT_FOUND

last_attempt_timestring

The UTC time for the latest log delivery attempt.

last_successful_attempt_timestring

The UTC time for the latest successful log delivery.

messagestringRequired

Informative message about the latest log delivery attempt. If the log delivery fails with USER_FAILURE, error details will be provided for fixing misconfigurations in cloud permissions.

View as Markdown

Response

{
  "log_delivery_configuration": {
    "config_id": "string",
    "config_name": "string",
    "log_type": "BILLABLE_USAGE",
    "output_format": "CSV",
    "account_id": "string",
    "credentials_id": "string",
    "storage_configuration_id": "string",
    "workspace_ids_filter": [
      0
    ],
    "delivery_path_prefix": "string",
    "delivery_start_time": "string",
    "status": "ENABLED",
    "creation_time": 0,
    "update_time": 0,
    "log_delivery_status": {}
  }
}

Get all log delivery configurations GA

GET /api/2.0/accounts/{account_id}/log-delivery

Gets all <Databricks> log delivery configurations associated with an account specified by ID.

API scopes: billing

Parameters

account_idstringuuidRequiredpath

<Databricks> account ID.

AWS

<Databricks> account ID of any type. For non-E2 account types, get your account ID from the Accounts Console.

credentials_idstringuuidquery

The Credentials id to filter the search results with

storage_configuration_idstringuuidquery

The Storage Configuration id to filter the search results with

statusstringquery

The log delivery status to filter the search results with

Values:

• ENABLED
• DISABLED

page_tokenstringquery

A page token received from a previous get all budget configurations call. This token can be used to retrieve the subsequent page. Requests first page if absent.

Response

Returns a list of LogDeliveryConfiguration objects.

View as Markdown

Response

{
  "log_delivery_configurations": [
    {
      "config_id": "string",
      "config_name": "string",
      "log_type": "BILLABLE_USAGE",
      "output_format": "CSV",
      "account_id": "string",
      "credentials_id": "string",
      "storage_configuration_id": "string",
      "workspace_ids_filter": [
        0
      ],
      "delivery_path_prefix": "string",
      "delivery_start_time": "string",
      "status": "ENABLED",
      "creation_time": 0,
      "update_time": 0,
      "log_delivery_status": {}
    }
  ],
  "next_page_token": "string"
}

Create a new log delivery configuration GA

POST /api/2.0/accounts/{account_id}/log-delivery

Creates a new <Databricks> log delivery configuration to enable delivery of the specified type of logs to your storage location. This requires that you already created a credential object (which encapsulates a cross-account service IAM role) and a storage configuration object (which encapsulates an S3 bucket).

For full details, including the required IAM role policies and bucket policies, see Deliver and access billable usage logs or Configure audit logging.

Note: There is a limit on the number of log delivery configurations available per account (each limit applies separately to each log type including billable usage and audit logs). You can create a maximum of two enabled account-level delivery configurations (configurations without a workspace filter) per type. Additionally, you can create two enabled workspace-level delivery configurations per workspace for each log type, which means that the same workspace ID can occur in the workspace filter for no more than two delivery configurations per log type.

You cannot delete a log delivery configuration, but you can disable it (see Enable or disable log delivery configuration).

API scopes: billing

Parameters

account_idstringuuidRequiredpath

<Databricks> account ID.

AWS

<Databricks> account ID of any type. For non-E2 account types, get your account ID from the Accounts Console.

Request body

log_delivery_configurationobjectRequired

Show child attributesHide child attributes

config_idstringuuid

The unique UUID of log delivery configuration

config_namestring

The optional human-readable name of the log delivery configuration. Defaults to empty.

log_typestringRequired

Log delivery type. Supported values are:

• BILLABLE_USAGE — Configure billable usage log delivery. For the CSV schema, see the View billable usage.
• AUDIT_LOGS — Configure audit log delivery. For the JSON schema, see Configure audit logging

Values:

• BILLABLE_USAGE
• AUDIT_LOGS

output_formatstringRequired

The file type of log delivery.

• If log_type is BILLABLE_USAGE, this value must be CSV. Only the CSV (comma-separated values) format is supported. For the schema, see the View billable usage
• If log_type is AUDIT_LOGS, this value must be JSON. Only the JSON (JavaScript Object Notation) format is supported. For the schema, see the Configuring audit logs.

Values:

• CSV
• JSON

credentials_idstringuuidRequired

The ID for a method:credentials/create that represents the AWS IAM role with policy and trust relationship as described in the main billable usage documentation page. See Configure billable usage delivery.

storage_configuration_idstringuuidRequired

The ID for a method:storage/create that represents the S3 bucket with bucket policy as described in the main billable usage documentation page. See Configure billable usage delivery.

workspace_ids_filterarray of int64

Optional filter that specifies workspace IDs to deliver logs for. By default the workspace filter is empty and log delivery applies at the account level, delivering workspace-level logs for all workspaces in your account, plus account level logs. You can optionally set this field to an array of workspace IDs (each one is an int64) to which log delivery should apply, in which case only workspace-level logs relating to the specified workspaces are delivered. If you plan to use different log delivery configurations for different workspaces, set this field explicitly. Be aware that delivery configurations mentioning specific workspaces won't apply to new workspaces created in the future, and delivery won't include account level logs. For some types of <Databricks> deployments there is only one workspace per account ID, so this field is unnecessary.

delivery_path_prefixstring

The optional delivery path prefix within Amazon S3 storage. Defaults to empty, which means that logs are delivered to the root of the bucket. This must be a valid S3 object key. This must not start or end with a slash character.

delivery_start_timestring

This field applies only if log_type is BILLABLE_USAGE. This is the optional start month and year for delivery, specified in YYYY-MM format. Defaults to current year and month. BILLABLE_USAGE logs are not available for usage before March 2019 (2019-03).

statusstring

Status of log delivery configuration. Set to ENABLED (enabled) or DISABLED (disabled). Defaults to ENABLED. You can enable or disable the configuration later. Deletion of a configuration is not supported, so disable a log delivery configuration that is no longer needed.

Values:

• ENABLED
• DISABLED

creation_timeint64

Time in epoch milliseconds when the log delivery configuration was created.

update_timeint64

Time in epoch milliseconds when the log delivery configuration was updated.

log_delivery_statusobject

The LogDeliveryStatus of this log delivery configuration

Show child attributesHide child attributes

statusstringRequired

Enum that describes the status. Possible values are:

• CREATED: There were no log delivery attempts since the config was created.
• SUCCEEDED: The latest attempt of log delivery has succeeded completely.
• USER_FAILURE: The latest attempt of log delivery failed because of misconfiguration of customer provided permissions on role or storage.
• SYSTEM_FAILURE: The latest attempt of log delivery failed because of an <Databricks> internal error. Contact support if it doesn't go away soon.
• NOT_FOUND: The log delivery status as the configuration has been disabled since the release of this feature or there are no workspaces in the account.

Values:

• CREATED
• SUCCEEDED
• USER_FAILURE
• SYSTEM_FAILURE
• NOT_FOUND

last_attempt_timestring

The UTC time for the latest log delivery attempt.

last_successful_attempt_timestring

The UTC time for the latest successful log delivery.

messagestringRequired

Informative message about the latest log delivery attempt. If the log delivery fails with USER_FAILURE, error details will be provided for fixing misconfigurations in cloud permissions.

Response

log_delivery_configurationobject

The created log delivery configuration

Show child attributesHide child attributes

config_idstringuuid

The unique UUID of log delivery configuration

config_namestring

The optional human-readable name of the log delivery configuration. Defaults to empty.

log_typestringRequired

Log delivery type. Supported values are:

• BILLABLE_USAGE — Configure billable usage log delivery. For the CSV schema, see the View billable usage.
• AUDIT_LOGS — Configure audit log delivery. For the JSON schema, see Configure audit logging

Values:

• BILLABLE_USAGE
• AUDIT_LOGS

output_formatstringRequired

The file type of log delivery.

• If log_type is BILLABLE_USAGE, this value must be CSV. Only the CSV (comma-separated values) format is supported. For the schema, see the View billable usage
• If log_type is AUDIT_LOGS, this value must be JSON. Only the JSON (JavaScript Object Notation) format is supported. For the schema, see the Configuring audit logs.

Values:

• CSV
• JSON

account_idstringuuidRequired

<Databricks> account ID.

AWS

<Databricks> account ID of any type. For non-E2 account types, get your account ID from the Accounts Console.

credentials_idstringuuidRequired

The ID for a method:credentials/create that represents the AWS IAM role with policy and trust relationship as described in the main billable usage documentation page. See Configure billable usage delivery.

storage_configuration_idstringuuidRequired

The ID for a method:storage/create that represents the S3 bucket with bucket policy as described in the main billable usage documentation page. See Configure billable usage delivery.

workspace_ids_filterarray of int64

Optional filter that specifies workspace IDs to deliver logs for. By default the workspace filter is empty and log delivery applies at the account level, delivering workspace-level logs for all workspaces in your account, plus account level logs. You can optionally set this field to an array of workspace IDs (each one is an int64) to which log delivery should apply, in which case only workspace-level logs relating to the specified workspaces are delivered. If you plan to use different log delivery configurations for different workspaces, set this field explicitly. Be aware that delivery configurations mentioning specific workspaces won't apply to new workspaces created in the future, and delivery won't include account level logs. For some types of <Databricks> deployments there is only one workspace per account ID, so this field is unnecessary.

delivery_path_prefixstring

The optional delivery path prefix within Amazon S3 storage. Defaults to empty, which means that logs are delivered to the root of the bucket. This must be a valid S3 object key. This must not start or end with a slash character.

delivery_start_timestring

This field applies only if log_type is BILLABLE_USAGE. This is the optional start month and year for delivery, specified in YYYY-MM format. Defaults to current year and month. BILLABLE_USAGE logs are not available for usage before March 2019 (2019-03).

statusstring

Status of log delivery configuration. Set to ENABLED (enabled) or DISABLED (disabled). Defaults to ENABLED. You can enable or disable the configuration later. Deletion of a configuration is not supported, so disable a log delivery configuration that is no longer needed.

Values:

• ENABLED
• DISABLED

creation_timeint64

Time in epoch milliseconds when the log delivery configuration was created.

update_timeint64

Time in epoch milliseconds when the log delivery configuration was updated.

log_delivery_statusobject

The LogDeliveryStatus of this log delivery configuration

Show child attributesHide child attributes

statusstringRequired

Enum that describes the status. Possible values are:

• CREATED: There were no log delivery attempts since the config was created.
• SUCCEEDED: The latest attempt of log delivery has succeeded completely.
• USER_FAILURE: The latest attempt of log delivery failed because of misconfiguration of customer provided permissions on role or storage.
• SYSTEM_FAILURE: The latest attempt of log delivery failed because of an <Databricks> internal error. Contact support if it doesn't go away soon.
• NOT_FOUND: The log delivery status as the configuration has been disabled since the release of this feature or there are no workspaces in the account.

Values:

• CREATED
• SUCCEEDED
• USER_FAILURE
• SYSTEM_FAILURE
• NOT_FOUND

last_attempt_timestring

The UTC time for the latest log delivery attempt.

last_successful_attempt_timestring

The UTC time for the latest successful log delivery.

messagestringRequired

Informative message about the latest log delivery attempt. If the log delivery fails with USER_FAILURE, error details will be provided for fixing misconfigurations in cloud permissions.

View as Markdown

Request

{
  "log_delivery_configuration": {
    "config_id": "string",
    "config_name": "string",
    "log_type": "BILLABLE_USAGE",
    "output_format": "CSV",
    "credentials_id": "string",
    "storage_configuration_id": "string",
    "workspace_ids_filter": [
      0
    ],
    "delivery_path_prefix": "string",
    "delivery_start_time": "string",
    "status": "ENABLED",
    "creation_time": 0,
    "update_time": 0,
    "log_delivery_status": {}
  }
}

Response

{
  "log_delivery_configuration": {
    "config_id": "string",
    "config_name": "string",
    "log_type": "BILLABLE_USAGE",
    "output_format": "CSV",
    "account_id": "string",
    "credentials_id": "string",
    "storage_configuration_id": "string",
    "workspace_ids_filter": [
      0
    ],
    "delivery_path_prefix": "string",
    "delivery_start_time": "string",
    "status": "ENABLED",
    "creation_time": 0,
    "update_time": 0,
    "log_delivery_status": {}
  }
}

Enable or disable log delivery configuration GA

PATCH /api/2.0/accounts/{account_id}/log-delivery/{config_id}

Enables or disables a log delivery configuration. Deletion of delivery configurations is not supported, so disable log delivery configurations that are no longer needed. Note that you can't re-enable a delivery configuration if this would violate the delivery configuration limits described under Create log delivery.

API scopes: billing

Parameters

account_idstringuuidRequiredpath

<Databricks> account ID of any type. For non-E2 account types, get your account ID from the Accounts Console.

config_idstringuuidRequiredpath

The log delivery configuration id of customer

Request body

statusstringRequired

Status of log delivery configuration. Set to ENABLED (enabled) or DISABLED (disabled). Defaults to ENABLED. You can enable or disable the configuration later. Deletion of a configuration is not supported, so disable a log delivery configuration that is no longer needed.

Values:

• ENABLED
• DISABLED

View as Markdown

Request

{
  "status": "ENABLED"
}

Response

{}