Deliver and access billable usage logs

Preview

This feature is in Public Preview.

As a Databricks account owner or account admin), you can configure daily delivery of billable usage logs in CSV file format to an AWS S3 storage bucket, where you can make the data available for usage analysis. Databricks delivers a separate CSV file for each workspace in your account. This CSV file includes historical data about the workspace’s cluster usage in Databricks Units (DBUs), sortable by cluster ID, billing SKU, cluster creator, cluster tags, and more. For a description of each CSV file column, see CSV file schema.

If your account is on the E2 version of the platform, account owners and account admins can view usage in graph or table form on the Usage page in the account console for E2, which is in Public Preview. That page also includes a usage chart that displays account usage in DBUs, grouped by workload type, and allows you to directly download usage data in CSV format.

Note

All new Databricks accounts and most existing accounts are now E2. If you are unsure which account type you have, contact your Databricks representative.

If your account is on another version of the platform, account owners can view usage on the legacy account console’s Usage Overview tab.

You can optionally deliver logs to an AWS account other than the account used for the IAM role that you create for log delivery. This allows flexibility, for example setting up workspaces from multiple AWS accounts to deliver to the same S3 bucket. This option requires that you configure an S3 bucket policy that references a cross-account IAM role. Instructions and a policy template are provided in this article.

Account owner and account admin access to the logs depends on how you set up the S3 bucket. Databricks delivers logs to your S3 bucket with AWS’s built-in BucketOwnerFullControl Canned ACL so that account owners and designees can download the logs directly. To support bucket ownership for newly-created objects, you must set your bucket’s S3 Object Ownership setting to the value Bucket owner preferred.

Important

If instead you set your bucket’s S3 Object Ownership setting to Object writer, new objects such as your logs remain owned by the uploading account, which is by default the IAM role you created and specified to access your bucket. This can make it difficult to access the logs, because you cannot access them from the AWS console or automation tools that you authenticated with as the bucket owner.

Databricks recommends that you review Security Best Practices for S3 for guidance around protecting the data in your bucket from unwanted access.

Configuration options

When you configure billable usage log delivery, you have the following options if your account is on the E2 version of the platform:

  • Share the same configuration (log delivery S3 bucket and IAM role) for all workspaces in the account. This is the default.
  • Use separate configurations for each workspace in the account.
  • Use separate configurations for different groups of workspaces, each sharing a configuration.

If your workspaces were not created using the Account API or account console for E2 accounts, you can only have one workspace per account. You must therefore create unique storage and credential configuration objects for each account, but you can reuse an S3 bucket or IAM role between these configuration objects.

Note

Even though you use the Account API to configure log delivery, you can configure log delivery with any workspace, including workspaces that were not created using the Account API.

High-level flow

The high-level flow of billable usage log delivery:

  1. Configure storage: In AWS, create a new AWS S3 bucket. Using Databricks APIs, call the Account API to create a storage configuration object that uses the bucket name.

    Note

    To deliver logs to an AWS account other than the account used for the IAM role that you create for log delivery, you need to add an S3 bucket policy. You do not add the policy in this step, but in a later one.

  2. Configure credentials: In AWS, create the appropriate AWS IAM role. Using Databricks APIs, call the Account API to create a credentials configuration object that uses the IAM role’s ARN. The role policy can specify a path prefix for log delivery within your S3 bucket. You can choose to define an IAM role to include multiple path prefixes if you want log delivery configurations for different workspaces that share the S3 bucket but use different path prefixes.

  3. Optional cross-account support To deliver logs to an AWS account other than the account of the IAM role that you create for log delivery, add an S3 bucket policy. This policy references IDs for the cross-account IAM role that you created in the previous step.

  4. Call the log delivery API: Call the Account API to create a log delivery configuration that uses the credential and storage configuration objects from previous steps. This step lets you specify if you want to associate the log delivery configuration for whole account (current and future workspaces) or for a specific set of workspaces.

  5. Access the CSV files for analysis: The delivery location is <bucket-name>/<prefix>/billable-usage/csv/, where <prefix> is the name of the optional delivery path prefix you set up during log delivery configuration. Files are named workspaceId=<workspace-id>-usageMonth=<month>.csv. Files are delivered daily by overwriting the month’s CSV file for each workspace. You can import this data into Databricks for analysis. There is also a sample notebook that you can use to run a usage analysis dashboard based on these CSV files.

Important

There is a limit on the number of log delivery configurations that you can create for an account. You can create a maximum of two enabled configurations that use the account level (no workspace filter) and two enabled configurations for every specific workspace (a workspaceId can occur in the workspace filter for two configurations). You cannot delete a log delivery configuration, but you can disable it. You can re-enable a disabled configuration, but the request fails if it violates the limits previously described.

Requirements

  • Account owner (or account admin, if you are on an E2 account) email address and password to authenticate with the APIs.

    The email address and password are both case sensitive.

  • Account ID.

    For accounts on the E2 version of the platform, get your account ID from the user profile drop-down in the account console. For non-E2 accounts, get your account ID from the account console’s Usage Overview tab. Contact your Databricks representative if you cannot find your account ID.

How to authenticate to the APIs

The APIs described in this article are published on the accounts.cloud.databricks.com base endpoint for all AWS regional deployments.

Use the following base URL for API requests: https://accounts.cloud.databricks.com/api/2.0/

This REST API requires HTTP basic authentication, which involves setting the HTTP header Authorization. In this article, username refers to your account owner (or account admin, if you are on an E2 account) email address. The email address is case sensitive. There are several ways to provide your credentials to tools such as curl.

  • Pass your username and account password separately in the headers of each request in <username>:<password> syntax.

    For example:

    curl -X GET -u `<username>:<password>` -H "Content-Type: application/json" \
     'https://accounts.cloud.databricks.com/api/2.0/accounts/<account-id>/<endpoint>'
    
  • Apply base64 encoding to your <username>:<password> string and provide it directly in the HTTP header:

    curl -X GET -H "Content-Type: application/json" \
      -H 'Authorization: Basic <base64-username-pw>'
      'https://accounts.cloud.databricks.com/api/2.0/accounts/<account-id>/<endpoint>'
    
  • Create a .netrc file with machine, login, and password properties:

    machine accounts.cloud.databricks.com
    login <username>
    password <password>
    

    To invoke the .netrc file, use -n in your curl command:

    curl -n -X GET 'https://accounts.cloud.databricks.com/api/2.0/accounts/<account-id>/workspaces'
    

    This article’s examples use this authentication style.

For the complete API reference, see Account API.

Step 1: Configure storage

Databricks delivers the billable usage data to an S3 bucket in your account. You can configure multiple workspaces to use a single S3 bucket, or you can define different workspaces (or groups of workspaces) to use different buckets.

This procedure describes how to set up a single configuration object with a common configuration for one or more workspaces in the account. To use different storage locations for different workspaces, repeat the procedures in this article for each workspace or group of workspaces.

  1. Create the S3 bucket, following the instructions in Configure AWS storage.

    Important

    To deliver logs to an AWS account other than the one used for your Databricks workspace, you must add an S3 bucket policy. You do not add the bucket policy in this step. See Step 3: Optional cross-account support.

  2. Create a Databricks storage configuration record that represents your new S3 bucket. Specify your S3 bucket by calling the create new storage configuration API (POST /accounts/<account-id>/storage-configurations).

    Pass the following:

    • storage_configuration_name — New unique storage configuration name.
    • root_bucket_info — A JSON object that contains a bucket_name field that contains your S3 bucket name.

    Copy the storage_configuration_id value returned in the response body. You will use it to create the log delivery configuration in a later step.

    For example:

    curl -X POST -n \
        'https://accounts.cloud.databricks.com/api/2.0/accounts/<databricks-account-id>/storage-configurations' \
      -d '{
        "storage_configuration_name": "databricks-workspace-storageconf-v1",
        "root_bucket_info": {
          "bucket_name": "my-company-example-bucket"
        }
      }'
    

    Response:

    {
      "storage_configuration_id": "<databricks-storage-config-id>",
      "account_id": "<databricks-account-id>",
      "root_bucket_info": {
        "bucket_name": "my-company-example-bucket"
      },
      "storage_configuration_name": "databricks-workspace-storageconf-v1",
      "creation_time": 1579754875555
    }
    

Step 2: Configure credentials

This procedure describes how to set up a single configuration object with a common configuration for one or more workspaces in the account. To use different credentials for different workspaces, repeat the procedures in this article for each workspace or group of workspaces.

Note

To use different S3 bucket names, you need to create separate IAM roles.

  1. Log into your AWS Console as a user with administrator privileges and go to the IAM service.

  2. Click the Roles tab in the sidebar.

  3. Click Create role.

    1. In Select type of trusted entity, click AWS service.

    2. In Common Use Cases, click EC2.

    3. Click the Next: Permissions button.

    4. Click the Next: Tags button.

    5. Click the Next: Review button.

    6. In the Role name field, enter a role name.

      Role name
    7. Click Create role. The list of roles displays.

  4. In the list of roles, click the role you created.

  5. Add an inline policy.

    1. On the Permissions tab, click Add inline policy.

      Inline policy
    2. In the policy editor, click the JSON tab.

      JSON editor
    3. Copy this access policy and modify it. Replace the following values in the policy with your own configuration values:

      • <s3-bucket-name>: The bucket name of your AWS S3 bucket.
      • <s3-bucket-path-prefix>: (Optional) The path to the delivery location in the S3 bucket. If unspecified, the logs are delivered to the root of the bucket. This path must match the delivery_path_prefix argument when you call the log delivery API.
      {
        "Version":"2012-10-17",
        "Statement":[
          {
            "Effect":"Allow",
            "Action":[
              "s3:GetBucketLocation"
            ],
            "Resource":[
              "arn:aws:s3:::<s3-bucket-name>"
            ]
          },
          {
            "Effect":"Allow",
            "Action":[
              "s3:PutObject",
              "s3:GetObject",
              "s3:DeleteObject",
              "s3:PutObjectAcl",
              "s3:AbortMultipartUpload"
            ],
            "Resource":[
              "arn:aws:s3:::<s3-bucket-name>/<s3-bucket-path-prefix>/",
              "arn:aws:s3:::<s3-bucket-name>/<s3-bucket-path-prefix>/*"
            ]
          },
          {
            "Effect":"Allow",
            "Action":[
              "s3:ListBucket",
              "s3:ListMultipartUploadParts",
              "s3:ListBucketMultipartUploads"
            ],
            "Resource":"arn:aws:s3:::<s3-bucket-name>",
            "Condition":{
              "StringLike":{
                "s3:prefix":[
                  "<s3-bucket-path-prefix>",
                  "<s3-bucket-path-prefix>/*"
                ]
              }
            }
          }
        ]
      }
      

      You can customize the policy use of the path prefix:

      • If you do not want to use the bucket path prefix, remove <s3-bucket-path-prefix>/ (including the final slash) from the policy each time it appears.
      • If you want log delivery configurations for different workspaces that share the S3 bucket but use different path prefixes, you can define an IAM role to include multiple path prefixes. There are two separate parts of the policy that reference <s3-bucket-path-prefix>. In each case, duplicate the two adjacent lines that reference the path prefix. Repeat each pair of lines for every new path prefix, for example:
      {
        "Resource":[
          "arn:aws:s3:::<mybucketname>/field-team/",
          "arn:aws:s3:::<mybucketname>/field-team/*",
          "arn:aws:s3:::<mybucketname>/finance-team/",
          "arn:aws:s3:::<mybucketname>/finance-team/*"
        ]
      }
      
    4. Click Review policy.

    5. In the Name field, enter a policy name.

    6. Click Create policy.

    7. If you use service control policies to deny certain actions at the AWS account level, ensure that sts:AssumeRole is whitelisted so Databricks can assume the cross-account role.

  6. On the role summary page, click the Trust Relationships tab.

  7. Paste this access policy into the editor and replace the following values in the policy with your own configuration values:

    <databricks-account-id>: Your Databricks account ID.

    {
      "Version":"2012-10-17",
      "Statement":[
        {
          "Effect":"Allow",
          "Principal":{
            "AWS":"arn:aws:iam::414351767826:role/SaasUsageDeliveryRole-prod-IAMRole-3PLHICCRR1TK"
          },
          "Action":"sts:AssumeRole",
          "Condition":{
            "StringEquals":{
              "sts:ExternalId":[
                "<databricks-account-id>"
              ]
            }
          }
        }
      ]
    }
    
  8. In the role summary, copy the Role ARN and save it for a later step.

    Role ARN
  9. Create a Databricks credentials configuration ID for your AWS role. Call the Create credential configuration API (POST /accounts/<account-id>/credentials). This request establishes cross-account trust and returns a reference ID to use when you create a new workspace.

    Replace <account-id> with your Databricks account ID. In the request body:

    • Set credentials_name to a name that is unique within your account.
    • Set aws_credentials to an object that contains an sts_role property. That object must specify the role_arn for the role you’ve created.

    The response body will include a credentials_id field, which is the Databricks credentials configuration ID that you need to create the new workspace. Copy this field so you can use it to create the log delivery configuration in a later step.

    For example:

     curl -X POST -n \
       'https://accounts.cloud.databricks.com/api/2.0/accounts/<databricks-account-id>/credentials' \
       -d '{
       "credentials_name": "databricks-credentials-v1",
       "aws_credentials": {
         "sts_role": {
           "role_arn": "arn:aws:iam::<aws-account-id>:role/my-company-example-role"
         }
       }
     }'
    

    Example response:

     {
       "credentials_id": "<databricks-credentials-id>",
       "account_id": "<databricks-account-id>",
       "aws_credentials": {
         "sts_role": {
           "role_arn": "arn:aws:iam::<aws-account-id>:role/my-company-example-role",
           "external_id": "<databricks-account-id>"
         }
       },
       "credentials_name": "databricks-credentials-v1",
       "creation_time": 1579753556257
     }
    

    Copy the credentials_id field from the response for later use.

Step 3: Optional cross-account support

If your S3 bucket is in the same AWS account as the IAM role used for log delivery, skip this step.

To deliver logs to an AWS account other than the account used for the IAM role that you create for log delivery, add the S3 bucket policy shown below. This policy references IDs for the cross-account IAM role that you created in the previous step.

  1. In the AWS Console, go to the S3 service.

  2. Click the bucket name.

  3. Click the Permissions tab.

  4. Click the Bucket Policy button.

    Bucket policy button
  5. Copy and modify this bucket policy.

    Replace <s3-bucket-name> with the S3 bucket name. Replace <customer-iam-role-id> with the role ID of your newly-created IAM role. Replace <s3-bucket-path-prefix> with the bucket path prefix you want. See the notes after the policy sample for information about customizing the path prefix.

    {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Effect": "Allow",
              "Principal": {
                  "AWS": ["arn:aws:iam::<customer-iam-role-id>"]
              },
              "Action": "s3:GetBucketLocation",
              "Resource": "arn:aws:s3:::<s3-bucket-name>"
          },
          {
              "Effect": "Allow",
              "Principal": {
                  "AWS": "arn:aws:iam::<customer-iam-role-id>"
              },
              "Action": [
                  "s3:PutObject",
                  "s3:GetObject",
                  "s3:DeleteObject",
                  "s3:PutObjectAcl",
                  "s3:AbortMultipartUpload",
                  "s3:ListMultipartUploadParts"
              ],
              "Resource": [
                  "arn:aws:s3:::<s3-bucket-name>/<s3-bucket-path-prefix>/",
                  "arn:aws:s3:::<s3-bucket-name>/<s3-bucket-path-prefix>/*"
              ]
          },
          {
              "Effect": "Allow",
              "Principal": {
                  "AWS": "arn:aws:iam::<customer-iam-role-id>"
              },
              "Action": "s3:ListBucket",
              "Resource": "arn:aws:s3:::<s3-bucket-name>",
              "Condition": {
                  "StringLike": {
                      "s3:prefix": [
                          "<s3-bucket-path-prefix>",
                          "<s3-bucket-path-prefix>/*"
                      ]
                  }
              }
          }
      ]
    }
    

    You can customize the policy use of the path prefix:

    • If you do not want to use the bucket path prefix, remove <s3-bucket-path-prefix>/ (including the final slash) from the policy each time it appears.

    • If you want log delivery configurations for multiple workspaces that share the same S3 bucket but use different path prefixes, you can define an IAM role to include multiple path prefixes. Two parts of the policy reference <s3-bucket-path-prefix>. In each place, duplicate the two adjacent lines that reference the path prefix. Repeat each pair of lines for each new path prefix. For example:

      {
        "Resource":[
          "arn:aws:s3:::<mybucketname>/field-team/",
          "arn:aws:s3:::<mybucketname>/field-team/*",
          "arn:aws:s3:::<mybucketname>/finance-team/",
          "arn:aws:s3:::<mybucketname>/finance-team/*"
        ]
      }
      

Step 4: Call the log delivery API

To configure log delivery, call the Log delivery configuration API (POST /accounts/<account-id>/log-delivery).

You need the following values that you copied in the previous steps:

  • credentials_id: Your Databricks credential configuration ID, which represents your cross-account role credentials.
  • storage_configuration_id: Your Databricks storage configuration ID, which represents your root S3 bucket.

Also set the following fields:

  • log_type: Always set to BILLABLE_USAGE.

  • output_format: Always set to CSV. For details of the CSV file format, see Download usage as a CSV file.

  • delivery_path_prefix: (Optional) Set to the path prefix. This must match the path prefix that you used in your role policy.

  • workspace_ids_filter: (Optional) By default, this log configuration applies to all workspaces associated with your account ID. For some types of deployments there is only one workspace per account ID so this field is unnecessary. If your account was created originally for workspace creation with the Account API, you may have multiple workspaces associated with your account ID. You can optionally set this field to array of workspace IDs that this configuration applies to. If you plan to use different log delivery configurations for different workspaces, set this explicitly rather than leaving it blank. If you leave this blank and your account ID is associated in the future with additional workspaces, this configuration also applies to the new workspaces. A workspace might apply to more than one log delivery configuration, in which case the logs are written to multiple locations.

    Important

    There is a limit on the number of log delivery configurations that you can create for an account. You can create a maximum of two enabled configurations that use the account level (no workspace filter) and two enabled configurations for every specific workspace (a workspaceId can occur in the workspace filter for two configurations). You cannot delete a log delivery configuration, but you can disable it. You can re-enable a disabled configuration, but the request fails if it violates the limits previously described.

  • delivery_start_time: (Optional) The month and year at which log delivery starts. Defaults to current month. Format is text in YYYY-MM format. You can enter any month and year from 2019-03 on.

For example:

curl -X POST -n \
  'https://accounts.cloud.databricks.com/api/2.0/accounts/<databricks-account-id>/log-delivery' \
  -d '{
  "log_delivery_configuration": {
    "log_type": "BILLABLE_USAGE",
    "config_name": "billable usage config",
    "output_format": "CSV",
    "credentials_id": "<databricks-credentials-id>",
    "storage_configuration_id": "<databricks-storage-config-id>",
    "delivery_path_prefix": "usage-data",
    "delivery_start_time": "2020-06",
    "workspace_ids_filter": [
        6383650456894062,
        4102272838062927
    ]
    }
}'

Example response:

{
    "log_delivery_configuration": {
        "config_id": "<config-id>",
        "config_name": "billable usage config",
        "log_type": "BILLABLE_USAGE",
        "output_format": "CSV",
        "account_id": "<account-id>",
        "credentials_id": "<databricks-credentials-id>",
        "storage_configuration_id": "<databricks-storage-config-id>",
        "workspace_ids_filter": [
            6383650456894062,
            4102272838062927
        ],
        "delivery_path_prefix": "usage-data",
        "delivery_start_time": "2020-06",
        "status": "ENABLED",
        "creation_time": 1591638409000,
        "update_time": 1593108904000
    }
}

Additional features of the log delivery APIs

The log delivery APIs have additional features. See the API reference documentation for details.

Additional operations include:

  • Get all log delivery configurations

  • Get a log delivery configuration by ID

  • Enable or disable a log delivery configuration by ID

    Important

    There is a limit on the number of log delivery configurations that you can create for an account. You can create a maximum of two enabled configurations that use the account level (no workspace filter) and two enabled configurations for every specific workspace (a workspaceId can occur in the workspace filter for two configurations). You cannot delete a log delivery configuration, but you can disable it. You can re-enable a disabled configuration, but the request fails if it violates the limits previously described.

Step 5: Access the log files for analysis

Log files are delivered to <bucket-name>/<prefix>/billable-usage/csv/, where <prefix> is the name of the optional delivery path prefix you set up during log delivery configuration. Files are named workspaceId=<workspace-id>-usageMonth=<month>.csv. Files are delivered daily by overwriting the month’s CSV file for each workspace.

CSV file schema

Column Type Description Example
workspaceId string ID of the workspace. 1234567890123456
timestamp datetime End of the hour for the provided usage. 2019-02-22T09:59:59Z
clusterId string ID of the cluster. 0405-020048-brawl507
clusterName string User-provided name for the cluster. Shared Autoscaling
clusterNodeType string Instance type of the cluster. m4.16xlarge
clusterOwnerUserId string ID of the user who created the cluster. 12345678901234
clusterCustomTags string (“-escaped json) Custom tags associated with the cluster during this hour. "{""dept"":""mktg"",""op_phase"":""dev""}"
sku string Billing SKU. See the Billing SKUs table for a list of values. STANDARD_ALL_PURPOSE_COMPUTE
dbus double Number of DBUs used by the user during this hour. 1.2345
machineHours double Total number of machine hours used by all containers in the cluster. 12.345
clusterOwnerUserName string Username (email) of the user who created the cluster. user@yourcompany.com
tags string (“-escaped json) Default and custom cluster tags, and default and custom instance pool tags (if applicable) associated with the cluster during this hour. See Cluster tags and Pool tags. This is a superset of the clusterCustomTags column. "{""dept"":""mktg"",""op_phase"":""dev"", ""Vendor"":""Databricks"", ""ClusterId"":""0405-020048-brawl507"", ""Creator"":""user@yourcompany.com""}"

Billing SKUs

  • ENTERPRISE_ALL_PURPOSE_COMPUTE
  • ENTERPRISE_JOBS_COMPUTE
  • ENTERPRISE_JOBS_LIGHT_COMPUTE
  • ENTERPRISE_SQL_COMPUTE
  • PREMIUM_ALL_PURPOSE_COMPUTE
  • PREMIUM_JOBS_COMPUTE
  • PREMIUM_JOBS_LIGHT_COMPUTE
  • PREMIUM_SQL_COMPUTE
  • STANDARD_ALL_PURPOSE_COMPUTE
  • STANDARD_JOBS_COMPUTE
  • STANDARD_JOBS_LIGHT_COMPUTE

Usage billed before March 2020 uses the following SKUs:

  • LIGHT_AUTOMATED_NON_OPSEC
  • LIGHT_AUTOMATED_OPSEC
  • STANDARD_AUTOMATED_NON_OPSEC
  • STANDARD_AUTOMATED_OPSEC
  • STANDARD_INTERACTIVE_NON_OPSEC
  • STANDARD_INTERACTIVE_OPSEC

Analyze usage data in Databricks

This section describes how to make the data in the billable usage CSV file available to Databricks for analysis. It describes options for creating a usage table and includes a sample notebook that you can use to run a usage analysis dashboard.

The CSV file uses a format that is standard for commercial spreadsheet applications but requires a modification to be read by Apache Spark. You must use option("escape", "\"") when you create the usage table in Databricks.

Total DBUs are the sum of the dbus column.

Import using the Create Table UI

You can use the Create New Table UI to import the CSV file into Databricks for analysis.

Create Spark DataFrame

You can also use the following code to create the usage table from a path to the CSV file:

df = (spark.
      read.
      option("header", "true").
      option("inferSchema", "true").
      option("escape", "\"").
      csv("/FileStore/tables/usage_data.csv"))

df.createOrReplaceTempView("usage")

If the file is stored in an S3 bucket, for example when it is used with log delivery, the code will look like the following. You can specify a file path or a directory. If you pass a directory, all files are imported. The following example specifies a file.

df = (spark.
      read.
      option("header", "true").
      option("inferSchema", "true").
      option("escape", "\"").
      load("s3://<bucketname>/<pathprefix>/billable-usage/csv/workspaceId=<workspace-id>-usageMonth=<month>.csv"))

df.createOrReplaceTempView("usage")

The following example imports a directory of billable usage files:

df = (spark.
      read.
      option("header", "true").
      option("inferSchema", "true").
      option("escape", "\"").
      load("s3://<bucketname>/<pathprefix>/billable-usage/csv/"))

df.createOrReplaceTempView("usage")

Create a Delta table

To create a Delta table from the DataFrame (df) in the previous example, use the following code:

(df.write
    .format("delta")
    .mode("overwrite")
    .saveAsTable("database_name.table_name")
)

Warning

The saved Delta table is not updated automatically when you add or replace new CSV files. If you need the latest data, re-run these commands before you use the Delta table.

Usage analysis dashboard notebook

If you use billable usage delivery, you can use the following notebook to run a usage analysis dashboard by providing a path to the S3 bucket where your CSV files are stored and entering report parameters in a widget.

The widget that you use to enter report parameters appears above the first notebook cell when you import the notebook to your Databricks workspace. The widget does not appear in the browser-only view of the notebook. Here’s an image of the widget:

usage dashboard widget

Usage analysis dashboard notebook

Open notebook in new tab