Skip to main content
Last updated on

Databricks REST API

This page describes the Databricks REST API, how to call it, and some best practices.

For complete reference for the Databricks REST API, see Databricks REST API reference.

note

With the exception of advanced scenarios, Databricks recommends using the Databricks SDKs or the Databricks CLI instead of the Databricks REST API to programmatically manage Databricks objects.

Workspace vs Account REST APIs​

Databricks provides two sets of REST APIs. Workspace APIs manage resources inside a single workspace, such as clusters, jobs, notebooks, and Unity Catalog objects, and you call them using your workspace URL as the host. Account APIs manage account-wide resources, such as user and group provisioning, workspace creation, network and billing configuration, and account-level Unity Catalog settings, and you call them using your account console login URL and account ID.

For the operations available in each set, see the workspace API reference and the account API reference.

Call a REST API​

A Databricks REST API call includes the following components:

  • Depending on whether it is a workspace or account endpoint, either:
  • The REST API operation type, such as GET, POST, PATCH, or DELETE.
  • The REST API operation path, such as /api/2.0/clusters/get.
  • Databricks authentication information, such as a Databricks OAuth token.
  • Any request payload or request query parameters that are supported by the REST API operation, such as a cluster's ID.

For information about how to structure a REST API request and how to parse response payloads for your preferred developer tool, see your provider's documentation.

Example 1: Get clusters​

The following example calls the Cluster, List endpoint to return a list of available clusters. It assumes that the DATABRICKS_HOST environment variable is set to your Databricks workspace URL and DATABRICKS_TOKEN is set to a Databricks token.

Bash
curl -X GET "$DATABRICKS_HOST/api/2.0/clusters/list" \
  -H "Authorization: Bearer $DATABRICKS_TOKEN"
Python
import requests
import os

headers = {"Authorization": f"Bearer {os.getenv('DATABRICKS_TOKEN')}"}
response = requests.get(f"{os.getenv('DATABRICKS_HOST')}/api/2.0/clusters/list", headers=headers)

print(response.json())

Example 2: Run a job​

The following example calls the Job, Run Now endpoint to trigger a dry run of an existing job. It assumes that the DATABRICKS_HOST environment variable is set to your Databricks workspace URL and DATABRICKS_TOKEN is set to a Databricks token.

Bash
curl -X POST "$DATABRICKS_HOST/api/2.1/jobs/run-now" \
  -H "Authorization: Bearer $DATABRICKS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "job_id": 45678,
    "notebook_params": {
      "dry_run": "true",
      "start_date": "2026-08-27"
    }
  }'
Python
import requests
import os

url = f"{os.getenv('DATABRICKS_HOST')}/api/2.1/jobs/run-now"
headers = {
    "Authorization": f"Bearer {os.getenv('DATABRICKS_TOKEN')}",
    "Content-Type": "application/json"
}
payload = {
    "job_id": 45678,
    "notebook_params": {"dry_run": "true", "start_date": "2026-08-27"}
}

response = requests.post(url, headers=headers, json=payload)
print(f"Run ID: {response.json().get('run_id')}")

Example 3: Return account users​

The following example calls the Account User, List endpoint to return users in the Databricks account identified by <account_id>:

Bash
curl -X GET '<databricks-account-login-url>/api/2.0/identity/accounts/<account_id>/users' \
  --header "Authorization: Bearer $OAUTH_TOKEN"
Python
import requests
import os

url = "<databricks-account-login-url>/api/2.0/identity/accounts/<account_id>/users"
headers = {"Authorization": f"Bearer {os.getenv('OAUTH_TOKEN')}"}

response = requests.get(url, headers=headers)
print(response.json())

Best practices​

The following sections describe some performance best practices as the data in your workspace grows.

Paginate LIST API responses​

LIST APIs return results in pages instead of a single large response. To retrieve a complete result set, request the first page, then use the token in the response to request each subsequent page until no token is returned.

To page through a complete result set:

  • Set max_results=0 in your request. This lets the server choose an appropriate page size, which is more efficient than requesting a fixed number of results per page.
  • Read the next_page_token field from each response. To request the next page, pass its value in the page_token query parameter of your next request.
  • Repeat until a response omits next_page_token or returns it as an empty value. That response is the last page.
  • Don't include page_token in the first request. Add it only for follow-up requests.

The following example uses this pattern to retrieve every table in a schema from the Unity Catalog Table, List endpoint. The same loop works for any LIST API. Only the endpoint and the name of the array field in the response change. For example, the Grants endpoint returns results in a privilege_assignments array instead of tables.

Python
import requests

base_url = "https://example.cloud.databricks.com"  # No trailing slash
bearer_token = "<your-personal-access-token>"
catalog_name = "main"
schema_name = "default"


def list_tables(base_url, bearer_token, catalog_name, schema_name):
    endpoint = f"{base_url}/api/2.1/unity-catalog/tables"
    headers = {"Authorization": f"Bearer {bearer_token}"}
    params = {
        "catalog_name": catalog_name,
        "schema_name": schema_name,
        "max_results": 0,  # Let the server choose the page size.
    }

    tables = []
    while True:
        response = requests.get(endpoint, headers=headers, params=params)
        response.raise_for_status()
        body = response.json()

        tables.extend(body.get("tables", []))

        # Stop when the response no longer includes a page token.
        page_token = body.get("next_page_token")
        if not page_token:
            break
        params["page_token"] = page_token

    return tables

Handle 429 rate-limit responses​

Databricks enforces rate limits on REST API calls to keep workspaces responsive under heavy load. Limits are applied per endpoint and per workspace to support fair usage and availability. A request that exceeds the rate limit returns an HTTP 429 Too Many Requests response.

Handle 429 responses gracefully by retrying with exponential backoff and jitter:

  • Exponential backoff: After a 429, wait before retrying, and double the wait time after each subsequent 429. Set a maximum wait time and a maximum number of retries so that a request doesn't retry indefinitely.
  • Jitter: Add a small random amount of time to each wait. Jitter spreads out retries from multiple clients so that they don't all retry at the same moment and cause repeated bursts of traffic.
  • If a response includes a Retry-After header, wait at least that long before retrying.

Most HTTP client libraries can apply this retry behavior for you. For background on the algorithm, see Exponential backoff and jitter.

For the rate limits that apply to specific APIs, see the API rate limits in API rate limits.

Trim response fields for performance​

Some LIST APIs return fields that are expensive to compute or that make responses large. When you don't need these fields, use the request parameters that omit them to reduce response size and improve latency.

For example, the Unity Catalog Tables API supports the following parameters:

  • omit_properties=true: Omits the properties field from each table in the response.
  • omit_columns=true: Omits the columns field from each table in the response.

If you're listing tables only to retrieve their names, setting both parameters returns a smaller response and lists tables faster. Check the REST API reference for the field-trimming parameters that each endpoint supports.

Additional resources​

On this page