Choose a performance mode for serverless jobs and pipelines

Serverless jobs and pipelines run in one of the following performance modes:

• Performance-optimized mode (default): Databricks keeps a pool of warm compute ready, so workloads typically start in less than a minute and get more compute quickly when they need it. Use this mode for interactive and time-sensitive work, where a person or a downstream process is waiting on the result.
• Standard mode: Workloads typically start within 4 to 6 minutes and consume fewer Databricks units (DBUs). Use this mode for scheduled and batch work that can wait a few minutes to start in exchange for lower cost.

You choose the mode for each job and each pipeline schedule. To find the workloads where standard mode would save the most, see Find jobs and pipelines to switch to standard mode.

This page compares the modes, explains how to choose and set a mode, and shows how to check which mode a run used and what it cost.

note

Standard mode is a performance setting for serverless compute. It's unrelated to standard access mode for classic compute.

Compare the performance modes

The following table compares the two performance modes:

	Performance-optimized mode (default)	Standard mode
Startup time	Typically less than a minute. Databricks keeps warm compute ready for the workload.	Typically 4 to 6 minutes, depending on compute availability.
Run time	Shorter for workloads that scale up, because compute is added from warm capacity.	Can be longer for workloads that scale up, because compute is added more slowly.
Cost	Consumes more DBUs for the same workload, in exchange for fast and predictable starts.	Consumes fewer DBUs for the same workload, at the same price per DBU and on the same SKU.
Best for	Interactive and time-sensitive work: runs that someone is waiting on, development and testing, jobs with a service-level agreement or freshness target, event-triggered and frequent jobs, and short jobs.	Scheduled and batch work that can absorb a few minutes of startup latency, such as nightly batch jobs, backfills, and one-time loads.

	Performance-optimized mode (default)	Standard mode
Startup time	Typically less than a minute. Databricks keeps warm compute ready for the workload.	Typically 4 to 6 minutes, depending on compute availability.
Run time	Shorter for workloads that scale up, because compute is added from warm capacity.	Can be longer for workloads that scale up, because compute is added more slowly.
Cost	Consumes more DBUs for the same workload, in exchange for fast and predictable starts.	Consumes fewer DBUs for the same workload, at the same price per DBU and on the same SKU.
Best for	Interactive and time-sensitive work: runs that someone is waiting on, development and testing, jobs with a service-level agreement or freshness target, event-triggered and frequent jobs, and short jobs.	Scheduled and batch work that can absorb a few minutes of startup latency, such as nightly batch jobs, backfills, and one-time loads.

Performance-optimized mode is the default for serverless jobs and pipelines, whether you create them in the UI, with the API, with Declarative Automation Bundles, with Terraform, or with the SDKs. Refreshes of materialized views and streaming tables that run on a schedule from the table definition have their own setting, which defaults to standard mode. See Select a performance mode for scheduled refreshes.

Both modes run your workload on the same serverless runtime and produce the same results. For pricing, see the Lakeflow Jobs pricing page.

Choose a mode for your workload

Performance-optimized mode is the default. To decide whether to switch a workload to standard mode, ask whether a person or a downstream process is waiting on the run. If one is, keep performance-optimized mode. If the workload can start a few minutes later without affecting anyone, standard mode costs less.

Use performance-optimized mode for the following workloads:

• Interactive work, where someone waits for the result, such as ad hoc runs with Run now and runs that a user starts from an app or via an API call.
• Development, testing, and validation of jobs and pipelines, where fast feedback matters.
• Workloads with a service-level agreement or a data freshness target, or with downstream consumers such as dashboards, alerts, and other jobs that need results quickly.
• Jobs that a file arrival trigger or a table update trigger starts, where the goal is to process new data promptly.
• Jobs that run every few minutes, where slower starts could delay or queue the next run.
• Short jobs, where a few minutes of startup time would be a large share of the total run time.
• Workloads whose compute needs grow during the run, which finish sooner when compute is added quickly.

Standard mode is a good fit for the following workloads:

• Nightly, hourly, and other scheduled batch jobs whose results aren't needed right away.
• Pipelines that refresh tables on a schedule, when consumers don't need the refresh to finish quickly.
• Backfills and one-time data loads.
• Jobs that an external orchestrator triggers on a schedule, when nothing downstream is waiting on them.

The performance mode setting doesn't apply to interactive serverless notebooks.

The performance mode is a job-level setting. If a job mixes time-sensitive tasks with tasks that can wait, split the tasks into separate jobs. To orchestrate the jobs together, use a Run Job task. The job that a Run Job task runs uses its own performance mode.

Try standard mode on an existing job

To measure the effect of standard mode on a job before you commit to it, do the following:

1. Switch the job to standard mode. See Switch a job to standard mode.
2. Let the job run on its normal schedule for several runs.
3. Compare the DBUs per run and the run duration with runs from before the switch. See Measure the cost of each mode.
4. If the job no longer meets its latency requirements, switch it back to performance-optimized mode. The change applies to the next run.

Use the job's scheduled runs for the comparison. Extra manual runs of a production job can write duplicate data.

Support for performance mode by workload type

You can directly configure performance mode for the following types of serverless compute workloads:

• Serverless notebook, Python script, Python wheel, JAR, and dbt tasks, including notebooks that a task runs with dbutils.notebook.run().
• Serverless pipelines run from a pipeline task.
• Refreshes of materialized views and streaming tables that a SQL task starts, or that a notebook or dbt task on a SQL warehouse starts. The mode doesn't change the SQL warehouse.

The following workloads run in performance-optimized mode:

• Triggered pipeline updates that you start manually. The exception is a selective full refresh, which can run in standard mode. See Switch a pipeline to standard mode.
• Continuous pipelines that use the pipeline's built-in continuous mode. To choose the mode, run the pipeline with a continuous job. See Switch a pipeline to standard mode.

For the following workloads, you set the performance mode somewhere other than the job settings:

• One-time runs that you create with the Jobs API runs/submit operation. To set the performance mode, set performance_target in the request body. If you omit the parameter, the run uses performance-optimized mode.
• Materialized views and streaming tables that refresh on a schedule from the table definition. To set the performance mode, update the refresh schedule in Catalog Explorer. These refreshes use standard mode by default. Setting the mode for these refreshes is in Beta. To use it, a workspace admin must turn on the System-Managed Job for Materialized Views & Streaming Tables preview on the Previews page. See Manage Databricks previews and Select a performance mode for scheduled refreshes.

The performance mode doesn't apply to interactive serverless notebooks, tasks that run on classic compute, SQL warehouses, or tasks that run on AI Runtime.

Switch a job to standard mode

note

• To change the performance mode of a job, you need the CAN MANAGE permission on the job.
• To set the mode in Declarative Automation Bundles, you need Databricks CLI version 0.241.0 or above.

You set the performance mode for each job. The mode applies to all serverless tasks in the job, and a change applies to runs that start after you save it.

UI

To switch a job to standard mode in the Jobs UI, do the following:

1. In your workspace, click Jobs & Pipelines in the sidebar.
2. Click the name of the job.
3. In the Job details pane, turn off Performance optimized.

Performance optimized shows as off in the Job details pane, and runs that start after the change use standard mode.

To switch back to performance-optimized mode, turn on Performance optimized. You can change the setting only when the job has at least one task that the mode applies to. See Support for performance mode by workload type.

You can also choose the mode for a single run:

• Run now with different settings: Runs the job one time in the mode that you choose, without changing the job. See Run a job with different settings.
• Repair run: By default, a repair uses the same mode as the run that it repairs. You can choose a different mode for the repair. See Re-run failed and skipped tasks.

When you schedule a notebook from the notebook, choose the mode in the schedule dialog. See Create and manage scheduled notebook jobs.

Declarative Automation Bundles

To use standard mode for a job in a bundle, set performance_target on the job resource. The following example uses standard mode, and uses performance-optimized mode in the dev target for faster feedback during development:

YAML

resources:
  jobs:
    nightly_etl:
      name: nightly_etl
      performance_target: STANDARD
      tasks:
        - task_key: transform
          notebook_task:
            notebook_path: ../src/transform.ipynb

targets:
  dev:
    resources:
      jobs:
        nightly_etl:
          performance_target: PERFORMANCE_OPTIMIZED

If you omit performance_target, the job uses performance-optimized mode. For more about job resources and target overrides, see job and Override with target settings.

REST API

To switch an existing job to standard mode, call Update job settings with performance_target in new_settings. The following example assumes that the DATABRICKS_HOST and DATABRICKS_TOKEN environment variables are set, as described in Perform personal access token authentication:

Bash

curl -X POST "$DATABRICKS_HOST/api/2.2/jobs/update" \
  -H "Authorization: Bearer $DATABRICKS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "job_id": 123456789,
    "new_settings": {
      "performance_target": "STANDARD"
    }
  }'

The following list describes how performance_target works in other requests:

• Create a job: Sets the mode for the new job. If you omit the field, the job uses PERFORMANCE_OPTIMIZED.
• Reset a job: Replaces all job settings. To keep standard mode, include performance_target in the new settings.
• Run now and Repair a run: Sets the mode for that run only.
• Submit a one-time run: Sets the mode for the run. If you omit the field, the run uses PERFORMANCE_OPTIMIZED.

Databricks CLI

To switch an existing job to standard mode, run databricks jobs update with the job ID in the JSON request body:

Bash

databricks jobs update --json '{"job_id": 123456789, "new_settings": {"performance_target": "STANDARD"}}'

To run a job one time in standard mode without changing the job, run the following:

Bash

databricks jobs run-now 123456789 --performance-target STANDARD --no-wait

Python SDK

To switch one or more existing jobs to standard mode with the Databricks SDK for Python, run the following:

Python

from databricks.sdk import WorkspaceClient
from databricks.sdk.service import jobs

w = WorkspaceClient()

# Replace with the IDs of the jobs to switch to standard mode.
job_ids = [123456789, 234567890]

for job_id in job_ids:
    w.jobs.update(
        job_id=job_id,
        new_settings=jobs.JobSettings(performance_target=jobs.PerformanceTarget.STANDARD),
    )

Terraform

To use standard mode for a job that you manage with the Databricks Terraform provider, set performance_target on the databricks_job resource:

Hcl

resource "databricks_job" "nightly_etl" {
  name               = "nightly_etl"
  performance_target = "STANDARD"

  task {
    task_key = "transform"
    notebook_task {
      notebook_path = "/Workspace/Users/someone@example.com/transform"
    }
  }
}

If a bundle or Terraform deploys a job, change the mode in the configuration. The next deployment overwrites changes that you make in the UI or with the API. Jobs that a bundle deploys are read-only in the Jobs UI.

Switch a pipeline to standard mode

A serverless pipeline uses the performance mode of the job that runs it. When you schedule a pipeline from the pipeline page, the schedule creates a job, so you choose the mode on the schedule.

To create a pipeline schedule that uses standard mode, do the following:

1. In your workspace, click Jobs & Pipelines in the sidebar.
2. Click the name of the pipeline.
3. Click Schedule. If the pipeline already has a schedule, click Add schedule.
4. Configure the trigger and the job name. See Schedule a pipeline with the pipeline UI.
5. Clear the Performance optimized checkbox.
6. Click Create.

The schedule creates a job that runs the pipeline in standard mode. To confirm, open that job and check that Performance optimized is off in the Job details pane.

To change the mode of an existing schedule, or of any other job that runs the pipeline, switch that job to standard mode. See Switch a job to standard mode.

Triggered pipeline updates that you start manually, for example with Start on the pipeline page, run in performance-optimized mode. A selective full refresh can run in standard mode. After you select the tables to refresh, click Run now with different settings, turn on Run pipeline with full table refresh, and turn off Performance optimized.

A continuous pipeline that uses the pipeline's built-in continuous mode runs in performance-optimized mode, and you can't change its mode. To run a continuous pipeline in standard mode, run it with a continuous job and turn off Performance optimized for that job. A continuous job picks up a new mode when its run restarts. To apply a change right away, click Restart run. See Run a continuous pipeline with a continuous job and Run jobs continuously.

For materialized views and streaming tables that refresh on a schedule from the table definition, set the mode on the refresh schedule in Catalog Explorer. See Select a performance mode for scheduled refreshes.

Check which mode a run used

The mode that a run uses can differ from the job's setting. For example, you can choose a different mode for a single run, and the mode doesn't apply to some task types. To check which mode a run used, use one of the following methods:

• Jobs UI: Open the run. On the Job run details page, the details pane shows Performance optimized as Enabled for performance-optimized mode or Disabled for standard mode. The Task run details page shows the same field for each task.
• Jobs API: The effective_performance_target field in the Get a single job run response shows the mode that the run used.
• System tables: The product_features.performance_target column in the billable usage system table records the mode for serverless jobs and pipelines. For pipeline updates that a job triggers, the trigger_details.job_task.performance_target column in the pipeline update timeline table also records the mode.

Measure the cost of each mode

Use the billable usage system table to find workloads to switch and to measure what standard mode saves. Usage records can take several hours to appear in the table. The queries in this section use list prices, so the costs don't reflect discounts.

Find jobs and pipelines to switch to standard mode

The following query lists the serverless jobs and pipelines that used performance-optimized mode in the last 30 days, sorted by list cost. Workloads near the top of the results that can tolerate a few minutes of startup latency are good candidates for standard mode.

SQL

SELECT
  u.workspace_id,
  u.billing_origin_product AS product,
  u.usage_metadata.job_id AS job_id,
  u.usage_metadata.dlt_pipeline_id AS pipeline_id,
  MAX(u.usage_metadata.job_name) AS job_name,
  COUNT(DISTINCT u.usage_metadata.job_run_id) AS job_runs,
  SUM(u.usage_quantity) AS dbus,
  SUM(u.usage_quantity * lp.pricing.default) AS list_cost
FROM system.billing.usage AS u
INNER JOIN system.billing.list_prices AS lp
  ON u.cloud = lp.cloud
  AND u.sku_name = lp.sku_name
  AND u.usage_start_time >= lp.price_start_time
  AND (u.usage_end_time <= lp.price_end_time OR lp.price_end_time IS NULL)
WHERE
  u.billing_origin_product IN ('JOBS', 'DLT')
  AND u.product_features.is_serverless
  AND u.product_features.performance_target = 'PERFORMANCE_OPTIMIZED'
  AND u.usage_unit = 'DBU'
  AND u.usage_date >= DATEADD(day, -30, current_date())
GROUP BY
  u.workspace_id,
  u.billing_origin_product,
  u.usage_metadata.job_id,
  u.usage_metadata.dlt_pipeline_id
ORDER BY list_cost DESC
LIMIT 50

To switch a pipeline that a job runs, switch the job. See Switch a job to standard mode.

Compare a job's usage before and after you switch

After a job has run in both modes, the following query compares the DBUs that its tasks consume per run in each mode. Replace <job-id> with the ID of the job:

SQL

SELECT
  u.product_features.performance_target AS performance_mode,
  COUNT(DISTINCT u.usage_metadata.job_run_id) AS runs,
  SUM(u.usage_quantity) AS dbus,
  SUM(u.usage_quantity) / COUNT(DISTINCT u.usage_metadata.job_run_id) AS dbus_per_run
FROM system.billing.usage AS u
WHERE
  u.billing_origin_product = 'JOBS'
  AND u.product_features.is_serverless
  AND u.usage_unit = 'DBU'
  AND u.usage_metadata.job_id = '<job-id>'
  AND u.usage_date >= DATEADD(day, -30, current_date())
GROUP BY ALL

The query counts only usage that's billed to the job (JOBS). Pipeline tasks are billed as pipeline usage (DLT), so the results don't include them. To compare a pipeline that a pipeline task runs, filter on u.billing_origin_product = 'DLT' and u.usage_metadata.dlt_pipeline_id = '<pipeline-id>' instead, and count u.usage_metadata.dlt_update_id instead of u.usage_metadata.job_run_id.

DBU consumption per run also depends on how much data each run processes, so compare runs that process similar amounts of data. To compare run durations, open the job and view its runs. See Monitor Lakeflow Jobs.

To report DBU consumption by mode across all jobs and pipelines, see Report on DBUs consumed by performance mode.

Frequently asked questions (FAQ) about serverless performance modes

Does standard mode have a different price?

No. Both modes bill under the same SKU at the same price per DBU. Standard mode costs less because it consumes fewer DBUs.

Can I set a default mode for all jobs in a workspace?

No. You set the mode for each job and each pipeline schedule. To switch many jobs, use the Jobs API, the Databricks SDK for Python, Declarative Automation Bundles, or Terraform. See Switch a job to standard mode.

Can tasks in the same job use different modes?

No. The mode applies to the whole job. To run some tasks in each mode, put them in separate jobs, and use a Run Job task to orchestrate them.

When does a mode change take effect?

A change applies to runs that start after you save it. A run that's in progress keeps the mode that it started with. For a continuous job, click Restart run to apply the change right away.

Why did a run use a different mode than its job?

The run might have started with a mode for that run only, for example from Run now with different settings, from a repair, or with performance_target in a run now request. To check the mode that a run used, see Check which mode a run used.

Why does a job use standard mode when I didn't choose it?

Until September 2025, standard mode was the default for new serverless jobs and pipeline schedules that you created in the UI. Jobs that the UI created in standard mode before the change keep that mode until you change it. See the September 2025 release notes. To switch a job to performance-optimized mode, see Switch a job to standard mode.

Why does my job take longer than 6 minutes to start in standard mode?

Startup time in standard mode depends on compute availability and varies from run to run. Four to six minutes is typical, not a maximum. If a job needs a predictable start time, use performance-optimized mode.

Can I use standard mode for jobs that an external tool triggers?

Yes. Set the mode on the job, and have the tool trigger the job with run now. If the tool submits one-time runs with runs/submit, the runs use performance-optimized mode unless the request sets performance_target.

Additional resources

• Run your Lakeflow Jobs with serverless compute for workflows
• Configure a serverless pipeline
• Best practices for serverless compute
• Monitor the cost of serverless compute
• Lakeflow Jobs pricing