Compute
Databricks compute refers to the selection of computing resources available on Databricks to run your data engineering, data science, and analytics workloads. Choose from serverless compute for on-demand scaling, classic compute for customizable resources, or SQL warehouses for optimized analytics.
You can view and manage compute resources in the Compute section of your workspace:
Serverless compute
On-demand, automatically managed compute that scales based on your workload requirements.
Topic | Description |
---|---|
Interactive Python and SQL execution in notebooks with automatic scaling and no infrastructure management. | |
Run Lakeflow Jobs without without configuring or deploying infrastructure. Automatically provisions and scales compute resources. | |
Run Lakeflow Declarative Pipelines without configuring or deploying infrastructure. Automatically provisions and scales compute resources. | |
Understanding limitations and requirements for serverless workloads and supported configurations. |
Classic compute
Provisioned compute resources that you create, configure, and manage for your workloads.
Topic | Description |
---|---|
Overview of who can access and create classic compute resources. | |
Create and configure compute for interactive data analysis in notebooks or automated workflows with Lakeflow Jobs. | |
Multi-user compute with shared resources for cost-effective collaboration. Lakeguard provides secure user isolation. | |
Compute resource assigned to a single user or group. | |
Pre-configured instances that reduce compute startup time and provide cost savings for frequent workloads. |
SQL warehouses
Optimized compute resources for specific use cases and advanced functionality. SQL warehouses can be configured as serverless or classic.
Topic | Description |
---|---|
Optimized compute for SQL queries, analytics, and business intelligence workloads with serverless or classic options. | |
Understanding the differences between serverless and classic SQL warehouse options to choose the right type for your workloads. |
Additional topics
Topic | Description |
---|---|
High-performance query engine that accelerates SQL workloads and provides faster data processing. | |
Security framework that provides data governance and access control for compute resources. |
For information about working with compute using the command line or APIs, see What is the Databricks CLI? and the Databricks REST API reference.