Compute selection recommendations
This page explains how to choose the right compute type for your workload. Depending on your permissions, you might have the option to choose or create a number of different types of compute resources.
For guidance on configuring classic compute resources, see Compute configuration recommendations.
Choose compute for your interactive notebooks
Compute type | When to use |
---|---|
Serverless compute | Generally recommended. Offers faster start-up, automatic scaling, and lower cost. |
Serverless SQL warehouse | For SQL-based analytics and reporting. Optimized for SQL query performance. |
Classic all-purpose compute | When you need RDD APIs, GPU instances, or R language support. |
For more on notebook compute, see Notebook compute resources.
Choose compute for your jobs
When configuring a job, select a compute resource for each task.
Compute type | When to use |
---|---|
Serverless compute | Recommended choice for most automated workloads. Fewer settings to manage, faster start-up, automatic scaling, and lower cost. Choose either performance optimized or standard performance mode based on your latency and cost needs. |
SQL warehouse | For SQL tasks. Use a warehouse type that fits your latency and cost needs. |
Classic jobs compute | For non-SQL jobs that require custom cluster settings not available in serverless. |
Classic all-purpose compute | Generally avoid. Not optimized for automated workloads. |
To view a table of recommended compute types for each task type, see What is the recommended compute for each task?.
Choose compute for pipelines
When configuring a pipeline, either use serverless compute or a classic pipeline.
Compute type | When to use |
---|---|
Serverless compute | Recommended choice for most automated workloads. Fewer settings to manage, faster start-up, automatic scaling, and lower cost. Choose either performance optimized or standard performance mode based on your latency and cost needs. |
Classic pipeline compute | When you access to a feature not supported in serverless or when you are working with the legacy Hive metastore. |
Choose a SQL warehouse type for your SQL workloads
There are three SQL warehouse types you can choose from:
Warehouse type | When to use |
---|---|
Serverless SQL warehouse | Recommended for BI, ETL, and exploratory analysis. Startup typically 2–6 seconds and scales rapidly. |
Pro SQL warehouse | Use when serverless is not available or you require custom networking (for example, federation or hybrid). |
Classic SQL warehouse | Entry-level performance. Use for basic interactive exploration when serverless or pro are not options. |
For more considerations on choosing a SQL warehouse type, see SQL warehouse types.