Enable serverless SQL warehouses

This article explains how to enable serverless SQL warehouses for your workspace.

Note

Serverless SQL warehouses do not have public IP addresses. For more architectural information, see Databricks architecture overview.

Requirements

Your Databricks account must not be on a free trial.
Your Databricks workspace must be on the Premium plan or above.
Your workspace control plane and serverless compute plane must be in a region that supports Databricks SQL Serverless. See Databricks clouds and regions.
Your workspace must not use S3 access policies.
Your workspace must not use an external Hive legacy metastore. See Remove Hive metastore credentials to enable serverless.

Note

Your workspace can use AWS Glue as the workspace legacy metastore. See Use AWS Glue Data Catalog as a metastore (legacy).

Enable serverless compute in your account

Note

If your account was created after March 28, 2022, serverless compute is enabled by default for your workspaces.

Go to the feature enablement tab of the account console settings page.
A banner at the top of the page prompts you to accept the additional terms. Once you read the terms, click Accept. If you do not see the banner asking you to accept the terms, this step has been completed already.

After you’ve accepted the terms, your account is enabled for serverless.

Update instance profile for serverless compute (optional)

If your workspace uses an instance profile created before June 24, 2022 for data access, you may need to update the instance profile’s trust relationship to enable serverless. See Confirm or set up an AWS instance profile to use with your serverless SQL warehouses.

You do not need to complete this step if you are using Unity Catalog for data governance.

Remove Hive metastore credentials to enable serverless

You cannot deploy serverless SQL warehouses if Hive metastore credentials are defined at the workspace level. To remove legacy Hive metastore credentials:

Click your username in the top bar of the workspace and select Settings from the drop-down.
Click the Compute tab.
Click Manage next to SQL warehouses.
In the Data Access Configuration field, locate and delete the Hive metastore credentials.
Click Save changes.

Monitor usage with tags

Like any other compute resource in Databricks, you can apply tags to serverless SQL warehouses to monitor usage and cost. You can view usage in the account console or access downloadable logs. See Monitor usage in the account console.

Limitations

Serverless warehouses have the following limitations:

Cluster policies, including spot instance policies are unsupported. See Create and manage compute policies.
Customer-managed VPCs are not applicable to compute resources for serverless SQL warehouses. See Configure a customer-managed VPC.
Serverless SQL warehouses use private connectivity between the Databricks control plane and the serverless compute plane in nearly all cases. Some legacy us-east-1 region workspaces use an alternative secure network connection.
Although the serverless compute plane does not use the secure cluster connectivity relay for the classic compute plane, serverless SQL warehouses do not have public IP addresses.
Serverless SQL warehouses do not use customer-managed keys for EBS storage encryption. See Serverless compute and customer-managed keys.