Skip to main content
Last updated on

Set up serverless SQL warehouses

This page explains how to set up serverless SQL warehouses for your workspace.

note

Serverless SQL warehouses do not have public IP addresses. For more architectural information, see High-level architecture.

Serverless enablement overview​

Serverless is enabled by default in most accounts. Serverless might not be enabled in the following scenarios:

  • Workspace does not meet requirements: The workspace must meet all requirements listed on this page. Ineligible workspaces show serverless as disabled with a tooltip explaining why.
  • Account has a granted postponement: If an account has a granted postponement, serverless is not enabled by default and new workspaces in that account inherit the postponement status. You can request workspace-level enablement without enabling the entire account. Contact your Databricks account team to lift a postponement.
  • Account has not accepted the Terms of Service: Serverless requires acceptance of additional terms in the account console. See Enable serverless compute in your account.

Requirements​

note

Your workspace can use AWS Glue as the workspace legacy metastore. See Use AWS Glue Data Catalog as a metastore (legacy).

Enable serverless compute in your account​

note

If your account was created after March 28, 2022, serverless compute is already enabled for your eligible workspaces. You can verify the status in the feature enablement tab of the account console settings page.

  1. Go to the feature enablement tab of the account console settings page.
  2. A banner at the top of the page prompts you to accept the additional terms. Read the terms, then click Accept. If you don't see the banner, this step has already been completed.

After you accept the terms, your account is enabled for serverless.

Update instance profile for serverless compute (optional)​

If your workspace uses an instance profile created before June 24, 2022 for data access, update the instance profile's trust relationship to enable serverless. See Confirm or set up an AWS instance profile to use with your serverless SQL warehouses.

You don't need to complete this step if you are using Unity Catalog for data governance.

Remove Hive metastore credentials to enable serverless​

You can't deploy serverless SQL warehouses if Hive metastore credentials are defined at the workspace level. To remove legacy Hive metastore credentials:

  1. Click your username in the top bar of the workspace and select Settings from the drop-down.
  2. Click the Compute tab.
  3. Click Manage next to SQL warehouses.
  4. In the Data Access Configuration field, locate and delete the Hive metastore credentials.
  5. Click Save changes.

Monitor usage with tags​

Like any other compute resource in Databricks, you can apply tags to serverless SQL warehouses to monitor usage and cost. You can view usage in the account console or access downloadable logs. See Usage dashboards.

Limitations​

Serverless warehouses have the following limitations:

  • Cluster policies, including spot instance policies, are not supported. See Create and manage compute policies.
  • Customer-managed VPCs are not applicable to compute resources for serverless SQL warehouses. See Configure a customer-managed VPC.
  • Serverless SQL warehouses use private connectivity between the Databricks control plane and the serverless compute plane in nearly all cases. Some legacy us-east-1 region workspaces use an alternative secure network connection.
  • Although the serverless compute plane does not use the secure cluster connectivity relay for the classic compute plane, serverless SQL warehouses do not have public IP addresses.
  • Serverless SQL warehouses don't use customer-managed keys for EBS storage encryption. See Serverless compute and customer-managed keys.