Networking

This article introduces networking configurations for the deployment and management of Databricks accounts and workspaces.

Note

Starting October 7, 2024, Databricks will charge customers for networking costs incurred from serverless compute resources connecting to external resources. Over the next few months, serverless network billing will roll out in phases, which might result in gradual billing changes. For more information on billing, see Databricks pricing.

Databricks architecture overview

Databricks operates out of a control plane and a compute plane.

  • The control plane includes the backend services that Databricks manages in your Databricks account. The web application is in the control plane.

  • The compute plane is where your data is processed. There are two types of compute planes depending on the compute that you are using.

    • For classic Databricks compute, the compute resources are in your AWS account in what is called the classic compute plane. This refers to the network in your AWS account and its resources. Classic compute plan resources are in the region that your workspace is in.

    • For serverless compute, the serverless compute resources run in a serverless compute plane in your Databricks account. Serverless compute plan resources are in the same cloud region as your workspace’s classic compute plane. You select this region when creating a workspace.

To learn more about classic compute and serverless compute, see Types of compute. For additional architecture information, see Databricks architecture overview.

Secure network connectivity

Databricks provides a secure networking environment by default, but if your organization has additional needs, you can configure network connectivity features between the different networking connections shown in the diagram below.

Network connectivity overview diagram

  1. Users and applications to Databricks: You can configure features to control access and provide private connectivity between users and their Databricks workspaces. See Users to Databricks networking.

  2. The control plane and the classic compute plane: Classic compute resources, such as clusters, are deployed in are in your AWS account and connect to the control plane. You can use classic network connectivity features to deploy classic compute plane resources in your own virtual private cloud and to enable private connectivity from the clusters to the control plane. See Classic compute plane networking.

  3. The serverless compute plane and storage: You can configure firewalls on your resources to allow access from Databricks serverless compute plane. See Serverless compute plane networking.

You can configure your AWS storage networking features to secure the connection between the classic compute plane and S3. For more information, see Configure Databricks S3 commit service-related settings and Networking recommendations for Lakehouse Federation.

Connectivity between the control plane and the serverless compute plane is always over the cloud network backbone and not the public internet.