Private Service Connect concepts
This page provides a general overview of Private Service Connect at Databricks on Google Cloud. Private Service Connect creates a private, secure connection between your Databricks resources and your Google Cloud services, ensuring your network traffic isn't exposed to the public internet.
Databricks supports the following types of Private Service Connect connectivity on Google Cloud:
- Inbound: Secures connections from users to workspaces
- Outbound (serverless): Secures connections from Databricks serverless compute to resources in your Virtual Private Cloud (VPC)
- Classic compute plane: Secures connections from classic compute to the control plane
Unlike AWS PrivateLink and Azure Private Link, Databricks on Google Cloud does not currently support inbound Private Service Connect for performance-intensive services (Zerobus Ingest, Lakebase Autoscaling). General inbound and classic compute plane connectivity are generally available; serverless outbound connectivity is in Private Preview.
Private connectivity overview
Private Service Connect enables secure, private connectivity from your Google Cloud VPC networks and on-premises networks to Databricks services, keeping your traffic isolated from the public internet. End-to-end private networking helps you meet security and compliance requirements and reduces the risk of data exfiltration.
You can enable inbound and classic compute plane Private Service Connect connections independently or together, depending on your security and compliance requirements. Enabling both privatizes the user and classic compute traffic paths, but it doesn't by itself block public access to the workspace. To block public access, disable public access in your private access settings object. If you instead leave public access enabled, you can use IP access lists to control which public IP addresses can reach the workspace. See Configure Inbound Private Service Connect.
With Private Service Connect, you can:
- Block data access from unauthorized networks or the public internet when using the Databricks web application or APIs.
- Lower the risk of data exfiltration by restricting network exposure to approved private endpoints only.
Terminology
The following Google Cloud and Databricks terms describe Private Service Connect configuration:
Term | Description |
|---|---|
Google Cloud Private Service Connect | A Google Cloud feature that provides private connectivity between VPC networks and Google Cloud services. Databricks uses Private Service Connect for private connections between your VPCs and Databricks services. |
Host project | In Shared VPC setups, the Google Cloud project where VPCs are created. Used for both the classic compute plane VPC and the transit VPC. |
Service project | In Shared VPC setups, the Google Cloud project that contains the workspace compute resources. |
Private Service Connect endpoint (VPC endpoint) | A private connection from a VPC network to a service published by Databricks. Endpoints have internal IP addresses in a consumer VPC network and are accessible to clients in that network. |
Transit VPC | The VPC network that hosts the inbound Private Service Connect endpoint and routes traffic from clients to Databricks. In Google Cloud terms, this is typically the VPC used for inbound Private Service Connect endpoints. |
Classic compute plane VPC | The VPC network that hosts the compute resources of your Databricks workspace. In Google Cloud terms, this is typically the VPC used for classic compute plane Private Service Connect. |
Service attachment | A Google Cloud resource that publishes a service to consumers. Databricks publishes regional service attachments that you connect to from your VPC. |
Private workspace | A workspace where the classic compute plane VMs have no public IP addresses. On Google Cloud, Databricks creates clusters without public IPs by default. |
Choose the right Private Service Connect implementation
Use this guide to determine which implementation fits your needs.
Consideration | Inbound only | Classic compute plane only | Complete private isolation |
|---|---|---|---|
Primary security goal | Only authorized individuals can access my Databricks resources. | Lock down classic compute plane | Maximum isolation (secure everything) |
User connectivity | Private or public | Public (internet) | Private only |
Cluster connectivity to control plane | Public (standard secure path) | Private (required) | Private (required) |
Prerequisites | Enterprise plan, customer-managed VPC | Enterprise plan, customer-managed VPC | Enterprise plan, customer-managed VPC |
Required service attachments | Workspace endpoint ( | Workspace endpoint and SCC relay endpoint ( | Workspace endpoint and SCC relay endpoint |
Relative cost | Cost per endpoint and data transfer | Cost per endpoint and data transfer | Higher cost (all endpoints) |
Outbound (serverless) Private Service Connect is a separate, additive capability that you can layer on any of the configurations above. See Outbound connectivity (serverless).
Inbound connectivity
Inbound Private Service Connect secures the connection from users to the Databricks workspace. Traffic routes through a Private Service Connect endpoint in your transit VPC instead of public IPs. Inbound Private Service Connect provides secure access to:
- Databricks web application
- REST API
- Databricks Connect API
See Configure inbound Private Service Connect.
Outbound connectivity (serverless)
Outbound Private Service Connect enables private connectivity from Databricks serverless compute to resources in your VPC. Unlike inbound and classic compute plane Private Service Connect, which secure connections to Databricks, outbound Private Service Connect secures connections from serverless compute to your resources.
Outbound Private Service Connect uses Network Connectivity Configurations (NCCs), which are account-level regional constructs that manage private endpoint creation at scale. NCCs can be attached to multiple workspaces in the same region.
This feature is in Private Preview. To try it, reach out to your Databricks contact.
Databricks on Google Cloud supports serverless outbound Private Service Connect to resources in your VPC through a customer-managed Google Cloud service attachment fronted by an internal load balancer. See Configure private connectivity to resources in your VPC.
The serverless compute plane on Google Cloud does not support Private Service Connect connectivity to Google-managed services. Serverless compute reaches Google APIs such as Google Cloud Storage and BigQuery through Private Google Access. If VPC Service Controls block that access, use the classic compute plane instead.
Key concepts for outbound connectivity
Key concepts for outbound connectivity
- Network Connectivity Configuration (NCC): An account-level regional construct that manages private endpoints and controls how serverless compute accesses customer resources.
- Private endpoint rule: A rule on an NCC that defines a specific service attachment and the domain names that serverless compute uses to reach it.
- Customer service attachment: A Google Cloud service attachment that you create in your VPC, fronted by an internal load balancer. Databricks creates a Private Service Connect endpoint that connects to this service attachment.
- Workspace attachment model: NCCs can be attached to up to 50 workspaces in the same region.
- Limits and quotas (Private Preview):
- Up to 10 NCCs per account per region
- Up to 10 private endpoint objects per region
- Up to 50 workspaces per NCC
Classic compute plane private connectivity
Classic compute plane Private Service Connect secures the connection from Databricks clusters to the control plane. Clusters connect to the control plane for REST APIs and secure cluster connectivity relay.
Classic compute plane Private Service Connect addresses:
- Compliance requirements: Helps meet strict regulatory and corporate compliance mandates that require all internal cloud traffic to remain on a private network.
- Network perimeter hardening: Implementing classic compute plane Private Service Connect alongside VPC Service Controls for Google APIs reduces data exfiltration risk by removing the path from clusters to the public internet.
See Configure classic compute plane Private Service Connect.
You can set up classic compute plane private connectivity independently. It doesn't require inbound or serverless connectivity.
Virtual private clouds for private connectivity
Private connectivity uses two VPC networks:
- Transit VPC: Functions as a central hub for user connectivity. Contains the inbound Private Service Connect endpoints required for client access to workspaces.
- Classic compute plane VPC: A customer-managed VPC that hosts your Databricks workspace and the classic compute plane Private Service Connect endpoints.
In a single-network topology, one VPC can serve both functions. In that case, all client access from the VPC must go over the inbound Private Service Connect connection. See Enable Private Service Connect for your workspace for details.
You cannot mix workspaces that use inbound Private Service Connect with workspaces that do not within the same transit VPC. All workspaces in a transit VPC must be of the same type. This is a constraint of Google Cloud DNS resolution.
Subnet allocation and sizing
Plan subnets in each VPC to support Private Service Connect endpoints alongside workspace deployment.
- Transit VPC: Allocate a dedicated subnet for inbound Private Service Connect endpoints.
- Classic compute plane VPC:
- Workspace subnet: The Databricks workspace requires its own subnet. See Configure a customer-managed VPC.
- Private Service Connect endpoint subnet: A separate subnet (for example,
/28) for the classic compute plane Private Service Connect endpoints. Endpoint IP ranges cannot overlap with the workspace subnet ranges.
Databricks Private Service Connect endpoints
Databricks publishes two regional service attachments that you connect to from your VPC:
- Workspace endpoint: Service attachment suffix
plproxy-psc-endpoint-all-ports. This endpoint has a dual role. Classic compute plane Private Service Connect uses it to reach the control plane for REST APIs. Inbound Private Service Connect uses it to connect your transit VPC to the workspace web application and REST APIs. - SCC relay endpoint: Service attachment suffix
ngrok-psc-endpoint. Only classic compute plane Private Service Connect uses this endpoint, for the secure cluster connectivity (SCC) relay.
For service attachment URIs by region, see Private Service Connect (PSC) attachment URIs and project numbers.
Within the same region in a VPC, Databricks recommends at most one endpoint per endpoint type. Share one workspace endpoint across all workspaces that use the VPC, and one SCC relay endpoint for all classic compute plane connectivity from that VPC.
For Private Service Connect endpoints, note the following:
- Shared endpoints: Multiple workspaces that use the same VPC can share Private Service Connect endpoints, because they are VPC-level resources.
- Region-specific: Private Service Connect endpoints are region-specific. Workspaces in different regions require separate Private Service Connect endpoint configurations.
- Quota: Up to 10 Private Service Connect endpoints of each type (the workspace endpoint and the SCC relay endpoint) per region, per host project, for up to 20 total. The limit applies per host project (the Google Cloud project where the VPC is created), not per VPC. Contact your account team if this is a constraint.
Key considerations
Before you configure Private Service Connect, keep the following in mind:
- Customer-managed VPC is required: You cannot use Private Service Connect with a Databricks-managed VPC.
- No cross-region connectivity: All Private Service Connect components must be in the same region as your workspace, including the transit VPC, the classic compute plane VPC, the workspace, the Private Service Connect endpoints, and the Private Service Connect endpoint subnets.
- DNS resolution: Inbound Private Service Connect requires private DNS records in your transit VPC. See Configure DNS for Google Cloud inbound Private Service Connect.
- VPC Service Controls integration: You can combine Private Service Connect with VPC Service Controls to further restrict access to Google APIs and Cloud Storage from your Databricks projects. For serverless compute, the NCC exposes stable Google Cloud project numbers that you add to your VPC Service Controls ingress rules. See Private Service Connect (PSC) attachment URIs and project numbers.
- Firewall implications: The classic compute plane VPC cannot reach the public internet by default. If you use a firewall to control egress, you must allow the URLs that Databricks needs, or provide a Cloud NAT or private mirror for package repositories and the metastore. See Limit network egress for your workspace using a firewall.
- Context-based ingress control: When public access remains enabled alongside inbound Private Service Connect, use context-based ingress control to restrict access based on identity, request type, and network source. See Context-based ingress control.
- No performance-intensive service support: Databricks on Google Cloud does not currently support inbound Private Service Connect for performance-intensive services such as Zerobus Ingest or Lakebase Autoscaling.