Skip to main content
Last updated on

Configure private connectivity to resources in your VPC

note

There are currently no networking charges for serverless features. In a later release, you might be charged. Databricks will provide advance notice for networking pricing changes.

See Understand Databricks networking costs.

Preview

This feature is in Private Preview. To try it, reach out to your Databricks contact.

This page explains how to use the Databricks account console to configure Private Service Connect connections from serverless compute to resources in your VPC through a customer-managed Google Cloud service attachment.

Configuring private connectivity for serverless compute provides:

  • A dedicated and private connection: Your private endpoint is exclusively tied to your Databricks account, so access to your VPC resources is restricted to authorized workspaces only. This creates a secure, dedicated communication channel.
  • Enhanced data exfiltration mitigation: Databricks serverless with Unity Catalog offers built-in data exfiltration protection. Private Service Connect adds a network-layer defense. Placing your VPC resources in a private subnet and controlling access through dedicated private endpoints reduces the risk of unauthorized data movement outside your controlled network.

Requirements​

  • Your account and workspace must be on the Enterprise plan.
  • You are the account admin of your Databricks account.
  • You have at least one workspace using serverless compute. For supported regions, see Serverless availability.
  • The serverless_gcp_pl preview must be enabled on your account. To request access, contact your Databricks account team. Allowlisting can take up to seven days.
  • Your Google Cloud service attachment is fronted by an internal load balancer in your VPC, in the same region as your NCC.
  • Each Databricks account can have up to 10 NCCs per region.
  • Each region can have up to 10 private endpoints, distributed across one or more NCCs.
  • Each NCC can be attached to up to 50 workspaces.
  • Each private endpoint rule supports up to 10 domain names.
  • Private Service Connect from serverless compute applies to all serverless clusters in workspaces attached to the NCC.
tip

Share an NCC across workspaces that are in the same region and belong to the same business unit or environment, so they reuse the same private connectivity configuration.

Configure private connectivity in Google Cloud​

Enable the preview​

After your account is allowlisted (see Requirements), enable the preview named "Outbound (serverless) GCP Private Link to customer-managed resources" on your workspace. See Manage workspace-level previews.

Create a Google Cloud service attachment for your resource​

Create a service attachment in your VPC that publishes your resource (database, internal service, application) through an internal load balancer.

  1. Create a regional internal TCP load balancer that targets your resource. For instructions, see the Google Cloud internal load balancer documentation.
  2. Publish the load balancer as a service attachment. For instructions, see Publish a service in the Google Cloud documentation.
  3. Note the full service attachment URI in the format projects/<project-id>/regions/<region>/serviceAttachments/<name>. You need this URI when you create the private endpoint rule.

The service attachment must be in the same region as the NCC you create in the next step.

Create or use an existing NCC​

The NCC defines the private connectivity settings for your workspaces. Skip this step if a regional NCC already exists.

  1. As an account admin, go to the account console.
  2. In the sidebar, click Security.
  3. Click the Networking tab.
  4. Under Network Connectivity Configurations, click Add Network Connectivity Configuration.
  5. Enter a name for the NCC.
  6. Choose the region. This must match your workspace region and your service attachment region.
  7. Click Add.

Create a private endpoint rule​

This step creates a Databricks-managed Private Service Connect endpoint that connects to your service attachment.

  1. In the account console, click Security, then Networking.
  2. Select the NCC you created in the previous step.
  3. On the Private endpoint rules tab, click Add private endpoint rule.
  4. In the GCP service attachment URI field, paste the full URI from the service attachment you created.
  5. In the Domain names field, add the domain names that serverless compute uses to reach your resource. These domains must resolve directly to the backend resources. Domains that use private-use TLDs (such as .internal, .localhost, or .test) are not supported.
  6. Click Add.
  7. Confirm that the Status column for your new rule is CREATING, then refreshes to PENDING once the endpoint is provisioned.

Get the PSC endpoint URI​

Wait a few minutes for the endpoint to provision, then capture the PSC endpoint URI from the rule. You need this URI to approve the endpoint on Google Cloud in the next step.

Refresh the Private endpoint rules tab. When the rule shows the endpoint name, click it to view the PSC endpoint URI field. The URI has the format https://www.googleapis.com/compute/v1/projects/<databricks-project>/regions/<region>/forwardingRules/<id>.

Approve the endpoint on Google Cloud​

This step happens in your Google Cloud account, not in Databricks. You add the PSC endpoint URI from the previous step to the consumer accept list on your service attachment.

  1. In the Google Cloud console, go to Private Service Connect.
  2. Select the service attachment you created earlier.
  3. In the Consumers section, approve the pending connection that matches the PSC endpoint URI.

Confirm the endpoint status​

Verify the connection from the Databricks side.

  1. Refresh the Private endpoint rules tab in the account console.
  2. Confirm that the Status column for your rule is ESTABLISHED.

This usually happens within a few minutes of the Google Cloud approval. If the status doesn't update, refresh the page.

Attach the NCC to one or more workspaces​

Skip this step if your workspace is already attached to the NCC.

  1. In the account console, click Workspaces in the left navigation.
  2. Select a workspace.
  3. Click Update Workspace.
  4. Under Network Connectivity Configuration, select the NCC from the dropdown.
  5. Repeat for each workspace that should use this NCC.
note

NCCs are regional objects. You can only attach an NCC to workspaces in the same region.

Verify connectivity​

After serverless compute restarts, test that it can reach your resource through the private endpoint. The simplest check is to open a TCP connection to your load balancer port from a notebook.

In a notebook attached to a serverless cluster in an NCC-attached workspace, run:

Bash
%sh nc -vz <domain-name> <port>

For example, nc -vz staging-database.example.com 3306 for a MySQL load balancer. A successful connection confirms that the private endpoint is routing traffic to your VPC resource.

Limitations​

  • The Google Cloud private endpoint rule is not available in the Databricks Terraform provider. You can create the NCC and the workspace binding with Terraform, but you must create the private endpoint rule in the account console.
  • Domain names with private-use TLDs (.internal, .localhost, .test) are not supported.

Additional resources​

On this page