Skip to main content
Last updated on

Configure DNS for Google Cloud inbound Private Service Connect

Configure DNS to route user requests through your private network when using inbound Private Service Connect for Databricks workspaces. This page covers the DNS resolution model and step-by-step setup instructions for a Cloud DNS private zone in your transit Virtual Private Cloud (VPC).

note

This page covers inbound Private Service Connect DNS configuration. If you also enable classic compute plane Private Service Connect, make sure the workspace URL and data plane URL resolve consistently to the same Private Service Connect endpoint from the VPC that hosts your classic compute resources. See Best practice: consistent DNS resolution for classic compute plane Private Service Connect.

Architecture overview​

Inbound Private Service Connect on Google Cloud routes traffic from your transit VPC to a Private Service Connect endpoint that targets the Databricks workspace service attachment. Your private DNS zone resolves the workspace URL, the data plane URL, and the browser authentication URL to the IP address of the Private Service Connect endpoint in the transit VPC.

You can share one transit VPC across multiple workspaces in the same region, but the transit VPC must contain only workspaces that use inbound Private Service Connect or only workspaces that do not. Google Cloud DNS resolution does not support mixing both types in a single transit VPC.

DNS resolution with Private Service Connect​

Without Private Service Connect, workspace URLs resolve to public IP addresses through the regional Databricks frontend. After you attach a private access settings object to a workspace, Databricks adds an intermediate psc.gcp.databricks.com hostname to the resolution chain. Override this hostname in your private DNS zone to point to the private IP of your Private Service Connect endpoint.

Your private DNS zone must resolve three records to the private IP of your Private Service Connect endpoint:

  • The workspace URL (<workspace-id>.<n>.gcp.databricks.com), which browsers and clients in the transit VPC use to reach the workspace.
  • The data plane URL (dp-<workspace-id>.<n>.gcp.databricks.com), which classic compute and some control plane services use.
  • The browser authentication URL (<region>.psc-auth.gcp.databricks.com), which web browser logins require. REST API-only access doesn't need this record.

In these URLs, <n> is a single-digit number from 0 to 9 that Databricks assigns to your workspace, not a region code. Use the exact value from your workspace URL, which you can find in the browser address bar when you view the workspace or in the account console workspace list.

For workspaces that use either classic compute plane or inbound Private Service Connect, the intermediate Private Service Connect hostname is <region>.psc.gcp.databricks.com. This hostname separates Private Service Connect-enabled workspace traffic from other Databricks services that don't support Private Service Connect, such as the account console.

Prerequisites​

Before you configure DNS, verify that you have:

  • A Databricks workspace deployed with inbound Private Service Connect. See Configure Inbound Private Service Connect.
  • A Private Service Connect endpoint in your transit VPC connected to the plproxy-psc-endpoint-all-ports service attachment for your region. Note the private IP address of the endpoint.
  • The workspace URL for your deployed workspace. Get it from the browser address bar in the workspace UI or from the workspace list in the account console.
  • Google Cloud permissions to create Cloud DNS zones and records in the project that hosts the transit VPC.

Configure a Cloud DNS private zone​

Create a Cloud DNS private zone for the gcp.databricks.com domain and add A records that resolve each Databricks URL to the private IP of your Private Service Connect endpoint.

Step 1: Create a Cloud DNS private zone​

  1. In the Google Cloud console, go to Cloud DNS zones.
  2. Click Create zone.
  3. For Zone type, select Private.
  4. For DNS name, enter gcp.databricks.com.
  5. For Networks, select your transit VPC network.
  6. Click Create.

Step 2: Add A records for the workspace URLs​

Add three A records in the new zone, all pointing to the private IP of your inbound Private Service Connect endpoint. The following examples assume the endpoint IP is 10.0.0.2 and the workspace ID is 33333333333333.3.

Record name

Record type

Value

<workspace-id>.<n>.gcp.databricks.com

A

<PL endpoint IP>

dp-<workspace-id>.<n>.gcp.databricks.com

A

<PL endpoint IP>

<region>.psc-auth.gcp.databricks.com

A

<PL endpoint IP>

Record name

Record type

Value

<workspace-id>.<n>.gcp.databricks.com

A

<PL endpoint IP>

dp-<workspace-id>.<n>.gcp.databricks.com

A

<PL endpoint IP>

<region>.psc-auth.gcp.databricks.com

A

<PL endpoint IP>

For a workspace at 33333333333333.3.gcp.databricks.com in the us-east4 region, the records are:

  • 33333333333333.3.gcp.databricks.com → 10.0.0.2
  • dp-33333333333333.3.gcp.databricks.com → 10.0.0.2
  • us-east4.psc-auth.gcp.databricks.com → 10.0.0.2
note

If you only need inbound Private Service Connect from the transit network for REST API access (no browser-based logins), you can omit the psc-auth.gcp.databricks.com record. Most deployments require it.

Step 3: Verify the records in the console​

In the Cloud DNS zone view, confirm that the three A records appear and resolve to the same Private Service Connect endpoint IP.

Verification​

After you create the DNS records, verify that workspace URLs resolve to the Private Service Connect endpoint IP from a client in the transit VPC.

From a VM in the transit VPC, run nslookup against each URL:

Bash
$ nslookup 33333333333333.3.gcp.databricks.com
Address: 10.0.0.2

$ nslookup dp-33333333333333.3.gcp.databricks.com
Address: 10.0.0.2

$ nslookup us-east4.psc-auth.gcp.databricks.com
Address: 10.0.0.2

Each command must return the private IP of your inbound Private Service Connect endpoint.

Common issues​

DNS resolution returns public IPs: The Cloud DNS private zone isn't associated with the transit VPC, or queries aren't reaching it. Verify the zone's network bindings.

Browser login redirects fail or hang: The <region>.psc-auth.gcp.databricks.com record is missing or points to a different IP than the workspace URL. Add or update the record.

Cluster startup fails after enabling classic compute plane Private Service Connect: The workspace URL and the data plane URL must resolve to the same Private Service Connect endpoint from within the classic compute plane VPC. If they resolve to different endpoints, cluster nodes can't reach the control plane consistently. See Best practice: consistent DNS resolution.

Best practice: consistent DNS resolution for classic compute plane Private Service Connect​

If you configure both inbound and classic compute plane Private Service Connect, the workspace URL and the data plane URL (dp-<workspace-id>.<n>.gcp.databricks.com) must resolve to the same Private Service Connect endpoint from within each VPC that uses them.

When a compute node starts, Databricks uses both URLs to reach the control plane. If they resolve to different endpoints, this mismatch can conflict with your private access settings policy and cause compute failures.

To verify, run the following from a VM in the classic compute plane VPC. Both commands must return the same private IP:

Bash
nslookup <workspace-url>
nslookup dp-<workspace-id>.<n>.gcp.databricks.com

If they don't match, update the Cloud DNS zone in the classic compute plane VPC (or the transit VPC if it doubles as the classic compute plane VPC) so both records point to the same Private Service Connect endpoint.

What's next​

On this page