Skip to main content

Configure DNS for Google Cloud inbound Private Service Connect

Beta

Private access to account-level resources is in Beta. Enable Context-Based Ingress: Account Policy to get started.

Configure DNS to route user requests through your private network when using inbound Private Service Connect for Databricks workspaces and account-level resources. This page covers the DNS resolution model and step-by-step setup instructions for a Cloud DNS private zone in your transit Virtual Private Cloud (VPC).

note

This page covers inbound Private Service Connect DNS configuration. If you also enable classic compute plane Private Service Connect, make sure the workspace URL and data plane URL resolve consistently to the same Private Service Connect endpoint from the VPC that hosts your classic compute resources. See Best practice: consistent DNS resolution for classic compute plane Private Service Connect.

Architecture overview​

Inbound Private Service Connect on Google Cloud routes traffic from your transit VPC to a Private Service Connect endpoint that targets the Databricks workspace service attachment. Your private DNS zone resolves the workspace URL, the data plane URL, and the browser authentication URL to the IP address of the Private Service Connect endpoint in the transit VPC.

You can share one transit VPC across multiple workspaces in the same region, but the transit VPC must contain only workspaces that use inbound Private Service Connect or only workspaces that do not. Google Cloud DNS resolution does not support mixing both types in a single transit VPC.

DNS resolution with Private Service Connect​

Without Private Service Connect, workspace URLs resolve to public IP addresses through the regional Databricks frontend. After you attach a private access settings object to a workspace, Databricks adds an intermediate <region>.psc.gcp.databricks.com hostname to the resolution chain. Override this hostname in your private DNS zone to point to the private IP of your Private Service Connect endpoint.

Your private DNS zone must resolve three records to the private IP of your Private Service Connect endpoint:

  • The workspace URL (<workspace-id>.<n>.gcp.databricks.com), which browsers and clients in the transit VPC use to reach the workspace.
  • The data plane URL (dp-<workspace-id>.<n>.gcp.databricks.com), which classic compute and some control plane services use.
  • The browser authentication URL (<region>.psc-auth.gcp.databricks.com), which web browser logins require. REST API-only access doesn't need this record.

In these URLs, <n> is a single-digit number from 0 to 9 that Databricks assigns to your workspace, not a region code. Use the exact value from your workspace URL, which you can find in the browser address bar when you view the workspace or in the account console workspace list.

For workspaces that use either classic compute plane or inbound Private Service Connect, the intermediate Private Service Connect hostname is <region>.psc.gcp.databricks.com. This hostname separates Private Service Connect-enabled workspace traffic from other Databricks services.

Custom URLs (<my-custom-account-name>.databricks.com) and accounts.gcp.databricks.com do not include the psc hostname in their DNS resolution chain and must be resolved separately. See Add records for account-level resources and custom URLs.

Prerequisites​

Before you configure DNS, verify that you have:

  • A Databricks workspace deployed with inbound Private Service Connect, or a General Access endpoint registered for account-level resources. See Configure inbound Private Service Connect for workspaces and Configure inbound Private Service Connect for account-level resources.
  • A Private Service Connect endpoint in your transit VPC connected to the plproxy-psc-endpoint-all-ports service attachment for your region. Note the private IP address of the endpoint.
  • The workspace URL for your deployed workspace. Get it from the browser address bar in the workspace UI or from the workspace list in the account console.
  • Google Cloud permissions to create Cloud DNS zones and records in the project that hosts the transit VPC.

Configure a Cloud DNS private zone​

Create a Cloud DNS private zone for the gcp.databricks.com domain and add A records that resolve each Databricks URL to the private IP of your Private Service Connect endpoint.

Create a Cloud DNS private zone​

  1. In the Google Cloud console, go to Cloud DNS zones.
  2. Click Create zone.
  3. For Zone type, select Private.
  4. For DNS name, enter gcp.databricks.com.
  5. For Networks, select your transit VPC network.
  6. Click Create.

Add A records for the workspace URLs​

Add three A records in the new zone, all pointing to the private IP of your inbound Private Service Connect endpoint. The following examples assume the endpoint IP is 10.0.0.2 and the workspace ID is 33333333333333.3.

Record name

Record type

Value

<workspace-id>.<n>.gcp.databricks.com

A

<PL endpoint IP>

dp-<workspace-id>.<n>.gcp.databricks.com

A

<PL endpoint IP>

<region>.psc-auth.gcp.databricks.com

A

<PL endpoint IP>

Record name

Record type

Value

<workspace-id>.<n>.gcp.databricks.com

A

<PL endpoint IP>

dp-<workspace-id>.<n>.gcp.databricks.com

A

<PL endpoint IP>

<region>.psc-auth.gcp.databricks.com

A

<PL endpoint IP>

For a workspace at 33333333333333.3.gcp.databricks.com in the us-east4 region, the records are:

  • 33333333333333.3.gcp.databricks.com → 10.0.0.2
  • dp-33333333333333.3.gcp.databricks.com → 10.0.0.2
  • us-east4.psc-auth.gcp.databricks.com → 10.0.0.2
note

If you only need inbound Private Service Connect from the transit network for REST API access (no browser-based logins), you can omit the psc-auth.gcp.databricks.com record. Most deployments require it.

Add records for account-level resources and custom URLs​

Add the following records to route account-level resources and your custom URL to a registered General Access endpoint.

  • accounts.gcp.databricks.com: Add an A record for accounts.gcp.databricks.com to the gcp.databricks.com zone that points to the private IP of a registered General Access endpoint. This record is required, because the gcp.databricks.com private zone overrides public DNS resolution for every name in the domain. It also enables private unified login. See Configure inbound Private Service Connect with unified login.

  • Custom URL: Required if you use custom URLs. Create a separate Cloud DNS private zone for your custom URL in your transit VPC, and add an A record that points to the private IP of a registered General Access endpoint. Your custom URL can serve the UI and API of all of your workspaces in any region, and account-level resources.

    Configuration

    Value

    Zone type

    Private

    DNS name

    <my-custom-account-name>.databricks.com

    Record name

    Leave blank

    Record type

    A

    Record value

    <PL endpoint IP>

    Configuration

    Value

    Zone type

    Private

    DNS name

    <my-custom-account-name>.databricks.com

    Record name

    Leave blank

    Record type

    A

    Record value

    <PL endpoint IP>

Verify the records in the console​

In the Cloud DNS zone views, confirm that the A records appear and resolve to the expected Private Service Connect endpoint IPs.

Example configurations​

The following examples illustrate common routing patterns. You can choose any routing model that fits your security, compliance, or networking requirements.

note

All workspaces in a region share <region>.psc-auth.gcp.databricks.com, so each Cloud DNS private zone can route it to only one endpoint. Browser login uses this URL only to return the Google sign-in response to the workspace URL, so its endpoint doesn't need to match the endpoint of the workspace URL, and you don't need to allow it in workspace-level policies. Route it to any endpoint that your users can reach.

Example 1: Unified endpoint for production, non-production, and account-level resources (single-endpoint approach)​

In this example, all workspaces and account-level resources are accessed through a single General Access endpoint.

Configure the following routing and context-based ingress policies:

Cloud DNS record

Routed to endpoint

Context-based ingress configuration

Custom URL (<my-custom-account-name>.databricks.com zone)

Registered endpoint 1

In the account-level policy, add an allow rule for endpoint 1 to access account-level resources.

Workspace records (workspace URL, data plane URL, and <region>.psc-auth.gcp.databricks.com) for all workspaces

Registered endpoint 1

Create a workspace-level policy that allows ingress from endpoint 1, and attach it to all workspaces.

Cloud DNS record

Routed to endpoint

Context-based ingress configuration

Custom URL (<my-custom-account-name>.databricks.com zone)

Registered endpoint 1

In the account-level policy, add an allow rule for endpoint 1 to access account-level resources.

Workspace records (workspace URL, data plane URL, and <region>.psc-auth.gcp.databricks.com) for all workspaces

Registered endpoint 1

Create a workspace-level policy that allows ingress from endpoint 1, and attach it to all workspaces.

With this configuration, users can access over inbound Private Service Connect:

  • Account-level resources, such as the account console and account-level Genie One, using your custom URL.
  • All workspaces using your custom URL.
  • All workspaces using workspace-specific URLs.

Example 2: Separate endpoints for production, non-production, and account-level resources (multi-endpoint approach)​

In this example, production workspaces, non-production workspaces, and account-level resources are accessed through separate isolated endpoints.

Configure the following routing and context-based ingress policies:

Cloud DNS record

Routed to endpoint

Context-based ingress configuration

Custom URL (<my-custom-account-name>.databricks.com zone)

Registered endpoint 1

In the account-level policy, add an allow rule for endpoint 1 to access account-level resources.

Workspace URL and data plane URL records for production workspaces

Registered endpoint 2

Create a workspace-level policy that allows ingress from endpoint 2, and attach it to your production workspaces.

Workspace URL and data plane URL records for non-production workspaces

Registered endpoint 3

Create a workspace-level policy that allows ingress from endpoint 3, and attach it to your non-production workspaces.

<region>.psc-auth.gcp.databricks.com for each region

Registered endpoint 2 or 3

None. See the note at the start of this section.

Cloud DNS record

Routed to endpoint

Context-based ingress configuration

Custom URL (<my-custom-account-name>.databricks.com zone)

Registered endpoint 1

In the account-level policy, add an allow rule for endpoint 1 to access account-level resources.

Workspace URL and data plane URL records for production workspaces

Registered endpoint 2

Create a workspace-level policy that allows ingress from endpoint 2, and attach it to your production workspaces.

Workspace URL and data plane URL records for non-production workspaces

Registered endpoint 3

Create a workspace-level policy that allows ingress from endpoint 3, and attach it to your non-production workspaces.

<region>.psc-auth.gcp.databricks.com for each region

Registered endpoint 2 or 3

None. See the note at the start of this section.

With this configuration, users can access over inbound Private Service Connect:

  • Account-level resources, such as the account console and account-level Genie One, using your custom URL.
  • Production workspaces using production workspace-specific URLs.
  • Non-production workspaces using non-production workspace-specific URLs.

Example 3: Separate endpoints for workspaces in different regions, and account-level resources (multi-endpoint approach)​

In this example, workspaces in region A, workspaces in region B, and account-level resources are accessed through separate isolated endpoints.

Configure the following routing and context-based ingress policies:

Cloud DNS record

Routed to endpoint

Context-based ingress configuration

Custom URL (<my-custom-account-name>.databricks.com zone)

Registered endpoint 1

In the account-level policy, add an allow rule for endpoint 1 to access account-level resources.

Workspace records for workspaces in region A, including <region A>.psc-auth.gcp.databricks.com

Registered endpoint 2

Create a workspace-level policy that allows ingress from endpoint 2, and attach it to your workspaces in region A.

Workspace records for workspaces in region B, including <region B>.psc-auth.gcp.databricks.com

Registered endpoint 3

Create a workspace-level policy that allows ingress from endpoint 3, and attach it to your workspaces in region B.

Cloud DNS record

Routed to endpoint

Context-based ingress configuration

Custom URL (<my-custom-account-name>.databricks.com zone)

Registered endpoint 1

In the account-level policy, add an allow rule for endpoint 1 to access account-level resources.

Workspace records for workspaces in region A, including <region A>.psc-auth.gcp.databricks.com

Registered endpoint 2

Create a workspace-level policy that allows ingress from endpoint 2, and attach it to your workspaces in region A.

Workspace records for workspaces in region B, including <region B>.psc-auth.gcp.databricks.com

Registered endpoint 3

Create a workspace-level policy that allows ingress from endpoint 3, and attach it to your workspaces in region B.

With this configuration, users can access over inbound Private Service Connect:

  • Account-level resources, such as the account console and account-level Genie One, using your custom URL.
  • Workspaces in region A using workspace-specific URLs.
  • Workspaces in region B using workspace-specific URLs.

Example 4: Shared endpoint for production and account-level resources, separate endpoint for non-production (multi-endpoint approach)​

In this example, production workspaces and account-level resources share one isolated endpoint, while non-production workspaces use a separate endpoint.

Configure the following routing and context-based ingress policies:

Cloud DNS record

Routed to endpoint

Context-based ingress configuration

Custom URL (<my-custom-account-name>.databricks.com zone)

Registered endpoint 1

In the account-level policy, add an allow rule for endpoint 1 to access account-level resources.

Workspace URL and data plane URL records for production workspaces

Registered endpoint 1

Also create a workspace-level policy that allows ingress from endpoint 1, and attach it to your production workspaces.

Workspace URL and data plane URL records for non-production workspaces

Registered endpoint 2

Create a workspace-level policy that allows ingress from endpoint 2, and attach it to your non-production workspaces.

<region>.psc-auth.gcp.databricks.com for each region

Registered endpoint 1 or 2

None. See the note at the start of this section.

Cloud DNS record

Routed to endpoint

Context-based ingress configuration

Custom URL (<my-custom-account-name>.databricks.com zone)

Registered endpoint 1

In the account-level policy, add an allow rule for endpoint 1 to access account-level resources.

Workspace URL and data plane URL records for production workspaces

Registered endpoint 1

Also create a workspace-level policy that allows ingress from endpoint 1, and attach it to your production workspaces.

Workspace URL and data plane URL records for non-production workspaces

Registered endpoint 2

Create a workspace-level policy that allows ingress from endpoint 2, and attach it to your non-production workspaces.

<region>.psc-auth.gcp.databricks.com for each region

Registered endpoint 1 or 2

None. See the note at the start of this section.

With this configuration, users can access over inbound Private Service Connect:

  • Account-level resources, such as the account console and account-level Genie One, using your custom URL.
  • Production workspaces using your custom URL.
  • Production workspaces using production workspace-specific URLs.
  • Non-production workspaces using non-production workspace-specific URLs.

Production and account-level traffic remain isolated from non-production workspace traffic by using separate endpoints.

In all of the above configurations, also add the required accounts.gcp.databricks.com record to the gcp.databricks.com zone (see Add records for account-level resources and custom URLs). You can resolve it to any registered General Access endpoint. If you want to access account-level resources privately through accounts.gcp.databricks.com without using your custom URL, configure the following routing and context-based ingress policy:

Cloud DNS record

Routed to endpoint

Context-based ingress configuration

accounts.gcp.databricks.com (in the gcp.databricks.com zone)

Registered endpoint 1

In the account-level policy, add an allow rule for endpoint 1 to access account-level resources.

Cloud DNS record

Routed to endpoint

Context-based ingress configuration

accounts.gcp.databricks.com (in the gcp.databricks.com zone)

Registered endpoint 1

In the account-level policy, add an allow rule for endpoint 1 to access account-level resources.

tip

Keep the following guidelines in mind:

  • Your custom URL, workspace-specific URLs, and the account URL can all route to the same endpoint, or to separate endpoints based on your isolation requirements.
  • Your custom URL can be used to access both workspaces and account-level resources.
  • Use context-based ingress to control which workspaces and account-level resources each registered endpoint can access. See Context-based ingress control.

Verification​

After you create the DNS records, verify that workspace URLs resolve to the Private Service Connect endpoint IP from a client in the transit VPC.

From a VM in the transit VPC, run nslookup against each URL:

Bash
$ nslookup 33333333333333.3.gcp.databricks.com
Address: 10.0.0.2

$ nslookup dp-33333333333333.3.gcp.databricks.com
Address: 10.0.0.2

$ nslookup us-east4.psc-auth.gcp.databricks.com
Address: 10.0.0.2

$ nslookup accounts.gcp.databricks.com
Address: 10.0.0.2

$ nslookup <my-custom-account-name>.databricks.com
Address: 10.0.0.2

Each command must return the private IP of the Private Service Connect endpoint you routed that URL to.

Common issues​

DNS resolution returns public IPs: The Cloud DNS private zone isn't associated with the transit VPC, or queries aren't reaching it. Verify the zone's network bindings.

Account console or custom URL doesn't load from the transit VPC: The accounts.gcp.databricks.com record or the custom URL zone is missing, or the endpoint isn't allowed in the account-level context-based ingress policy. See Add records for account-level resources and custom URLs and Manage account context-based ingress policies.

Browser login redirects fail or hang: The <region>.psc-auth.gcp.databricks.com record is missing, or points to an endpoint that the browser can't reach. Add or update the record.

Cluster startup fails after enabling classic compute plane Private Service Connect: The workspace URL and the data plane URL must resolve to the same Private Service Connect endpoint from within the classic compute plane VPC. If they resolve to different endpoints, cluster nodes can't reach the control plane consistently. See Best practice: consistent DNS resolution.

Best practice: consistent DNS resolution for classic compute plane Private Service Connect​

If you configure both inbound and classic compute plane Private Service Connect, the workspace URL and the data plane URL (dp-<workspace-id>.<n>.gcp.databricks.com) must resolve to the same Private Service Connect endpoint from within each VPC that uses them.

When a compute node starts, Databricks uses both URLs to reach the control plane. If they resolve to different endpoints, this mismatch can conflict with your private access settings policy and cause compute failures.

To verify, run the following from a VM in the classic compute plane VPC. Both commands must return the same private IP:

Bash
nslookup <workspace-url>
nslookup dp-<workspace-id>.<n>.gcp.databricks.com

If they don't match, update the Cloud DNS zone in the classic compute plane VPC (or the transit VPC if it doubles as the classic compute plane VPC) so both records point to the same Private Service Connect endpoint.

What's next​