Update workspace network configuration

Preview

This feature is in Public Preview.

Update or reconfigure an existing workspace for Private Service Connect

This page provides step-by-step instructions for updating the network configuration of an existing Databricks workspace to use Private Service Connect, change an existing workspace's front-end or back-end Private Service Connect settings, or modify its underlying network configuration, like expanding a subnet's IP range.

Why update your network configuration?

Updating your workspace network configuration provides the following enhanced security and connectivity capabilities:

• Enhanced security: Use Private Service Connect to establish private connections between your VPC and Databricks services.
• Flexible networking: Modify subnet IP ranges and network configurations to meet your organization's requirements.
• Service integration: Connect your workspace to other Google Cloud services using private endpoints.
• Custom connectivity: Configure front-end and back-end Private Service Connect settings for your specific use case.

Before you begin

• To prevent disruption and ensure a safe update, terminate all running clusters and jobs in the workspace. You can restart them after the update is complete.
• You must have the necessary permissions on the GCP project that hosts the VPC used by the workspace. These permissions include compute.firewalls.get, compute.networks.get, compute.projects.get, compute.subnetworks.get, resourcemanager.projects.get, which allow you to make changes to the network configuration. For information on these permissions, see Required permissions for workspace creation.

Change the subnet for an existing workspace

To change the IP range used by your workspace, you must create a new subnet with the desired range, create a new network configuration object, and then update your workspace to use this new network configuration.

Step 1: Create cloud resources

Create the necessary cloud resources in your Google Cloud project and Databricks account:

1. Create a new subnet: Create a subnet in your VPC with the new CIDR range you intend to use in your Google Cloud project.
2. Create a network configuration object: Create the network configuration object against the subnet in your Databricks account. See Step 1: Create a subnet.

Step 2: Configure the firewall

Create a new ingress firewall rule for the new subnet. You must complete this step before you update the workspace to use the new CIDR range.

In the Google Cloud console, create a firewall rule with the following values:

Field	Value
Rule name	db-<subnet-name>-ingress
Direction	Ingress
Target	Network tag databricks-<workspace-id>
Priority	1000
Source filter	IPv4 range: the primary CIDR range of the new subnet
Protocols and ports	Allow all

Replace <subnet-name> with the name of the new subnet and <workspace-id> with your workspace ID.

For more information about firewall configuration, see Limit network egress for your workspace using a firewall.

Step 3: Grant the workspace service account the Databricks Network Role v2

Before you update the workspace, you must grant the workspace service account the Databricks Network Role v2 on the new subnet. If you skip this step, all VM launches fail due to permission issues.

To find your workspace service account, check the IAM grants on the old subnet. The service account has the format db-<workspace-id>@prod-gcp-<geo-region>.iam.gserviceaccount.com.

1. Get the existing IAM policy from the current subnet:

   Bash

   gcloud compute networks subnets get-iam-policy <current-subnet> \
     --region=<region> \
     --project=<project>

   The command returns a policy JSON like the following:

   JSON

   {
     "bindings": [
       {
         "role": "projects/<project>/roles/<databricks-network-role-v2>",
         "members": ["serviceAccount:db-<workspace-id>@prod-gcp-<geo-region>.iam.gserviceaccount.com"]
       }
     ]
   }

2. Use the policy binding spec as a template. Create a new JSON file (for example, policy.json) with the same role and member, targeting the new subnet.

3. Apply the policy to the new subnet:

   Bash

   gcloud compute networks subnets set-iam-policy <new-subnet> policy.json \
     --region=<region>

Step 4: Update the workspace using the API

Update the workspace using the Databricks Accounts REST API:

1. Authentication: Authenticate to the Accounts API. See Authorize access to Databricks resources. On Google Cloud, you can also use Authenticate with Google ID tokens.

2. Execute the REST API:

   • Endpoint: PATCH /api/2.0/accounts/{account_id}/workspaces/{workspace_id}
   • Path Parameters:
     • account_id (required, uuid)
     • workspace_id (required, int64)
   • Query Parameters:
     • update_mask (required, string)

3. Request example:

   Use the network_id of the new network configuration object you created in Step 1.

   HTTP

   PATCH /api/2.0/accounts/{account_id}/workspaces/{workspace_id}?update_mask=network_id
   Content-Type: application/json
   
   {
       "network_id": "fd0cc5bc-683c-47e9-b15e-144d7744a496"
   }

4. Response example:

   JSON

   {
     "workspace_id": 3389895405782117,
     "workspace_name": "example-ws",
     "creation_time": 1722542958272,
     "deployment_name": "3389895405782117.7",
     "workspace_status": "PROVISIONING",
     "account_id": "2e0b6f3b-e1dc-43ea-93f5-7d55b9d7b780",
     "workspace_status_message": "Workspace resources are being set up.",
     "network_id": "fd0cc5bc-683c-47e9-b15e-144d7744a496",
     "pricing_tier": "ENTERPRISE",
     "location": "us-east4",
     "cloud": "gcp",
     "identity_federation_info": {
       "enable_identity_federation": true
     },
     "gke_config": {
       "connectivity_type": "PRIVATE_NODE_PUBLIC_MASTER",
       "master_ip_range": "10.103.0.0/28"
     },
     "cloud_resource_container": {
       "gcp": {
         "project_id": "databricks-dev-entd-shared"
       }
     }
   }

Step 5: Restart clusters and jobs

After the workspace update is complete, wait at least 10 minutes for the changes to propagate before you restart your clusters and jobs.

Migrate a non-Private Service Connect workspace or change the Private Service Connect configuration of an existing workspace

Complete these steps if you are migrating a non-Private Service Connect workspace or changing the Private Service Connect configuration of an existing one.

Step 1: Create or update cloud resources

Create or update the necessary cloud resources:

1. VPC endpoints: See Step 2: Create VPC endpoints.
2. Private access settings: See Create private access settings.
3. Network configuration: See Step 6: Create a network configuration.

You need the IDs for these resources in Step 2.

Step 2: Update the workspace using the API

To apply the changes, send a PATCH request to the Accounts API.

• Authentication: Authenticate to the Accounts API. See Authorize access to Databricks resources. On Google Cloud, you can also use Authenticate with Google ID tokens.
• Endpoint: PATCH /api/2.0/accounts/{account_id}/workspaces/{workspace_id}
• Query Parameter: You must include the update_mask query parameter to specify which fields to update.
  • Example: update_mask=network_id,private_access_settings_id,workspace_name,custom_tags
• Request Body: Include the fields you want to update in the JSON body. Supported fields include network_id, private_access_settings_id, workspace_name, and custom_tags.

The following example request updates a workspace's network configuration, private access settings, name, and custom tags:

HTTP

PATCH /api/2.0/accounts/{account_id}/workspaces/{workspace_id}?update_mask=network_id,private_access_settings_id,workspace_name,custom_tags
Content-Type: application/json

{
  "network_id": "6f855d93-8c45-4a4c-a0c9-56e35e16d715",
  "private_access_settings_id": "be123b84-c964-4285-8b03-b7fe961588dc",
  "workspace_name": "updated-ws-name",
  "custom_tags": {
    "test_key": "test_value"
  }
}

The response shows a PROVISIONING status because the workspace runs updates asynchronously. The workspace returns to RUNNING after all updates are complete.

JSON

{
  "workspace_id": 3389895405782117,
  "workspace_name": "example-ws",
  "creation_time": 1722542958272,
  "deployment_name": "3389895405782117.7",
  "workspace_status": "PROVISIONING",
  "account_id": "2e0b6f3b-e1dc-43ea-93f5-7d55b9d7b780",
  "workspace_status_message": "Workspace resources are being set up.",
  "network_id": "fd0cc5bc-683c-47e9-b15e-144d7744a496",
  "pricing_tier": "ENTERPRISE",
  "private_access_settings_id": "3b3bbcb5-46bd-4b03-944e-97eb44ed7991",
  "location": "us-east4",
  "cloud": "gcp",
  "identity_federation_info": {
    "enable_identity_federation": true
  },
  "gke_config": {
    "connectivity_type": "PRIVATE_NODE_PUBLIC_MASTER",
    "master_ip_range": "10.103.0.0/28"
  },
  "cloud_resource_container": {
    "gcp": {
      "project_id": "databricks-dev-entd-shared"
    }
  }
}

Step 3: Configure DNS (optional)

This step is required if the workspace is a Private Service Connect workspace after the update. To allow users to access the new workspace URL, you must configure DNS. See Step 9: Configure DNS.

Step 4: Configure the firewall (optional)

This step is required if you updated the workspace with a network configuration object that uses a different CIDR range.

Update your firewall's ingress rules to allow traffic from the new subnet CIDR ranges. Verify that the Databricks ingress rule source is updated to reflect the new CIDR range. For details, see Limit network egress for your workspace using a firewall.

Step 5: Restart clusters and jobs

After the workspace update is complete and any necessary DNS or firewall changes are configured, you can restart your clusters and jobs.