Skip to main content

Configure a firewall for serverless compute access

This article describes how to configure a firewall for serverless compute using the SAP Databricks account console UI.

Overview of firewall enablement for serverless compute​

Serverless network connectivity is managed with network connectivity configurations (NCCs). Account admins create NCCs in the account console, and an NCC can be attached to one or more workspaces. NCCs are account-level regional constructs that are used to manage private endpoint creation and firewall enablement at scale.

An NCC contains a list of IPs. When an NCC is attached to a workspace, serverless compute in that workspace uses one of those IP addresses to connect your resources. You can allow list those networks on your resource firewalls.

NCC firewall enablement is supported from serverless SQL warehouses, jobs, notebooks, and model serving endpoints.

What is a network connectivity configuration (NCC)?​

Serverless network connectivity is managed with network connectivity configurations (NCC). NCCs are account-level regional constructs that are used to manage private endpoints creation and firewall enablement at scale.

Account admins create NCCs in the account console and an NCC can be attached to one or more workspaces to enable firewalls for resources. An NCC contains a list of stable IP addresses. When an NCC is attached to a workspace, serverless compute in that workspace uses one of those IP addresses to connect the cloud resource. You can allow list those networks on your resource firewall.

Creating a resource firewall also affects connectivity from the classic compute plane to your resource. You must also allow list the networks on your resource firewalls to connect to them from classic compute resources.

NCC firewall enablement is not supported for Amazon S3 or Amazon DynamoDB. When reading or writing to Amazon S3 buckets in the same region as your workspace, serverless compute resources use direct access to S3 using AWS gateway endpoints. This applies when serverless compute reads and writes to your workspace storage bucket in your AWS account and to other S3 data sources in the same region.

note

Databricks uses S3 gateway endpoints, private IPs, and public IPs to connect to resources based on their location and type. These connectivity methods are generally available unless explicitly stated otherwise.

Requirements​

  • You must be a Databricks account admin.
  • Each NCC can be attached to up to 50 workspaces.

Step 1: Create a network connectivity configuration and copy the stable IPs​

Databricks recommends sharing NCCs among workspaces in the same business unit and those sharing the same region.

Network connectivity configuration.

  1. As an account admin, go to the account console.
  2. In the sidebar, click Cloud Resources.
  3. Click Network.
  4. Click Network Connectivity Configuration.
  5. Click Add Network Connectivity Configuration.
  6. Type a name for the NCC.
  7. Choose the region. This must match your workspace region.
  8. Click Add.
  9. Click the Default Rules tab.
  10. Under Stable IPs, click Copy all IPs and save the list of IPs.

Step 2: Attach an NCC to workspaces​

You can attach an NCC to up to 50 workspaces in the same region as the NCC.

  1. In the account console sidebar, click Workspaces.
  2. Click your workspace's name.
  3. Click Update workspace.
  4. In the Network Connectivity Configuration field, select your NCC. If it's not visible, confirm that you've selected the same region for both the workspace and the NCC.
  5. Click Update.
  6. Wait 10 minutes for the change to take effect.
  7. Restart any running serverless compute resources in the workspace.

Step 3: Update your resource access rules to allowlist the IPs​

Add the stable IPs to your resource access rules.

Creating a storage firewall also affects connectivity from classic compute plane resources to resources. You must also update your resource access rules to allowlist the IPs to connect to them from classic compute resources.

NCC firewall enablement is not supported for Amazon S3 or Amazon DynamoDB. When reading or writing to Amazon S3 buckets in the same region as your workspace, serverless compute resources use direct access to S3 using AWS gateway endpoints. This applies when serverless SQL compute reads and writes to your workspace storage bucket in your AWS account and to other S3 data sources in the same region.

Last updated on