Secure cluster connectivity

Preview

This feature is in Public Preview. It is not available on all Databricks deployment types and subscriptions. Contact your Databricks representative to request access.

Some deployment types and Databricks subscriptions include optional secure cluster connectivity. With secure cluster connectivity, customer VPCs have no open ports and Databricks Runtime workers have no public IP addresses. In some APIs, this is referred to as No Public IP (NPIP):

  • At a network level, each cluster initiates a connection to the control plane secure cluster connectivity relay (proxy) during cluster creation. The cluster establishes this connection using port 443 (HTTPS) and a different IP address than is used for the Web application and REST API.
  • Actions that the control plane logically initiates, such as starting new Databricks Runtime jobs or performing cluster administration, are sent as requests to the cluster through this reverse tunnel.
  • The data plane (the customer VPC) does not have open ports and Databricks Runtime workers do not have public IP addresses.

Benefits:

  • Easy network administration — Less complexity because there is no need for port configuration on security groups or configuring network peering.
  • Easier approval — Because of better security and simpler network administration, it is easier for Information Security teams to approve Databricks as a PaaS provider.
Secure cluster connectivity

Using secure cluster connectivity

To use secure cluster connectivity for a workspace, you must create a new workspace using the Multi-workspace API. You cannot add secure cluster connectivity to an existing workspace.