AWS firewall restrictions

In AWS, Databricks launches the cluster in a VPC created and managed by Databricks in the customer’s account. For additional security, workers that belong to a cluster can only communicate with other workers that belong to the same cluster. Workers cannot talk to any other EC2 instances or other AWS services running in the Databricks VPC. If you have any AWS service running on the same VPC as that of the Databricks cluster, you may not be able to talk to the service because of this firewall restriction.

Databricks recommends to run such services outside of the Databricks VPC and peer with that VPC to connect to those services.