Permissions in cross-account IAM roles
This page lists permissions in the cross-account IAM role needed for workspace deployment and the purpose of each role.
If your AWS account has organization-level service control policies (SCPs) that deny the AssumeRole action or deny EC2/VPC access, the cross-account role setup might fail despite having the correct IAM policy. Verify that your organization's SCPs allow the required permissions listed on this page.
The permissions are different based on how you configure your VPC.
IAM permissions for Databricks-managed VPCs
Databricks requires the following list of IAM permissions to operate and manage clusters in an effective manner. This configuration applies only to workspaces that use the default (Databricks-managed) VPC. To create the AWS cross-account role policy for use with the default Databricks-managed VPC, see Create a credential configuration.
The following table lists Databricks IAM cross-account role permissions in the default configuration, the resources that they control, and the purpose for each permission.
AWS IAM permission | AWS resource | Purpose |
|---|---|---|
| Elastic IP address | Allocates an Elastic IP that is associated with the NAT Gateway used in secure cluster connectivity |
| Network Interface | Assigns a private IP to EC2 instance. |
| DHCP | Associates a set of DHCP options (or no DHCP options) with a VPC. |
| InstanceProfile | Associate an instance profile with a running EC2 instance. This allows a Databricks pool instance to be used by clusters with different instance profiles throughout its lifetime in the pool. |
| RouteTable | Associates a subnet with a route table. |
| InternetGateway | Attaches an Internet gateway to a VPC, enabling connectivity between the Internet and the VPC. This is currently required to connect to S3 buckets and update code for the workers and spark containers. |
| EBS Volume | Attaches volume for EBS auto-scaling. |
| SecurityGroup | Adds egress rules to the security groups if required. |
| SecurityGroup | Adds ingress rules to the security groups. |
| SpotInstance | Cancels spot instances. |
| Dhcp | Creates DHCP options. |
| EC2 Fleet | Creates an EC2 fleet (used with Databricks fleet clusters). |
| InternetGateway | Creates an Internet Gateway. |
| Launch Template | Creates a launch template (used with Databricks fleet clusters). |
| Launch Template | Creates a launch template version (used with Databricks fleet clusters). |
| NatGateway | Creates a NAT gateway. |
| Route | Creates routes during workspace setup. |
| RouteTable | Creates routes during workspace setup. |
| SecurityGroup | Creates security groups during initial setup |
| Subnet | Create subnets for the VPC during workspace setup. |
| Tags | Adds tags on Databricks resources. |
| EBS Volume | Creates volume. |
| VPC | Creates the Databricks-managed VPC. |
| VPCEndpoint | Creates VPC endpoints as part of configuring the VPC. |
| DHCPOptions | Deletes DHCPOptions |
| EC2 Fleet | Deletes an EC2 fleet (used with Databricks fleet clusters) |
| InternetGateway | Deletes Internet Gateway during workspace deletion. |
| Launch Template | Deletes a launch template and all its versions. |
| Launch Template | Deletes a version from a launch template (used with Databricks fleet clusters). |
| NatGateway | Deletes NAT gateway as needed to setup the secure cluster connectivity relay. |
| Route | Deletes routes. |
| RouteTable | Deletes route table. |
| SecurityGroup | Deletes security groups during workspace deletion. |
| Subnet | Deletes subnet. |
| Tags | Removes tags from cluster resources to allows Databricks pool instances to be reused by clusters with different tags. |
| EBS Volume | Deletes a volume for EBS auto-scaling. See this page. |
| VPC | Deletes the VPC when customers during workspace deletion. |
| VPCEndpoints | Deletes the VPC endpoints during workspace deletion. |
| AvailabilityZones | Gets a list of Availability Zones in a region so that Databricks can deploy resources in that zone. |
| EC2 Fleet | Lists events in an EC2 fleet (used with Databricks fleet clusters). |
| EC2 Fleet | Lists instances in an EC2 fleet (used with Databricks fleet clusters). |
| EC2 Fleet | Describes details of an EC2 fleet (used with Databricks fleet clusters). |
| InstanceProfile | Checks the current instance profile that is set on an EC2 instance so that the right profile is set on a Databricks pool instance before it's reused by a cluster. |
| Instance | Confirms that Databricks AWS instances are healthy. |
| Instance | Confirms that Databricks AWS instances are healthy. |
| InternetGateway | Describes InternetGateway to confirm that Databricks AWS instances have a route to the internet. |
| Launch Template | Deletes a launch template (used with Databricks fleet clusters). |
| Launch Template | Describes details of launch template versions (used with Databricks fleet clusters). |
| NATGateway | Describes a NAT Gateway to confirm that Databricks AWS instances have a route to the internet in the secure cluster connectivity architecture. |
| PrefixList | Creates a prefix list ID to create an outbound security group rule that allows traffic from a VPC so that Databricks can access an AWS service through a gateway VPC endpoint. |
| Instance | Describes Reserved Instance pricing in support of AWS spot instance pricing. |
| RouteTable | Confirms that the route tables are set up correctly in the Databricks-managed VPC. |
| SecurityGroup | Confirms that AWS security groups are set up correctly. |
| Instance | Describes spot instances. |
| SpotInstance | Describes spot instances. |
| Subnet | Confirms that subnets are setup correctly in Databricks VPC. |
| Volume | Lists volumes. |
| VPC | Confirm that the workspace's VPC was set up correctly. |
| InternetGateway | Detaches the Databricks created Internet Gateway during workspace deletion. |
| InstanceProfile | Disassociates an instance profile from an EC2 instance so that xDatabricks pool instances can be used by clusters with different instance profiles. |
| RouteTable | Detaches the Databricks created route table during workspace deletion. |
| Launch Templates | Gets config of a launch template (used with Databricks fleet clusters). |
| Availability Zones | Gets list of AZs with best spot capacity for given instance type(s). |
| EC2 Fleet | Modifies an EC2 fleet (used with Databricks fleet clusters). |
| Launch Templates | Modifies an existing launch template (used with Databricks fleet clusters). |
| VPCAttribute | Configures the Databricks-managed VPC. |
| Address | Detach the Databricks created address during workspace deletion. |
| InstanceProfile | Swaps one instance profile for another on an EC2 instance so that Databricks pool instances can be used by clusters with different instance profiles. |
| SpotInstance | Requests spot instances. |
| SecurityGroup | Updates Databricks-managed security groups if required. |
| SecurityGroup | Updates security groups. |
| Instance | Launches AWS instances to create Spark Clusters. Also leveraged during scaling up an existing Spark cluster. |
| Instance | Terminates Spark EC2 nodes during cluster scale down or to terminate a given Spark cluster. |
| ServiceLinkedRole | Sets up support for spot instances. |
| RolePolicy | Configures Databricks to use spot instances. |
IAM permissions for customer-managed VPC
If you use a customer-managed VPC, there's a smaller set of permissions needed for the cross-account IAM role. This feature requires the Premium plan or above.
To create the AWS cross-account role policy for use with a customer-managed VPC, see Create a credential configuration.
The permissions can be scoped down further if needed. To create the AWS cross-account role policy for use with a customer-managed VPC with additional custom restrictions on resources, see Create a credential configuration.
The following table lists Databricks IAM cross-account role permissions for a customer-managed VPC, the resources that they control, and the purpose for each permission.
AWS IAM permission | AWS resource | Purpose |
|---|---|---|
| InstanceProfile | Associates an instance profile with a running EC2 instance so that a Databricks pool instance can be used by clusters with different instance profiles throughout its lifetime in the pool. |
| Volume | Attaches a volume. |
| SecurityGroup | Add egress rules to the security groups if required. |
| SecurityGroup | Adds ingress rules to the security groups. |
| SpotInstance | Cancels spot instances. |
| EC2 Fleet | Creates an EC2 fleet (used with Databricks fleet clusters). |
| Launch Template | Creates a launch template (used with Databricks fleet clusters). |
| Tags | Adds tags on Databricks resources. |
| Volume | Creates a volume. |
| EC2 Fleet | Deletes an EC2 fleet (used with Databricks fleet clusters) |
| Launch Template | Deletes a launch template and all its versions. |
| Launch Template | Deletes a version from a launch template (used with Databricks fleet clusters). |
| Tags | Removes tags from cluster resources so that Databricks pool instances can be reused by clusters with different tags. |
| Volume | Deletes a volume. |
| AvailabilityZones | Gets a list of Availability Zones in a region so that Databricks can deploy the resources in that zone. |
| InstanceProfile | Checks the current instance profile set on an EC2 instance to confirm that the right profile is set on a Databricks pool instance before it's reused by a cluster. |
| Instance | Confirms that Databricks AWS instances are healthy. |
| Instance | Confirm that Databricks AWS instances are healthy. |
| InternetGateway | Describes InternetGateway to confirm that Databricks AWS instances have a route to the internet. |
| EC2 Fleet | Lists events in an EC2 fleet (used with Databricks fleet clusters). |
| EC2 Fleet | Lists instances in an EC2 fleet (used with Databricks fleet clusters). |
| EC2 Fleet | Describes details of an EC2 fleet (used with Databricks fleet clusters). |
| Launch Template | Deletes a launch template (used with Databricks fleet clusters). |
| Launch Template | Describes details of launch template versions (used with Databricks fleet clusters). |
| NATGateway | Describes NATGateway to confirm that Databricks AWS instances have a route to the internet in the secure cluster connectivity architecture. |
| NetworkAcl | Confirms the correct Network ACL setup. |
| PrefixList | Gets a list of prefix list IDs to create an outbound security group rule that allows traffic from a VPC to access an AWS service through a gateway VPC endpoint. |
| Instance | Gets Reserved Instance pricing as the starting point for AWS spot instance pricing. |
| RouteTable | Confirms that route tables are set up correctly in the VPC. |
| SecurityGroup | Confirms that AWS security groups are set up correctly. |
| Instance | Describes spot instance. |
| SpotInstance | Describes spot instances. |
| Subnet | Confirms that subnets are setup correctly in the VPC. |
| Volume | List volumes. |
| VPC | Describes VPC attributes including but not limited to |
| VPC | Confirms that the Databricks workspace VPC was created. |
| Volume | Detaches an EBS volume from EC2 instances during cluster shutdown. |
| InstanceProfile | Disassociates an instance profile from an EC2 instance so that pool instances can be used by clusters with different instance profiles. |
| Launch Templates | Gets config of a launch template (used with Databricks fleet clusters). |
| EC2 Fleet | Modifies an EC2 fleet (used with Databricks fleet clusters). |
| Launch Templates | Modifies an existing launch template (used with Databricks fleet clusters). |
| InstanceProfile | Swaps one instance profile for another on an EC2 instance so that pool instances can be used by clusters with different instance profiles. |
| SpotInstance | Requests spot instances. |
| SecurityGroup | Updates Databricks-managed security groups if required |
| SecurityGroup | Updates security groups. |
| Instance | Launches AWS instances to create Spark Clusters. Also used to scale up an existing Spark cluster. |
| Instance | Terminates Spark EC2 nodes during cluster scale down or to terminate a Spark cluster. |
| ServiceLinkedRole | Sets up support for spot instances. |
| RolePolicy | Configures Databricks to use spot instances. |