Create a cross-account IAM role

To create a new workspace, you must set up an IAM cross-account role so that Databricks can access your AWS account.

This IAM role includes a policy that varies depending on your deployment type. The configuration procedure is the same for all options, regardless of policy.

Create a cross-account role and an access policy

  1. Get your Databricks external ID (account ID). You need it when you create the AWS cross-account IAM role in your AWS account.

    Go to the account console and click the down arrow next to your username in the upper right corner. Under Account ID, select and copy the ID.

    You can also find this ID (labeled External ID) by going to Account Settings > Credential configurations > Add credential configuration.

  2. Log into your AWS Console as a user with administrator privileges and go to the IAM service.

  3. Click the Roles tab in the sidebar.

  4. Click Create role.

    1. In Select type of trusted entity, click the Another AWS account tile.

      Trusted entity type
    2. In the Account ID field, enter the Databricks account ID 414351767826.

      This is not the Account ID you copied from the Databricks account console.

    3. Select the Require external ID checkbox.

    4. In the External ID field, enter your Databricks account ID, which you copied from the Databricks account console.

      Important

      Protect your account ID like a credential.

    5. Click the Next: Permissions button.

    6. Click the Next: Tags button.

    7. Click the Next: Review button.

    8. In the Role name field, enter a role name.

      Role name
    9. Click Create role. The list of roles displays.

  5. In the list of roles, click the role you created.

  6. Add an inline policy.

    1. On the Permissions tab, click Add inline policy.

      Inline policy
    2. In the policy editor, click the JSON tab.

      JSON editor
    3. Copy the access policy that is appropriate for your VPC network deployment:

      Deployment Policy
      Customer-managed VPC with default policy restrictions: Launch Databricks workspaces in your own VPC. [Your VPC, default]
      Customer-managed VPC with custom policy restrictions: Launch Databricks workspaces in your own VPC with policy restrictions by account ID, VPC ID, region, and security group. [Your VPC, custom]
      Databricks-managed VPC: Launch Databricks workspaces in a Databricks-managed VPC. [Databricks VPC]

      Select the tab that corresponds with your policy and click Copy. For the tab labeled custom, modify the policy as instructed.

      Important

      All of these policies assume the workspace uses secure cluster connectivity, which is sometimes referred to as No Public IP (NPIP). Secure cluster connectivity is the default as of September 1, 2020 for workspaces created with the Account API:

      Customer-managed VPC with secure cluster connectivity (no public IP / NPIP) with default restrictions

      {
      "Version": "2012-10-17",
      "Statement": [{
            "Sid": "Stmt1403287045000",
            "Effect": "Allow",
            "Action": [
                "ec2:AssociateIamInstanceProfile",
                "ec2:AttachVolume",
                "ec2:AuthorizeSecurityGroupEgress",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:CancelSpotInstanceRequests",
                "ec2:CreateTags",
                "ec2:CreateVolume",
                "ec2:DeleteTags",
                "ec2:DeleteVolume",
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeIamInstanceProfileAssociations",
                "ec2:DescribeInstanceStatus",
                "ec2:DescribeInstances",
                "ec2:DescribeInternetGateways",
                "ec2:DescribeNatGateways",
                "ec2:DescribeNetworkAcls",
                "ec2:DescribePrefixLists",
                "ec2:DescribeReservedInstancesOfferings",
                "ec2:DescribeRouteTables",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSpotInstanceRequests",
                "ec2:DescribeSpotPriceHistory",
                "ec2:DescribeSubnets",
                "ec2:DescribeVolumes",
                "ec2:DescribeVpcAttribute",
                "ec2:DescribeVpcs",
                "ec2:DetachVolume",
                "ec2:DisassociateIamInstanceProfile",
                "ec2:ReplaceIamInstanceProfileAssociation",
                "ec2:RequestSpotInstances",
                "ec2:RevokeSecurityGroupEgress",
                "ec2:RevokeSecurityGroupIngress",
                "ec2:RunInstances",
                "ec2:TerminateInstances"
            ],
            "Resource": [
              "*"
            ]
          },
          {
            "Effect": "Allow",
            "Action": [
                "iam:CreateServiceLinkedRole",
                "iam:PutRolePolicy"
            ],
            "Resource": "arn:aws:iam::*:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot",
            "Condition": {
              "StringLike": {
                  "iam:AWSServiceName": "spot.amazonaws.com"
              }
            }
          }]
      }
      

      Customer-managed VPC with secure cluster connectivity (no public IP / NPIP) with custom restrictions for account ID, VPC ID, region, and security group.

      Replace the following values in the policy with your own configuration values:

      • ACCOUNTID — Your AWS account ID, which is a number.

      • VPCID — ID of your AWS VPC in which you want to launch workspaces.

        Important

        If you add a VPC ID restriction, you cannot reuse the cross-account IAM role or reference credentials ID (credentials_id) for other workspaces hosted in other VPCs. For those other VPCs, you must create separate roles, policies, credentials objects.

      • REGION — AWS region name for your VPC deployment, for example us-west-2.

      • SECURITYGROUPID — ID of your AWS security group. When you add a security group restriction, you cannot reuse the cross-account IAM role or reference credentials ID (credentials_id) for any other workspaces. For those other workspaces, you must create separate roles, policies, and credentials objects.

        Note

        If you use immutable security groups, remove these permissions from the policy: ec2:AuthorizeSecurityGroupEgress, ec2:AuthorizeSecurityGroupIngress, ec2:RevokeSecurityGroupEgress, ec2:RevokeSecurityGroupIngress. Modify the policy to remove the final JSON object with Sid value VpcNonresourceSpecificActions.

      {
      "Version": "2012-10-17",
      "Statement": [
          {
          "Sid": "NonResourceBasedPermissions",
          "Effect": "Allow",
          "Action": [
              "ec2:CancelSpotInstanceRequests",
              "ec2:DescribeAvailabilityZones",
              "ec2:DescribeIamInstanceProfileAssociations",
              "ec2:DescribeInstanceStatus",
              "ec2:DescribeInstances",
              "ec2:DescribeInternetGateways",
              "ec2:DescribeNatGateways",
              "ec2:DescribeNetworkAcls",
              "ec2:DescribePrefixLists",
              "ec2:DescribeReservedInstancesOfferings",
              "ec2:DescribeRouteTables",
              "ec2:DescribeSecurityGroups",
              "ec2:DescribeSpotInstanceRequests",
              "ec2:DescribeSpotPriceHistory",
              "ec2:DescribeSubnets",
              "ec2:DescribeVolumes",
              "ec2:DescribeVpcAttribute",
              "ec2:DescribeVpcs",
              "ec2:CreateTags",
              "ec2:DeleteTags",
              "ec2:RequestSpotInstances"
          ],
          "Resource": [
              "*"
          ]
          },
          {
          "Sid": "InstancePoolsSupport",
          "Effect": "Allow",
          "Action": [
              "ec2:AssociateIamInstanceProfile",
              "ec2:DisassociateIamInstanceProfile",
              "ec2:ReplaceIamInstanceProfileAssociation"
          ],
          "Resource": "arn:aws:ec2:REGION:ACCOUNTID:instance/*",
          "Condition": {
              "StringEquals": {
              "ec2:ResourceTag/Vendor": "Databricks"
              }
          }
          },
          {
          "Sid": "AllowEc2RunInstancePerTag",
          "Effect": "Allow",
          "Action": "ec2:RunInstances",
          "Resource": [
              "arn:aws:ec2:REGION:ACCOUNTID:volume/*",
              "arn:aws:ec2:REGION:ACCOUNTID:instance/*"
          ],
          "Condition": {
              "StringEquals": {
              "aws:RequestTag/Vendor": "Databricks"
              }
          }
          },
          {
          "Sid": "AllowEc2RunInstanceImagePerTag",
          "Effect": "Allow",
          "Action": "ec2:RunInstances",
          "Resource": [
              "arn:aws:ec2:REGION:ACCOUNTID:image/*"
          ],
          "Condition": {
              "StringEquals": {
              "aws:ResourceTag/Vendor": "Databricks"
              }
          }
          },
          {
          "Sid": "AllowEc2RunInstancePerVPCid",
          "Effect": "Allow",
          "Action": "ec2:RunInstances",
          "Resource": [
              "arn:aws:ec2:REGION:ACCOUNTID:network-interface/*",
              "arn:aws:ec2:REGION:ACCOUNTID:subnet/*",
              "arn:aws:ec2:REGION:ACCOUNTID:security-group/*"
          ],
          "Condition": {
              "StringEquals": {
              "ec2:vpc": "arn:aws:ec2:REGION:ACCOUNTID:vpc/VPCID"
              }
          }
          },
          {
          "Sid": "AllowEc2RunInstanceOtherResources",
          "Effect": "Allow",
          "Action": "ec2:RunInstances",
          "NotResource": [
              "arn:aws:ec2:REGION:ACCOUNTID:image/*",
              "arn:aws:ec2:REGION:ACCOUNTID:network-interface/*",
              "arn:aws:ec2:REGION:ACCOUNTID:subnet/*",
              "arn:aws:ec2:REGION:ACCOUNTID:security-group/*",
              "arn:aws:ec2:REGION:ACCOUNTID:volume/*",
              "arn:aws:ec2:REGION:ACCOUNTID:instance/*"
          ]
          },
          {
          "Sid": "EC2TerminateInstancesTag",
          "Effect": "Allow",
          "Action": [
              "ec2:TerminateInstances"
          ],
          "Resource": [
              "arn:aws:ec2:REGION:ACCOUNTID:instance/*"
          ],
          "Condition": {
              "StringEquals": {
              "ec2:ResourceTag/Vendor": "Databricks"
              }
          }
          },
          {
          "Sid": "EC2AttachDetachVolumeTag",
          "Effect": "Allow",
          "Action": [
              "ec2:AttachVolume",
              "ec2:DetachVolume"
          ],
          "Resource": [
              "arn:aws:ec2:REGION:ACCOUNTID:instance/*",
              "arn:aws:ec2:REGION:ACCOUNTID:volume/*"
          ],
          "Condition": {
              "StringEquals": {
              "ec2:ResourceTag/Vendor": "Databricks"
              }
          }
          },
          {
          "Sid": "EC2CreateVolumeByTag",
          "Effect": "Allow",
          "Action": [
              "ec2:CreateVolume"
          ],
          "Resource": [
              "arn:aws:ec2:REGION:ACCOUNTID:volume/*"
          ],
          "Condition": {
              "StringEquals": {
              "aws:RequestTag/Vendor": "Databricks"
              }
          }
          },
          {
          "Sid": "EC2DeleteVolumeByTag",
          "Effect": "Allow",
          "Action": [
              "ec2:DeleteVolume"
          ],
          "Resource": [
              "arn:aws:ec2:REGION:ACCOUNTID:volume/*"
          ],
          "Condition": {
              "StringEquals": {
              "ec2:ResourceTag/Vendor": "Databricks"
              }
          }
          },
          {
          "Effect": "Allow",
          "Action": [
              "iam:CreateServiceLinkedRole",
              "iam:PutRolePolicy"
          ],
          "Resource": "arn:aws:iam::*:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot",
          "Condition": {
              "StringLike": {
              "iam:AWSServiceName": "spot.amazonaws.com"
              }
          }
          },
          {
          "Sid": "VpcNonresourceSpecificActions",
          "Effect": "Allow",
          "Action": [
              "ec2:AuthorizeSecurityGroupEgress",
              "ec2:AuthorizeSecurityGroupIngress",
              "ec2:RevokeSecurityGroupEgress",
              "ec2:RevokeSecurityGroupIngress"
          ],
          "Resource": "arn:aws:ec2:REGION:ACCOUNTID:security-group/SECURITYGROUPID",
          "Condition": {
              "StringEquals": {
                  "ec2:vpc": "arn:aws:ec2:REGION:ACCOUNTID:vpc/VPCID"
              }
          }
          }
      ]
      }
      

      Databricks-managed VPC using secure cluster connectivity (NPIP). This is the most typical deployment type. If you have a legacy workspace that does not use secure cluster connectivity (NPIP), contact your Databricks representative.

      {
      "Version": "2012-10-17",
      "Statement": [{
            "Sid": "Stmt1403287045000",
            "Effect": "Allow",
            "Action": [
              "ec2:AllocateAddress",
              "ec2:AssociateDhcpOptions",
              "ec2:AssociateIamInstanceProfile",
              "ec2:AssociateRouteTable",
              "ec2:AttachInternetGateway",
              "ec2:AttachVolume",
              "ec2:AuthorizeSecurityGroupEgress",
              "ec2:AuthorizeSecurityGroupIngress",
              "ec2:CancelSpotInstanceRequests",
              "ec2:CreateDhcpOptions",
              "ec2:CreateInternetGateway",
              "ec2:CreateNatGateway",
              "ec2:CreateRoute",
              "ec2:CreateRouteTable",
              "ec2:CreateSecurityGroup",
              "ec2:CreateSubnet",
              "ec2:CreateTags",
              "ec2:CreateVolume",
              "ec2:CreateVpc",
              "ec2:CreateVpcEndpoint",
              "ec2:DeleteDhcpOptions",
              "ec2:DeleteInternetGateway",
              "ec2:DeleteNatGateway",
              "ec2:DeleteRoute",
              "ec2:DeleteRouteTable",
              "ec2:DeleteSecurityGroup",
              "ec2:DeleteSubnet",
              "ec2:DeleteTags",
              "ec2:DeleteVolume",
              "ec2:DeleteVpc",
              "ec2:DeleteVpcEndpoints",
              "ec2:DescribeAvailabilityZones",
              "ec2:DescribeIamInstanceProfileAssociations",
              "ec2:DescribeInstanceStatus",
              "ec2:DescribeInstances",
              "ec2:DescribeInternetGateways",
              "ec2:DescribeNatGateways",
              "ec2:DescribePrefixLists",
              "ec2:DescribeReservedInstancesOfferings",
              "ec2:DescribeRouteTables",
              "ec2:DescribeSecurityGroups",
              "ec2:DescribeSpotInstanceRequests",
              "ec2:DescribeSpotPriceHistory",
              "ec2:DescribeSubnets",
              "ec2:DescribeVolumes",
              "ec2:DescribeVpcs",
              "ec2:DetachInternetGateway",
              "ec2:DisassociateIamInstanceProfile",
              "ec2:DisassociateRouteTable",
              "ec2:ModifyVpcAttribute",
              "ec2:ReleaseAddress",
              "ec2:ReplaceIamInstanceProfileAssociation",
              "ec2:RequestSpotInstances",
              "ec2:RevokeSecurityGroupEgress",
              "ec2:RevokeSecurityGroupIngress",
              "ec2:RunInstances",
              "ec2:TerminateInstances"
            ],
            "Resource": [
              "*"
            ]
          },
          {
            "Effect": "Allow",
            "Action": [
                "iam:CreateServiceLinkedRole",
                "iam:PutRolePolicy"
            ],
            "Resource": "arn:aws:iam::*:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot",
            "Condition": {
              "StringLike": {
                  "iam:AWSServiceName": "spot.amazonaws.com"
              }
            }
          }]
      }
      
    4. Click Review policy.

    5. In the Name field, enter a policy name.

    6. Click Create policy.

    7. If you use Service Control Policies to deny certain actions at the AWS account level, ensure that sts:AssumeRole is whitelisted so Databricks can assume the cross-account role.

  7. In the role summary, copy the Role ARN.

    Role ARN