Create a cross-account IAM role for launching multiple workspaces

Preview

The Multi-workspace API is in Public Preview.

To create a new workspace using the Multi-workspace (MWS) API, you must set up an IAM cross-account role so that Databricks can access your AWS account. This IAM role includes a policy that varies depending on your deployment type. The configuration procedure is the same for all options, regardless of policy.

Note

If you are configuring a new Databricks account and are not using the Multi-workspace API, see instead Configure your AWS account.

Requirements

You need the multi-workspace master account ID for your Databricks account, which should have been shared with you in the Multi-workspace API welcome email. If you do not know it, contact your Databricks representative. See Creating a workspace with Multi-workspace API.

Create a cross-account role and an access policy

  1. Log into your AWS Console as a user with administrator privileges and go to the IAM service.

  2. Click the Roles tab in the sidebar.

  3. Click Create role.

    1. In Select type of trusted entity, click the Another AWS account tile.

      Trusted entity type
    2. In the Account ID field, enter the Databricks account ID 414351767826.

    3. Select the Require external ID checkbox.

    4. In the External ID field, enter your multi-workspace master account ID.

      Important

      Protect your multi-workspace master account ID like a credential.

    5. Click the Next: Permissions button.

    6. Click the Next: Tags button.

    7. Click the Next: Review button.

    8. In the Role name field, enter a role name.

      Role name
    9. Click Create role. The list of roles displays.

  4. In the list of roles, click the role you created.

  5. Add an inline policy.

    1. On the Permissions tab, click Add inline policy.

      Inline policy
    2. In the policy editor, click the JSON tab.

      JSON editor
    3. Copy the access policy that is appropriate for your VPC network deployment:

      Deployment Policy
      Customer-managed VPC with default policy restrictions: Launch Databricks workspaces in your own VPC. [Your VPC, default]
      Customer-managed VPC with custom policy restrictions: Launch Databricks workspaces in your own VPC with policy restrictions by account ID, VPC ID, region, and security group. [Your VPC, custom]
      Databricks-managed VPC: Launch Databricks workspaces in a Databricks-managed VPC. [Databricks VPC]

      Select the tab that corresponds with your policy and click Copy. For the tab labeled custom, modify the policy as instructed:

      Customer-managed VPC with secure cluster connectivity (no public IP / NPIP) with default restrictions

      {
      "Version": "2012-10-17",
      "Statement": [{
            "Sid": "Stmt1403287045000",
            "Effect": "Allow",
            "Action": [
                "ec2:AssociateIamInstanceProfile",
                "ec2:AttachVolume",
                "ec2:AuthorizeSecurityGroupEgress",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:CancelSpotInstanceRequests",
                "ec2:CreateKeyPair",
                "ec2:CreatePlacementGroup",
                "ec2:CreateTags",
                "ec2:CreateVolume",
                "ec2:DeleteKeyPair",
                "ec2:DeletePlacementGroup",
                "ec2:DeleteTags",
                "ec2:DeleteVolume",
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeIamInstanceProfileAssociations",
                "ec2:DescribeInstanceStatus",
                "ec2:DescribeInstances",
                "ec2:DescribeInternetGateways",
                "ec2:DescribeNatGateways",
                "ec2:DescribeNetworkAcls",
                "ec2:DescribePlacementGroups",
                "ec2:DescribePrefixLists",
                "ec2:DescribeReservedInstancesOfferings",
                "ec2:DescribeRouteTables",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSpotInstanceRequests",
                "ec2:DescribeSpotPriceHistory",
                "ec2:DescribeSubnets",
                "ec2:DescribeVolumes",
                "ec2:DescribeVpcAttribute",
                "ec2:DescribeVpcs",
                "ec2:DetachVolume",
                "ec2:DisassociateIamInstanceProfile",
                "ec2:ReplaceIamInstanceProfileAssociation",
                "ec2:RequestSpotInstances",
                "ec2:RevokeSecurityGroupEgress",
                "ec2:RevokeSecurityGroupIngress",
                "ec2:RunInstances",
                "ec2:TerminateInstances"
            ],
            "Resource": [
              "*"
            ]
          },
          {
            "Effect": "Allow",
            "Action": [
                "iam:CreateServiceLinkedRole",
                "iam:PutRolePolicy"
            ],
            "Resource": "arn:aws:iam::*:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot",
            "Condition": {
              "StringLike": {
                  "iam:AWSServiceName": "spot.amazonaws.com"
              }
            }
          }]
      }
      

      Customer-managed VPC with secure cluster connectivity (no public IP / NPIP) with custom restrictions for account ID, VPC ID, region, and security group.

      Replace the following values in the policy with your own configuration values:

      • ACCOUNTID — Your AWS account ID, which is a number.

      • VPCID — ID of your AWS VPC in which you want to launch workspaces.

        Important

        If you add a VPC ID restriction, you cannot reuse the cross-account IAM role or reference credentials ID (credentials_id) for other workspaces hosted in other VPCs. For those other VPCs, you must create separate roles, policies, credentials objects.

      • REGION — AWS region name for your VPC deployment, for example us-west-2.

      • SECURITYGROUPID — ID of your AWS security group. When you add a security group restriction, you cannot reuse the cross-account IAM role or reference credentials ID (credentials_id) for any other workspaces. For those other workspaces, you must create separate roles, policies, and credentials objects.

        Note

        If you use immutable security groups, remove these permissions from the policy: ec2:AuthorizeSecurityGroupEgress, ec2:AuthorizeSecurityGroupIngress, ec2:RevokeSecurityGroupEgress, ec2:RevokeSecurityGroupIngress. Modify the policy to remove the final JSON object with Sid value VpcNonresourceSpecificActions.

      {
      "Version": "2012-10-17",
      "Statement": [
          {
          "Sid": "NonResourceBasedPermissions",
          "Effect": "Allow",
          "Action": [
              "ec2:CancelSpotInstanceRequests",
              "ec2:DescribeAvailabilityZones",
              "ec2:DescribeIamInstanceProfileAssociations",
              "ec2:DescribeInstanceStatus",
              "ec2:DescribeInstances",
              "ec2:DescribeInternetGateways",
              "ec2:DescribeNatGateways",
              "ec2:DescribeNetworkAcls",
              "ec2:DescribePlacementGroups",
              "ec2:DescribePrefixLists",
              "ec2:DescribeReservedInstancesOfferings",
              "ec2:DescribeRouteTables",
              "ec2:DescribeSecurityGroups",
              "ec2:DescribeSpotInstanceRequests",
              "ec2:DescribeSpotPriceHistory",
              "ec2:DescribeSubnets",
              "ec2:DescribeVolumes",
              "ec2:DescribeVpcAttribute",
              "ec2:DescribeVpcs",
              "ec2:CreatePlacementGroup",
              "ec2:DeletePlacementGroup",
              "ec2:CreateKeyPair",
              "ec2:DeleteKeyPair",
              "ec2:CreateTags",
              "ec2:DeleteTags",
              "ec2:RequestSpotInstances"
          ],
          "Resource": [
              "*"
          ]
          },
          {
          "Sid": "InstancePoolsSupport",
          "Effect": "Allow",
          "Action": [
              "ec2:AssociateIamInstanceProfile",
              "ec2:DisassociateIamInstanceProfile",
              "ec2:ReplaceIamInstanceProfileAssociation"
          ],
          "Resource": "arn:aws:ec2:REGION:ACCOUNTID:instance/*",
          "Condition": {
              "StringEquals": {
              "ec2:ResourceTag/Vendor": "Databricks"
              }
          }
          },
          {
          "Sid": "AllowEc2RunInstancePerTag",
          "Effect": "Allow",
          "Action": "ec2:RunInstances",
          "Resource": [
              "arn:aws:ec2:REGION:ACCOUNTID:volume/*",
              "arn:aws:ec2:REGION:ACCOUNTID:instance/*"
          ],
          "Condition": {
              "StringEquals": {
              "aws:RequestTag/Vendor": "Databricks"
              }
          }
          },
          {
          "Sid": "AllowEc2RunInstanceImagePerTag",
          "Effect": "Allow",
          "Action": "ec2:RunInstances",
          "Resource": [
              "arn:aws:ec2:REGION:ACCOUNTID:image/*"
          ],
          "Condition": {
              "StringEquals": {
              "aws:ResourceTag/Vendor": "Databricks"
              }
          }
          },
          {
          "Sid": "AllowEc2RunInstancePerVPCid",
          "Effect": "Allow",
          "Action": "ec2:RunInstances",
          "Resource": [
              "arn:aws:ec2:REGION:ACCOUNTID:network-interface/*",
              "arn:aws:ec2:REGION:ACCOUNTID:subnet/*",
              "arn:aws:ec2:REGION:ACCOUNTID:security-group/*"
          ],
          "Condition": {
              "StringEquals": {
              "ec2:vpc": "arn:aws:ec2:REGION:ACCOUNTID:vpc/VPCID"
              }
          }
          },
          {
          "Sid": "AllowEc2RunInstanceOtherResources",
          "Effect": "Allow",
          "Action": "ec2:RunInstances",
          "NotResource": [
              "arn:aws:ec2:REGION:ACCOUNTID:image/*",
              "arn:aws:ec2:REGION:ACCOUNTID:network-interface/*",
              "arn:aws:ec2:REGION:ACCOUNTID:subnet/*",
              "arn:aws:ec2:REGION:ACCOUNTID:security-group/*",
              "arn:aws:ec2:REGION:ACCOUNTID:volume/*",
              "arn:aws:ec2:REGION:ACCOUNTID:instance/*"
          ]
          },
          {
          "Sid": "EC2TerminateInstancesTag",
          "Effect": "Allow",
          "Action": [
              "ec2:TerminateInstances"
          ],
          "Resource": [
              "arn:aws:ec2:REGION:ACCOUNTID:instance/*"
          ],
          "Condition": {
              "StringEquals": {
              "ec2:ResourceTag/Vendor": "Databricks"
              }
          }
          },
          {
          "Sid": "EC2AttachDetachVolumeTag",
          "Effect": "Allow",
          "Action": [
              "ec2:AttachVolume",
              "ec2:DetachVolume"
          ],
          "Resource": [
              "arn:aws:ec2:REGION:ACCOUNTID:instance/*",
              "arn:aws:ec2:REGION:ACCOUNTID:volume/*"
          ],
          "Condition": {
              "StringEquals": {
              "ec2:ResourceTag/Vendor": "Databricks"
              }
          }
          },
          {
          "Sid": "EC2CreateVolumeByTag",
          "Effect": "Allow",
          "Action": [
              "ec2:CreateVolume"
          ],
          "Resource": [
              "arn:aws:ec2:REGION:ACCOUNTID:volume/*"
          ],
          "Condition": {
              "StringEquals": {
              "aws:RequestTag/Vendor": "Databricks"
              }
          }
          },
          {
          "Sid": "EC2DeleteVolumeByTag",
          "Effect": "Allow",
          "Action": [
              "ec2:DeleteVolume"
          ],
          "Resource": [
              "arn:aws:ec2:REGION:ACCOUNTID:volume/*"
          ],
          "Condition": {
              "StringEquals": {
              "ec2:ResourceTag/Vendor": "Databricks"
              }
          }
          },
          {
          "Effect": "Allow",
          "Action": [
              "iam:CreateServiceLinkedRole",
              "iam:PutRolePolicy"
          ],
          "Resource": "arn:aws:iam::*:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot",
          "Condition": {
              "StringLike": {
              "iam:AWSServiceName": "spot.amazonaws.com"
              }
          }
          },
          {
          "Sid": "VpcNonresourceSpecificActions",
          "Effect": "Allow",
          "Action": [
              "ec2:AuthorizeSecurityGroupEgress",
              "ec2:AuthorizeSecurityGroupIngress",
              "ec2:RevokeSecurityGroupEgress",
              "ec2:RevokeSecurityGroupIngress"
          ],
          "Resource": "arn:aws:ec2:REGION:ACCOUNTID:security-group/SECURITYGROUPID",
          "Condition": {
              "StringEquals": {
                  "ec2:vpc": "arn:aws:ec2:REGION:ACCOUNTID:vpc/VPCID"
              }
          }
          }
      ]
      }
      

      Databricks-managed VPC and not using secure cluster connectivity (NPIP)

      {
      "Version": "2012-10-17",
      "Statement": [
          {
          "Sid": "Stmt1403287045000",
          "Effect": "Allow",
          "Action": [
              "ec2:AssociateDhcpOptions",
              "ec2:AssociateIamInstanceProfile",
              "ec2:AssociateRouteTable",
              "ec2:AttachInternetGateway",
              "ec2:AttachVolume",
              "ec2:AuthorizeSecurityGroupEgress",
              "ec2:AuthorizeSecurityGroupIngress",
              "ec2:CancelSpotInstanceRequests",
              "ec2:CreateDhcpOptions",
              "ec2:CreateInternetGateway",
              "ec2:CreateKeyPair",
              "ec2:CreateRoute",
              "ec2:CreateSecurityGroup",
              "ec2:CreateSubnet",
              "ec2:CreateTags",
              "ec2:CreateVolume",
              "ec2:CreateVpc",
              "ec2:DeleteInternetGateway",
              "ec2:DeleteKeyPair",
              "ec2:DeleteRoute",
              "ec2:DeleteRouteTable",
              "ec2:DeleteSecurityGroup",
              "ec2:DeleteSubnet",
              "ec2:DeleteTags",
              "ec2:DeleteVolume",
              "ec2:DeleteVpc",
              "ec2:DescribeAvailabilityZones",
              "ec2:DescribeIamInstanceProfileAssociations",
              "ec2:DescribeInstanceStatus",
              "ec2:DescribeInstances",
              "ec2:DescribeInternetGateways",
              "ec2:DescribePrefixLists",
              "ec2:DescribeReservedInstancesOfferings",
              "ec2:DescribeRouteTables",
              "ec2:DescribeSecurityGroups",
              "ec2:DescribeSpotInstanceRequests",
              "ec2:DescribeSpotPriceHistory",
              "ec2:DescribeSubnets",
              "ec2:DescribeVolumes",
              "ec2:DescribeVpcs",
              "ec2:DetachInternetGateway",
              "ec2:DisassociateIamInstanceProfile",
              "ec2:ModifyVpcAttribute",
              "ec2:ReplaceIamInstanceProfileAssociation",
              "ec2:RequestSpotInstances",
              "ec2:RevokeSecurityGroupEgress",
              "ec2:RevokeSecurityGroupIngress",
              "ec2:RunInstances",
              "ec2:TerminateInstances",
              "ec2:CreatePlacementGroup",
              "ec2:DeletePlacementGroup",
              "ec2:DescribePlacementGroups"
          ],
          "Resource": [
              "*"
          ]
          },
          {
          "Effect": "Allow",
          "Action": [
              "iam:CreateServiceLinkedRole",
              "iam:PutRolePolicy"
          ],
          "Resource": "arn:aws:iam::*:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot",
          "Condition": {
              "StringLike": {
              "iam:AWSServiceName": "spot.amazonaws.com"
              }
          }
          }
      ]
      }
      
    4. Click Review policy.

    5. In the Name field, enter a policy name.

    6. Click Create policy.

    7. If you use Service Control Policies to deny certain actions at the AWS account level, ensure that sts:AssumeRole is whitelisted so Databricks can assume the cross-account role.

  6. In the role summary, copy the Role ARN.

    Role ARN