Create an IAM role for workspace deployment

This article describes how to:

Create and configure a cross-account IAM role for Databricks workspace deployment. This role gives Databricks limited access to your AWS account for the purposes of creating and managing compute and VPC resources.
Use the Databricks account console to create a credential configuration that references the IAM role.

Requirements

You need to be a Databricks account admin.

Automate IAM role creation

You can automate the IAM role creation by using one of the following automation options:

The AWS Quick Start (CloudFormation) to deploy your workspace. This is the recommended workspace deployment method.
The Databricks Terraform provider. See Create Databricks workspaces using Terraform.

Manual IAM role creation

The following steps apply to a custom AWS workspace deployment. You only need to follow these steps if you are deploying a workspace using the Custom AWS configuration option.

Step 1: Create a cross-account IAM role
Step 2: Create an access policy
Step 3: Create a credential configuration for the role in Databricks

Step 1: Create a cross-account IAM role

Get your Databricks account ID. See Locate your account ID.
Log into your AWS Console as a user with administrator privileges and go to the IAM console.
Click the Roles tab in the sidebar.
Click Create role.
1. In Select type of trusted entity, click the AWS account tile.
2. Select the Another AWS account checkbox.
3. In the Account ID field, enter the Databricks account ID 414351767826. This is not the Account ID you copied from the Databricks account console. If you are are using Databricks on AWS GovCloud use the Databricks account ID 044793339203.
4. Select the Require external ID checkbox.
5. In the External ID field, enter your Databricks account ID, which you copied from the Databricks account console.
6. Click the Next button.
7. In the Add Permissions page, click the Next button. You should now be on the Name, review, and create page.
8. In the Role name field, enter a role name.
9. Click Create role. The list of roles appears.

Step 2: Create an access policy

The access policy you add to the role depends on your Amazon VPC (Virtual Private Cloud) deployment type. For information about how Databricks uses each permission, see IAM permissions for Databricks-managed VPCs. Use the policy instructions that describe your deployment:

Option 1: Default. A single VPC that Databricks creates and configures in your AWS account. This is the default configuration.
Option 2: Customer-managed VPC with default restrictions. Create your Databricks workspaces in your own VPC, using a feature known as customer-managed VPC.
Option 3: Customer-managed VPC with custom restrictions. Create your Databricks workspaces in your own VPC with custom restrictions for account ID, VPC ID, AWS Region, and security group.

Option 1: Default deployment policy

In the Roles section of the IAM console, click the IAM role you created in Step 1.
Click the Add permissions drop-down and select Create inline policy.
In the policy editor, click the JSON tab.

Copy and paste the following access policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1403287045000",
      "Effect": "Allow",
      "Action": [
        "ec2:AllocateAddress",
        "ec2:AssignPrivateIpAddresses",
        "ec2:AssociateDhcpOptions",
        "ec2:AssociateIamInstanceProfile",
        "ec2:AssociateRouteTable",
        "ec2:AttachInternetGateway",
        "ec2:AttachVolume",
        "ec2:AuthorizeSecurityGroupEgress",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:CancelSpotInstanceRequests",
        "ec2:CreateDhcpOptions",
        "ec2:CreateFleet",
        "ec2:CreateInternetGateway",
        "ec2:CreateLaunchTemplate",
        "ec2:CreateLaunchTemplateVersion",
        "ec2:CreateNatGateway",
        "ec2:CreateRoute",
        "ec2:CreateRouteTable",
        "ec2:CreateSecurityGroup",
        "ec2:CreateSubnet",
        "ec2:CreateTags",
        "ec2:CreateVolume",
        "ec2:CreateVpc",
        "ec2:CreateVpcEndpoint",
        "ec2:DeleteDhcpOptions",
        "ec2:DeleteFleets",
        "ec2:DeleteInternetGateway",
        "ec2:DeleteLaunchTemplate",
        "ec2:DeleteLaunchTemplateVersions",
        "ec2:DeleteNatGateway",
        "ec2:DeleteRoute",
        "ec2:DeleteRouteTable",
        "ec2:DeleteSecurityGroup",
        "ec2:DeleteSubnet",
        "ec2:DeleteTags",
        "ec2:DeleteVolume",
        "ec2:DeleteVpc",
        "ec2:DeleteVpcEndpoints",
        "ec2:DescribeAvailabilityZones",
        "ec2:DescribeFleetHistory",
        "ec2:DescribeFleetInstances",
        "ec2:DescribeFleets",
        "ec2:DescribeIamInstanceProfileAssociations",
        "ec2:DescribeInstanceStatus",
        "ec2:DescribeInstances",
        "ec2:DescribeInternetGateways",
        "ec2:DescribeLaunchTemplates",
        "ec2:DescribeLaunchTemplateVersions",
        "ec2:DescribeNatGateways",
        "ec2:DescribePrefixLists",
        "ec2:DescribeReservedInstancesOfferings",
        "ec2:DescribeRouteTables",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSpotInstanceRequests",
        "ec2:DescribeSpotPriceHistory",
        "ec2:DescribeSubnets",
        "ec2:DescribeVolumes",
        "ec2:DescribeVpcs",
        "ec2:DetachInternetGateway",
        "ec2:DisassociateIamInstanceProfile",
        "ec2:DisassociateRouteTable",
        "ec2:GetLaunchTemplateData",
        "ec2:GetSpotPlacementScores",
        "ec2:ModifyFleet",
        "ec2:ModifyLaunchTemplate",
        "ec2:ModifyVpcAttribute",
        "ec2:ReleaseAddress",
        "ec2:ReplaceIamInstanceProfileAssociation",
        "ec2:RequestSpotInstances",
        "ec2:RevokeSecurityGroupEgress",
        "ec2:RevokeSecurityGroupIngress",
        "ec2:RunInstances",
        "ec2:TerminateInstances"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "iam:CreateServiceLinkedRole",
        "iam:PutRolePolicy"
      ],
      "Resource": "arn:aws:iam::*:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot",
      "Condition": {
        "StringLike": {
          "iam:AWSServiceName": "spot.amazonaws.com"
        }
      }
    }
  ]
}

Click Review policy.
In the Name field, enter a policy name.
Click Create policy.
(Optional) If you use Service Control Policies to deny certain actions at the AWS account level, ensure that sts:AssumeRole is allowlisted so Databricks can assume the cross-account role.
In the role summary, copy the Role ARN to add to Databricks.

Option 2: Customer-managed VPC with default restrictions policy

Log into your AWS Console as a user with administrator privileges and go to the IAM console.
Click the Roles tab in the sidebar.
In the list of roles, click the cross-account IAM role that you created in Step 1.
Click the Add permissions drop-down and select Create inline policy.
In the policy editor, click the JSON tab.

Copy and paste the following access policy.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1403287045000",
      "Effect": "Allow",
      "Action": [
        "ec2:AssociateIamInstanceProfile",
        "ec2:AttachVolume",
        "ec2:AuthorizeSecurityGroupEgress",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:CancelSpotInstanceRequests",
        "ec2:CreateTags",
        "ec2:CreateVolume",
        "ec2:DeleteTags",
        "ec2:DeleteVolume",
        "ec2:DescribeAvailabilityZones",
        "ec2:DescribeIamInstanceProfileAssociations",
        "ec2:DescribeInstanceStatus",
        "ec2:DescribeInstances",
        "ec2:DescribeInternetGateways",
        "ec2:DescribeNatGateways",
        "ec2:DescribeNetworkAcls",
        "ec2:DescribePrefixLists",
        "ec2:DescribeReservedInstancesOfferings",
        "ec2:DescribeRouteTables",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSpotInstanceRequests",
        "ec2:DescribeSpotPriceHistory",
        "ec2:DescribeSubnets",
        "ec2:DescribeVolumes",
        "ec2:DescribeVpcAttribute",
        "ec2:DescribeVpcs",
        "ec2:DetachVolume",
        "ec2:DisassociateIamInstanceProfile",
        "ec2:ReplaceIamInstanceProfileAssociation",
        "ec2:RequestSpotInstances",
        "ec2:RevokeSecurityGroupEgress",
        "ec2:RevokeSecurityGroupIngress",
        "ec2:RunInstances",
        "ec2:TerminateInstances",
        "ec2:DescribeFleetHistory",
        "ec2:ModifyFleet",
        "ec2:DeleteFleets",
        "ec2:DescribeFleetInstances",
        "ec2:DescribeFleets",
        "ec2:CreateFleet",
        "ec2:DeleteLaunchTemplate",
        "ec2:GetLaunchTemplateData",
        "ec2:CreateLaunchTemplate",
        "ec2:DescribeLaunchTemplates",
        "ec2:DescribeLaunchTemplateVersions",
        "ec2:ModifyLaunchTemplate",
        "ec2:DeleteLaunchTemplateVersions",
        "ec2:CreateLaunchTemplateVersion",
        "ec2:AssignPrivateIpAddresses",
        "ec2:GetSpotPlacementScores"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "iam:CreateServiceLinkedRole",
        "iam:PutRolePolicy"
      ],
      "Resource": "arn:aws:iam::*:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot",
      "Condition": {
        "StringLike": {
          "iam:AWSServiceName": "spot.amazonaws.com"
        }
      }
    }
  ]
}

Click Review policy.
In the Name field, enter a policy name.
Click Create policy.
(Optional) If you use Service Control Policies to deny certain actions at the AWS account level, ensure that sts:AssumeRole is allowlisted so Databricks can assume the cross-account role.
In the role summary, copy the Role ARN.

Option 3: Customer-managed VPC with custom policy restrictions

Note

The Databricks production AWS account from which Amazon Machine Images (AMI) are sourced is 601306020600. You can use this account ID to create custom access policies that restrict which AMIs can be used within your AWS account. For more information, contact your Databricks account team.

Log into your AWS Console as a user with administrator privileges and go to the IAM console.
Click the Roles tab in the sidebar.
In the list of roles, click the cross-account IAM role that you created for in Step 1.
Click the Add permissions dropdown then Create inline policy.
In the policy editor, click the JSON tab.

Copy and paste the following access policy.

Replace the following values in the policy with your own configuration values:

ACCOUNTID — Your AWS account ID, which is a number.
VPCID — ID of the AWS VPC where you want to launch workspaces.
REGION — AWS Region name for your VPC deployment, for example us-west-2.
SECURITYGROUPID — ID of your AWS security group. When you add a security group restriction, you cannot reuse the cross-account IAM role or reference a credentials ID (credentials_id) for any other workspaces. For those other workspaces, you must create separate roles, policies, and credentials objects.

Note

If you have custom requirements configured for security groups with your customer-managed vpc, contact your Databricks account team for assistance with IAM policy customizations.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "NonResourceBasedPermissions",
      "Effect": "Allow",
      "Action": [
        "ec2:AssignPrivateIpAddresses",
        "ec2:CancelSpotInstanceRequests",
        "ec2:DescribeAvailabilityZones",
        "ec2:DescribeIamInstanceProfileAssociations",
        "ec2:DescribeInstanceStatus",
        "ec2:DescribeInstances",
        "ec2:DescribeInternetGateways",
        "ec2:DescribeNatGateways",
        "ec2:DescribeNetworkAcls",
        "ec2:DescribePrefixLists",
        "ec2:DescribeReservedInstancesOfferings",
        "ec2:DescribeRouteTables",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSpotInstanceRequests",
        "ec2:DescribeSpotPriceHistory",
        "ec2:DescribeSubnets",
        "ec2:DescribeVolumes",
        "ec2:DescribeVpcAttribute",
        "ec2:DescribeVpcs",
        "ec2:CreateTags",
        "ec2:DeleteTags",
        "ec2:GetSpotPlacementScores",
        "ec2:RequestSpotInstances",
        "ec2:DescribeFleetHistory",
        "ec2:ModifyFleet",
        "ec2:DeleteFleets",
        "ec2:DescribeFleetInstances",
        "ec2:DescribeFleets",
        "ec2:CreateFleet",
        "ec2:DeleteLaunchTemplate",
        "ec2:GetLaunchTemplateData",
        "ec2:CreateLaunchTemplate",
        "ec2:DescribeLaunchTemplates",
        "ec2:DescribeLaunchTemplateVersions",
        "ec2:ModifyLaunchTemplate",
        "ec2:DeleteLaunchTemplateVersions",
        "ec2:CreateLaunchTemplateVersion"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "InstancePoolsSupport",
      "Effect": "Allow",
      "Action": [
        "ec2:AssociateIamInstanceProfile",
        "ec2:DisassociateIamInstanceProfile",
        "ec2:ReplaceIamInstanceProfileAssociation"
      ],
      "Resource": "arn:aws:ec2:REGION:ACCOUNTID:instance/*",
      "Condition": {
        "StringEquals": {
          "ec2:ResourceTag/Vendor": "Databricks"
        }
      }
    },
    {
      "Sid": "AllowEc2RunInstancePerTag",
      "Effect": "Allow",
      "Action": "ec2:RunInstances",
      "Resource": [
        "arn:aws:ec2:REGION:ACCOUNTID:volume/*",
        "arn:aws:ec2:REGION:ACCOUNTID:instance/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/Vendor": "Databricks"
        }
      }
    },
    {
      "Sid": "AllowEc2RunInstanceImagePerTag",
      "Effect": "Allow",
      "Action": "ec2:RunInstances",
      "Resource": [
        "arn:aws:ec2:REGION:ACCOUNTID:image/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/Vendor": "Databricks"
        }
      }
    },
    {
      "Sid": "AllowEc2RunInstancePerVPCid",
      "Effect": "Allow",
      "Action": "ec2:RunInstances",
      "Resource": [
        "arn:aws:ec2:REGION:ACCOUNTID:network-interface/*",
        "arn:aws:ec2:REGION:ACCOUNTID:subnet/*",
        "arn:aws:ec2:REGION:ACCOUNTID:security-group/*"
      ],
      "Condition": {
        "StringEquals": {
          "ec2:vpc": "arn:aws:ec2:REGION:ACCOUNTID:vpc/VPCID"
        }
      }
    },
    {
      "Sid": "AllowEc2RunInstanceOtherResources",
      "Effect": "Allow",
      "Action": "ec2:RunInstances",
      "NotResource": [
        "arn:aws:ec2:REGION:ACCOUNTID:image/*",
        "arn:aws:ec2:REGION:ACCOUNTID:network-interface/*",
        "arn:aws:ec2:REGION:ACCOUNTID:subnet/*",
        "arn:aws:ec2:REGION:ACCOUNTID:security-group/*",
        "arn:aws:ec2:REGION:ACCOUNTID:volume/*",
        "arn:aws:ec2:REGION:ACCOUNTID:instance/*"
      ]
    },
    {
      "Sid": "EC2TerminateInstancesTag",
      "Effect": "Allow",
      "Action": [
        "ec2:TerminateInstances"
      ],
      "Resource": [
        "arn:aws:ec2:REGION:ACCOUNTID:instance/*"
      ],
      "Condition": {
        "StringEquals": {
          "ec2:ResourceTag/Vendor": "Databricks"
        }
      }
    },
    {
      "Sid": "EC2AttachDetachVolumeTag",
      "Effect": "Allow",
      "Action": [
        "ec2:AttachVolume",
        "ec2:DetachVolume"
      ],
      "Resource": [
        "arn:aws:ec2:REGION:ACCOUNTID:instance/*",
        "arn:aws:ec2:REGION:ACCOUNTID:volume/*"
      ],
      "Condition": {
        "StringEquals": {
          "ec2:ResourceTag/Vendor": "Databricks"
        }
      }
    },
    {
      "Sid": "EC2CreateVolumeByTag",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateVolume"
      ],
      "Resource": [
        "arn:aws:ec2:REGION:ACCOUNTID:volume/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/Vendor": "Databricks"
        }
      }
    },
    {
      "Sid": "EC2DeleteVolumeByTag",
      "Effect": "Allow",
      "Action": [
        "ec2:DeleteVolume"
      ],
      "Resource": [
        "arn:aws:ec2:REGION:ACCOUNTID:volume/*"
      ],
      "Condition": {
        "StringEquals": {
          "ec2:ResourceTag/Vendor": "Databricks"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": [
        "iam:CreateServiceLinkedRole",
        "iam:PutRolePolicy"
      ],
      "Resource": "arn:aws:iam::*:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot",
      "Condition": {
        "StringLike": {
          "iam:AWSServiceName": "spot.amazonaws.com"
        }
      }
    },
    {
      "Sid": "VpcNonresourceSpecificActions",
      "Effect": "Allow",
      "Action": [
        "ec2:AuthorizeSecurityGroupEgress",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:RevokeSecurityGroupEgress",
        "ec2:RevokeSecurityGroupIngress"
      ],
      "Resource": "arn:aws:ec2:REGION:ACCOUNTID:security-group/SECURITYGROUPID",
      "Condition": {
        "StringEquals": {
          "ec2:vpc": "arn:aws:ec2:REGION:ACCOUNTID:vpc/VPCID"
        }
      }
    }
  ]
}

Click Review policy.
In the Name field, enter a policy name.
Click Create policy.
(Optional) If you use Service Control Policies to deny certain actions at the AWS account level, ensure that sts:AssumeRole is allowlisted so Databricks can assume the cross-account role.
In the role summary, copy the Role ARN.

Step 3: Create a credential configuration in Databricks

When you have created the IAM role, you can tell Databricks about it by creating a credential configuration that uses that role’s ID.

To create a credential configuration:

In the account console, click Cloud resources.
Click Credential configuration.
Click Add credential configuration.
In the Credential configuration name field, enter a human-readable name for your new credential configuration.
In the Role ARN field, enter your role’s ARN.
Click Add.

Validation is not run during credential configuration creation. Some errors are detected only when you use the configuration to create a new workspace. These errors can include an invalid ARN or incorrect permissions for the role, among others.

Delete a credential configuration

Credential configurations cannot be edited after creation. If the configuration has incorrect data or if you no longer need it, delete the credential configuration:

In the account console, click Cloud resources.
Click Credential configuration.
On the credential configuration row, click the Actions menu icon, and select Delete.

You can also click the credential configuration name and click Delete on the pop-up dialog.
On the confirmation dialog, click Confirm Delete.