Cluster access control

Note

Access control is available only in the Premium plan (or, for customers who subscribed to Databricks before March 3, 2020, the Operational Security package).

By default, all users can create and modify clusters unless an administrator enables cluster access control. With cluster access control, permissions determine a user’s abilities. This article describes the permissions.

Before you can use cluster access control, a Databricks admin must enable it for the workspace. See Enable cluster access control for your workspace.

Types of permissions

You can configure two types of cluster permissions:

• The Allow unrestricted cluster creation entitlement controls your ability to create clusters.

• Cluster-level permissions control your ability to use and modify a specific cluster.

When cluster access control is enabled:

• An administrator can configure whether a user can create clusters.

• Any user with Can Manage permission for a cluster can configure whether a user can attach to, restart, resize, and manage that cluster.

Cluster-level permissions

There are four permission levels for a cluster: No Permissions, Can Attach To, Can Restart, and Can Manage. The table lists the abilities for each permission.

Important

Users with Can Attach To permissions can view the service account keys in the log4j file. Use caution when granting this permission level.

Ability	No Permissions	Can Attach To	Can Restart	Can Manage
Attach notebook to cluster		x	x	x
View Spark UI		x	x	x
View cluster metrics		x	x	x
View driver logs		x (see note)	x (see note)	x
Terminate cluster			x	x
Start cluster			x	x
Restart cluster			x	x
Edit cluster				x
Attach library to cluster				x
Resize cluster				x
Modify permissions				x

Note

• Secrets are not redacted from the Spark driver log streams stdout and stderr. To protect secrets that might appear in those driver log streams such that only users with the Can Manage permission on the cluster can view them, set the cluster’s Spark configuration property spark.databricks.acl.needAdminPermissionToViewLogs true.

• You have Can Manage permission for any cluster that you create.

Configure cluster-level permissions

Note

This section describes how to manage permissions using the UI. You can also use the Permissions API 2.0.

Cluster access control must be enabled and you must have Can Manage permission for the cluster.

1. Click Compute in the sidebar.

2. Click the name of the cluster you want to modify.

3. Click Permissions at the top of the page.

4. In the Permission settings for <cluster name> dialog, you can:

   • Select users and groups from the Add Users and Groups drop-down and assign permission levels for them.

   • Update cluster permissions for users and groups that have already been added, using the drop-down menu beside a user or group name.

   

5. Click Done.

Example: using cluster-level permissions to enforce cluster configurations

One benefit of cluster access control is the ability to enforce cluster configurations so that users cannot change them.

For example, configurations that admins might want to enforce include:

• Tags to charge back costs

• IAM roles to control access to data

• Spot instances to save costs

• Standard libraries

Databricks recommends the following workflow for organizations that need to lock down cluster configurations:

1. Disable Allow unrestricted cluster creation for all users.

   

   Note

   This entitlement cannot be removed from admin users.

2. After you create all of the cluster configurations that you want your users to use, give the users who need access to a given cluster Can Restart permission. This allows a user to freely start and stop the cluster without having to set up all of the configurations manually.

   

Terraform integration

You can manage permissions in a fully automated setup using Databricks Terraform provider and databricks_permissions:

resource "databricks_group" "auto" {
  display_name = "Automation"
}

resource "databricks_group" "eng" {
  display_name = "Engineering"
}

resource "databricks_group" "ds" {
  display_name = "Data Science"
}

data "databricks_spark_version" "latest" {}

data "databricks_node_type" "smallest" {
  local_disk = true
}

resource "databricks_cluster" "shared_autoscaling" {
  cluster_name            = "Shared Autoscaling"
  spark_version           = data.databricks_spark_version.latest.id
  node_type_id            = data.databricks_node_type.smallest.id
  autotermination_minutes = 60
  autoscale {
    min_workers = 1
    max_workers = 10
  }
}

resource "databricks_permissions" "cluster_usage" {
  cluster_id = databricks_cluster.shared_autoscaling.cluster_id

  access_control {
    group_name       = databricks_group.auto.display_name
    permission_level = "CAN_ATTACH_TO"
  }

  access_control {
    group_name       = databricks_group.eng.display_name
    permission_level = "CAN_RESTART"
  }

  access_control {
    group_name       = databricks_group.ds.display_name
    permission_level = "CAN_MANAGE"
  }
}