# List Cluster Compliance For Policy Launch stage: GA `GET /api/2.0/policies/clusters/list-compliance` Returns the policy compliance status of all clusters that use a given policy. Clusters could be out of compliance if their policy was updated after the cluster was last edited. API scopes: clusters ## Query parameters - `policy_id` (string, optional) Canonical unique identifier for the cluster policy. - `page_token` (string, optional) A page token that can be used to navigate to the next page or previous page as returned by `next_page_token` or `prev_page_token`. - `page_size` (int32, optional) Use this field to specify the maximum number of results to be returned by the server. The server may further constrain the maximum number of results returned in a single page. Default: `20` Constraints: `[ 1 .. 100 ]` ## Returns - `clusters` (array of object, optional) A list of clusters and their policy compliance statuses. - `cluster_id` (string, optional) Canonical unique identifier for a cluster. - `is_compliant` (boolean, optional) Whether this cluster is in compliance with the latest version of its policy. - `violations` (object, optional) An object containing key-value mappings representing the first 200 policy validation errors. The keys indicate the path where the policy validation error is occurring. The values indicate an error message describing the policy validation error. - `pending_enforcement` (object, optional) Information about the pending enforcement for the cluster. Only present if a pending enforcement is scheduled for the cluster. - `target_spec` (object, optional) The new configuration to apply upon cluster termination or restart. - `cluster_name` (string, optional) Cluster name requested by the user. This doesn't have to be unique. If not specified at creation, the cluster name will be an empty string. For job clusters, the cluster name is automatically set based on the job and job run IDs. - `spark_version` (string, optional) The Spark version of the cluster, e.g. `3.3.x-scala2.11`. A list of available Spark versions can be retrieved by using the [clusters/sparkVersions](https://docs.databricks.com/api/workspace/clusters/sparkversions) API call. - `spark_conf` (object, optional) An object containing a set of optional, user-specified Spark configuration key-value pairs. Users can also pass in a string of extra JVM options to the driver and the executors via `spark.driver.extraJavaOptions` and `spark.executor.extraJavaOptions` respectively. - `aws_attributes` (object, optional) Attributes related to clusters running on Amazon Web Services. If not specified at cluster creation, a set of default values will be used. - `azure_attributes` (object, optional) Attributes related to clusters running on Microsoft Azure. If not specified at cluster creation, a set of default values will be used. - `gcp_attributes` (object, optional) Attributes related to clusters running on Google Cloud Platform. If not specified at cluster creation, a set of default values will be used. - `node_type_id` (string, optional) This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads. A list of available node types can be retrieved by using the [clusters/listNodeTypes](https://docs.databricks.com/api/workspace/clusters/listnodetypes) API call. - `driver_node_type_id` (string, optional) The node type of the Spark driver. Note that this field is optional; if unset, the driver node type will be set as the same value as `node_type_id` defined above. This field, along with node_type_id, should not be set if virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and virtual_cluster_size are specified, driver_node_type_id and node_type_id take precedence. - `worker_node_type_flexibility` (object, optional) Flexible node type configuration for worker nodes. - `driver_node_type_flexibility` (object, optional) Flexible node type configuration for the driver node. - `ssh_public_keys` (array of string, optional) SSH public key contents that will be added to each Spark node in this cluster. The corresponding private keys can be used to login with the user name `ubuntu` on port `2200`. Up to 10 keys can be specified. - `custom_tags` (object, optional) Additional tags for cluster resources. will tag all cluster resources (e.g., AWS instances and EBS volumes) with these tags in addition to `default_tags`. Notes: - Currently, allows at most 45 custom tags - Clusters can only reuse cloud resources if the resources' tags are a subset of the cluster tags - `cluster_log_conf` (object, optional) The configuration for delivering spark logs to a long-term storage destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes) are supported. Only one destination can be specified for one cluster. If the conf is given, the logs will be delivered to the destination every `5 mins`. The destination of driver logs is `$destination/$clusterId/driver`, while the destination of executor logs is `$destination/$clusterId/executor`. - `spark_env_vars` (object, optional) An object containing a set of optional, user-specified environment variable key-value pairs. Please note that key-value pair of the form (X,Y) will be exported as is (i.e., `export X='Y'`) while launching the driver and workers. In order to specify an additional set of `SPARK_DAEMON_JAVA_OPTS`, we recommend appending them to `$SPARK_DAEMON_JAVA_OPTS` as shown in the example below. This ensures that all default databricks managed environmental variables are included as well. Example Spark environment variables: `{"SPARK_WORKER_MEMORY": "28000m", "SPARK_LOCAL_DIRS": "/local_disk0"}` or `{"SPARK_DAEMON_JAVA_OPTS": "$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true"}` - `autotermination_minutes` (int32, optional) Automatically terminates the cluster after it is inactive for this time in minutes. If not set, this cluster will not be automatically terminated. If specified, the threshold must be between 10 and 10000 minutes. Users can also set this value to 0 to explicitly disable automatic termination. - `enable_elastic_disk` (boolean, optional) Autoscaling Local Storage: when enabled, this cluster will dynamically acquire additional disk space when its Spark workers are running low on disk space. **AWS:** Autoscaling Local Storage: when enabled, this cluster will dynamically acquire additional disk space when its Spark workers are running low on disk space. This feature requires specific AWS permissions to function correctly - refer to the User Guide for more details. - `init_scripts` (array of object, optional) The configuration for storing init scripts. Any number of destinations can be specified. The scripts are executed sequentially in the order provided. If `cluster_log_conf` is specified, init script logs are sent to `//init_scripts`. - `docker_image` (object, optional) Custom docker image BYOC - `instance_pool_id` (string, optional) The optional ID of the instance pool to which the cluster belongs. - `single_user_name` (string, optional) Single user name if data_security_mode is `SINGLE_USER` - `policy_id` (string, optional) The ID of the cluster policy used to create the cluster if applicable. - `enable_local_disk_encryption` (boolean, optional) Whether to enable LUKS on cluster VMs' local disks - `driver_instance_pool_id` (string, optional) The optional ID of the instance pool for the driver of the cluster belongs. The pool cluster uses the instance pool with id (instance_pool_id) if the driver pool is not assigned. - `workload_type` (object, optional) - `data_security_mode` (string, optional) Possible values: `NONE`, `SINGLE_USER`, `USER_ISOLATION`, `LEGACY_TABLE_ACL`, `LEGACY_PASSTHROUGH`, `LEGACY_SINGLE_USER`, `LEGACY_SINGLE_USER_STANDARD`, `DATA_SECURITY_MODE_STANDARD`, `DATA_SECURITY_MODE_DEDICATED`, `DATA_SECURITY_MODE_AUTO` - `runtime_engine` (string, optional) Determines the cluster's runtime engine, either standard or Photon. This field is not compatible with legacy `spark_version` values that contain `-photon-`. Remove `-photon-` from the `spark_version` and set `runtime_engine` to `PHOTON`. If left unspecified, the runtime engine defaults to standard unless the spark_version contains -photon-, in which case Photon will be used. Possible values: `NULL`, `STANDARD`, `PHOTON` - `kind` (string, optional) Possible values: `COMPUTE_KIND_UNSPECIFIED`, `CLASSIC_PREVIEW` - `use_ml_runtime` (boolean, optional) This field can only be used when `kind = CLASSIC_PREVIEW`. `effective_spark_version` is determined by `spark_version` (DBR release), this field `use_ml_runtime`, and whether `node_type_id` is gpu node or not. - `is_single_node` (boolean, optional) This field can only be used when `kind = CLASSIC_PREVIEW`. When set to true, will automatically set single node related `custom_tags`, `spark_conf`, and `num_workers` - `remote_disk_throughput` (int32, optional) If set, what the configurable throughput (in Mb/s) for the remote disk is. Currently only supported for GCP HYPERDISK_BALANCED disks. - `total_initial_remote_disk_size` (int32, optional) If set, what the total initial volume size (in GB) of the remote disks should be. Supported for GCP. - `dependency_mode` (string, optional, Beta) Controls dependency configuration for the cluster. Possible values: `DEPENDENCY_MODE_UNSPECIFIED`, `DEPENDENCY_MODE_ENVIRONMENTS`, `DEPENDENCY_MODE_CLUSTER_LIBRARIES`, `DEPENDENCY_MODE_AUTO` - `num_workers` (int32, required) Number of worker nodes that this cluster should have. A cluster has one Spark Driver and `num_workers` Executors for a total of `num_workers` + 1 Spark nodes. Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in `spark_info` will gradually increase from 5 to 10 as the new nodes are provisioned. - `autoscale` (object, required) Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later. - `initiate_time` (string, optional) The time the pending enforcement was initiated. - `enforcement_status` (string, optional) Whether the pending enforcement will be applied. A pending enforcement begins in `ACTIVE` state. If the enforcement fails to apply too many times, the state transitions to `INACTIVE`. Afterwards, the enforcement must be re-scheduled to become `ACTIVE` again. Possible values: `ENFORCEMENT_STATUS_UNSPECIFIED`, `ACTIVE`, `INACTIVE` - `target_changes` (array of object, optional) A list of changes that will be made to the cluster configuration when the pending enforcement is applied. - `field` (string, optional) The field where this change would be made. Example: `spark_version` - `previous_value` (string, optional) The previous value of this field before enforcing policy compliance (either a number, a boolean, or a string) converted to a string. This is intended to be read by a human. The type of the field can be retrieved by reading the settings field in the API response. Example: `10.4.x-scala2.12` - `new_value` (string, optional) The new value of this field after enforcing policy compliance (either a number, a boolean, or a string) converted to a string. This is intended to be read by a human. The typed new value of this field can be retrieved by reading the settings field in the API response. Example: `14.3.x-scala2.12` - `initiator_user` (string, optional) The user who initiated the pending enforcement. - `next_page_token` (string, optional) This field represents the pagination token to retrieve the next page of results. If the value is "", it means no further results for the request. - `prev_page_token` (string, optional) This field represents the pagination token to retrieve the previous page of results. If the value is "", it means no further results for the request. ## Response ```json { "clusters": [ { "cluster_id": "string", "is_compliant": true, "violations": { "custom_tags.test_tag": "Validation failed for custom_tags.test_tag, the value cannot be present" }, "pending_enforcement": {} } ], "next_page_token": "string", "prev_page_token": "string" } ```