Policy

View as Markdown

Describes a Cluster Policy entity.

Policy object

Describes a Cluster Policy entity.

policy_idstring: Canonical unique identifier for the Cluster Policy.

creator_user_namestring: Creator user name. The field won't be included in the response if the user has already been deleted.

created_at_timestampint64: Creation time. The timestamp (in millisecond) when this Cluster Policy was created.

is_defaultboolean: If true, policy is a default policy created and managed by <Databricks>. Default policies cannot be deleted, and their policy families cannot be changed.

namestring: Cluster Policy name requested by the user. This has to be unique. Length must be between 1 and 100 characters.

definitionstring: Policy definition document expressed in Databricks Cluster Policy Definition Language.

descriptionstring: Additional human-readable description of the cluster policy.
Constraints: <= 1000 characters

policy_family_idstring

ID of the policy family. The cluster policy's policy definition inherits the policy family's policy definition.

Cannot be used with definition. Use policy_family_definition_overrides instead to customize the policy definition.

policy_family_definition_overridesstring

Policy definition JSON document expressed in Databricks Policy Definition Language. The JSON document must be passed as a string and cannot be embedded in the requests.

You can use this to customize the policy definition inherited from the policy family. Policy rules specified here are merged into the inherited policy definition.

max_clusters_per_userint64: Max number of clusters per user that can be active using this policy. If not present, there is no max limit.
Constraints: >= 1

librariesarray of object

A list of libraries to be installed on the next cluster restart that uses this policy. The maximum number of libraries is 500.

Show child attributesHide child attributes

jarstring: URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example: { "jar": "/Workspace/path/to/library.jar" }, { "jar" : "/Volumes/path/to/library.jar" } or { "jar": "s3://my-bucket/library.jar" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.
Azure
URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example: { "jar": "/Workspace/path/to/library.jar" }, { "jar" : "/Volumes/path/to/library.jar" } or { "jar": "abfss://my-bucket/library.jar" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.
GCP
URI of the jar library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example: { "jar": "/Workspace/path/to/library.jar" }, { "jar" : "/Volumes/path/to/library.jar" } or { "jar": "gs://my-bucket/library.jar" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.

eggstring: Deprecated. URI of the egg library to install. Installing Python egg files is deprecated and is not supported in Databricks Runtime 14.0 and above.

pypiobject

Specification of a PyPi library to be installed. For example: { "package": "simplejson" }

Show child attributesHide child attributes

packagestring: The name of the pypi package to install. An optional exact version specification is also supported. Examples: "simplejson" and "simplejson==3.8.0".

repostring: The repository where the package can be found. If not specified, the default pip index is used.

mavenobject

Specification of a maven library to be installed. For example: { "coordinates": "org.jsoup:jsoup:1.7.2" }

Show child attributesHide child attributes

coordinatesstring: Gradle-style maven coordinates. For example: "org.jsoup:jsoup:1.7.2".

repostring: Maven repo to install the Maven package from. If omitted, both Maven Central Repository and Spark Packages are searched.

exclusionsarray of string

List of dependences to exclude. For example: ["slf4j:slf4j", "*:hadoop-client"].

Maven dependency exclusions: https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html.

cranobject

Specification of a CRAN library to be installed as part of the library

Show child attributesHide child attributes

packagestring: The name of the CRAN package to install.

repostring: The repository where the package can be found. If not specified, the default CRAN repo is used.

whlstring: URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example: { "whl": "/Workspace/path/to/library.whl" }, { "whl" : "/Volumes/path/to/library.whl" } or { "whl": "s3://my-bucket/library.whl" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.
Azure
URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example: { "whl": "/Workspace/path/to/library.whl" }, { "whl" : "/Volumes/path/to/library.whl" } or { "whl": "abfss://my-bucket/library.whl" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.
GCP
URI of the wheel library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example: { "whl": "/Workspace/path/to/library.whl" }, { "whl" : "/Volumes/path/to/library.whl" } or { "whl": "gs://my-bucket/library.egg" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.

requirementsstring: URI of the requirements.txt file to install. Only Workspace paths and Unity Catalog Volumes paths are supported. For example: { "requirements": "/Workspace/path/to/requirements.txt" } or { "requirements" : "/Volumes/path/to/requirements.txt" }

View as Markdown

The Policy object

{
  "policy_id": "string",
  "creator_user_name": "string",
  "created_at_timestamp": 0,
  "is_default": true,
  "name": "string",
  "definition": {
    "custom_tags.test_tag": {
      "type": "fixed",
      "value": "test_value"
    }
  },
  "description": "string",
  "policy_family_id": "string",
  "policy_family_definition_overrides": {
    "custom_tags.test_tag": {
      "type": "fixed",
      "value": "test_value"
    }
  },
  "max_clusters_per_user": 0,
  "libraries": [
    {
      "jar": "string",
      "egg": "string",
      "pypi": {},
      "maven": {},
      "cran": {},
      "whl": "string",
      "requirements": "string"
    }
  ]
}

Get

GET /api/2.0/policies/clusters/get

Get a cluster policy entity. Creation and editing is available to admins only.

API scopes: clusters

Parameters

policy_idstringquery: Canonical unique identifier for the Cluster Policy.

Response

Returns the Policy object.

View as Markdown

Response

{
  "policy_id": "string",
  "creator_user_name": "string",
  "created_at_timestamp": 0,
  "is_default": true,
  "name": "string",
  "definition": {
    "custom_tags.test_tag": {
      "type": "fixed",
      "value": "test_value"
    }
  },
  "description": "string",
  "policy_family_id": "string",
  "policy_family_definition_overrides": {
    "custom_tags.test_tag": {
      "type": "fixed",
      "value": "test_value"
    }
  },
  "max_clusters_per_user": 0,
  "libraries": [
    {
      "jar": "string",
      "egg": "string",
      "pypi": {},
      "maven": {},
      "cran": {},
      "whl": "string",
      "requirements": "string"
    }
  ]
}

List

GET /api/2.0/policies/clusters/list

Returns a list of policies accessible by the requesting user.

API scopes: clusters

Parameters

sort_orderstringquery

The order in which the policies get listed.

DESC - Sort result list in descending order.
ASC - Sort result list in ascending order.

Default: DESC

Values: DESC, ASC

sort_columnstringquery

The cluster policy attribute to sort by.

POLICY_CREATION_TIME - Sort result list by policy creation time.
POLICY_NAME - Sort result list by policy name.

Values: POLICY_CREATION_TIME, POLICY_NAME

Response

Returns a list of Policy objects.

View as Markdown

Response

{
  "policies": [
    {
      "policy_id": "string",
      "creator_user_name": "string",
      "created_at_timestamp": 0,
      "is_default": true,
      "name": "string",
      "definition": {
        "custom_tags.test_tag": {
          "type": "fixed",
          "value": "test_value"
        }
      },
      "description": "string",
      "policy_family_id": "string",
      "policy_family_definition_overrides": {
        "custom_tags.test_tag": {
          "type": "fixed",
          "value": "test_value"
        }
      },
      "max_clusters_per_user": 0,
      "libraries": [
        {}
      ]
    }
  ]
}

Create

POST /api/2.0/policies/clusters/create

Creates a new policy with prescribed settings.

API scopes: clusters

Request body

namestring: Cluster Policy name requested by the user. This has to be unique. Length must be between 1 and 100 characters.

definitionstring: Policy definition document expressed in Databricks Cluster Policy Definition Language.

descriptionstring: Additional human-readable description of the cluster policy.
Constraints: <= 1000 characters

policy_family_idstring

ID of the policy family. The cluster policy's policy definition inherits the policy family's policy definition.

Cannot be used with definition. Use policy_family_definition_overrides instead to customize the policy definition.

policy_family_definition_overridesstring

Policy definition JSON document expressed in Databricks Policy Definition Language. The JSON document must be passed as a string and cannot be embedded in the requests.

You can use this to customize the policy definition inherited from the policy family. Policy rules specified here are merged into the inherited policy definition.

max_clusters_per_userint64: Max number of clusters per user that can be active using this policy. If not present, there is no max limit.
Constraints: >= 1

librariesarray of object

A list of libraries to be installed on the next cluster restart that uses this policy. The maximum number of libraries is 500.

Show child attributesHide child attributes

jarstringRequired: URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example: { "jar": "/Workspace/path/to/library.jar" }, { "jar" : "/Volumes/path/to/library.jar" } or { "jar": "s3://my-bucket/library.jar" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.
Azure
URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example: { "jar": "/Workspace/path/to/library.jar" }, { "jar" : "/Volumes/path/to/library.jar" } or { "jar": "abfss://my-bucket/library.jar" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.
GCP
URI of the jar library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example: { "jar": "/Workspace/path/to/library.jar" }, { "jar" : "/Volumes/path/to/library.jar" } or { "jar": "gs://my-bucket/library.jar" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.

eggstringRequired: Deprecated. URI of the egg library to install. Installing Python egg files is deprecated and is not supported in Databricks Runtime 14.0 and above.

pypiobjectRequired

Specification of a PyPi library to be installed. For example: { "package": "simplejson" }

Show child attributesHide child attributes

packagestring: The name of the pypi package to install. An optional exact version specification is also supported. Examples: "simplejson" and "simplejson==3.8.0".

repostring: The repository where the package can be found. If not specified, the default pip index is used.

mavenobjectRequired

Specification of a maven library to be installed. For example: { "coordinates": "org.jsoup:jsoup:1.7.2" }

Show child attributesHide child attributes

coordinatesstring: Gradle-style maven coordinates. For example: "org.jsoup:jsoup:1.7.2".

repostring: Maven repo to install the Maven package from. If omitted, both Maven Central Repository and Spark Packages are searched.

exclusionsarray of string

List of dependences to exclude. For example: ["slf4j:slf4j", "*:hadoop-client"].

Maven dependency exclusions: https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html.

cranobjectRequired

Specification of a CRAN library to be installed as part of the library

Show child attributesHide child attributes

packagestring: The name of the CRAN package to install.

repostring: The repository where the package can be found. If not specified, the default CRAN repo is used.

whlstringRequired: URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example: { "whl": "/Workspace/path/to/library.whl" }, { "whl" : "/Volumes/path/to/library.whl" } or { "whl": "s3://my-bucket/library.whl" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.
Azure
URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example: { "whl": "/Workspace/path/to/library.whl" }, { "whl" : "/Volumes/path/to/library.whl" } or { "whl": "abfss://my-bucket/library.whl" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.
GCP
URI of the wheel library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example: { "whl": "/Workspace/path/to/library.whl" }, { "whl" : "/Volumes/path/to/library.whl" } or { "whl": "gs://my-bucket/library.egg" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.

requirementsstringRequired: URI of the requirements.txt file to install. Only Workspace paths and Unity Catalog Volumes paths are supported. For example: { "requirements": "/Workspace/path/to/requirements.txt" } or { "requirements" : "/Volumes/path/to/requirements.txt" }

Response

policy_idstring: Canonical unique identifier for the cluster policy.

View as Markdown

Request

{
  "definition": "{ \"custom_tags.test_tag\": { \"type\": \"fixed\", \"value\": \"test_value\" } }",
  "libraries": [
    {
      "jar": "dbfs:/Test/Example.jar"
    }
  ],
  "name": "Test policy"
}

Response

{
  "policy_id": "string"
}

Update

POST /api/2.0/policies/clusters/edit

Update an existing policy for cluster. This operation may make some clusters governed by the previous policy invalid.

API scopes: clusters

Request body

policy_idstring: The ID of the policy to update.

namestring: Cluster Policy name requested by the user. This has to be unique. Length must be between 1 and 100 characters.

definitionstring: Policy definition document expressed in Databricks Cluster Policy Definition Language.

descriptionstring: Additional human-readable description of the cluster policy.
Constraints: <= 1000 characters

policy_family_idstring

ID of the policy family. The cluster policy's policy definition inherits the policy family's policy definition.

Cannot be used with definition. Use policy_family_definition_overrides instead to customize the policy definition.

policy_family_definition_overridesstring

Policy definition JSON document expressed in Databricks Policy Definition Language. The JSON document must be passed as a string and cannot be embedded in the requests.

You can use this to customize the policy definition inherited from the policy family. Policy rules specified here are merged into the inherited policy definition.

max_clusters_per_userint64: Max number of clusters per user that can be active using this policy. If not present, there is no max limit.
Constraints: >= 1

librariesarray of object

A list of libraries to be installed on the next cluster restart that uses this policy. The maximum number of libraries is 500.

Show child attributesHide child attributes

jarstringRequired: URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example: { "jar": "/Workspace/path/to/library.jar" }, { "jar" : "/Volumes/path/to/library.jar" } or { "jar": "s3://my-bucket/library.jar" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.
Azure
URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example: { "jar": "/Workspace/path/to/library.jar" }, { "jar" : "/Volumes/path/to/library.jar" } or { "jar": "abfss://my-bucket/library.jar" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.
GCP
URI of the jar library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example: { "jar": "/Workspace/path/to/library.jar" }, { "jar" : "/Volumes/path/to/library.jar" } or { "jar": "gs://my-bucket/library.jar" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.

eggstringRequired: Deprecated. URI of the egg library to install. Installing Python egg files is deprecated and is not supported in Databricks Runtime 14.0 and above.

pypiobjectRequired

Specification of a PyPi library to be installed. For example: { "package": "simplejson" }

Show child attributesHide child attributes

packagestring: The name of the pypi package to install. An optional exact version specification is also supported. Examples: "simplejson" and "simplejson==3.8.0".

repostring: The repository where the package can be found. If not specified, the default pip index is used.

mavenobjectRequired

Specification of a maven library to be installed. For example: { "coordinates": "org.jsoup:jsoup:1.7.2" }

Show child attributesHide child attributes

coordinatesstring: Gradle-style maven coordinates. For example: "org.jsoup:jsoup:1.7.2".

repostring: Maven repo to install the Maven package from. If omitted, both Maven Central Repository and Spark Packages are searched.

exclusionsarray of string

List of dependences to exclude. For example: ["slf4j:slf4j", "*:hadoop-client"].

Maven dependency exclusions: https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html.

cranobjectRequired

Specification of a CRAN library to be installed as part of the library

Show child attributesHide child attributes

packagestring: The name of the CRAN package to install.

repostring: The repository where the package can be found. If not specified, the default CRAN repo is used.

whlstringRequired: URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example: { "whl": "/Workspace/path/to/library.whl" }, { "whl" : "/Volumes/path/to/library.whl" } or { "whl": "s3://my-bucket/library.whl" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.
Azure
URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example: { "whl": "/Workspace/path/to/library.whl" }, { "whl" : "/Volumes/path/to/library.whl" } or { "whl": "abfss://my-bucket/library.whl" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.
GCP
URI of the wheel library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example: { "whl": "/Workspace/path/to/library.whl" }, { "whl" : "/Volumes/path/to/library.whl" } or { "whl": "gs://my-bucket/library.egg" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.

requirementsstringRequired: URI of the requirements.txt file to install. Only Workspace paths and Unity Catalog Volumes paths are supported. For example: { "requirements": "/Workspace/path/to/requirements.txt" } or { "requirements" : "/Volumes/path/to/requirements.txt" }

View as Markdown

Request

{
  "definition": "{ \"custom_tags.test_tag\": { \"type\": \"fixed\", \"value\": \"test_value\" } }",
  "libraries": [
    {
      "jar": "dbfs:/Test/Example.jar"
    }
  ],
  "name": "Test policy",
  "policy_id": "ABCD000000000000"
}

{
  "name": "Test policy",
  "policy_family_definition_overrides": "{ \"custom_tags.test_tag\": { \"type\": \"fixed\", \"value\": \"test_value\" } }",
  "policy_family_id": "personal-vm",
  "policy_id": "ABCD000000000000"
}

Response

{}

Delete

POST /api/2.0/policies/clusters/delete

Delete a policy for a cluster. Clusters governed by this policy can still run, but cannot be edited.

API scopes: clusters

Request body

policy_idstring: The ID of the policy to delete.

Policy object​

Get​

Parameters​

Response​

List​

Parameters​

Response​

Create​

Request body​

Response​

Update​

Request body​

Delete​

Request body​

Policy object

Get

Parameters

Response

List

Parameters

Response

Create

Request body

Response

Update

Request body

Delete

Request body