Library
A library to install on a cluster. Exactly one of the library-type fields must be set.
Library object
A library to install on a cluster. Exactly one of the library-type fields must be set.
- jarstring
URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "s3://my-bucket/library.jar" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.AzureURI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "abfss://my-bucket/library.jar" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.GCPURI of the jar library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "gs://my-bucket/library.jar" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.
- eggstringDeprecated
Deprecated: Installing Python egg files is not supported in Databricks Runtime 14.0 and above. Use a wheel (
whl) or a PyPI package (pypi) instead. The value is the URI of the egg file to install.
- pypiobject
Specification of a PyPI library to be installed. For example:
{ "package": "simplejson" }Show child attributesHide child attributes
- packagestring
The name of the PyPI package to install. An optional exact version specification is also supported. Examples: "simplejson" and "simplejson==3.8.0".
- repostring
The repository where the package can be found. If not specified, the default pip index is used.
- mavenobject
Specification of a maven library to be installed. For example:
{ "coordinates": "org.jsoup:jsoup:1.7.2" }Show child attributesHide child attributes
- coordinatesstring
Gradle-style maven coordinates. For example: "org.jsoup:jsoup:1.7.2".
- repostring
Maven repo to install the Maven package from. If omitted, both Maven Central Repository and Spark Packages are searched.
- exclusionsarray of string
List of dependencies to exclude. For example:
["slf4j:slf4j", "*:hadoop-client"].Maven dependency exclusions: https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html.
- cranobject
Specification of a CRAN library to be installed.
Show child attributesHide child attributes
- packagestring
The name of the CRAN package to install.
- repostring
The repository where the package can be found. If not specified, the default CRAN repo is used.
- whlstring
URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "s3://my-bucket/library.whl" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.AzureURI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "abfss://my-bucket/library.whl" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.GCPURI of the wheel library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "gs://my-bucket/library.whl" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.
- requirementsstring
URI of the requirements.txt file to install. Only Workspace paths and Unity Catalog Volumes paths are supported. For example:
{ "requirements": "/Workspace/path/to/requirements.txt" }or{ "requirements" : "/Volumes/path/to/requirements.txt" }. Installing a requirements file requires Databricks Runtime 15.0 or above.
Get status GA
GET
Get the status of libraries on a cluster. A status is returned for all libraries installed on this cluster via the API or the libraries UI. The order of returned libraries is as follows:
- Libraries set to be installed on this cluster, in the order that the libraries were added to the cluster, are returned first.
- Libraries that were previously requested to be installed on this cluster but are now marked for removal, in no particular order, are returned last.
API scopes: libraries
Parameters
- cluster_idstringcluster_idRequiredquery
Unique identifier of the cluster whose status should be retrieved.
Response
- cluster_idstring
Unique identifier for the cluster.
- library_statusesarray of object
Status of all libraries on the cluster.
Show child attributesHide child attributes
- libraryobject
The library whose status is reported.
Show child attributesHide child attributes
- jarstring
URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "s3://my-bucket/library.jar" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.AzureURI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "abfss://my-bucket/library.jar" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.GCPURI of the jar library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "gs://my-bucket/library.jar" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.
- eggstringDeprecated
Deprecated: Installing Python egg files is not supported in Databricks Runtime 14.0 and above. Use a wheel (
whl) or a PyPI package (pypi) instead. The value is the URI of the egg file to install.
- pypiobject
Specification of a PyPI library to be installed. For example:
{ "package": "simplejson" }Show child attributesHide child attributes
- packagestringRequired
The name of the PyPI package to install. An optional exact version specification is also supported. Examples: "simplejson" and "simplejson==3.8.0".
- repostring
The repository where the package can be found. If not specified, the default pip index is used.
- mavenobject
Specification of a maven library to be installed. For example:
{ "coordinates": "org.jsoup:jsoup:1.7.2" }Show child attributesHide child attributes
- coordinatesstringRequired
Gradle-style maven coordinates. For example: "org.jsoup:jsoup:1.7.2".
- repostring
Maven repo to install the Maven package from. If omitted, both Maven Central Repository and Spark Packages are searched.
- exclusionsarray of string
List of dependencies to exclude. For example:
["slf4j:slf4j", "*:hadoop-client"].Maven dependency exclusions: https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html.
- cranobject
Specification of a CRAN library to be installed.
Show child attributesHide child attributes
- packagestringRequired
The name of the CRAN package to install.
- repostring
The repository where the package can be found. If not specified, the default CRAN repo is used.
- whlstring
URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "s3://my-bucket/library.whl" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.AzureURI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "abfss://my-bucket/library.whl" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.GCPURI of the wheel library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "gs://my-bucket/library.whl" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.
- requirementsstring
URI of the requirements.txt file to install. Only Workspace paths and Unity Catalog Volumes paths are supported. For example:
{ "requirements": "/Workspace/path/to/requirements.txt" }or{ "requirements" : "/Volumes/path/to/requirements.txt" }. Installing a requirements file requires Databricks Runtime 15.0 or above.
- statusstring
Status of installing the library on the cluster. The status starts at PENDING and ends at INSTALLED, FAILED or SKIPPED. Intermediate states are not always reported. A library stays UNINSTALL_ON_RESTART after an uninstall request until the cluster restarts. While the cluster is not running, libraries are reported as PENDING, and libraries marked for uninstall are omitted.
PENDINGRESOLVINGINSTALLINGINSTALLEDFAILEDUNINSTALL_ON_RESTARTSKIPPEDRESTORED
- messagesarray of string
All the info and warning messages that have occurred so far for this library.
- is_library_for_all_clustersboolean
Whether the library was set to be installed on all clusters via the libraries UI.
Add a library GA
POST
Add libraries to install on a cluster. The installation is asynchronous; it happens in the background after the completion of this request.
API scopes: libraries
Request body
- cluster_idstringcluster_idRequired
Unique identifier for the cluster on which to install these libraries.
- librariesarray of objectRequired
The libraries to install.
Show child attributesHide child attributes
- jarstring
URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "s3://my-bucket/library.jar" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.AzureURI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "abfss://my-bucket/library.jar" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.GCPURI of the jar library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "gs://my-bucket/library.jar" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.
- eggstringDeprecated
Deprecated: Installing Python egg files is not supported in Databricks Runtime 14.0 and above. Use a wheel (
whl) or a PyPI package (pypi) instead. The value is the URI of the egg file to install.
- pypiobject
Specification of a PyPI library to be installed. For example:
{ "package": "simplejson" }Show child attributesHide child attributes
- packagestringRequired
The name of the PyPI package to install. An optional exact version specification is also supported. Examples: "simplejson" and "simplejson==3.8.0".
- repostring
The repository where the package can be found. If not specified, the default pip index is used.
- mavenobject
Specification of a maven library to be installed. For example:
{ "coordinates": "org.jsoup:jsoup:1.7.2" }Show child attributesHide child attributes
- coordinatesstringRequired
Gradle-style maven coordinates. For example: "org.jsoup:jsoup:1.7.2".
- repostring
Maven repo to install the Maven package from. If omitted, both Maven Central Repository and Spark Packages are searched.
- exclusionsarray of string
List of dependencies to exclude. For example:
["slf4j:slf4j", "*:hadoop-client"].Maven dependency exclusions: https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html.
- cranobject
Specification of a CRAN library to be installed.
Show child attributesHide child attributes
- packagestringRequired
The name of the CRAN package to install.
- repostring
The repository where the package can be found. If not specified, the default CRAN repo is used.
- whlstring
URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "s3://my-bucket/library.whl" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.AzureURI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "abfss://my-bucket/library.whl" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.GCPURI of the wheel library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "gs://my-bucket/library.whl" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.
- requirementsstring
URI of the requirements.txt file to install. Only Workspace paths and Unity Catalog Volumes paths are supported. For example:
{ "requirements": "/Workspace/path/to/requirements.txt" }or{ "requirements" : "/Volumes/path/to/requirements.txt" }. Installing a requirements file requires Databricks Runtime 15.0 or above.
Uninstall libraries GA
POST
Set libraries to uninstall from a cluster. The libraries won't be uninstalled until the cluster is restarted. A request to uninstall a library that is not currently installed is ignored.
API scopes: libraries
Request body
- cluster_idstringcluster_idRequired
Unique identifier for the cluster on which to uninstall these libraries.
- librariesarray of objectRequired
The libraries to uninstall.
Show child attributesHide child attributes
- jarstring
URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "s3://my-bucket/library.jar" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.AzureURI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "abfss://my-bucket/library.jar" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.GCPURI of the jar library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "gs://my-bucket/library.jar" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.
- eggstringDeprecated
Deprecated: Installing Python egg files is not supported in Databricks Runtime 14.0 and above. Use a wheel (
whl) or a PyPI package (pypi) instead. The value is the URI of the egg file to install.
- pypiobject
Specification of a PyPI library to be installed. For example:
{ "package": "simplejson" }Show child attributesHide child attributes
- packagestringRequired
The name of the PyPI package to install. An optional exact version specification is also supported. Examples: "simplejson" and "simplejson==3.8.0".
- repostring
The repository where the package can be found. If not specified, the default pip index is used.
- mavenobject
Specification of a maven library to be installed. For example:
{ "coordinates": "org.jsoup:jsoup:1.7.2" }Show child attributesHide child attributes
- coordinatesstringRequired
Gradle-style maven coordinates. For example: "org.jsoup:jsoup:1.7.2".
- repostring
Maven repo to install the Maven package from. If omitted, both Maven Central Repository and Spark Packages are searched.
- exclusionsarray of string
List of dependencies to exclude. For example:
["slf4j:slf4j", "*:hadoop-client"].Maven dependency exclusions: https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html.
- cranobject
Specification of a CRAN library to be installed.
Show child attributesHide child attributes
- packagestringRequired
The name of the CRAN package to install.
- repostring
The repository where the package can be found. If not specified, the default CRAN repo is used.
- whlstring
URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "s3://my-bucket/library.whl" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.AzureURI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "abfss://my-bucket/library.whl" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.GCPURI of the wheel library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "gs://my-bucket/library.whl" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.
- requirementsstring
URI of the requirements.txt file to install. Only Workspace paths and Unity Catalog Volumes paths are supported. For example:
{ "requirements": "/Workspace/path/to/requirements.txt" }or{ "requirements" : "/Volumes/path/to/requirements.txt" }. Installing a requirements file requires Databricks Runtime 15.0 or above.
Get all statuses GA
GET
Get the status of all libraries on all clusters in the workspace. A status is returned for all libraries installed on each cluster via the API or the libraries UI. If the workspace has more library statuses than the response limit allows, the request fails; call the per-cluster status method for each cluster instead.
API scopes: libraries
Response
- statusesarray of object
A list of cluster statuses.
Show child attributesHide child attributes
- cluster_idstring
Unique identifier for the cluster.
- library_statusesarray of object
Status of all libraries on the cluster.
Show child attributesHide child attributes
- libraryobject
The library whose status is reported.
Show child attributesHide child attributes
- jarstring
URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "s3://my-bucket/library.jar" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.AzureURI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "abfss://my-bucket/library.jar" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.GCPURI of the jar library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example:
{ "jar": "/Workspace/path/to/library.jar" },{ "jar" : "/Volumes/path/to/library.jar" }or{ "jar": "gs://my-bucket/library.jar" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.
- eggstringDeprecated
Deprecated: Installing Python egg files is not supported in Databricks Runtime 14.0 and above. Use a wheel (
whl) or a PyPI package (pypi) instead. The value is the URI of the egg file to install.
- pypiobject
Specification of a PyPI library to be installed. For example:
{ "package": "simplejson" }Show child attributesHide child attributes
- packagestringRequired
The name of the PyPI package to install. An optional exact version specification is also supported. Examples: "simplejson" and "simplejson==3.8.0".
- repostring
The repository where the package can be found. If not specified, the default pip index is used.
- mavenobject
Specification of a maven library to be installed. For example:
{ "coordinates": "org.jsoup:jsoup:1.7.2" }Show child attributesHide child attributes
- coordinatesstringRequired
Gradle-style maven coordinates. For example: "org.jsoup:jsoup:1.7.2".
- repostring
Maven repo to install the Maven package from. If omitted, both Maven Central Repository and Spark Packages are searched.
- exclusionsarray of string
List of dependencies to exclude. For example:
["slf4j:slf4j", "*:hadoop-client"].Maven dependency exclusions: https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html.
- cranobject
Specification of a CRAN library to be installed.
Show child attributesHide child attributes
- packagestringRequired
The name of the CRAN package to install.
- repostring
The repository where the package can be found. If not specified, the default CRAN repo is used.
- whlstring
URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "s3://my-bucket/library.whl" }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.AzureURI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and ADLS URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "abfss://my-bucket/library.whl" }. If ADLS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with a Microsoft Entra ID service principal to access the ADLS URI.GCPURI of the wheel library to be installed. Supported URIs include Workspace path, UC Volumes path, and GCS URIs. For example:
{ "whl": "/Workspace/path/to/library.whl" },{ "whl" : "/Volumes/path/to/library.whl" }or{ "whl": "gs://my-bucket/library.whl" }. If GCS is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM service account to access the GCS URI.
- requirementsstring
URI of the requirements.txt file to install. Only Workspace paths and Unity Catalog Volumes paths are supported. For example:
{ "requirements": "/Workspace/path/to/requirements.txt" }or{ "requirements" : "/Volumes/path/to/requirements.txt" }. Installing a requirements file requires Databricks Runtime 15.0 or above.
- statusstring
Status of installing the library on the cluster. The status starts at PENDING and ends at INSTALLED, FAILED or SKIPPED. Intermediate states are not always reported. A library stays UNINSTALL_ON_RESTART after an uninstall request until the cluster restarts. While the cluster is not running, libraries are reported as PENDING, and libraries marked for uninstall are omitted.
PENDINGRESOLVINGINSTALLINGINSTALLEDFAILEDUNINSTALL_ON_RESTARTSKIPPEDRESTORED
- messagesarray of string
All the info and warning messages that have occurred so far for this library.
- is_library_for_all_clustersboolean
Whether the library was set to be installed on all clusters via the libraries UI.