Libraries CLI (legacy)
Important
This documentation has been retired and might not be updated.
This information applies to legacy Databricks CLI versions 0.18 and below. Databricks recommends that you use newer Databricks CLI version 0.205 or above instead. See What is the Databricks CLI?. To find your version of the Databricks CLI, run databricks -v
.
To migrate from Databricks CLI version 0.18 or below to Databricks CLI version 0.205 or above, see Databricks CLI migration.
You run Databricks libraries CLI subcommands by appending them to databricks libraries
. These subcommands call the Libraries API.
databricks libraries -h
Usage: databricks libraries [OPTIONS] COMMAND [ARGS]...
Utility to interact with libraries.
Options:
-v, --version [VERSION]
-h, --help Show this message and exit.
Commands:
all-cluster-statuses Get the status of all libraries.
cluster-status Get the status of all libraries for a cluster.
Options:
--cluster-id CLUSTER_ID Can be found in the URL at https://<databricks-instance>/?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration.
install Install a library on a cluster.
Options:
--cluster-id CLUSTER_ID Can be found in the URL at https://<databricks-instance>/?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration.
--jar TEXT JAR on DBFS or S3 or WASB.
--egg TEXT Egg on DBFS or S3 or WASB.
--whl TEXT Wheel or zipped wheelhouse on DBFS or S3 or WASB. Supported in CLI 0.8.2 and above.
--maven-coordinates TEXT Maven coordinates in the form of GroupId:ArtifactId:Version (i.e.org.jsoup:jsoup:1.7.2).
--maven-repo TEXT Maven repository to install the Maven package from. If omitted, both Maven Repository and Spark Packages are searched.
--maven-exclusion TEXT List of dependences to exclude. For example: --maven-exclusion "slf4j:slf4j" --maven-exclusion "*:hadoop-client".
--pypi-package TEXT The name of the PyPI package to install. An optional exact version specification is also supported. Examples "simplejson" and "simplejson==3.8.0".
--pypi-repo TEXT The repository where the package can be found. If not specified, the default pip index is used.
--cran-package TEXT The name of the CRAN package to install.
--cran-repo TEXT The repository where the package can be found. If not specified, the default CRAN repo is used.
list Shortcut to `all-cluster-statuses` or `cluster-status`.
Options:
--cluster-id CLUSTER_ID Can be found in the URL at https://<databricks-instance>/?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration.
uninstall Uninstall a library on a cluster.
Options:
--cluster-id CLUSTER_ID Can be found in the URL at https://<databricks-instance>/?o=<16-digit-number>#/setting/clusters/$CLUSTER_ID/configuration. [required]
--all Uninstall all libraries.
--jar TEXT JAR on DBFS or S3 or WASB.
--egg TEXT Egg on DBFS or S3 or WASB.
--whl TEXT Wheel or zipped wheelhouse on DBFS or S3 or WASB. Supported in CLI 0.8.2 and above.
--maven-coordinates TEXT Maven coordinates in the form of GroupId:ArtifactId:Version (i.e.org.jsoup:jsoup:1.7.2).
--maven-repo TEXT Maven repository to install the Maven package from. If omitted, both Maven Repository and Spark Packages are searched.
--maven-exclusion TEXT List of dependences to exclude. For example: --maven-exclusion "slf4j:slf4j" --maven-exclusion "*:hadoop-client".
--pypi-package TEXT The name of the PyPI package to install. An optional exact version specification is also supported. Examples "simplejson" and "simplejson==3.8.0".
--pypi-repo TEXT The repository where the package can be found. If not specified, the default pip index is used.
--cran-package TEXT The name of the CRAN package to install.
--cran-repo TEXT The repository where the package can be found. If not specified, the default CRAN repo is used.
List the status of all libraries on all clusters
To display usage documentation, run databricks libraries all-cluster-statuses --help
.
databricks libraries all-cluster-statuses
{
"statuses": [
{
"cluster_id": "1234-567890-lest123",
"library_statuses": [
{
"library": {
"jar": "dbfs:/FileStore/jars/bbf81650_a62b_4b7a_b47e_7bdd9505792a-SparkJDBC42.jar"
},
"status": "INSTALLED",
"is_library_for_all_clusters": true
},
...
]
},
...
]
}
List the status of all libraries on a cluster
To display usage documentation, run databricks libraries cluster-status --help
or databricks libraries list --help
.
databricks libraries cluster-status --cluster-id 1234-567890-lest123
Or:
databricks libraries list --cluster-id 1234-567890-lest123
{
"cluster_id": "1234-567890-lest123",
"library_statuses": [
{
"library": {
"jar": "dbfs:/FileStore/jars/bbf81650_a62b_4b7a_b47e_7bdd9505792a-SparkJDBC42.jar"
},
"status": "INSTALLED",
"is_library_for_all_clusters": false
},
...
]
}
Install a library on a cluster
To display usage documentation, run databricks libraries install --help
.
databricks libraries install --cluster-id 1234-567890-lest123 --jar dbfs:/test-dir/test.jar
If successful, no output is displayed.
Uninstall a library from a cluster
To display usage documentation, run databricks libraries uninstall --help
.
databricks libraries uninstall --cluster-id 1234-567890-lest123 --jar dbfs:/test-dir/test.jar
WARNING: Uninstalling libraries requires a cluster restart.
databricks clusters restart --cluster-id 1234-567890-lest123