You run Databricks DBFS CLI commands appending them to
databricks fs (or the alias
dbfs), prefixing all DBFS paths with
databricks fs -h
Usage: databricks fs [OPTIONS] COMMAND [ARGS]... Utility to interact with DBFS. DBFS paths are all prefixed with dbfs:/. Local paths can be absolute or local. Options: -v, --version -h, --help Show this message and exit. Commands: configure cp Copies files to and from DBFS. Options: -r, --recursive --overwrite Overwrites files that exist already. ls Lists files in DBFS. Options: --absolute Displays absolute paths. -l Displays full information including size and file type. mkdirs Makes directories in DBFS. mv Moves a file between two DBFS paths. rm Removes files from dbfs. Options: -r, --recursive
For operations that delete more than 10k files, we discourage using the DBFS CLI, but advise you to perform such operations in the context of a cluster, using File system utilities.
dbutils.fs covers the functional scope of the DBFS REST API, but from notebooks. Running such operations using notebooks provides better control and manageability, such as selective deletes, and the possibility to automate periodic delete jobs.
dbfs cp test.txt dbfs:/test.txt # Or recursively dbfs cp -r test-dir dbfs:/test-dir