fs
command group
Note
This information applies to Databricks CLI versions 0.205 and above, which are in Public Preview. To find your version of the Databricks CLI, run databricks -v
.
The fs
command group within the Databricks CLI allows you to automate volumes in Unity Catalog and to automate Databricks File System (DBFS) objects.
You run fs
commands by appending them to databricks fs
. To display help for the fs
command, run databricks fs -h
.
fs
commands require volume paths to begin with dbfs:/Volumes
and require directory and file paths in DBFS to begin with dbfs:/
.
Important
To install the Databricks CLI, see Install or update the Databricks CLI. To configure authentication for the Databricks CLI, see Authentication for the Databricks CLI.
List the contents of a directory
To list the contents of a directory in a volume or in DBFS, use the ls
command. Specify the path to the directory. The following examples list the names of the objects found in the specified volume’s root or in the DBFS root:
databricks fs ls dbfs:/Volumes/main/default/my-volume
databricks fs ls dbfs:/
To display full information including object types, sizes, modification times since Epoch in milliseconds, as well as the objects’ names, use the --long
or -l
option. The following examples list the full information of the objects found in the specified volume’s root or in a tmp
directory within the DBFS root:
databricks fs ls dbfs:/Volumes/main/default/my-volume -l
databricks fs ls dbfs:/tmp -l
To display full object paths, use the --absolute
option. The following examples list the full information of the objects, and the objects’ full paths, found in the specified volume’s root or in a tmp
directory within the DBFS root:
databricks fs ls dbfs:/Volumes/main/default/my-volume -l --absolute
databricks fs ls dbfs:/tmp -l --absolute
Output the contents of a file
To output the contents of a file, use the cat
command. Specify the path to the file in DBFS. The following examples output the contents of the file named babynames.csv
found in the specified volume’s root or in a tmp
directory within the DBFS root:
databricks fs cat dbfs:/Volumes/main/default/my-volume/babynames.csv
databricks fs cat dbfs:/tmp/babynames.csv
The following examples return errors, as they try to output the contents of a directory instead of a file:
databricks fs cat dbfs:/Volumes/main/default/my-volume
databricks fs cat dbfs:/tmp
Create a directory
To create a directory, use the mkdir
command. Specify the path to the directory to be created in a volume or in DBFS. If the directory already exists, nothing happens. The following examples create a directory named squirrel-data
within the specified volume’s root or in a directory named tmp
within the DBFS root:
databricks fs mkdir dbfs:/Volumes/main/default/my-volume/squirrel-data
databricks fs mkdir dbfs:/tmp/squirrel-data
Copy a directory or a file
To copy a directory or a file, use the cp
command. Specify the paths to the source directory or file to copy and its destination. You can copy directories and files between the local filesystem and DBFS, and you can copy directories and files between DBFS paths. To overwrite existing files, use the --overwrite
option. To recursively copy files within a directory, use the --recursive
or -r
option.
The cp
command assumes file:/
, if file:/
is omitted.
The following examples copy a directory named squirrel-data
and its contents, from a local filesystem path to a squirrels
directory within the specified volume’s root or the DBFS root.
databricks fs cp /Users/<username>/squirrel-data dbfs:/Volumes/main/default/my-volume/squirrels -r
databricks fs cp /Users/<username>/squirrel-data dbfs:/squirrels -r
The following example copies a file named squirrels.csv
from a local filesystem path to a directory named squirrel-data
within the specified volume’s root or the DBFS root. If the file already exists in the destination, it is overwritten.
databricks fs cp /Users/<username>/squirrels.csv dbfs:/Volumes/main/default/my-volume/squirrel-data --overwrite
databricks fs cp /Users/<username>/squirrels.csv dbfs:/squirrel-data --overwrite
Remove a directory
To remove a directory, use the rm
command. Specify the path to the existing directory in DBFS. The following examples remove a directory named squirrel-data
from the specified volume’s root or from a tmp
directory in the DBFS root:
databricks fs rm dbfs:/Volumes/main/default/my-volume/squirrel-data
databricks fs rm dbfs:/tmp/squirrel-data
If the directory does not exist, nothing happens, and no error is returned.
If the directory exists but is not empty, an error is returned. To remove a non-empty directory and all of its contents, use the --recursive
or -r
option. The following examples remove a non-empty directory named squirrel-data
from the specified volume’s root or from a tmp
directory in the DBFS root:
databricks fs rm dbfs:/Volumes/main/default/my-volume/squirrel-data -r
databricks fs rm dbfs:/tmp/squirrel-data -r