Skip to main content

File

View as Markdown

The Files API is a standard HTTP API that allows you to read, write, list, and delete files and directories by referring to their URI. The API makes working with file content as raw bytes easier and more efficient.

The API supports Unity Catalog volumes, where files and directories to operate on are specified using their volume URI path, which follows the format /Volumes/<catalog_name>/<schema_name>/<volume_name>/<path_to_file>.

The Files API has two distinct endpoints, one for working with files (/fs/files) and another one for working with directories (/fs/directories). Both endpoints use the standard HTTP methods GET, HEAD, PUT, and DELETE to manage files and directories specified using their URI path. The path is always absolute.

Use of Files API may incur Databricks data transfer charges.

Get Public Preview

HEAD /api/2.0/fs/files{file_path}

Get the metadata of a file. The response HTTP headers contain the metadata. There is no response body.

API scopes: files

Path parameters

file_pathstringRequiredpath

The absolute path of the file.

Examples:

  • /Volumes/my-catalog/my-schema/my-volume/directory/file.txt : A file

Header parameters

Rangestring

The range of bytes to retrieve. The range is inclusive and zero-based, see RFC 9110 for further details.

Examples:

  • bytes=0-499 : Retrieve the first 500 bytes
  • bytes=500- : Retrieve the 501st byte onwards
If-Unmodified-Sincestring

Download the file only if it has not been modified since the specified timestamp. If it has, a 412 Precondition Failed error will be returned. See RFC 9110 for further details.

Examples:

  • Wed, 24 Jan 2024 14:10:56 GMT : The time that the file must be unmodified since

Responses

200

Request completed successfully.

Response headers
content-lengthint64

The length of the HTTP response body in bytes.

content-typestring

Example: "application/octet-stream"

last-modifiedHTTP-date

Example: "Wed, 24 Jan 2024 14:10:56 GMT"

The last modified time of the file in HTTP-date (RFC 7231) format.

206

Request successful. Partial content returned.

Response headers
content-lengthint64

The length of the HTTP response body in bytes.

content-rangestring

Example: "bytes 0-499/500"

The position of the content of the response body within the complete file. Further information about this header is available in RFC 9110.

content-typestring

Example: "application/octet-stream"

last-modifiedHTTP-date

Example: "Wed, 24 Jan 2024 14:10:56 GMT"

The last modified time of the file in HTTP-date (RFC 7231) format.

Create Public Preview

PUT /api/2.0/fs/files{file_path}

Uploads a file of up to 5 GiB. The file contents should be sent as the request body as raw bytes (an octet stream); do not encode or otherwise modify the bytes before sending. The contents of the resulting file will be exactly the bytes sent in the request body. If the request is successful, there is no response body.

API scopes: files

Path parameters

file_pathstringRequiredpath

The absolute path of the file.

Examples:

  • /Volumes/my-catalog/my-schema/my-volume/directory/file.txt : A file

Query parameters

overwritebooleanquery

If true or unspecified, an existing file will be overwritten. If false, an error will be returned if the path points to an existing file.

Responses

204

Request successful, but no content returned.

Delete Public Preview

DELETE /api/2.0/fs/files{file_path}

Deletes a file. If the request is successful, there is no response body.

API scopes: files

Path parameters

file_pathstringRequiredpath

The absolute path of the file.

Examples:

  • /Volumes/my-catalog/my-schema/my-volume/directory/file.txt : A file

Responses

204

Request successful, but no content returned.

Create Directory Public Preview

PUT /api/2.0/fs/directories{directory_path}

Creates an empty directory. If necessary, also creates any parent directories of the new, empty directory (like the shell command mkdir -p). If called on an existing directory, returns a success response; this method is idempotent (it will succeed if the directory already exists).

API scopes: files

Path parameters

directory_pathstringRequiredpath

The absolute path of a directory.

Examples:

  • /Volumes/my-catalog/my-schema/my-volume/directory/ : A directory

Responses

204

Request successful, but no content returned.

Delete Directory Public Preview

DELETE /api/2.0/fs/directories{directory_path}

Deletes an empty directory.

To delete a non-empty directory, first delete all of its contents. This can be done by listing the directory contents and deleting each file and subdirectory recursively.

API scopes: files

Path parameters

directory_pathstringRequiredpath

The absolute path of a directory.

Examples:

  • /Volumes/my-catalog/my-schema/my-volume/directory/ : A directory

Responses

204

Request successful, but no content returned.

Download File Public Preview

GET /api/2.0/fs/files{file_path}

Downloads a file. The file contents are the response body. This is a standard HTTP file download, not a JSON RPC. It supports the Range and If-Unmodified-Since HTTP headers.

API scopes: files

Path parameters

file_pathstringRequiredpath

The absolute path of the file.

Examples:

  • /Volumes/my-catalog/my-schema/my-volume/directory/file.txt : A file

Header parameters

Rangestring

The range of bytes to retrieve. The range is inclusive and zero-based, see RFC 9110 for further details.

Examples:

  • bytes=0-499 : Retrieve the first 500 bytes
  • bytes=500- : Retrieve the 501st byte onwards
If-Unmodified-Sincestring

Download the file only if it has not been modified since the specified timestamp. If it has, a 412 Precondition Failed error will be returned. See RFC 9110 for further details.

Examples:

  • Wed, 24 Jan 2024 14:10:56 GMT : The time that the file must be unmodified since

Responses

200

Request completed successfully.

Response headers
content-lengthint64

The length of the HTTP response body in bytes.

content-typestring

Example: "application/octet-stream"

last-modifiedHTTP-date

Example: "Wed, 24 Jan 2024 14:10:56 GMT"

The last modified time of the file in HTTP-date (RFC 7231) format.

206

Request successful. Partial content returned.

Response headers
content-lengthint64

The length of the HTTP response body in bytes.

content-rangestring

Example: "bytes 0-499/500"

The position of the content of the response body within the complete file. Further information about this header is available in RFC 9110.

content-typestring

Example: "application/octet-stream"

last-modifiedHTTP-date

Example: "Wed, 24 Jan 2024 14:10:56 GMT"

The last modified time of the file in HTTP-date (RFC 7231) format.

Get Directory Metadata Public Preview

HEAD /api/2.0/fs/directories{directory_path}

Get the metadata of a directory. The response HTTP headers contain the metadata. There is no response body.

This method is useful to check if a directory exists and the caller has access to it.

If you wish to ensure the directory exists, you can instead use PUT, which will create the directory if it does not exist, and is idempotent (it will succeed if the directory already exists).

API scopes: files

Path parameters

directory_pathstringRequiredpath

The absolute path of a directory.

Examples:

  • /Volumes/my-catalog/my-schema/my-volume/directory/ : A directory

Responses

200

Request completed successfully.

List Directory Contents Public Preview

GET /api/2.0/fs/directories{directory_path}

Returns the contents of a directory. If there is no directory at the specified path, the API returns an HTTP 404 error.

API scopes: files

Path parameters

directory_pathstringRequiredpath

The absolute path of a directory.

Examples:

  • /Volumes/my-catalog/my-schema/my-volume/directory/ : A directory

Query parameters

page_sizeint64query

The maximum number of directory entries to return. The response may contain fewer entries. If the response contains a next_page_token, there may be more entries, even if fewer than page_size entries are in the response.

We recommend not to set this value unless you are intentionally listing less than the complete directory contents.

If unspecified, at most 1000 directory entries will be returned. The maximum value is 1000. Values above 1000 will be coerced to 1000.

Default: 1000

Examples:

  • page_size=0 : Passing zero implies the maximum page size
  • page_size=100 : User defined page size
page_tokenstringquery

An opaque page token which was the next_page_token in the response of the previous request to list the contents of this directory. Provide this token to retrieve the next page of directory entries. When providing a page_token, all other parameters provided to the request must match the previous request. To list all of the entries in a directory, it is necessary to continue requesting pages of entries until the response contains no next_page_token. Note that the number of entries returned must not be used to determine when the listing is complete.

Response

contentsarray of object

Array of DirectoryEntry.

Show child attributesHide child attributes
file_sizeint64

The length of the file in bytes. This field is omitted for directories.

is_directoryboolean

True if the path is a directory.

last_modifiedint64

Last modification time of given file in milliseconds since unix epoch.

namestring

The name of the file or directory. This is the last component of the path.

pathstring

The absolute path of the file or directory.

next_page_tokenstring

A token, which can be sent as page_token to retrieve the next page.

Responses

200

Request completed successfully.

The response includes a list of items and pagination information. If next_page_token is set, there are more results.