Files

View as Markdown

The Files API is a standard HTTP API that allows you to read, write, list, and delete files and directories by referring to their URI. The API makes working with file content as raw bytes easier and more efficient.

The API supports Unity Catalog volumes, where you specify the files and directories to operate on using their volume URI path, which follows the format /Volumes/<catalog_name>/<schema_name>/<volume_name>/<path_to_file>. The path is always absolute.

The Files API has two distinct endpoints: one for working with files (/fs/files) and another for working with directories (/fs/directories). Both endpoints use the standard HTTP methods GET, HEAD, PUT, DELETE, and POST to manage files and directories specified by their URI path.

Files API 3.0 requests must use the /api/3.0/fs prefix and must be sent to the regional hostname of the workspace. Using Files API 3.0 requires network connectivity from your network to Databricks and from Databricks to your storage.

Use of the Files API may incur Databricks data transfer charges.

Upload a filePUT /api/3.0/fs/files{file_path}Get file metadataHEAD /api/3.0/fs/files{file_path}Download a fileGET /api/3.0/fs/files{file_path}Delete a fileDELETE /api/3.0/fs/files{file_path}Perform an action on a filePOST /api/3.0/fs/files{file_path}Create a directoryPUT /api/3.0/fs/directories{directory_path}Get directory metadataHEAD /api/3.0/fs/directories{directory_path}List directory contentsGET /api/3.0/fs/directories{directory_path}Delete a directoryDELETE /api/3.0/fs/directories{directory_path}

Upload a file Beta

PUT /api/3.0/fs/files{file_path}

Uploads a complete file of up to 5000 MiB in a single request, or one part of a file within a multipart upload session. The contents should be sent in the request body as raw bytes (an octet stream). Do not encode or otherwise modify the bytes before sending.

For a single-request upload, which uploads a file in a single operation, omit the multipart query parameters. The resulting file is exactly the bytes sent. To upload one part of a file within a multipart upload session, send upload_id and part_number as query parameters. The response then includes the part's ETag. Store it and pass it to the complete-upload action.

Files larger than 5000 MiB, or files whose total size is unknown at the start of the upload, must be uploaded as a multipart upload: initiate the session with Perform an action on a file (action=initiate-upload), send the parts here with upload_id and part_number, then assemble them with Perform an action on a file (action=complete-upload).

API scopes: files

Parameters

file_pathstringRequiredpath

The absolute path of the file.

overwritebooleanquery

Only applies when performing a single-request upload. If true or unspecified, an existing file is overwritten. If false, returns a 409 FileAlreadyExists error if the path points to an existing file.

upload_idstringquery

Only applies when performing a multipart upload. The upload ID returned by the initiate-upload action. When present together with part_number, this request uploads one part of that upload instead of performing a single-request upload.

part_numberint32query

Only applies when performing a multipart upload. The sequence number of this part, starting at 1. Valid range: [1, 10000]. Parts are assembled in ascending order when the upload is completed.

Request headers

Content-Lengthint64Required

The size of the request body in bytes.

Content-Typestring

Only applies when performing a single-request upload. Optional content type to associate with the file. When provided, the server stores it verbatim and returns it on later GET and HEAD responses. When omitted, the server defaults to application/octet-stream.

If-Matchstring

Only applies when performing a single-request upload. Upload only if the current ETag matches the provided ETag. For this operation only the If-Match: <etag> form is supported. See RFC 9110.

If-None-Matchstring

Only applies when performing a single-request upload. Upload only if the current ETag does not match the provided ETag. For this operation only the If-None-Match: * form is supported, which uploads only if no file already exists at the path. See RFC 9110.

Request body

The file contents, or the contents of one part of a multipart upload, as raw bytes.

Success status code

200 OK

Response headers

ETagstring

The entity tag (ETag) of the uploaded content. For a single-request upload, this is the ETag of the new file. You can send it back in a later If-Match or If-None-Match request. For a multipart upload, it identifies the uploaded part. Store it and send it back in the complete-upload request.

View as Markdown

Get file metadata Beta

HEAD /api/3.0/fs/files{file_path}

Get the metadata of a file. The response headers contain the metadata, including ETag and Last-Modified. There is no response body. Supports the Range header and the If-Match, If-None-Match, If-Modified-Since, and If-Unmodified-Since conditional headers with the same semantics as Download a file.

API scopes: files

Parameters

file_pathstringRequiredpath

The absolute path of the file.

Request headers

Rangestring

The range of bytes whose metadata to return, as for Download a file. The range is inclusive and zero-based. A range that starts past the end of the file returns 416 Range Not Satisfiable. Suffix ranges, such as bytes=-500, are not supported and return 400 Bad Request. Range headers with invalid syntax, reversed ranges, or multiple ranges are ignored, and the metadata for the full file is returned (200 OK). See RFC 9110.

If-Unmodified-Sincestring

Return metadata only if the file has not been modified since the specified time. If it has, a 412 Precondition Failed error is returned. See RFC 9110.

If-Matchstring

Return metadata only if the file's current ETag matches one of the supplied entity tags. On a mismatch, a 412 Precondition Failed error is returned. See RFC 9110.

If-None-Matchstring

Return metadata only if the file's current ETag does not match the supplied entity tags. If it matches, a 304 Not Modified response is returned with no body. See RFC 9110.

If-Modified-Sincestring

Return metadata only if the file has been modified since the specified time. If it has not, a 304 Not Modified response is returned with no body. See RFC 9110.

Success status code

200 OK, or 206 Partial Content for a range request

Response headers

Content-Lengthint64

The size in bytes of the body that the matching GET request returns: the whole file, or the requested range for a 206 Partial Content response.

Content-Typestring

The content type of the file.

Example: application/octet-stream

Content-Rangestring

Returned on a 206 Partial Content response. Indicates where in the whole file the requested range belongs, and the total size of the file. On a 416 Range Not Satisfiable response, the value is bytes */<file size>. See RFC 9110.

Example: bytes 0-1048575/10485760

Accept-Rangesstring

Advertises that the server supports range requests for the file. See RFC 9110.

Example: bytes

Last-ModifiedstringHTTP-date

The last modified time of the file, as an HTTP-date.

Example: Wed, 24 Jan 2024 14:10:56 GMT

ETagstring

The entity tag (ETag) of the file. You can send it back in a later If-Match or If-None-Match request.

View as Markdown

Download a file Beta

GET /api/3.0/fs/files{file_path}

Downloads a file. The file contents are the response body. This is a standard HTTP file download, not a JSON RPC. It supports the Range header and the If-Match, If-None-Match, If-Modified-Since, and If-Unmodified-Since conditional headers. Successful responses include the file's ETag and Last-Modified headers.

API scopes: files

Parameters

file_pathstringRequiredpath

The absolute path of the file.

Request headers

Rangestring

The range of bytes to retrieve. The range is inclusive and zero-based. A range that starts past the end of the file returns 416 Range Not Satisfiable. Suffix ranges, such as bytes=-500, are not supported and return 400 Bad Request. Range headers with invalid syntax, reversed ranges, or multiple ranges are ignored, and the full file is returned (200 OK). See RFC 9110.

If-Unmodified-Sincestring

Download the file only if it has not been modified since the specified time. If it has, a 412 Precondition Failed error is returned. See RFC 9110.

If-Matchstring

Download the file only if its current ETag matches one of the supplied entity tags. On a mismatch, a 412 Precondition Failed error is returned. See RFC 9110.

If-None-Matchstring

Download the file only if its current ETag does not match the supplied entity tags. If it matches, a 304 Not Modified response is returned with no body. See RFC 9110.

If-Modified-Sincestring

Download the file only if it has been modified since the specified time. If it has not, a 304 Not Modified response is returned with no body. See RFC 9110.

Success status code

200 OK, or 206 Partial Content for a range request

Response headers

Content-Lengthint64

The size of the response body in bytes: the whole file, or the requested range for a 206 Partial Content response.

Content-Typestring

The content type of the file.

Example: application/octet-stream

Content-Rangestring

Returned on a 206 Partial Content response. Indicates where in the whole file the returned bytes belong, and the total size of the file. On a 416 Range Not Satisfiable response, the value is bytes */<file size>. See RFC 9110.

Example: bytes 0-1048575/10485760

Accept-Rangesstring

Advertises that the server supports range requests for the file. See RFC 9110.

Example: bytes

Last-ModifiedstringHTTP-date

The last modified time of the file, as an HTTP-date.

Example: Wed, 24 Jan 2024 14:10:56 GMT

ETagstring

The entity tag (ETag) of the file. You can send it back in a later If-Match or If-None-Match request.

Response body

The file contents, or the requested range of them, as raw bytes.

View as Markdown

Delete a file Beta

DELETE /api/3.0/fs/files{file_path}

Delete a file. Delete is idempotent: if the file does not exist, it returns 204 No Content.

API scopes: files

Parameters

file_pathstringRequiredpath

The absolute path of the file.

Request headers

If-Matchstring

Optional precondition. With a specific ETag, the file is deleted only if its current ETag matches. On a mismatch, a 412 Precondition Failed error is returned. With the wildcard *, if the file is missing, 404 Not Found is returned instead of 204 No Content. See RFC 9110.

Success status code

204 No Content

View as Markdown

Perform an action on a file Beta

POST /api/3.0/fs/files{file_path}

Performs an action on the file, selected by the required action query parameter. The action determines which additional query parameters and request body apply (see the field descriptions below). The supported actions are:

• initiate-upload: start a multipart upload session for a large file. The response body contains an upload_id that identifies the session for the subsequent calls.
• complete-upload: assemble the uploaded parts into the file at the path. Requires upload_id and a body listing every part as {part_number, etag}, assembled in ascending part-number order. Part numbers range from 1 to 10000. Every part except the last must be at least 5 MiB, and each part can be at most 4000 MiB. The response headers include the ETag of the assembled file.
• abort-upload: cancel the multipart upload session. Requires upload_id. The target file is not modified.

API scopes: files

Parameters

file_pathstringRequiredpath

The absolute path of the target file.

actionstringRequiredquery

The action to perform. One of:

• initiate-upload: start a multipart upload session. The response body contains the upload_id.
• complete-upload: assemble the uploaded parts into the final file. Requires upload_id and a request body listing the parts.
• abort-upload: cancel the multipart upload session. Requires upload_id.

upload_idstringquery

The upload ID returned by the initiate-upload action. Required for complete-upload and abort-upload. Omit it for initiate-upload.

overwritebooleanquery

Applies only to initiate-upload. If true or unspecified, an existing file is overwritten when the upload is completed. If false, complete-upload returns 409 FileAlreadyExists when a file already exists at the path.

Request headers

Content-Typestring

Applies only to initiate-upload. Optional content type to associate with the file. The server stores it verbatim and returns it on later GET and HEAD responses. When omitted, the server defaults to application/octet-stream.

If-Matchstring

Applies only to initiate-upload, and is evaluated when the upload completes: complete-upload succeeds only if the file's current ETag matches the provided ETag. For this operation only the If-Match: <etag> form is supported. It is not supported on complete-upload or abort-upload. See RFC 9110.

If-None-Matchstring

Applies only to initiate-upload, and is evaluated when the upload completes: complete-upload succeeds only if no file exists at the path. For this operation only the If-None-Match: * form is supported. It is not supported on complete-upload or abort-upload. See RFC 9110.

Request body

Applies only to complete-upload. Lists every part uploaded during the session.

partsarray of object

The list of parts uploaded during the session. Include every uploaded part. Parts are assembled in ascending part_number order.

Show child attributesHide child attributes

part_numberint32Required

The sequence number of the part, starting at 1, as you sent it when you uploaded the part.

etagstringRequired

The ETag value returned in the response headers when the part was uploaded.

Success status code

200 OK

Response headers

ETagstring

Returned only in the response of complete-upload. The entity tag (ETag) of the assembled file. You can send it back in a later If-Match or If-None-Match request.

Response body

upload_idstring

Returned only in the response of initiate-upload. The upload ID used when uploading parts and when completing or aborting the upload.

View as Markdown

Initiate upload response body

{
  "upload_id": "dvOR6FE39mNb"
}

Complete upload request body

{
  "parts": [
    {
      "part_number": 1,
      "etag": "\"33a64df551425fcc55e4d42a148795d9f25f89d4\""
    },
    {
      "part_number": 2,
      "etag": "\"954fb694be262821177cfbc47f9d4a0aa06c349b\""
    }
  ]
}

Create a directory Beta

PUT /api/3.0/fs/directories{directory_path}

Creates an empty directory. If necessary, also creates any parent directories of the new, empty directory (like the shell command mkdir -p). If called on an existing directory, returns a success response. This method is idempotent: it succeeds if the directory already exists.

API scopes: files

Parameters

directory_pathstringRequiredpath

The absolute path of a directory.

Success status code

200 OK

View as Markdown

Get directory metadata Beta

HEAD /api/3.0/fs/directories{directory_path}

Get the metadata of a directory. The response headers contain the metadata. There is no response body.

This method is useful to check if a directory exists and you have access to it.

If you want to ensure the directory exists, you can instead use Create a directory, which creates the directory if it does not exist and is idempotent: it succeeds if the directory already exists.

API scopes: files

Parameters

directory_pathstringRequiredpath

The absolute path of a directory.

Success status code

200 OK

View as Markdown

List directory contents Beta

GET /api/3.0/fs/directories{directory_path}

Returns the contents of a directory. If there is no directory at the specified path, the API returns an HTTP 404 error.

API scopes: files

Parameters

directory_pathstringRequiredpath

The absolute path of a directory.

page_sizeint32query

The maximum number of directory entries to return. The response may contain fewer entries. If the response contains a next_page_token, there may be more entries, even if fewer than page_size entries are in the response.

We recommend not to set this value unless you are intentionally listing less than the complete directory contents.

If unspecified, at most 1000 directory entries are returned. The maximum value is 1000. Values above 1000 are coerced to 1000.

Default: 1000

page_tokenstringquery

An opaque page token, which was the next_page_token in the response of the previous request to list the contents of this directory. Provide this token to retrieve the next page of directory entries. When you provide a page_token, all other parameters of the request must match the previous request. To list all of the entries in a directory, continue requesting pages of entries until the response contains no next_page_token. Don't use the number of entries returned to determine when the listing is complete.

Success status code

200 OK

Response body

contentsarray of object

The entries in the directory.

Show child attributesHide child attributes

file_sizeint32int64

The length of the file in bytes. This field is omitted for directories.

is_directoryboolean

True if the path is a directory.

last_modifiedint32int64

Last modification time of the file in milliseconds since the Unix epoch.

namestring

The name of the file or directory. This is the last component of the path.

pathstring

The absolute path of the file or directory.

next_page_tokenstring

A token, which can be sent as page_token to retrieve the next page.

View as Markdown

Response body

{
  "contents": [
    {
      "file_size": 0,
      "is_directory": true,
      "last_modified": 0,
      "name": "string",
      "path": "string"
    }
  ],
  "next_page_token": "string"
}

Delete a directory Beta

DELETE /api/3.0/fs/directories{directory_path}

Delete an empty directory. Delete is idempotent: if the directory does not exist, it returns 204 No Content. To delete a non-empty directory, first delete all of its contents. You can do this by listing the directory contents and deleting each file and subdirectory recursively.

API scopes: files

Parameters

directory_pathstringRequiredpath

The absolute path of a directory.

Request headers

If-Matchstring

Optional precondition. Directories do not have ETags, so only the wildcard * is supported. With the wildcard *, if the directory is missing, 404 Not Found is returned instead of 204 No Content. See RFC 9110.

Success status code

204 No Content

View as Markdown