The FileStore

The FileStore is a special folder within Databricks File System - DBFS where you can save files and have them accessible to your web browser. You can use the FileStore to:

  • Save files, such as images and libraries, that are accessible within HTML and JavaScript when you call displayHTML.
  • Save output files that you want to download to your local desktop.

When you use certain Databricks features, Databricks creates the the following directories in the FileStore for you:

  • /FileStore/jars - stores libraries that you create and upload. If you delete files in this directory, libraries that reference these files in your workspace may not longer work. See Libraries for more information.
  • /FileStore/plots - stores images created in notebooks when you call display() on a Python or R plot object, such as a ggplot or matplotlib plot. If you delete files in this directory, you may have to regenerate those plots in the notebooks that reference them. See Matplotlib and ggplot in Python Notebooks for more information.
  • /FileStore/tables - stores the files that you upload via the Create Table UI. If you delete files in this directory, tables that you created from these files may no longer be accessible. See Accessing Data for more information.
  • /FileStore/import-stage - contains temporary files created when you import notebooks or DBC files. These temporary files disappear after the notebook import completes.

Save a file to the FileStore

To save a file to the FileStore, put it in the /FileStore directory in DBFS:

dbutils.fs.put("/FileStore/my-stuff/my-file.txt", "Contents of my file")

Files in the FileStore are accessible in your web browser via https://<databricks-instance-name>.cloud.databricks.com/files/. For example, the example above is accessible at:

https://<databricks-instance-name>.cloud.databricks.com/files/my-stuff/my-file.txt

Note

If you are on Community Edition you may need to replace https://community.cloud.databricks.com/files/my-stuff/my-file.txt with https://community.cloud.databricks.com/files/my-stuff/my-file.txt?o=###### where the number after o= is the same as in your Community Edition URL.

Embed static images in notebooks

You can use this location to embed static images into your notebooks. Refer to the files/ location to reference the /FileStore within a displayHTML():

displayHTML("<img src ='files/image.jpg/'>")

or Markdown image import syntax:

%md
![my_test_image](files/image.jpg)

You can upload static images using the DBFS REST API.

dbfs_path = "/FileStore/images/sparkui/"
mkdirs_payload = { 'path': dbfs_path }
resp = client.post('/dbfs/mkdirs', json_params = mkdirs_payload).json()
print(resp)

files = [f for f in os.listdir('.') if os.path.isfile(f)]
for f in files:
    if ".png" in f:
        print("Sending png file ...")
        image = dbfs_path + f
        print(image)
        files = {'file': open(f, 'rb')}
        put_payload = { 'path' : image, 'overwrite' : 'true' }
        # push the images to DBFS
        resp = client.post('/dbfs/put', json_params = put_payload, files_json = files).json()
        print(resp)

Use a Javascript library

This notebook shows how to use FileStore to contain a JavaScript library.