The FileStore

The FileStore is a special folder within Databricks File System - DBFS where you can save files and have them accessible to your web browser. You can use the FileStore to:

  • Save files, such as images and libraries, that are accessible within HTML and JavaScript when you call displayHTML.
  • Save output files that you want to download to your local desktop.

When you use certain features, Databricks creates the following directories in the FileStore:

  • /FileStore/jars - stores libraries that you upload. If you delete files in this directory, libraries that reference these files in your Workspace may no longer work.
  • /FileStore/plots - stores images created in notebooks when you call display() on a Python or R plot object, such as a ggplot or matplotlib plot. If you delete files in this directory, you may have to regenerate those plots in the notebooks that reference them. See Matplotlib and ggplot in Python Notebooks for more information.
  • /FileStore/tables - stores the files that you upload via the Create table UI. If you delete files in this directory, tables that you created from these files may no longer be accessible.
  • /FileStore/import-stage - contains temporary files created when you import notebooks or DBC files. These temporary files disappear after the notebook import completes.

Save a file to the FileStore

To save a file to the FileStore, put it in the /FileStore directory in DBFS:

dbutils.fs.put("/FileStore/my-stuff/my-file.txt", "Contents of my file")

Files stored in /FileStore are accessible in your web browser via https://<databricks-instance-name>.cloud.databricks.com/files/. For example, the file you stored in /FileStore/my-stuff/my-file.txt is accessible at:

https://<databricks-instance-name>.cloud.databricks.com/files/my-stuff/my-file.txt

Note

If you are on Community Edition you may need to replace https://community.cloud.databricks.com/files/my-stuff/my-file.txt with https://community.cloud.databricks.com/files/my-stuff/my-file.txt?o=###### where the number after o= is the same as in your Community Edition URL.

Embed static images in notebooks

You can use this location to embed static images into your notebooks. Refer to the files/ location to reference the /FileStore within a displayHTML():

displayHTML("<img src ='files/image.jpg/'>")

or Markdown image import syntax:

%md
![my_test_image](files/image.jpg)

You can upload static images using the DBFS REST API.

dbfs_path = "/FileStore/images/sparkui/"
mkdirs_payload = { 'path': dbfs_path }
resp = client.post('/dbfs/mkdirs', json_params = mkdirs_payload).json()
print(resp)

files = [f for f in os.listdir('.') if os.path.isfile(f)]
for f in files:
  if ".png" in f:
    print("Sending png file ...")
    image = dbfs_path + f
    print(image)
    files = {'file': open(f, 'rb')}
    put_payload = { 'path' : image, 'overwrite' : 'true' }
    # push the images to DBFS
    resp = client.post('/dbfs/put', json_params = put_payload, files_json = files).json()
    print(resp)

Scale static images

To scale the size of an image that you have saved to DBFS, copy the image to /FileStore and then resize using image parameters in displayHTML:

dbutils.fs.cp('dbfs:/user/experimental/MyImage-1.png','dbfs:/FileStore/images/')
displayHTML('''<img src="files/images/MyImage-1.png" style="width:600px;height:600px;">''')

Use a Javascript library

This notebook shows how to use FileStore to contain a JavaScript library.