Upload files to a Unity Catalog volume

The Upload Files to Volume UI allows you to upload files in any format to a Unity Catalog volume, including structured, semi-structured, and unstructured data.

  • In Databricks Runtime 13.2 and above, Databricks recommends using volumes to store .whl libraries for compute with assigned or shared access modes.

  • In Databricks Runtime 13.3 and above, Databricks recommends using volumes to store JARs and init scripts for compute with assigned or shared access modes.

For semi-structured or structured files, you can use Auto Loader or COPY INTO to create tables from the files. You can also run various machine learning and data science workloads on files uploaded to a volume. Furthermore, you can upload libraries, certificates, and other configuration files of arbitrary formats, such as .whl or .txt, that you want to use to configure cluster libraries, notebook-scoped libraries, or job dependencies.

You can access this UI in the following ways:

  • In the sidebar, click New > Add data > Upload Files to Volume.

  • In Catalog Explorer, click Add > Upload to volume. You can also upload files directly to a volume or to a directory in a volume while browsing volumes in Catalog Explorer.

  • From within a notebook, by clicking File > Upload files to Volume.

Note

Volumes are only supported in Databricks Runtime 13.2 and above. In Databricks Runtime 13.1 and below, operations against /Volumes paths might succeed, but might write data to ephemeral storage disks attached to compute clusters rather than persisting data to Unity Catalog volumes as expected.

Before you begin

Before you upload files to a Unity Catalog volume, you must have the following:

  • A workspace with Unity Catalog enabled. For more information, see Get started using Unity Catalog.

  • The WRITE VOLUME privilege on the volume you want to upload files to.

  • The USE SCHEMA privilege on the parent schema

  • The USE CATALOG privilege on the parent catalog.

For more information, see Unity Catalog privileges and securable objects.

Steps to upload files to a volume

To upload files to a Unity Catalog volume, do the following:

  1. Click New > Add Data.

  2. Click Upload Files to Volume.

  3. Select a volume or a directory inside a volume, or paste a volume path.

  4. Click the browse button or drag and drop files directly into the drop zone.