Data format options

Databricks has built-in keyword bindings for all of the data formats natively supported by Apache Spark. Databricks uses Delta Lake as the default protocol for reading and writing data and tables, whereas Apache Spark uses Parquet.

These articles provide an overview of many of the options and configurations available when you query data on Databricks.

The following data formats have built-in keyword configurations in Apache Spark DataFrames and SQL:

• Delta Lake
• Iceberg
• OpenSharing
• Parquet
• ORC
• JSON
• CSV
• Avro
• Text
• Binary
• XML

Databricks also provides a custom keyword for loading MLflow experiments.

Data formats with special considerations

Some data formats require additional configuration or special considerations for use:

• Databricks recommends loading images as binary data.
• Most formats support write compression via the compression option. See the compression section in each format's documentation for configuration details. Databricks can also directly read pre-compressed files in many formats, and you can unzip compressed files on Databricks if necessary.
  • Text-based (CSV, JSON, XML, text): none (default), bzip2, gzip, lz4, snappy, deflate, and zstd
  • Parquet: snappy (default), gzip, lzo, brotli, lz4, and zstd
  • ORC: snappy, zlib, and lzo
  • Avro: snappy (default), deflate, bzip2, xz, and zstandard

For more information about Apache Spark data sources, see Generic Load/Save Functions and Generic File Source Options.