Optimizations

This section contains information on optimizations available with Delta Lake on Databricks.

• Optimize performance with file management
  • Compaction (bin-packing)
  • Data skipping
  • Z-Ordering (multi-dimensional clustering)
  • Notebooks
  • Improving performance for interactive queries
  • Frequently asked questions (FAQ)
• Auto Optimize
  • How Auto Optimize works
  • Requirements
  • Usage
  • When to opt in and opt out
  • Example workflow: Streaming ingest with concurrent deletes or updates
  • Frequently asked questions (FAQ)
• Optimize performance with caching
  • Delta and Apache Spark caching
  • Delta cache consistency
  • Use Delta caching
  • Cache a subset of the data
  • Monitor the Delta cache
  • Configure the Delta cache
• Isolation levels
  • Set the isolation level
• Replicate MySQL tables to Delta tables
  • MySQL configuration
  • Set up a Structured Streaming job in Databricks
  • Advanced configuration
  • Limitations
• Optimize join performance
  • Range Join optimization
  • Skew Join optimization
• Optimized data transformation
  • Higher-order functions
  • Transform complex data types
• Table versioning