Databricks runtimes are the set of core components that run on Databricks clusters. Databricks offers several types of runtimes.
- Databricks Runtime
Databricks Runtime includes Apache Spark but also adds a number of components and updates that substantially improve the usability, performance, and security of big data analytics.
- Databricks Runtime for Machine Learning
Databricks Runtime ML is a variant of Databricks Runtime that adds multiple popular machine learning libraries, including TensorFlow, Keras, PyTorch, and XGBoost.
Photon is the Databricks native vectorized query engine that runs SQL workloads faster and reduces your total cost per workload.
- Databricks Light
Databricks Light provides a runtime option for jobs that don’t need the advanced performance, reliability, or autoscaling benefits provided by Databricks Runtime.
- Databricks Runtime for Genomics (Deprecated)
Databricks Runtime for Genomics is a variant of Databricks Runtime optimized for working with genomic and biomedical data.
You can choose from among the supported runtime versions when you create a cluster.
For information about the contents of each runtime variant, see the release notes.