Databricks runtimes are the set of core components that run on Databricks clusters. Databricks offers several types of runtimes.
- Databricks Runtime
Databricks Runtime includes Apache Spark but also adds a number of components and updates that substantially improve the usability, performance, and security of big data analytics.
- Databricks Runtime for Machine Learning
Databricks Runtime ML is a variant of Databricks Runtime that adds multiple popular machine learning libraries, including TensorFlow, Keras, PyTorch, and XGBoost.
- Databricks Runtime for Genomics
Databricks Runtime for Genomics is a variant of Databricks Runtime optimized for working with genomic and biomedical data.
- Databricks Runtime with Conda
Databricks Runtime with Conda is a variant of Databricks Runtime that provides an optimized list of default packages and a flexible Python environment that enables maximum control over packages.
- Databricks Light
Databricks Light provides a runtime option for jobs that don’t need the advanced performance, reliability, or autoscaling benefits provided by Databricks Runtime.
You can choose from among the supported runtime versions when you create a cluster.
For information about the contents of each runtime variant, see the release notes.