The Databricks Runtime for Machine Learning (Databricks Runtime ML) is a ready-to-go environment optimized for machine learning and data science. Databricks Runtime ML includes many external libraries, including TensorFlow, PyTorch, Horovod, scikit-learn and XGBoost, and provides extensions to improve performance, including GPU acceleration in XGBoost, distributed deep learning using HorovodRunner, and model checkpointing using a Databricks File System (DBFS) FUSE mount.
To use Databricks Runtime ML, select the ML version of the runtime when you create your cluster.
You can install additional libraries to create a custom environment for your notebook or cluster.
You can create GPU-enabled clusters to accelerate deep learning tasks. For information about creating GPU-enabled Databricks clusters, see GPU-enabled clusters. Databricks Runtime ML includes GPU hardware drivers and NVIDIA libraries such as CUDA.