Deep Learning Guide

Databricks provides an environment that makes it easy to build, train, and deploy deep learning (DL) models at scale. Many deep learning libraries are available in Databricks Runtime ML, a machine learning runtime that provides a ready-to-go environment for machine learning and data science. For deep learning libraries not included in Databricks Runtime ML, you can either install libraries as a Databricks library or use init scripts to install libraries on clusters upon creation.

Graphics processing units (GPUs) can accelerate deep learning tasks. For information about creating GPU-enabled Databricks clusters, see GPU-enabled Clusters. Databricks Runtime ML includes pre-installed GPU hardware drivers and NVIDIA libraries such as CUDA.

A typical DL workflow involves the phases data preparation, training, and inference. This section gives guidelines on deep learning in Databricks.