Microsoft Cognitive Toolkit (CNTK) is an open-source deep learning toolkit developed by Microsoft.
In the section below, we provide example notebooks to demonstrate how to install CNTK on Databricks using Cluster Node Initialization Scripts and how to run example CNTK programs. The content is based on CNTK tutorial 103A and 103B with minor modifications to run on Databricks. Thanks to the developers of CNTK for this tutorial!
See Integrating Deep Learning Libraries with Apache Spark for an example of integrating a deep learning library with Spark.
The recommended way to install CNTK is to install the official Python packages using Cluster Node Initialization Scripts.
The following script installs CNTK compiled for GPUs.
To test and migrate single-machine CNTK workflows, you can start with a driver-only cluster on Databricks by setting the number of workers to zero. Though Apache Spark is not functional under this setting, it is a cost-effective way to run single-machine CNTK workflows.