horovod-spark-estimator-keras(Python)

Import Notebook

Distributed training usin the Horovod Spark Estimator API with Keras

This notebook demonstrates how to use the Horovod Spark Estimator API with Keras

Requiremenmts

Databricks Runtime for Machine Learning version 9.1 LTS or above.

Load MNIST data from databricks-datasets

The following sets up the storage location for intermediate data and loads the MNIST dataset from databricks-datasets. This example uses the .limit() option. Using only limited data decreases the model's accuracy, you can remove this option for better accuracy.

Define the Keras model without any Horovod-specific parameters

Train a Horovod Spark Estimator on the DataFrame

Evaluate the model on the held-out test DataFrame