deep-learning-transfer-learning-keras(Python)
Import Notebook

Featurization using a pretrained model for transfer learning

This notebook demonstrates how to take a pre-trained deep learning model and use it to compute features for downstream models. This is sometimes called transfer learning since it allows transfering knowledge (i.e., the feature encoding) from the pre-trained model to a new model.

In this notebook:

  • The flowers example dataset
  • Distributed featurization using pandas UDFs
    • Load data using Apache Spark's binary files data source
    • Load and prepare a model for featurization
    • Compute features using a Scalar Iterator pandas UDF

This notebook does not take the final step of using those features to train a new model. For examples of training a simple model such as logistic regression, refer to the "Machine Learning" examples in the Databricks documentation.

Requirements:

  • Databricks Runtime for Machine Learning
import pandas as pd
from PIL import Image
import numpy as np
import io

import tensorflow as tf
from tensorflow.keras.applications.resnet50 import ResNet50, preprocess_input
from tensorflow.keras.preprocessing.image import img_to_array

from pyspark.sql.functions import col, pandas_udf, PandasUDFType

The flowers dataset

We use the flowers dataset from the TensorFlow team as our example dataset, which contains flower photos stored under five sub-directories, one per class. It is hosted under Databricks Datasets for easy access.

%fs ls /databricks-datasets/flower_photos

Featurization using pandas UDFs

This section shows the workflow of computing features using pandas UDFs. This workflow is flexible, supporting image preprocessing and custom models. It is also efficient since it takes advantage of pandas UDFs for performance.

The major steps are:

  1. Load DataFrame
  2. Prepare model
  3. Define image loading and featurization methods
  4. Apply the model in a pandas UDF

Load data

Load images using Spark's binary file data source. You could alternatively use Spark's image data source, but the binary file data source provides more flexibility in how you preprocess images.

images = spark.read.format("binaryFile") \
  .option("pathGlobFilter", "*.jpg") \
  .option("recursiveFileLookup", "true") \
  .load("/databricks-datasets/flower_photos")

display(images.limit(5))

Prepare your model

Download a model file for featurization, and truncate the last layer(s). This notebook uses ResNet50.

Spark workers need to access the model and its weights.

  • For moderately sized models (< 1GB in size), a good practice is to download the model to the Spark driver and then broadcast the weights to the workers. This notebook uses this approach.
  • For large models (> 1GB), it is best to load the model weights from distributed storage to workers directly.
model = ResNet50(include_top=False)
model.summary()  # verify that the top layer is removed

bc_model_weights = sc.broadcast(model.get_weights())

def model_fn():
  """
  Returns a ResNet50 model with top layer removed and broadcasted pretrained weights.
  """
  model = ResNet50(weights=None, include_top=False)
  model.set_weights(bc_model_weights.value)
  return model