Automatic feature lookup with MLflow models on Databricks

Model Serving can automatically look up feature values from published online stores or from online tables. This article describes how to work with online stores. For information about working with online tables, see Use online tables for real-time feature serving.

Requirements

  • The model must have been logged with FeatureEngineeringClient.log_model (for Feature Engineering in Unity Catalog) or FeatureStoreClient.log_model (for Workspace Feature Store, requires v0.3.5 and above).

  • The online store must be published with read-only credentials.

Note

You can publish the feature table at any time prior to model deployment, including after model training.

Automatic feature lookup

Databricks Model Serving supports automatic feature lookup from these online stores:

  • Amazon DynamoDB (v0.3.8 and above)

Automatic feature lookup is supported for the following data types:

  • IntegerType

  • FloatType

  • BooleanType

  • StringType

  • DoubleType

  • LongType

  • TimestampType

  • DateType

  • ShortType

  • DecimalType

  • ArrayType

  • MapType

Override feature values in online model scoring

All features required by the model (logged with FeatureEngineeringClient.log_model or FeatureStoreClient.log_model) are automatically looked up from online stores for model scoring. To override feature values when scoring a model using a REST API with Model Serving include the feature values as a part of the API payload.

Note

The new feature values must conform to the feature’s data type as expected by the underlying model.

Notebook examples: Unity Catalog

With Databricks Runtime 13.2 and above, any Delta table in Unity Catalog with a primary key can be used as a feature table. When you use a table registered in Unity Catalog as a feature table, all Unity Catalog capabilities are automatically available to the feature table.

This example notebook illustrates how to publish features to an online store and then serve a trained model that automatically looks up features from the online store.

Online Store with Unity Catalog example notebook

Open notebook in new tab

Notebook examples: Workspace Feature Store

This example notebook illustrates how to publish features to an online store and then serve a trained model that automatically looks up features from the online store.

Online Store example notebook

Open notebook in new tab