Skip to main content

Build feature tables for machine learning

Feature tables store the inputs that your models train on and score with, in Unity Catalog, so that training and inference use the same feature definitions, and you can trace each model to the features that it uses. This page uses the feature-store demo from dbdemos, a Python library from Databricks that installs demos into your workspace. The demo uses sample travel data. You run the first three of its notebooks, which build on each other:

Notebook

What it covers

01_Feature_store_introduction

Feature tables in Unity Catalog, feature joins to training data with FeatureLookup, and model training with the Feature Engineering client

02_Feature_store_advanced

Point-in-time lookups to prevent data leakage, feature publishing to an Online Feature Store, and feature serving endpoints

03_Feature_store_pipeline

Feature table creation and refresh with Lakeflow pipelines

Notebook

What it covers

01_Feature_store_introduction

Feature tables in Unity Catalog, feature joins to training data with FeatureLookup, and model training with the Feature Engineering client

02_Feature_store_advanced

Point-in-time lookups to prevent data leakage, feature publishing to an Online Feature Store, and feature serving endpoints

03_Feature_store_pipeline

Feature table creation and refresh with Lakeflow pipelines

For the demo overview, see Feature Store and Online Inference in the Databricks Demo Center.

The following video introduces the Databricks feature store:

Requirements​

You need the following before you install the demo:

  • A workspace that is enabled for Unity Catalog.
  • The USE CATALOG and CREATE SCHEMA privileges on the catalog where you install the demo. If the schema already exists, you must have the USE SCHEMA, CREATE TABLE, and CREATE MODEL privileges on it instead.
  • Compute that can install Python packages from the Python Package Index (PyPI).

Install the demo​

These steps install the feature-store demo into the catalog and schema that you choose, and then walk you through running its notebooks.

Install the dbdemos library​

The demo installer imports dbdemos, so set up the library in a notebook before you run the install cell:

  1. In your workspace, create a Python notebook, and attach it to compute.

  2. Run the following code in the first cell:

    Python
    %pip install dbdemos
  3. Run the following code in the next cell to restart Python so that the notebook can import the library:

    Python
    dbutils.library.restartPython()

Install the feature store demo​

Run the following code in a new cell. Replace <catalog> and <schema> with the catalog and schema where you want the demo to create its tables. If the schema doesn't exist, the installer creates it.

Python
import dbdemos

dbdemos.install('feature-store', catalog='<catalog>', schema='<schema>')

When the installation completes, the cell output lists links to the notebooks. If the installation fails, see Troubleshooting.

Run the notebooks in order​

Open and run 01_Feature_store_introduction, 02_Feature_store_advanced, and 03_Feature_store_pipeline in order. Each notebook builds on the tables and models that the previous notebook creates.

tip

Notebook 03 builds the feature tables with Lakeflow pipelines, so that they refresh on a schedule and stay governed in Unity Catalog. Adapt that pipeline for your own data before you write your own training code.

Databricks recommends Feature Views for new use cases. Feature tables fit workloads that need OpenSharing or a third-party online store, and you own the pipeline that refreshes them. See Choose an authoring strategy.

The Online Feature Store and the feature serving endpoint that notebook 02 creates incur charges while they exist. Delete them when you finish the demo. See Databricks Online Feature Stores.

Troubleshooting​

If the feature store demo doesn't install, match your error to one of these entries.

ModuleNotFoundError: No module named 'dbdemos'

The notebook imported dbdemos before you installed the library, or before Python restarted. Run %pip install dbdemos, run dbutils.library.restartPython(), and then run the installation cell again.

The installer can't create the schema or tables

You don't have the required privileges on the catalog or schema. Choose a catalog and schema where you have the privileges in Requirements, or ask the catalog owner or a metastore admin to grant them.

%pip install fails

The compute can't reach PyPI. Use compute that has outbound access to PyPI, or ask your workspace admin to allow it.

Next steps​