Build feature tables for machine learning
Feature tables store the inputs that your models train on and score with, in Unity Catalog, so that training and inference use the same feature definitions, and you can trace each model to the features that it uses. This page uses the feature-store demo from dbdemos, a Python library from Databricks that installs demos into your workspace. The demo uses sample travel data. You run the first three of its notebooks, which build on each other:
Notebook | What it covers |
|---|---|
Feature tables in Unity Catalog, feature joins to training data with | |
Point-in-time lookups to prevent data leakage, feature publishing to an Online Feature Store, and feature serving endpoints | |
Feature table creation and refresh with Lakeflow pipelines |
For the demo overview, see Feature Store and Online Inference in the Databricks Demo Center.
The following video introduces the Databricks feature store:
Requirements
You need the following before you install the demo:
- A workspace that is enabled for Unity Catalog.
- The
USE CATALOGandCREATE SCHEMAprivileges on the catalog where you install the demo. If the schema already exists, you must have theUSE SCHEMA,CREATE TABLE, andCREATE MODELprivileges on it instead. - Compute that can install Python packages from the Python Package Index (PyPI).
Online Feature Stores are available in us-central1, us-east4, and europe-west3 (Frankfurt). See Databricks Online Feature Stores.
Install the demo
These steps install the feature-store demo into the catalog and schema that you choose, and then walk you through running its notebooks.
Install the dbdemos library
The demo installer imports dbdemos, so set up the library in a notebook before you run the install cell:
-
In your workspace, create a Python notebook, and attach it to compute.
-
Run the following code in the first cell:
Python%pip install dbdemos -
Run the following code in the next cell to restart Python so that the notebook can import the library:
Pythondbutils.library.restartPython()
Install the feature store demo
Run the following code in a new cell. Replace <catalog> and <schema> with the catalog and schema where you want the demo to create its tables. If the schema doesn't exist, the installer creates it.
import dbdemos
dbdemos.install('feature-store', catalog='<catalog>', schema='<schema>')
When the installation completes, the cell output lists links to the notebooks. If the installation fails, see Troubleshooting.
Run the notebooks in order
Open and run 01_Feature_store_introduction, 02_Feature_store_advanced, and 03_Feature_store_pipeline in order. Each notebook builds on the tables and models that the previous notebook creates.
Notebook 03 builds the feature tables with Lakeflow pipelines, so that they refresh on a schedule and stay governed in Unity Catalog. Adapt that pipeline for your own data before you write your own training code.
Databricks recommends Feature Views for new use cases. Feature tables fit workloads that need OpenSharing, and you own the pipeline that refreshes them. See Choose an authoring strategy.
The Online Feature Store and the feature serving endpoint that notebook 02 creates incur charges while they exist. Delete them when you finish the demo. See Databricks Online Feature Stores.
Troubleshooting
If the feature store demo doesn't install, match your error to one of these entries.
ModuleNotFoundError: No module named 'dbdemos'
The notebook imported dbdemos before you installed the library, or before Python restarted. Run %pip install dbdemos, run dbutils.library.restartPython(), and then run the installation cell again.
The installer can't create the schema or tables
You don't have the required privileges on the catalog or schema. Choose a catalog and schema where you have the privileges in Requirements, or ask the catalog owner or a metastore admin to grant them.
%pip install fails
The compute can't reach PyPI. Use compute that has outbound access to PyPI, or ask your workspace admin to allow it.
Next steps
-
- Register a model in Unity Catalog
- Train or import a model, and register it in Unity Catalog
-
- Databricks Feature Store
- Learn how to create, publish, and serve features with feature engineering in Unity Catalog