Skip to main content

Serve Feature Views using Declarative Automation Bundles

Preview

This feature is in Private Preview. To try it, reach out to your Databricks contact.

Declarative Automation Bundles (formerly known as Databricks Asset Bundles) enable you to programmatically validate, deploy, and run Databricks resources such as Feature Views and the endpoints that serve them. See What are Declarative Automation Bundles?.

This tutorial builds a bundle that declares two streaming Feature Views, materializes them to an online store, and serves them through a FeatureSpec and a Feature Serving endpoint. You then query the endpoint, change which features it serves, and tear everything down.

The Feature Views and their materialization follow Manage Feature Views using Declarative Automation Bundles, which explains them in detail. To create a FeatureSpec and an endpoint with the Python API instead, see Serve Feature Views.

Each step shows the configuration in both YAML and Python. Pick a language and the page follows your choice.

Requirements​

To complete this tutorial, you need the following:

The Feature Views in this tutorial materialize in real-time mode, so a Kinesis Stream must use an enhanced fan-out (EFO) consumer, with consumerMode set to efo in its extra_options. See Amazon Kinesis.

Serving resources in a bundle​

A bundle serves Feature Views through two resource types:

  • FeatureSpec: a Unity Catalog function that lists the features to serve. It holds no feature values. An endpoint that serves it reads them from the online tables that the Feature Views are materialized to. See Serve features with a FeatureSpec, and for the configuration reference, feature_spec.
  • Feature Serving endpoint: a model serving endpoint that serves a FeatureSpec instead of a model. A bundle declares it as a model_serving_endpoints resource. For the configuration reference, see model_serving_endpoint.

Serve Feature Views from a bundle​

The following steps build and deploy the bundle, query the endpoint, change what it serves, and clean up.

Set up authentication​

First, set up authentication between Databricks CLI on your development machine and your Databricks workspace. This page assumes that you want to use OAuth user-to-machine (U2M) authentication and a corresponding Databricks configuration profile named DEFAULT for authentication.

note

U2M authentication is appropriate for trying out these steps in real time. For fully automated workflows, Databricks recommends that you use OAuth machine-to-machine (M2M) authentication instead. See the M2M authentication setup instructions in Authorize Databricks service principal access to Databricks with OAuth.

  1. Use the Databricks CLI to initiate OAuth token management locally by running the following command for each target workspace.

    In the following command, replace <workspace-url> with your Databricks workspace instance URL, for example https://dbc-a1b2345c-d6e7.cloud.databricks.com.

    Bash
    databricks auth login --host <workspace-url>
  2. The Databricks CLI prompts you to save the information that you entered as a Databricks configuration profile. Press Enter to accept the suggested profile name, or enter the name of a new or existing profile. Any existing profile with the same name is overwritten with the information that you entered. You can use profiles to quickly switch your authentication context across multiple workspaces.

    To get a list of any existing profiles, in a separate terminal or command prompt, use the Databricks CLI to run the command databricks auth profiles. To view a specific profile's existing settings, run the command databricks auth env --profile <profile-name>.

  3. In your web browser, complete the on-screen instructions to log in to your Databricks workspace.

  4. To view a profile's current OAuth token value and the token's upcoming expiration timestamp, run one of the following commands:

    • databricks auth token --host <workspace-url>
    • databricks auth token -p <profile-name>
    • databricks auth token --host <workspace-url> -p <profile-name>

    If you have multiple profiles with the same --host value, you might need to specify the --host and -p options together to help the Databricks CLI find the correct matching OAuth token information.

Create the bundle​

Create an empty directory for the bundle, with a databricks.yml file at its root and a resources directory beside it.

Each of the following steps adds one file under resources/:

Text
databricks.yml
resources/
streaming_features.yml
feature_spec.yml
serving_endpoint.yml

databricks.yml holds the bundle name, the include entry that loads every file under resources/, and the variables that the resources reference:

YAML
bundle:
name: feature-serving-quickstart

include:
- resources/*.yml

variables:
catalog:
description: The catalog that the Feature Views and the FeatureSpec are written to.
default: main
schema:
description: The schema that the Feature Views and the FeatureSpec are written to.
default: features_demo
stream:
description: The three-level name of a Stream registered in Unity Catalog.
default: main.features_demo.events
online_store:
description: The online store that serves the features.
default: my_online_store
endpoint_name:
description: The name of the Feature Serving endpoint.
default: event-features

Declare the Feature Views and their materialization​

Both Feature Views read the same Stream and aggregate per key over a trailing 90 minutes. events_90m counts events, and amount_sum_90m sums value.amount. One materialization set writes both to the online store in streaming mode. For what each configuration key means, see Step 4 of the Feature Views tutorial.

A streaming Feature View addresses an input column as a path into the stream record, so substitute the names your Stream's schema uses wherever the examples show value.*.

Create resources/streaming_features.yml:

YAML
resources:
features:
events_90m:
full_name: ${var.catalog}.${var.schema}.events_90m
description: Events per key over a trailing 90 minutes
source:
stream_source:
full_name: ${var.stream}
function:
aggregation_function:
count_function:
input: value.amount
time_window:
rolling:
window_duration: 5400s
entities:
- name: value.key
timeseries_column:
name: value.event_timestamp

amount_sum_90m:
full_name: ${var.catalog}.${var.schema}.amount_sum_90m
description: Sum of amount per key over a trailing 90 minutes
source:
stream_source:
full_name: ${var.stream}
function:
aggregation_function:
sum:
input: value.amount
time_window:
rolling:
window_duration: 5400s
entities:
- name: value.key
timeseries_column:
name: value.event_timestamp

materialization_sets:
serving_online:
materialized_features:
- feature_name: ${resources.features.events_90m.full_name}
online_store_config:
online_store_name: ${var.online_store}
catalog_name: ${var.catalog}
schema_name: ${var.schema}
table_name_prefix: serving_online
streaming_mode:
mode: STREAMING_MODE_TYPE_RTM
- feature_name: ${resources.features.amount_sum_90m.full_name}
online_store_config:
online_store_name: ${var.online_store}
catalog_name: ${var.catalog}
schema_name: ${var.schema}
table_name_prefix: serving_online
streaming_mode:
mode: STREAMING_MODE_TYPE_RTM

Add a FeatureSpec​

The FeatureSpec lists the features the endpoint serves. Each entry in features names one Feature View with declarative_feature.

A FeatureSpec can be created only after each Feature View it names is materialized to an online store. The ${resources.features...} references in declarative_feature make the spec depend on the Feature Views events_90m and amount_sum_90m, and lifecycle.depends_on adds the dependency that no reference expresses, on the materialization set serving_online. The bundle creates the spec after all three and deletes it before them. See Rules a FeatureSpec must satisfy.

Create resources/feature_spec.yml:

YAML
resources:
feature_specs:
event_features:
name: ${var.catalog}.${var.schema}.event_features
features:
- declarative_feature:
feature: ${resources.features.events_90m.full_name}
- declarative_feature:
feature: ${resources.features.amount_sum_90m.full_name}
lifecycle:
depends_on:
- resource: materialization_sets.serving_online

Add a Feature Serving endpoint​

The endpoint serves the FeatureSpec that entity_name names. Because entity_name references the spec's name, the bundle creates the endpoint after the spec, as the output in Deploy shows, and deletes it before the spec. With scale_to_zero_enabled, the endpoint scales its compute down to zero when it receives no requests.

Create resources/serving_endpoint.yml:

YAML
resources:
model_serving_endpoints:
event_endpoint:
name: ${var.endpoint_name}
config:
served_entities:
- entity_name: ${resources.feature_specs.event_features.name}
workload_size: Small
scale_to_zero_enabled: true

Deploy​

Check the configuration, and then review what the deployment does:

Bash
databricks bundle validate
databricks bundle plan

On a first deployment, all five resources report create:

Text
create feature_specs.event_features
create features.amount_sum_90m
create features.events_90m
create materialization_sets.serving_online
create model_serving_endpoints.event_endpoint

Plan: 5 to add, 0 to change, 0 to delete, 0 unchanged

Deploy the bundle to your workspace:

Bash
databricks bundle deploy

The deploy creates the resources in dependency order and returns when the endpoint is ready, which takes several minutes:

Text
Uploading bundle files to /Workspace/Users/someone@example.com/.bundle/feature-serving-quickstart/default/files...
Created features.events_90m
Created features.amount_sum_90m
Created materialization_sets.serving_online
Created feature_specs.event_features
Created model_serving_endpoints.event_endpoint
Files: 4 uploaded, 0 deleted
Resources: 5 created, 0 changed, 0 deleted, 0 unchanged

Query the endpoint​

Send the keys to look up as dataframe_records. A record names each key by the last segment of the Feature View's entity path, so value.key becomes key:

Bash
databricks serving-endpoints query event-features --json '{"dataframe_records": [{"key": 1280}, {"key": 1188}, {"key": 42}, {"key": 10001}]}'

The response holds one object for each record, in the order of the request. Each object holds one value for each Feature View, keyed by the Feature View's name without its catalog and schema:

JSON
{
"outputs": [
{
"events_90m": 10,
"amount_sum_90m": 10
},
{
"events_90m": 11,
"amount_sum_90m": 11
},
{
"events_90m": 9,
"amount_sum_90m": 10
},
{
"events_90m": 0,
"amount_sum_90m": null
}
]
}

A key with no events in the window returns 0 for the count and null for the sum, as key 10001 does. Every key returns these values until the materialization pipeline writes its first rows, which can be several minutes after databricks bundle deploy returns. For other ways to query an endpoint, see Query an endpoint.

Remove a Feature View from the FeatureSpec​

Any change to a FeatureSpec deletes and re-creates it. A running endpoint keeps serving the features in the FeatureSpec it was built with, so a change that keeps the spec's name does not affect the endpoint. See Update dependencies of a live Feature Serving endpoint.

To see this, remove amount_sum_90m from the spec and keep its name:

Diff
feature_specs:
event_features:
name: ${var.catalog}.${var.schema}.event_features
features:
- declarative_feature:
feature: ${resources.features.events_90m.full_name}
- - declarative_feature:
- feature: ${resources.features.amount_sum_90m.full_name}
lifecycle:

databricks bundle plan reports a recreate of the spec only:

Text
recreate feature_specs.event_features

Plan: 1 to add, 0 to change, 1 to delete, 4 unchanged

databricks bundle deploy asks for confirmation before it re-creates the spec:

Text
Uploading bundle files to /Workspace/Users/someone@example.com/.bundle/feature-serving-quickstart/default/files...

This action will result in the deletion or recreation of the following feature specs.
Grants on them that are not declared in the bundle are lost:
recreate resources.feature_specs.event_features

Would you like to proceed? [y/N]: y
Recreated feature_specs.event_features
Files: 1 uploaded, 0 deleted
Resources: 1 created, 0 changed, 1 deleted, 4 unchanged

The endpoint still returns both features:

Bash
databricks serving-endpoints query event-features --json '{"dataframe_records": [{"key": 1188}, {"key": 42}]}'
JSON
{
"outputs": [
{
"events_90m": 16,
"amount_sum_90m": 20
},
{
"events_90m": 12,
"amount_sum_90m": 11
}
]
}

Rename the FeatureSpec​

To change what the endpoint serves, give the FeatureSpec a new name. The endpoint's entity_name references the name, so the deploy also updates the endpoint, and the endpoint keeps answering requests with the features in the original FeatureSpec until the update completes.

Diff
feature_specs:
event_features:
- name: ${var.catalog}.${var.schema}.event_features
+ name: ${var.catalog}.${var.schema}.event_features_v2

databricks bundle plan reports a recreate of the spec and an update of the endpoint:

Text
recreate feature_specs.event_features
update model_serving_endpoints.event_endpoint

Plan: 1 to add, 1 to change, 1 to delete, 3 unchanged

databricks bundle deploy asks for confirmation, re-creates the spec, and returns when the endpoint has been updated:

Text
Uploading bundle files to /Workspace/Users/someone@example.com/.bundle/feature-serving-quickstart/default/files...

This action will result in the deletion or recreation of the following feature specs.
Grants on them that are not declared in the bundle are lost:
recreate resources.feature_specs.event_features

Would you like to proceed? [y/N]: y
Recreated feature_specs.event_features
Updated model_serving_endpoints.event_endpoint
Files: 1 uploaded, 0 deleted
Resources: 1 created, 1 changed, 1 deleted, 3 unchanged

The endpoint now returns only events_90m:

Bash
databricks serving-endpoints query event-features --json '{"dataframe_records": [{"key": 1188}]}'
JSON
{
"outputs": [
{
"events_90m": 68
}
]
}

Clean up​

The materialization pipeline keeps running, and the endpoint keeps serving, until you delete them. Tear the bundle down when you are finished:

Bash
databricks bundle destroy

databricks bundle destroy lists what it deletes and asks for confirmation:

Text
The following resources will be deleted:
delete resources.feature_specs.event_features
delete resources.features.amount_sum_90m
delete resources.features.events_90m
delete resources.materialization_sets.serving_online
delete resources.model_serving_endpoints.event_endpoint

This action will result in the deletion of the following Feature Views.
Deleting a Feature View that a FeatureSpec serves is not blocked, and leaves the FeatureSpec stale:
delete resources.features.amount_sum_90m
delete resources.features.events_90m

This action will result in the deletion of the following feature specs.
Feature Serving endpoints and training code that use them can no longer load them:
delete resources.feature_specs.event_features

All files and directories at the following location will be deleted: /Workspace/Users/someone@example.com/.bundle/feature-serving-quickstart/default

Would you like to proceed? [y/N]: y
Destroy: 5 deleted

Only the user who created a materialized feature can delete it, so run databricks bundle destroy as the same identity that deployed the bundle. See delete_materialized_feature().

Limitations​

  • When a Feature View is materialized in batch mode, its online table exists only after the materialization set's first run, so an endpoint deployed in the same databricks bundle deploy fails. Run databricks bundle deploy again after the first run.
  • Deleting a Feature View that a FeatureSpec serves is not blocked, and it leaves the FeatureSpec stale.

Next steps​