Skip to main content
Last updated on

Troubleshooting and limitations

Troubleshooting​

Error message: Database recommender_system does not exist in the Hive metastore.

A feature table is stored as a Delta table. The database is specified by the table name prefix, so a feature table recommender_system.customer_features will be stored in the recommender_system database.

To create the database, run:

%sql CREATE DATABASE IF NOT EXISTS recommender_system;

Error message: ModuleNotFoundError: No module named 'databricks.feature_engineering' or ModuleNotFoundError: No module named 'databricks.feature_store'

This error occurs when databricks-feature-engineering is not installed on the Databricks Runtime you are using.

databricks-feature-engineering is available on PyPI, and can be installed with:

%pip install databricks-feature-engineering

Error message: ModuleNotFoundError: No module named 'databricks.feature_store'

This error occurs when databricks-feature-store is not installed on the Databricks Runtime you are using.

note

For Databricks Runtime 14.3 and above, install databricks-feature-engineering instead via %pip install databricks-feature-engineering

databricks-feature-store is available on PyPI, and can be installed with:

%pip install databricks-feature-store

Error message: Invalid input. Data is not compatible with model signature. Cannot convert non-finite values...'

This error can occur when using a Feature Store-packaged model in Mosaic AI Model Serving. When providing custom feature values in an input to the endpoint, you must provide a value for the feature for each row in the input, or for no rows. You cannot provide custom values for a feature for only some rows.

Error message: No suitable online store found for feature tables

This error occurs when setting up a Mosaic AI Model Serving endpoint for a model with automatic feature lookup or feature serving. The error message shows up in the failed serving endpoint's events log. It indicates a disconnect between your offline feature table and the Online Feature Store.

Cause 1: Table not published​

Offline feature tables need to be available in an online store for lookup. The most common cause of the error is that the required feature table (listed in the error message) has not yet been published to the Online Feature Store.

Resolution: Follow the Online Feature Store instructions to publish the table(s) mentioned in the error.

Cause 2: Source table recreated (ID mismatch)​

Resolution: You can verify if the Table IDs match and fix the issue.

  1. Get the offline Feature Table ID:
    • Navigate to Catalog Explorer.
    • Locate your Feature Table.
    • Open the Details tab and find the Table ID.
  2. Get the Online Table Source ID:
    • Locate the corresponding Online Table in Catalog Explorer.
    • Open the Details tab.
    • Look under Properties to find the source_table_id.
  3. Compare and fix:
    • If the IDs don't match, the link is broken. This means the offline source table was recreated.
    • Choose one of the options below to fix the issue based on if the original offline table can be restored.

Option 1: Restore the previous offline table

  • Use ALTER TABLE RENAME TO command to change the name of the current table.
  • Use UNDROP TABLE WITH ID command to restore the previouse table using the source_table_id recorded in the Online Table property.
  • Update or recreate the serving endpoint to connect with the new online tables.

Option 2: Keep the current offline table and publish it to online store

  • Follow instructions to publish the feature table.
  • Update or recreate the serving endpoint to connect with the new online tables.
  • Optional: Clean up the old online table to avoid confusion.

Limitations​

  • A model can use at most 50 tables and 100 functions for training.

  • Databricks Runtime ML clusters are not supported when using Lakeflow Spark Declarative Pipelines as feature tables. Instead, use a standard access mode compute resource and manually install the client using pip install databricks-feature-engineering. You must also install any other required ML libraries.

    Python
    %pip install databricks-feature-engineering
  • Databricks legacy Workspace Feature Store does not support deleting individual features from a feature table.
  • A maximum of 100 on-demand features can be used in a model.