Migrate from legacy and third party online tables

This page describes how to migrate your existing online tables. You can migrate to the following:

• An online feature store
• A Lakebase synced table

important

Databricks online tables are no longer supported.

Databricks Online Feature Store (powered by Lakebase) is the recommended approach for online feature serving.

List all existing online tables

To see all existing online tables in your workspace, use a SQL query or a Python script.

SQL

Replace <workspace_url> and <workspace_id> with your workspace information.

SQL

SELECT
  CONCAT("https://<workspace_url>/pipelines/", usage_metadata.dlt_pipeline_id, "?o=<workspace_id>"),
  SUM(usage_quantity)
FROM
  system.billing.usage
WHERE
  usage_date > DATE_SUB(NOW(), 7)
  AND billing_origin_product = 'ONLINE_TABLES'
GROUP BY
  ALL;

Python

Python

from databricks.sdk import *
from databricks.sdk.service.catalog import *

w = WorkspaceClient()

result = []
for catalog in w.catalogs.list():
    for schema in w.schemas.list(catalog_name=catalog.name):
        for table in w.tables.list(catalog_name=catalog.name, schema_name=schema.name):
            if table.table_type == TableType.FOREIGN and table.data_source_format == DataSourceFormat.MYSQL_FORMAT and table.pipeline_id is not None:
                result.append((table.full_name, table.comment))

print(result)

Migrate online tables to online feature store for model or feature serving endpoints

info

After you publish your feature tables to Online Feature Store, any subsequent change to your serving endpoints - including scaling operations - automatically switches them to use Online Feature Store as the default source. Ensure your downstream systems are prepared for this change before publishing.

Step 1: Create an online feature store and publish the feature tables

Databricks recommends creating a single online store per workspace for testing and proof of concept. For production use cases or isolation requirements, you can provision additional stores.

Python

from databricks.feature_engineering import FeatureEngineeringClient

fe = FeatureEngineeringClient()

# Create a single online store that can support multiple feature tables
fe.create_online_store(
    name="online-feature-store",
    capacity="CU_1"
)

For details on publishing feature tables, see Databricks Online Feature Stores.

Step 2: Verify migration and clean up your online tables

After your next endpoint change, verify that the endpoint is using the new online store by checking if the endpoint events contain messages like Linked to Online Feature Store table: "table name". See Monitor model quality and endpoint health.

Once verified, delete your legacy online tables. See Delete an online table using the UI or Delete an online table using APIs.

Migrate online tables to synced tables for OLTP

info

Lakebase Provisioned is available in the following regions: us-east-1, us-east-2, us-west-2, eu-central-1, eu-west-1, ap-south-1, ap-southeast-1, ap-southeast-2.

Lakebase Provisioned uses provisioned compute that you manually scale. For feature comparison with the new Lakebase Autoscaling, see choosing between versions.

Step 1: Create a database instance

To start, create a Lakebase database instance to store your synced tables. See Create and manage a database instance.

Optionally, you can create a database catalog to use Unity Catalog privileges to manage data access. See Register your database in Unity Catalog.

Step 2: Create a synced table from the source table

A synced table is a Unity Catalog read-only Postgres table that automatically synchronizes data from a Unity Catalog table to your Lakebase database instance.

To migrate from online tables to synced tables, create a synced table from the source table of an online table:

1. In Catalog, select the online table you want to migrate to a synced table.
2. In the Overview tab, under the Description section, click the name of the Source table.
3. Create a synced table from the selected source table. See Reverse ETL with Lakebase instances.
   • You can store the synced table in the same catalog location as the existing online table.
   • You can share a pipeline between synced tables.
4. After your synced table is created, you can connect to the database instance and query it directly. See Connect and query.

Step 3: Clean up your online tables

After you create your synced tables, delete your online tables. See Delete an online table using the UI or Delete an online table using APIs.

Delete an online table using the UI

From the online table page, select Delete from the kebab menu.

Delete an online table using APIs

Databricks SDK - Python

Python

w.online_tables.delete('main.default.my_online_table')

REST API

Bash

curl --request DELETE \
  "https://xxx.databricks.com/api/2.0/online-tables/main.default.my_online_table" \
  --header "Authorization: Bearer xxx"

Deleting the online table stops any ongoing data synchronization and releases all its resources.