Archive traces to a Delta table
This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Databricks previews.
You can save traces and their associated assessments to a Unity Catalog Delta table for long-term storage and advanced analysis. This is useful for building custom dashboards, performing in-depth analytics on trace data, and maintaining a durable record of your application's behavior.
You must have the necessary permissions to write to the specified Unity Catalog Delta table. The target table will be created if it does not already exist.
If the table already exists, traces are appended to it.
Enable trace archiving
- MLflow API
- Databricks UI
To begin archiving traces for an experiment, use the enable_databricks_trace_archival function. You must specify the full name of the target Delta table, including catalog and schema. If you don't provide an experiment_id, archiving traces is enabled for the currently active experiment.
from mlflow.tracing.archival import enable_databricks_trace_archival
# Archive traces from a specific experiment to a Unity Catalog Delta table
enable_databricks_trace_archival(
delta_table_fullname="my_catalog.my_schema.archived_traces",
experiment_id="YOUR_EXPERIMENT_ID",
)
To archive traces for an experiment in the UI:
- Go to Experiments page in the Databricks workspace.
- Click Delta sync: Not enabled.
- Specify the full name of the target Delta table, including catalog and schema.
Disable trace archiving
- MLflow API
- Databricks UI
Stop archiving traces for an experiment at any time by using the disable_databricks_trace_archival function.
from mlflow.tracing.archival import disable_databricks_trace_archival
# Stop archiving traces for the specified experiment
disable_databricks_trace_archival(experiment_id="YOUR_EXPERIMENT_ID")
To disable trace archiving:
- Go to Experiments page in the Databricks workspace.
- Click Delta sync: Enabled > Disable syncing.
Next steps
- Monitor GenAI apps in production - Set up production monitoring.
- Building MLflow evaluation datasets - Use archived traces to build evaluation datasets.