Export MLflow traces to OpenTelemetry
MLflow tracing is built on OpenTelemetry (OTel). Because every trace MLflow captures is a set of OTel spans, you can export those spans outbound to any OpenTelemetry-compatible observability platform — such as Datadog, Grafana, or Splunk — over the standard OTLP protocol. You can send traces exclusively to an external collector, or use dual export to send them to both Databricks MLflow and your existing observability stack at the same time.
Export MLflow traces to an external OTel collector
MLflow can export traces outbound to any OpenTelemetry-compatible observability platform. Three modes are supported.
OTel-only export
To send traces exclusively to an external OTel collector (bypassing Databricks MLflow storage), set OTEL_EXPORTER_OTLP_TRACES_ENDPOINT before starting any trace:
import os
import mlflow
os.environ["OTEL_EXPORTER_OTLP_TRACES_ENDPOINT"] = "http://localhost:4317/v1/traces"
os.environ["OTEL_SERVICE_NAME"] = "<your-service-name>"
# Trace is exported ONLY to the OTel collector at http://localhost:4317/v1/traces
with mlflow.start_span(name="foo") as span:
span.set_inputs({"a": 1})
span.set_outputs({"b": 2})
Dual export (MLflow + OTel)
To export traces to both Databricks MLflow and an external OTel collector simultaneously, set MLFLOW_ENABLE_DUAL_EXPORT:
import os
import mlflow
os.environ["MLFLOW_ENABLE_DUAL_EXPORT"] = "true"
os.environ["OTEL_EXPORTER_OTLP_TRACES_ENDPOINT"] = "http://localhost:4317/v1/traces"
os.environ["OTEL_SERVICE_NAME"] = "my-ml-service"
mlflow.set_tracking_uri("databricks")
# Traces are exported to BOTH MLflow and the OTel collector
with mlflow.start_span(name="dual_export_example") as span:
span.set_inputs({"model": "gpt-4", "prompt": "Hello world"})
span.set_outputs({"response": "Generated response"})
Metrics export
MLflow exports OTel metrics when a metrics endpoint is configured. For the complete list of exported metrics, see the MLflow documentation.
import os
os.environ["OTEL_METRICS_EXPORTER"] = "otlp"
os.environ["OTEL_EXPORTER_OTLP_METRICS_ENDPOINT"] = "http://localhost:4317"
# Optional: configure export interval in milliseconds
os.environ["OTEL_METRIC_EXPORT_INTERVAL"] = "60000"
Supported platforms
MLflow uses the standard OTLP exporter and supports all OTLP exporter configurations. The following platforms have published OTel setup guides:
To use HTTP protocol instead of the default gRPC or to set custom headers:
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="http://localhost:4317/v1/traces"
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL="http/protobuf"
export OTEL_EXPORTER_OTLP_TRACES_HEADERS="api_key=12345"
Additional resources
- Export Langfuse traces to Databricks — Route Langfuse traces to Databricks over the OTLP endpoint.
- Custom OpenTelemetry instrumentation — Set OTel span attributes for a custom-instrumented agent and search ingested traces.
- Store OpenTelemetry traces in Unity Catalog — Store and manage traces in Unity Catalog tables.
- View traces in the Databricks MLflow UI — Search and filter traces in the MLflow UI.
- Observe and find issues — Query trace data at scale using Databricks SQL.
- Programmatic access to traces — Search and filter traces programmatically with the MLflow SDK.
- Enrich traces: tags, context, and feedback — Attach searchable tags to traces.
Next step: Observe and find issues