Add an MLflow experiment resource to a Databricks app

Add MLflow experiments as Databricks Apps resources to enable machine learning experiment tracking within your applications. MLflow experiments provide a structured way to organize and log training runs, track parameters, metrics, and artifacts throughout the model development lifecycle.

When you add an MLflow experiment as a resource, your app can:

• Log training runs with parameters and metrics
• Retrieve experiment data and compare model performance
• Access experiment metadata and run history
• Manage the ML lifecycle programmatically

Add an MLflow experiment resource

Before you add an MLflow experiment as a resource, review the app resource prerequisites.

1. When you create or edit an app, navigate to the Configure step.
2. In the App resources section, click + Add resource.
3. Select MLflow experiment as the resource type.
4. Choose an MLflow experiment from the list of available experiments in your workspace.
5. Select the appropriate permission level for your app:
   • Can read: Grants the app permission to view experiment metadata, runs, parameters, and metrics. Use for apps that display experiment results.
   • Can edit: Grants the app permission to modify experiment settings and metadata.
   • Can manage: Grants the app full administrative access to the experiment.
6. (Optional) Specify a custom resource key, which is how you reference the experiment in your app configuration. The default key is experiment.

When you add an MLflow experiment resource:

• Databricks grants your app's service principal the specified permissions on the selected experiment.
• The app can log training runs and access experiment data through the MLflow Tracking API.
• Access is scoped to the selected experiment only. Your app can't access other experiments unless you add them as separate resources.

Environment variables

When you deploy an app with an MLflow experiment resource, Databricks exposes the experiment ID through environment variables that you can reference using the valueFrom field in your app.yaml configuration.

Example configuration:

YAML

env:
  - name: MLFLOW_EXPERIMENT_ID
    valueFrom: experiment # Use your custom resource key if different

Using the experiment ID in your application:

Python

import os
import mlflow

# Access the experiment using the injected environment variable
experiment_id = os.getenv("MLFLOW_EXPERIMENT_ID")

# Set the experiment for tracking
mlflow.set_experiment(experiment_id=experiment_id)

# Log parameters and metrics
with mlflow.start_run():
    mlflow.log_param("learning_rate", 0.01)
    mlflow.log_metric("accuracy", 0.95)
    mlflow.log_artifact("model.pkl")

For more information, see Access environment variables from resources.

Remove an MLflow Experiment resource

When you remove an MLflow Experiment resource from an app, the app's service principal loses access to the experiment. The experiment itself remains unchanged and continues to be available for other users and applications that have appropriate permissions.

Best practices

Follow these best practices when you work with MLflow experiment resources:

• Organize experiments logically by project or model type to improve discoverability.
• Use consistent naming conventions for runs and parameters across your organization.
• Consider experiment retention policies and storage management for long-running projects.