Access the MLflow tracking server from outside Databricks

You may wish to log to the MLflow tracking server from your own applications or from the MLflow CLI.

This article describes the required configuration steps. Start by installing MLflow and configuring your credentials (Step 1). You can then either configure an application (Step 2) or configure the MLflow CLI (Step 3).

For information on how to launch and log to an open-source tracking server, see the MLflow open source documentation.

Step 1: Configure your environment

If you don’t have a Databricks account, you can try Databricks for free. See Get started with Databricks.

To configure your environment to access your Databricks hosted MLflow tracking server:

  1. Install MLflow using pip install mlflow.

  2. Configure authentication according to your Databricks subscription.

    • Community Edition. Do one of the following:

      • (Recommended) Use mlflow.login() to be prompted for your credentials.

        import mlflow

mlflow.login()

        The following is a response example. If the authentication succeeds, you see the message, “Successfully signed into Databricks!”.

        2023/10/25 22:59:27 ERROR mlflow.utils.credentials: Failed to sign in Databricks: default auth: cannot configure default credentials
Databricks Host (should begin with https://): https://community.cloud.databricks.com/
Username: weirdmouse@gmail.com
Password: ··········
2023/10/25 22:59:38 INFO mlflow.utils.credentials: Successfully signed in Databricks!

      • Specify credentials using environment variables:

        # Configure MLflow to communicate with a Databricks-hosted tracking server
export MLFLOW_TRACKING_URI=databricks
# Specify your Databricks username & password
export DATABRICKS_USERNAME="..."
export DATABRICKS_PASSWORD="..."

    • Databricks Platform. Do one of:

      • Generate a REST API token and create your credentials file using databricks configure --token.

      • Specify credentials using environment variables:

        # Configure MLflow to communicate with a Databricks-hosted tracking server
export MLFLOW_TRACKING_URI=databricks
# Specify the workspace hostname and token
export DATABRICKS_HOST="..."
export DATABRICKS_TOKEN="..."

Step 2: Configure MLflow applications

Configure MLflow applications to log to Databricks by setting the tracking URI to databricks, or databricks://<profileName>, if you specified a profile name via --profile while creating your credentials file. For example, you can achieve this by setting the MLFLOW_TRACKING_URI environment variable to “databricks”.

Step 3: Configure the MLflow CLI

Configure the MLflow CLI to communicate with a Databricks tracking server with the MLFLOW_TRACKING_URI environment variable. For example, to create an experiment using the CLI with the tracking URI databricks, run:

# Replace <your-username> with your Databricks username
export MLFLOW_TRACKING_URI=databricks
mlflow experiments create -n /Users/<your-username>/my-experiment