custom-dashboards-mlflow-search-api(Python)

Import Notebook

Building Dashboards with the MLflow Search API

This notebook demonstrates how to use the mlflow.search_runs API to generate custom dashboards.

The notebook contains the following sections:

Get existing experiment

  • Generate dashboards from existing experiments.

Dashboards

  • Visualize changes in evaluation metrics over time
    • The example utilizes Mean Absolute Error (MAE) to illustrate how to visualize changes in metrics over time. If you choose to run this notebook on your own experiment, be sure your experiment calculates and logs the MAE metric prior to trying out this example.
  • Track the number of runs kicked off by a particular user
  • Measure the total number of runs across all users

Requirements

This notebook requires a cluster running Databricks Runtime for Machine Learning.

Get existing experiment ID

To plot summaries of existing experiment data, you need the notebook path in which your experiment was run.

  1. Find and select the experiment of interest via the Experiments page in the left sidebar
  2. On the experiment page, select the copy icon to the right of the experiment path at the top of the page.
  3. Paste this path into the following code cell as your experiment_name variable-- replace the text experiment_notebook_path, but keep the single quotations.

The following shows how to get the experiment ID of an existing experiment in your workspace with the experiment notebook path and get_experiment_by_name().

Construct Graphs for the Dashboard

Visualize changes in evaluation metrics over time

This example tracks progress towards optimizing Mean Absolute Error (MAE) over recent days of experiment runs.

You may wish to modify earliest_start_time, which restricts which runs are displayed.

    13 rows

    You can add this chart to a dashboard by clicking the Dashboard icon at the top right of the cell and selecting Add to New Dashboard.

    Track the number of runs started by a specific user

    This example calculates the number of runs started by a specific user each day.

    You may also wish to change:

    • filter_string: Defaults to selecting runs started by you
    • earliest_start_time: Restricts which runs are displayed

      Enter a user in the widget at the top of this page (for example, "username@example.com"). Defaults to all users. <command-4112123844379294>:16: FutureWarning: Using .astype to convert from timezone-aware dtype to timezone-naive dtype is deprecated and will raise in a future version. Use obj.tz_localize(None) or obj.tz_convert('UTC').tz_localize(None) instead runs['start_time'] = runs['start_time'].astype('datetime64[ns]')

      13 rows

      Measure the total number of runs across all users

      This example calculates the total number of experiment runs created during each month of 2019.

        12 rows