Manage the Lifecycle of MLflow Models in MLflow Model Registry

Preview

This feature is in Public Preview.

MLflow Model Registry is a centralized model store and a UI and set of APIs that enable you to manage the full lifecycle of MLflow Models. Model Registry provides chronological model lineage (which MLflow experiment and run produced the model at a given time), model versioning, stage transitions (for example, from staging to production or archived), and model and model version annotations and descriptions.

There are two workflows to interact with the Model Registry: the Model Registry UI and the Model Registry API.

Model Registry concepts

  • Model: An MLflow Model logged from an experiment or run that is logged with one of the model flavor’s mlflow.<model-flavor>.log_model methods. Once logged, you can register the model with the Model Registry.
  • Registered model: An MLflow Model that has been registered with the Model Registry. The registered model has a unique name, versions, model lineage, and other metadata.
  • Model version: A version of a registered model. When a new model is added to the Model Registry, it is added as Version 1. Each model registered to the same model name increments the version number.
  • Model stage: A model version can be assigned one or more stages. MLflow provides predefined stages for the common use-cases None, Staging, Production, and Archived. With the appropriate permission you can transition a model version between stages or you can request a model stage transition.
  • Description: You can annotate a model’s intent, including description and any relevant information useful for the team such as algorithm description, dataset employed, or methodology.
  • Activities: Each registered model’s activities—such as request for stage transition—is recorded. The trace of activities provides lineage and auditability of the model’s evolution, from experimentation to staged versions to production.

Control access to models in Model Registry

To learn how to control access to models registered in Model Registry, see MLflow Model permissions.

Requirements

Model Registry UI workflow

This section describes how to register a model, edit a model version description, transition a model version, view model version activities, display registered models, and delete a model version or model using the Model Registry UI.

Register a model in the Model Registry

To register an existing logged model:

  1. Identify the MLflow run containing the model you want to register.

    1. Click the Experiment icon Experiment in the notebook tool bar.

      Notebook toolbar
    2. In the Experiment Runs sidebar, click the External Link icon next to the date of the run.

      Run Date

      The MLflow Run page displays all the run details, including parameters, metrics, tags, and list of artifacts.

  2. In the Artifacts section, click the directory named xxx-model.

    Register model
  3. Click the Register Model button at the far right.

  4. In the Register Model dialog, select Create New Model from the Model drop-down menu, and in the Model Name field, enter a model name, for example scikit-learn-power-forecasting.

    Create model
  5. Click Register. This registers a model named scikit-learn-power-forecasting, copies the model into a secure location managed by the MLflow Model Registry, and creates a model version: Version 1.

    After a few moments, the MLflow Run UI replaces the Register Model button with a link to the new registered model version.

    Create model
  6. Click the link to open the new model version in the Model Registry UI.

Alternately, you can use the Create Model button on the registered models page to create a new, empty model and then assign a logged model to it. Follow these steps:

  1. On the registered models page, click Create Model. Enter a name for the model and click Create.

  2. Follow Steps 1 through 3 in Register a model in the Model Registry.

  3. In the Register Model dialog, select the name of the model you created in Step 1 and click Register. This registers a model with the name you created, copies the model into a secure location managed by the MLflow Model Registry, and creates a model version: Version 1.

    After a few moments, the MLflow Run UI replaces the Register Model button with a link to the new registered model version. You can now select the model from the Model drop-down list in the Register Model dialog on the Experiment Runs page. You can also register new versions of the model by specifying its name in API commands like Create ModelVersion.

Model version page

The model version page displays information about a version of a registered model: creation time for the registered model, creator, last modified time, stage in its lifecycle, source run ID, description, pending requests for stage transitions, and a chronological order of all the activities pertaining to the model version.

Model version

To navigate to a model version page from the sidebar:

  1. Click the Models Icon icon in the sidebar.
  2. Click a model name.
  3. Click a model version.

Update a model version description

You can provide useful information about a model version by annotating it. For example, you may want to annotate a model with an overview of the problem, the methodology and algorithm used, or information about and links to datasets.

To update a description:

  1. Click the Description Edit Icon icon.

  2. Edit the details about the model version in the edit window.

    Edit model version description
  3. Click the Save button.

Transition a model version stage

A model version has one of the following stages: None, Staging, Production, and Archived. Staging is meant for model testing and validating, while Production is for model versions that have completed the testing or review processes and have been deployed to applications for live scoring. An Archived model version is assumed to be inactive, at which point you can consider deleting it.

Different versions of a model can be in different stages.

A user with appropriate permission can transition a model version between stages. If you have permission to transition a model version to a particular stage, you can make the transition directly. If you do not have permission, you can request a stage transition and a user that has permission to transition model versions can approve, reject, or cancel the request.

  1. To display the list of available model stages and your available options, in a model version page, click the Stage: <Stage> button and request or select a transition to another stage.

    Stage transition options
  2. Enter a comment, and click OK.

    Transition to staging

Transition a model version to the Production stage

After testing and validation, you can transition or request a transition to the Production stage.

Model Registry allows more than one version of the registered model in each stage; if you want to have only one version in Production, you can transition all versions of the model currently in Production to Archived by checking the checkbox.

Transition to production

Approve, reject, or cancel a model version stage transition request

A user without stage transition permission can request a stage transition. The request appears in the Pending Requests section in the model version page:

Transition to production

To approve, reject, or cancel a stage transition request, click the Approve, Reject, or Cancel link.

The creator of a transition request can also cancel the request.

View model version activities

To view all the transitions requested, approved, pending, and applied to a model version, go to the Activities section. This record of activities provides a lineage of the model’s lifecycle for auditing or inspection.

Registered model activities

Display registered models

MLflow Model Registry is the hub for all registered models. To display all registered models, click the Models Icon icon in the sidebar.

To search for a specific model, type in the model name in the search box.

Registered models search

Registered model page

The registered model page displays information about a registered model: creation and last modified times, an editable description, and a list of versions with the respective registration times, stages, and pending transition requests. To navigate to a registered model page:

  1. Click the Models Icon icon in the sidebar.

    Registered models
  2. Click a model name.

    Registered model

Delete a model version or a registered model

Warning

You cannot undo this action. You can transition a model version to the Archived stage rather than deleting it from the registry. When you delete a model, all model artifacts stored by the Model Registry and all the metadata associated with the registered model are deleted.

Note

You can only delete models and model versions in the None or Archived stage. If a registered model has versions in the Staging or Production stage, you must transition them to either the None or Archived stage before deleting the model.

To delete a model version:

  1. Click the Models Icon icon in the sidebar.

  2. Click a model name.

  3. Click a model version.

  4. Select the Button Down next to the version number.

    Delete model version
  5. Click the Delete button.

To delete a model:

  1. Click the Models Icon icon in the sidebar.
  2. Click a model name.
  3. Select the Button Down next to the model.
  4. Click the Delete button.

Model Registry API workflow

This section describes how to register a model, edit a model version description, transition a model version, view model version activities, display registered models, and delete a model version or model using the Model Registry API.

Requirements

  • Databricks Runtime 6.5 or above
  • Databricks Runtime 6.4 and lower, mlflow>=1.7.0

Register a model in the Model Registry

There are three programmatic ways to register a model in the Model Registry. All methods copy the model into a secure location managed by the MLflow Model Registry.

  • To log a model and register it with the specified name during an MLflow experiment, use the mlflow.<model-flavor>.log_model(...) method. If a registered model with the name doesn’t exist, the method registers a new model, creates Version 1, and returns a ModelVersion MLflow object. If a registered model with the name exists already, the method creates a new model version and returns the version object.

    with mlflow.start_run(run_name=<run-name>) as run:
      ...
      mlflow.<model-flavor>.log_model(<model-flavor>=<model>,
        artifact_path="<model-path>",
        registered_model_name="<model-name>"
      )
    
  • To register a model with the specified name after all your experiment runs complete and you have decided which model is most suitable to add to the registry, use the mlflow.register_model() method. For this method, you need the run ID for the mlruns:URI argument. If a registered model with the name doesn’t exist, the method registers a new model, creates Version 1, and returns a ModelVersion MLflow object. If a registered model with the name exists already, the method creates a new model version and returns the version object.

    result=mlflow.register_model("runs:<model-path>", "<model-name>")
    
  • To create a new registered model with the specified name, use the MLflow Client API create_registered_model() method. If the model name exists, this method throws an MLflowException.

    client = MlflowClient()
    result = client.create_registered_model("<model-name>")
    

Update model version description

To update a model version description, use the MLflow Client API update_model_version() method:

client = MlflowClient()
client.update_model_version(
  name="<model-name>",
  version=<model-version>,
  description="<description>"
)

Rename a model

To rename a registered model, use the MLflow Client API rename_registered_model() method:

client=MlflowClient()
client.rename_registered_model("<model-name>", "<new-model-name>")

Note

You can rename a registered model only if it has no versions, or all versions are in the None or Archived stage.

Transition a model version stage

Users with appropriate permissions can transition a model version to a new stage.

To update a model version stage to a new stage, use the MLflow Client API transition_model_version_stage() method:

  client = MlflowClient()
  client.transition_model_version_stage(
    name="<model-name>",
    version=<model-version>,
    stage="<stage>",
    description="<description>"
  )

The accepted values for <stage> are: "Staging"|"staging", "Archived"|"archived", "Production"|"production", "None"|"none".

List and search registered models

To retrieve a list of all registered models, use the MLflow Client API list_model_versions() method:

from pprint import pprint

client = MlflowClient()
for rm in client.list_registered_models():
  pprint(dict(rm), indent=4)

This outputs:

{   'creation_timestamp': 1582671933216,
    'description': None,
    'last_updated_timestamp': 1582671960712,
    'latest_versions': [<ModelVersion: creation_timestamp=1582671933246, current_stage='Production', description='A random forest model containing 100 decision trees trained in scikit-learn', last_updated_timestamp=1582671960712, name='sk-learn-random-forest-reg-model', run_id='ae2cc01346de45f79a44a320aab1797b', source='./mlruns/0/ae2cc01346de45f79a44a320aab1797b/artifacts/sklearn-model', status='READY', status_message=None, user_id=None, version=1>,
    <ModelVersion: creation_timestamp=1582671960628, current_stage='None', description=None, last_updated_timestamp=1582671960628, name='sk-learn-random-forest-reg-model', run_id='d994f18d09c64c148e62a785052e6723', source='./mlruns/0/d994f18d09c64c148e62a785052e6723/artifacts/sklearn-model', status='READY', status_message=None, user_id=None, version=2>],
    'name': 'sk-learn-random-forest-reg-model'}

You can also search for a specific model name and list its version details using MLflow Client API search_model_versions() method:

from pprint import pprint

client=MlflowClient()
[pprint(mv) for mv in client.search_model_versions("name=<model-name>")]

This outputs:

{   'creation_timestamp': 1582671933246,
    'current_stage': 'Production',
    'description': 'A random forest model containing 100 decision trees '
                   'trained in scikit-learn',
    'last_updated_timestamp': 1582671960712,
    'name': 'sk-learn-random-forest-reg-model',
    'run_id': 'ae2cc01346de45f79a44a320aab1797b',
    'source': './mlruns/0/ae2cc01346de45f79a44a320aab1797b/artifacts/sklearn-model',
    'status': 'READY',
    'status_message': None,
    'user_id': None,
    'version': 1 }

{   'creation_timestamp': 1582671960628,
    'current_stage': 'None',
    'description': None,
    'last_updated_timestamp': 1582671960628,
    'name': 'sk-learn-random-forest-reg-model',
    'run_id': 'd994f18d09c64c148e62a785052e6723',
    'source': './mlruns/0/d994f18d09c64c148e62a785052e6723/artifacts/sklearn-model',
    'status': 'READY',
    'status_message': None,
    'user_id': None,
    'version': 2 }

Delete a model version or registered model

Warning

You cannot undo this action. You can transition a model version to an Archive stage rather than deleting it from the registry. When you delete a model, all model artifacts stored by the Model Registry and all the metadata associated with the registered model are deleted.

Note

You can only delete models and model versions in the None or Archived stage. If a registered model has versions in the Staging or Production stage, you must transition them to either the None or Archived stage before deleting the model.

Delete a model version

To delete a model version, use the MLflow Client API delete_model_version() method:

# Delete versions 1,2, and 3 of the model
client = MlflowClient()
versions=[1, 2, 3]
for version in versions:
  client.delete_model_version(name="<model-name>", version=version)

Delete a model

To delete a model, use the MLflow Client API delete_registered_model() method:

client = MlflowClient()
client.delete_registered_model(name="<model-name>")

Example

For an example that illustrates how to use the Model Registry to build a machine learning application that forecasts the daily power output of a wind farm, see: