Trigger jobs when models are updated

Use model update triggers to automatically run a job when a model change occurs in Unity Catalog. This feature replaces the need for cron schedules or persistent clusters to track model updates.

Beta

Model update triggers are in Beta.

Model update triggers are designed for two primary personas:

• Data scientists can trigger jobs when a new model version is ready or when an alias is set, on a single model or across multiple models at a time. For example, you can run validation, testing, or promotion jobs automatically whenever a model they own changes. See Example: Validate models on new versions or aliases.
• Administrators can monitor all models across an entire metastore (or a schema). For example, admins can automatically audit every new model as they are created. See Example: Audit models across a schema or metastore.

How model update triggers work

A model update trigger monitors a scope in Unity Catalog for a model event and triggers a job run when a matching event occurs. When you configure a model update trigger, you choose a scope and a condition.

Available scopes include:

• Model: A single registered model.
• Schema: All models in a schema.
• Metastore: All models in the metastore. Only a metastore admin can configure a metastore-scoped trigger.

The condition determines when to trigger a run:

• Model is created: A new registered model is created in the scope.
• Model version is ready: A new model version becomes ready in the scope.
• Model alias is set: A specified alias is applied to a model version. You can specify up to 10 aliases per trigger, and the trigger responds when any of them are applied.

Model update triggers do not incur additional costs other than cloud provider costs.

Batched updates

Model update triggers poll for new events approximately once per minute. Changes detected during each interval are batched and passed to the next job run as job parameters. See Job parameters associated with model update triggers.

At most nine updates can be processed per job run. If more than nine events are received, the remaining events are processed in a subsequent run. See Limitations.

Before you begin

The following are required to use model update triggers:

• The workspace must have Unity Catalog enabled.
• You must have EXECUTE privilege on the target model or schema you want the trigger to monitor. Without EXECUTE, reads and writes to those models might fail.
• To configure a metastore-scoped trigger, you must be a metastore admin. You must also grant yourself EXECUTE permission on every current and future catalog in the metastore so the job can access any model in Unity Catalog.

Add a model update trigger

To add a model update trigger to an existing job:

1. In your Databricks workspace's sidebar, click Jobs & Pipelines.
2. In the list of jobs, click the name of the job to which you want to add a trigger.
3. In the Job details pane on the right, click Add trigger.
4. In Trigger type, select Model update.
5. Under Scope, select Model, Schema, or Metastore, and then specify the model or schema to monitor.
6. Under Condition, select the event to trigger on:
   • Model is created
   • Model version is ready
   • Model alias is set. Specify up to 10 aliases to monitor. The trigger responds when any of the specified aliases is set.
7. (Optional) Configure advanced options:
   • Minimum time between triggers in seconds: The minimum time to wait to trigger a run after a previous run. Models updated during this period trigger a run only after the waiting time expires. Use this setting to control the frequency of run creation.
   • Wait after last change in seconds: The time to wait to trigger a run after a model update. Another model update during this period resets the timer. This setting can be used when model updates come in batches, and the whole batch needs to be processed after all updates have arrived.
8. To validate the configuration, click Test trigger. If there are no errors, the button shows Success.
9. Click Save.

After you save the trigger, wait approximately one minute for the trigger to initialize. The message The trigger will be evaluated soon indicates that initialization is in progress. After initialization, any matching model event in the scope triggers a job run.

note

You can also configure model update triggers from the Jobs API. See Configure a model update trigger with the Jobs API.

Example: Validate models on new versions or aliases

To run a validation, testing, or promotion job when a model changes, configure a trigger with the Model scope and either the Model version is ready or Model alias is set condition. For alias changes, specify the aliases to monitor, such as prod and staging. To monitor several models at once, use the Schema scope instead. After the trigger initializes, a matching change to a monitored model triggers a job run.

Example: Audit models across a schema or metastore

To audit every model created in a schema or metastore, configure a trigger with the Schema or Metastore scope and the Model is created condition. After the trigger initializes, any model created in the scope triggers a job run that can record or validate the change.

Job parameters associated with model update triggers

When a model update trigger fires, information about the changes that triggered the run is available to the job run through the {{job.trigger.model.updates}} dynamic value reference. The value is a JSON list of the model updates in the batch:

JSON

[
  {
    "full_name": "model.full.name1",
    "version": 123,
    "alias_name": "prod"
  }
]

The fields are populated as follows:

• full_name: Always populated with the name of the model that was created or updated.
• version: Populated for Model version is ready events with the version that became ready, and for Model alias is set events with the version that had the alias set.
• alias_name: Populated only for Model alias is set events with the name of the alias that was set.

For example, the parameter value for each condition is as follows:

• Model is created:

  JSON

  [{ "full_name": "model.number.one" }, { "full_name": "model.number.two" }]

• Model version is ready:

  JSON

  [
    { "full_name": "model.number.one", "version": 7 },
    { "full_name": "model.number.two", "version": 3 }
  ]

• Model alias is set:

  JSON

  [
    { "full_name": "model.number.one", "version": 7, "alias_name": "prod" },
    { "full_name": "model.number.two", "version": 3, "alias_name": "staging" }
  ]

To make the updates available to a task, add a job parameter whose value is {{job.trigger.model.updates}}. To add a parameter, click Edit parameters in the job's side panel, then set the parameter value to {{job.trigger.model.updates}}. For more information on job parameters, see Parameterize jobs.

The following notebook reads a parameter named events and processes the model updates passed to the run:

Python

import json

json_list = dbutils.widgets.get("events")
data = json.loads(json_list)

for item in data:
    print(f"Full Name: {item['full_name']}, Version: {item.get('version')}, Alias Name: {item.get('alias_name')}")

Process model updates with a For each task

A For each task runs a nested task once per element in a list. For model update triggers, you can run one iteration per element in {{job.trigger.model.updates}}:

1. Configure the For each task to take {{job.trigger.model.updates}} as its input.
2. Configure the nested task to read the values for each input provided by the iteration.
3. Each update's values are available as widget parameters to the nested task.

The following notebook reads the per-iteration values in a nested task:

Python

full_name = dbutils.widgets.get("full_name")
version = dbutils.widgets.get("version")
alias_name = dbutils.widgets.get("alias_name")

print(f"Full Name: {full_name}, Version: {version}, Alias Name: {alias_name}")

Configure a model update trigger with the Jobs API

You can configure a model update trigger by adding a trigger object to a jobs/create, jobs/update, or jobs/reset operation. The following example uses jobs/update:

JSON

{
  "job_id": 574587036927544,
  "new_settings": {
    "trigger": {
      "pause_status": "UNPAUSED",
      "model": {
        "securable_name": "main.default",
        "condition": "MODEL_ALIAS_SET",
        "aliases": ["alias1", "alias2"],
        "min_time_between_triggers_seconds": 3600,
        "wait_after_last_change_seconds": 120
      }
    },
    "parameters": [
      {
        "default": "{{job.trigger.model.updates}}",
        "name": "events"
      }
    ]
  }
}

For the model trigger object:

• securable_name: The schema or model to monitor. Leave empty or omit it to monitor the entire metastore.
• condition: One of MODEL_CREATED, MODEL_VERSION_READY, or MODEL_ALIAS_SET.
• aliases: The aliases to monitor. Applies only to the MODEL_ALIAS_SET condition.

Receive notifications of failed model update triggers

To be notified if a model update trigger fails to evaluate, configure email or system destination notifications on job failure. See Add notifications on a job.

Limitations

Model update triggers have the following limitations:

• A maximum of 100 model update triggers can be configured per workspace. This limit can be raised on a case-by-case basis.
• At most nine events are processed during each triggered job run.
• Each model update trigger can monitor up to 10 aliases.
• If a run is triggered every minute continuously for an hour, the trigger disables itself and must be manually re-enabled.
• To configure a metastore-scoped trigger, you must be a metastore admin and must grant yourself the EXECUTE privilege on every current and future catalog in Unity Catalog for the job to access any model.

FAQ

When should I use deployment jobs versus model update triggers?

Use model update triggers when you need to:

• Monitor model creation events or alias changes.
• Monitor events at a scope broader than a single model, such as a schema or metastore.
• Monitor many models with a single trigger, instead of configuring a separate job for each model.

Use deployment jobs when you need tight UI coupling and activity logs about the relationship between model version creation and job runs.