Manage model serving endpoints

This article describes how to manage model serving endpoints using the Serving UI and REST API. See Serving endpoints in the REST API reference.

Get the status of the model endpoint

In the Serving UI, you can check the status of an endpoint from the Serving endpoint state indicator at the top of your endpoint’s details page.

You can use the REST API to check the status and details of an endpoint with the following:

GET /api/2.0/serving-endpoints/{name}

The following example gets the details of an endpoint that serves the first version of the ads1 model that is registered in the model registry. To specify a model from Unity Catalog, provide the full model name including parent catalog and schema such as, catalog.schema.example-model.

In the following example response, the state.ready field is “READY”, which means the endpoint is ready to receive traffic. The state.update_state field is NOT_UPDATING and pending_config is no longer returned because the update was finished successfully.

{
  "name": "workspace-model-endpoint",
  "creator": "customer@example.com",
  "creation_timestamp": 1666829055000,
  "last_updated_timestamp": 1666829055000,
  "state": {
    "ready": "READY",
    "update_state": "NOT_UPDATING"
  },
  "config": {
    "served_entities": [
      {
        "name": "ads1-1",
        "entity_name": "ads1",
        "entity_version": "1",
        "workload_size": "Small",
        "scale_to_zero_enabled": false,
        "state": {
          "deployment": "DEPLOYMENT_READY",
          "deployment_state_message": ""
        },
        "creator": "customer@example.com",
        "creation_timestamp": 1666829055000
      }
    ],
    "traffic_config": {
      "routes": [
        {
          "served_model_name": "ads1-1",
          "traffic_percentage": 100
        }
      ]
    },
    "config_version": 1
  },
  "id": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "permission_level": "CAN_MANAGE"
}

Delete a model serving endpoint

To disable serving for a model, you can delete the endpoint it’s served on.

You can delete an endpoint from the endpoint’s details page in the Serving UI.

  1. Click Serving on the sidebar.

  2. Click the endpoint you want to delete.

  3. Click the kebab menu at the top and select Delete.

Alternatively, you can use the REST API to delete a serving endpoint as follows:

DELETE /api/2.0/serving-endpoints/{name}

Debug your model serving endpoint

To debug any issues with the endpoint, you can fetch:

  • Model server container build logs

  • Model server logs

These logs are also accessible from the Endpoints UI in the Logs tab.

For the build logs for a served model you can use the following request:

GET /api/2.0/serving-endpoints/{name}/served-models/{served-model-name}/build-logs

{
  “config_version”: 1  // optional
}

For the model server logs for a serve model, you can use the following request:

GET /api/2.0/serving-endpoints/{name}/served-models/{served-model-name}/logs

{
  “config_version”: 1  // optional
}