Model Service

View as Markdown

A governed AI Gateway endpoint in Unity Catalog that routes inference requests to one or more destinations, such as a <Databricks> foundation model or an external model reached through a model provider service. Applies centralized access control, rate limits, and auditing to its traffic.

Get a model serviceGET /api/2.1/unity-catalog/{name=model-services/*}List model servicesGET /api/2.1/unity-catalog/model-servicesCreate a model servicePOST /api/2.1/unity-catalog/model-servicesUpdate a model servicePATCH /api/2.1/unity-catalog/{name=model-services/*}Delete a model serviceDELETE /api/2.1/unity-catalog/{name=model-services/*}

ModelService object

A governed AI Gateway endpoint in Unity Catalog that routes inference requests to one or more destinations, such as a <Databricks> foundation model or an external model reached through a model provider service. Applies centralized access control, rate limits, and auditing to its traffic.

namestringImmutable

Resource name of the model service. Format: model-services/{catalog}.{schema}.{model_service}. Each {...} component is capped at 255 characters individually. Server-derived on Create from parent + model_service_id; required and immutable on Update/Get/Delete.

effective_ownerstringOutput only

Owner of the model service.

metastore_idstringOutput only

Metastore hosting the model service.

create_timestringOutput only

Time the model service was created.

created_bystringOutput only

Creator identity.

update_timestringOutput only

Time the model service was last modified.

updated_bystringOutput only

Identity of the last updater.

commentstring

User-provided description.

Constraints: <= 65536 characters

configobject

Destinations, routing, rate limits, and payload logging configuration. Required on Create. On Update, provide this field when update_mask contains config or one of its subpaths.

Show child attributesHide child attributes

routingobject

Routing configuration: destinations and fallback.

Show child attributesHide child attributes

destinationsarray of object

Primary routing destinations. At most 10 are allowed. At least one is required on Create. On Update, provide this list when replacing the full config or updating config.routing.destinations; other granular routing updates do not require resending destinations. The intermediate config.routing mask path is not supported.

Show child attributesHide child attributes

namestring

User-facing label for this destination, used in routing references.

destination_typestring

Backing-model category. Provide the matching type-specific configuration and leave the other type-specific configurations unset.

Values:

• DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL
• DESTINATION_TYPE_PROVISIONED_THROUGHPUT_FOUNDATION_MODEL
• DESTINATION_TYPE_EXTERNAL_FOUNDATION_MODEL

traffic_percentageint32

Percentage of primary traffic sent to this destination, from 0 to 100. Required when there is more than one primary destination, in which case the primary percentages must sum to 100; a single primary destination receives all traffic. Fallback destinations are ordered and do not use this field.

pay_per_token_configobject

Configuration for a pay-per-token <Databricks> foundation model.

provisioned_throughput_configobject

Configuration for a provisioned-throughput <Databricks> foundation model.

external_model_configobject

Configuration for an external model reached through a model provider service.

is_deletedbooleanOutput only

Whether the destination's backing model or model provider service has been deleted. The destination remains visible so you can identify the broken dependency. Requests cannot use this destination until the backing resource is restored or the destination is replaced.

fallbackobject

Fallback routing applied after a primary destination fails. Fallback destinations are tried in the listed order.

Show child attributesHide child attributes

destinationsarray of object

Fallback destinations, tried in the listed order. At most 5 are allowed.

rate_limitsarray of object

Rate limits applied to requests routed through this model service.

Show child attributesHide child attributes

keystring

Scope of the rate limit. Depending on this value, the limit applies to a principal, the service as a whole, or each user by default.

Values:

• RATE_LIMIT_KEY_USER
• RATE_LIMIT_KEY_USER_GROUP
• RATE_LIMIT_KEY_SERVICE_PRINCIPAL
• RATE_LIMIT_KEY_SERVICE
• RATE_LIMIT_KEY_USER_DEFAULT

renewal_periodstring

Renewal period.

Values:

• RATE_LIMIT_RENEWAL_PERIOD_MINUTE
• RATE_LIMIT_RENEWAL_PERIOD_HOUR

principalstring

Principal this limit applies to: user email, group name, or service principal application ID. Required when key applies to a user, group, or service principal; otherwise it must be unset.

requestsint64

Maximum requests allowed in one renewal period. Leave unset for no request limit. Set to 0 to deny all requests.

tokensint64

Maximum tokens allowed in one renewal period. Leave unset for no token limit. Set to 0 to deny all requests.

inference_tableobject

Inference table configuration for payload logging.

Show child attributesHide child attributes

parentstring

Parent Unity Catalog schema where the inference table is created, in the form schemas/{catalog}.{schema}. Required when configuring an inference table. After the inference table is created, this field cannot be changed.

table_name_prefixstring

Prefix used to form the inference table's registered name. AI Gateway appends _payload; for example, table_name_prefix = "orders" creates orders_payload. If unset, the prefix defaults to the service name. Read table from the response for the resulting resource name. After the inference table is created, this field cannot be changed.

tablestringOutput only

Resolved UC table for payload logs. Format: tables/{catalog}.{schema}.{table}.

is_deletedbooleanOutput only

Whether the referenced inference table has been deleted. The configuration remains visible so you can identify the broken dependency. Payload logging cannot continue until the table is restored or the configuration is updated.

etagstringOutput only

Optimistic concurrency token returned on every read. To make an Update or Delete conditional, pass the last-read value in that request's etag field. In REST responses, this value is a base64 string; URL-encode it when setting the etag query parameter.

supported_api_typesarray of stringOutput onlyUnordered list

API types supported across this service's destinations, such as openai/v1/chat/completions, openai/v1/embeddings, and mlflow/v1/chat/completions. Derived from the backing models and providers at read time.

View as Markdown

The ModelService object

{
  "name": "string",
  "effective_owner": "string",
  "metastore_id": "string",
  "create_time": "string",
  "created_by": "string",
  "update_time": "string",
  "updated_by": "string",
  "comment": "string",
  "config": {
    "routing": {},
    "rate_limits": [
      {}
    ],
    "inference_table": {}
  },
  "etag": "string",
  "supported_api_types": [
    "string"
  ]
}

Get a model service GA

GET /api/2.1/unity-catalog/{name=model-services/*}

Returns the model service identified by its resource name.

You must be the owner of the model service or have EXECUTE, READ_METADATA, or MANAGE on it, plus USE_CATALOG on the parent catalog and USE_SCHEMA on the parent schema.

API scopes: unity-catalog

Parameters

namestringRequiredpath

Resource name of the model service. Format: model-services/{catalog}.{schema}.{model_service}. Each {...} component is capped at 255 characters individually.

Response

Returns the ModelService object.

View as Markdown

Response

{
  "name": "string",
  "effective_owner": "string",
  "metastore_id": "string",
  "create_time": "string",
  "created_by": "string",
  "update_time": "string",
  "updated_by": "string",
  "comment": "string",
  "config": {
    "routing": {},
    "rate_limits": [
      {}
    ],
    "inference_table": {}
  },
  "etag": "string",
  "supported_api_types": [
    "string"
  ]
}

List model services GA

GET /api/2.1/unity-catalog/model-services

Lists the model services in a Unity Catalog schema. Provide parent as schemas/{catalog}.{schema}. Results are paginated; pass the returned next_page_token to fetch subsequent pages.

Requires USE_CATALOG on the parent catalog and USE_SCHEMA on the parent schema. Only model services the caller can access (as owner or through EXECUTE, READ_METADATA, or MANAGE) are returned.

API scopes: unity-catalog

Parameters

parentstringquery

Parent schema to list within, in the form schemas/{catalog}.{schema}. Required. Each {...} component is capped at 255 characters individually.

page_sizeint32query

Maximum number of model services to return. Defaults to 100 when unset or 0; the maximum is 100. Use page_token to retrieve additional pages.

Constraints: <= 100

page_tokenstringquery

Opaque pagination token from the previous response.

viewstringquery

Fields to return for each service. FULL includes destinations, inference-table details, and rate-limit principal names. BASIC omits destinations and inference-table details and omits principal names from rate limits. Defaults to BASIC when unset.

Values:

• VIEW_UNSPECIFIED
• FULL
• BASIC

Response

Returns a list of ModelService objects.

View as Markdown

Response

{
  "model_services": [
    {
      "name": "string",
      "effective_owner": "string",
      "metastore_id": "string",
      "create_time": "string",
      "created_by": "string",
      "update_time": "string",
      "updated_by": "string",
      "comment": "string",
      "config": {},
      "etag": "string",
      "supported_api_types": [
        "string"
      ]
    }
  ],
  "next_page_token": "string"
}

Create a model service GA

POST /api/2.1/unity-catalog/model-services

Creates a model service in a Unity Catalog schema. A model service is a governed AI Gateway endpoint that routes inference requests to one or more model destinations. Specify its name in model_service_id.

You must be the owner of the parent schema or have the CREATE_SERVICE and USE_SCHEMA privileges on the parent schema and USE_CATALOG on the parent catalog. For every destination, you also need USE_CATALOG and USE_SCHEMA on its parent and EXECUTE on the referenced Unity Catalog model or model provider service. A provisioned-throughput destination additionally requires CAN_MANAGE on its Model Serving endpoint. Configuring an inference table additionally requires CREATE_TABLE.

API scopes: unity-catalog

Parameters

parentstringRequiredquery

Name of the parent schema. Format: schemas/{catalog}.{schema}. Each {...} component is capped at 255 characters individually.

model_service_idstringRequiredquery

Name for the model service, e.g. "my_model_service".

Request body

The model service to create. Do not set name; the server derives it from parent and model_service_id.

namestringImmutable

Resource name of the model service. Format: model-services/{catalog}.{schema}.{model_service}. Each {...} component is capped at 255 characters individually. Server-derived on Create from parent + model_service_id; required and immutable on Update/Get/Delete.

commentstring

User-provided description.

Constraints: <= 65536 characters

configobject

Destinations, routing, rate limits, and payload logging configuration. Required on Create. On Update, provide this field when update_mask contains config or one of its subpaths.

Show child attributesHide child attributes

routingobject

Routing configuration: destinations and fallback.

Show child attributesHide child attributes

destinationsarray of object

Primary routing destinations. At most 10 are allowed. At least one is required on Create. On Update, provide this list when replacing the full config or updating config.routing.destinations; other granular routing updates do not require resending destinations. The intermediate config.routing mask path is not supported.

Show child attributesHide child attributes

namestringRequired

User-facing label for this destination, used in routing references.

destination_typestringRequired

Backing-model category. Provide the matching type-specific configuration and leave the other type-specific configurations unset.

Values:

• DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL
• DESTINATION_TYPE_PROVISIONED_THROUGHPUT_FOUNDATION_MODEL
• DESTINATION_TYPE_EXTERNAL_FOUNDATION_MODEL

traffic_percentageint32

Percentage of primary traffic sent to this destination, from 0 to 100. Required when there is more than one primary destination, in which case the primary percentages must sum to 100; a single primary destination receives all traffic. Fallback destinations are ordered and do not use this field.

pay_per_token_configobject

Configuration for a pay-per-token <Databricks> foundation model.

provisioned_throughput_configobject

Configuration for a provisioned-throughput <Databricks> foundation model.

external_model_configobject

Configuration for an external model reached through a model provider service.

fallbackobject

Fallback routing applied after a primary destination fails. Fallback destinations are tried in the listed order.

Show child attributesHide child attributes

destinationsarray of object

Fallback destinations, tried in the listed order. At most 5 are allowed.

rate_limitsarray of object

Rate limits applied to requests routed through this model service.

Show child attributesHide child attributes

keystringRequired

Scope of the rate limit. Depending on this value, the limit applies to a principal, the service as a whole, or each user by default.

Values:

• RATE_LIMIT_KEY_USER
• RATE_LIMIT_KEY_USER_GROUP
• RATE_LIMIT_KEY_SERVICE_PRINCIPAL
• RATE_LIMIT_KEY_SERVICE
• RATE_LIMIT_KEY_USER_DEFAULT

renewal_periodstringRequired

Renewal period.

Values:

• RATE_LIMIT_RENEWAL_PERIOD_MINUTE
• RATE_LIMIT_RENEWAL_PERIOD_HOUR

principalstring

Principal this limit applies to: user email, group name, or service principal application ID. Required when key applies to a user, group, or service principal; otherwise it must be unset.

requestsint64

Maximum requests allowed in one renewal period. Leave unset for no request limit. Set to 0 to deny all requests.

tokensint64

Maximum tokens allowed in one renewal period. Leave unset for no token limit. Set to 0 to deny all requests.

inference_tableobject

Inference table configuration for payload logging.

Show child attributesHide child attributes

parentstringRequired

Parent Unity Catalog schema where the inference table is created, in the form schemas/{catalog}.{schema}. Required when configuring an inference table. After the inference table is created, this field cannot be changed.

table_name_prefixstring

Prefix used to form the inference table's registered name. AI Gateway appends _payload; for example, table_name_prefix = "orders" creates orders_payload. If unset, the prefix defaults to the service name. Read table from the response for the resulting resource name. After the inference table is created, this field cannot be changed.

Response

Returns the ModelService object.

View as Markdown

Request

{
  "name": "string",
  "comment": "string",
  "config": {
    "routing": {},
    "rate_limits": [
      {}
    ],
    "inference_table": {}
  }
}

Response

{
  "name": "string",
  "effective_owner": "string",
  "metastore_id": "string",
  "create_time": "string",
  "created_by": "string",
  "update_time": "string",
  "updated_by": "string",
  "comment": "string",
  "config": {
    "routing": {},
    "rate_limits": [
      {}
    ],
    "inference_table": {}
  },
  "etag": "string",
  "supported_api_types": [
    "string"
  ]
}

Update a model service GA

PATCH /api/2.1/unity-catalog/{name=model-services/*}

Updates a model service. Only the fields named in update_mask are changed; the resource name is immutable. Optionally supply an etag to make the update conditional on the model service not having changed since it was read.

You must be the owner of the model service or have MANAGE on it, plus USE_CATALOG on the parent catalog and USE_SCHEMA on the parent schema. When changing destinations, both you and the model service owner need USE_CATALOG and USE_SCHEMA on each destination's parent and EXECUTE on the referenced Unity Catalog model or model provider service. A provisioned-throughput destination additionally requires CAN_MANAGE for you and CAN_QUERY for the model service owner. Adding an inference table additionally requires CREATE_TABLE.

API scopes: unity-catalog

Parameters

namestringRequiredImmutablepath

Resource name of the model service. Format: model-services/{catalog}.{schema}.{model_service}. Each {...} component is capped at 255 characters individually. Server-derived on Create from parent + model_service_id; required and immutable on Update/Get/Delete.

update_maskstringRequiredquery

Fields to update. Use config to replace the entire configuration. The replacement must include every required field; any optional field you omit is cleared. To preserve sibling fields, use one or more granular paths: comment, config.routing.destinations, config.routing.fallback.destinations, config.rate_limits, or config.inference_table. Intermediate paths such as config.routing and config.routing.fallback, and wildcard paths such as *, are not supported.

etagstringquery

Optimistic concurrency token from the most recent read. When set, the update succeeds only if the resource has not changed. Leave unset for an unconditional update. For REST requests, URL-encode the base64 string returned by the API when setting the etag query parameter.

Request body

The model service with the updated field values. name identifies the resource (model-services/{catalog}.{schema}.{model_service}); only fields listed in update_mask are applied.

commentstring

User-provided description.

Constraints: <= 65536 characters

configobject

Destinations, routing, rate limits, and payload logging configuration. Required on Create. On Update, provide this field when update_mask contains config or one of its subpaths.

Show child attributesHide child attributes

routingobject

Routing configuration: destinations and fallback.

Show child attributesHide child attributes

destinationsarray of object

Primary routing destinations. At most 10 are allowed. At least one is required on Create. On Update, provide this list when replacing the full config or updating config.routing.destinations; other granular routing updates do not require resending destinations. The intermediate config.routing mask path is not supported.

Show child attributesHide child attributes

namestringRequired

User-facing label for this destination, used in routing references.

destination_typestringRequired

Backing-model category. Provide the matching type-specific configuration and leave the other type-specific configurations unset.

Values:

• DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL
• DESTINATION_TYPE_PROVISIONED_THROUGHPUT_FOUNDATION_MODEL
• DESTINATION_TYPE_EXTERNAL_FOUNDATION_MODEL

traffic_percentageint32

Percentage of primary traffic sent to this destination, from 0 to 100. Required when there is more than one primary destination, in which case the primary percentages must sum to 100; a single primary destination receives all traffic. Fallback destinations are ordered and do not use this field.

pay_per_token_configobject

Configuration for a pay-per-token <Databricks> foundation model.

provisioned_throughput_configobject

Configuration for a provisioned-throughput <Databricks> foundation model.

external_model_configobject

Configuration for an external model reached through a model provider service.

fallbackobject

Fallback routing applied after a primary destination fails. Fallback destinations are tried in the listed order.

Show child attributesHide child attributes

destinationsarray of object

Fallback destinations, tried in the listed order. At most 5 are allowed.

rate_limitsarray of object

Rate limits applied to requests routed through this model service.

Show child attributesHide child attributes

keystringRequired

Scope of the rate limit. Depending on this value, the limit applies to a principal, the service as a whole, or each user by default.

Values:

• RATE_LIMIT_KEY_USER
• RATE_LIMIT_KEY_USER_GROUP
• RATE_LIMIT_KEY_SERVICE_PRINCIPAL
• RATE_LIMIT_KEY_SERVICE
• RATE_LIMIT_KEY_USER_DEFAULT

renewal_periodstringRequired

Renewal period.

Values:

• RATE_LIMIT_RENEWAL_PERIOD_MINUTE
• RATE_LIMIT_RENEWAL_PERIOD_HOUR

principalstring

Principal this limit applies to: user email, group name, or service principal application ID. Required when key applies to a user, group, or service principal; otherwise it must be unset.

requestsint64

Maximum requests allowed in one renewal period. Leave unset for no request limit. Set to 0 to deny all requests.

tokensint64

Maximum tokens allowed in one renewal period. Leave unset for no token limit. Set to 0 to deny all requests.

inference_tableobject

Inference table configuration for payload logging.

Show child attributesHide child attributes

parentstringRequired

Parent Unity Catalog schema where the inference table is created, in the form schemas/{catalog}.{schema}. Required when configuring an inference table. After the inference table is created, this field cannot be changed.

table_name_prefixstring

Prefix used to form the inference table's registered name. AI Gateway appends _payload; for example, table_name_prefix = "orders" creates orders_payload. If unset, the prefix defaults to the service name. Read table from the response for the resulting resource name. After the inference table is created, this field cannot be changed.

Response

Returns the ModelService object.

View as Markdown

Request

{
  "comment": "string",
  "config": {
    "routing": {},
    "rate_limits": [
      {}
    ],
    "inference_table": {}
  }
}

Response

{
  "name": "string",
  "effective_owner": "string",
  "metastore_id": "string",
  "create_time": "string",
  "created_by": "string",
  "update_time": "string",
  "updated_by": "string",
  "comment": "string",
  "config": {
    "routing": {},
    "rate_limits": [
      {}
    ],
    "inference_table": {}
  },
  "etag": "string",
  "supported_api_types": [
    "string"
  ]
}

Delete a model service GA

DELETE /api/2.1/unity-catalog/{name=model-services/*}

Deletes the model service identified by its resource name. Optionally supply an etag to make the delete conditional on the model service not having changed since it was read.

You must be the owner of the model service or have MANAGE on it, plus USE_CATALOG on the parent catalog and USE_SCHEMA on the parent schema.

API scopes: unity-catalog

Parameters

namestringRequiredpath

Resource name of the model service. Format: model-services/{catalog}.{schema}.{model_service}. Each {...} component is capped at 255 characters individually.

etagstringquery

Optimistic concurrency token from the most recent read. When set, the delete succeeds only if the resource has not changed. Leave unset for an unconditional delete. For REST requests, URL-encode the base64 string returned by the API when setting the etag query parameter.

View as Markdown

Response

{}