Model Service

View as Markdown

A governed AI Gateway endpoint in Unity Catalog that routes inference requests to one or more model destinations (for example a foundation model or an external LLM reached through a ModelProviderService). Applies centralized access control, rate limits, guardrails, and auditing to the traffic it serves.

GetGET /api/2.1/unity-catalog/{name=model-services/*}ListGET /api/2.1/unity-catalog/model-servicesCreatePOST /api/2.1/unity-catalog/model-servicesUpdatePATCH /api/2.1/unity-catalog/{model_service.name=model-services/*}DeleteDELETE /api/2.1/unity-catalog/{name=model-services/*}

ModelService object

A governed AI Gateway endpoint in Unity Catalog that routes inference requests to one or more model destinations (for example a foundation model or an external LLM reached through a ModelProviderService). Applies centralized access control, rate limits, guardrails, and auditing to the traffic it serves.

namestring

Resource name of the model service. Format: model-services/{catalog}.{schema}.{model_service}. Each {...} component is capped at 255 characters individually. Server-derived on Create from parent + model_service_id; required and immutable on Update/Get/Delete.

ownerstring

The owner of the model service. Write-only; read owner via effective_owner.

effective_ownerstring

The resolved owner of the ModelService. Falls back to the caller's identity when owner is not explicitly set on creation.

metastore_idstring

Metastore hosting the model service.

create_timestring

When the model service was created.

created_bystring

Creator identity.

update_timestring

When the model service was last modified.

updated_bystring

Identity of the last updater.

commentstring

User-provided description.

Constraints: <= 65536 characters

configobject

Operational configuration: destinations, routing, rate limits, inference table. Required on CreateModelService; on UpdateModelService it is required only when config (or a config.* subpath) appears in update_mask.

Show child attributesHide child attributes

routingobject

Routing configuration: destinations, routing strategy, and fallback.

Show child attributesHide child attributes

destinationsarray of object

Primary routing destinations. At most 10 are allowed. At least one is required on CreateModelService; on UpdateModelService it is required only when config.routing (or a config.routing.* subpath) appears in update_mask.

Show child attributesHide child attributes

namestring

User-facing label for this destination, used in routing references.

destination_typestring

Backing-model category. Determines which oneof variant is populated.

Values: DESTINATION_TYPE_PAY_PER_TOKEN_FOUNDATION_MODEL, DESTINATION_TYPE_PROVISIONED_THROUGHPUT_FOUNDATION_MODEL, DESTINATION_TYPE_EXTERNAL_FOUNDATION_MODEL

traffic_percentageint32

Share of traffic sent to this destination, 0-100. Optional on fallback destinations; see FallbackConfig.

pay_per_token_configobject

provisioned_throughput_configobject

external_model_configobject

is_deletedboolean

True when the destination's backing UC entity (MODEL for foundation-model destinations, MODEL_PROVIDER_SERVICE for external destinations) has been deleted but the destination row still references it. The dangling destination is surfaced (not silently dropped) so callers can see the broken routing. Inference traffic through this destination fails closed (BAD_REQUEST / FAILED_PRECONDITION).

traffic_splittingobject

Marker message selecting request-based traffic splitting. Traffic is distributed according to each destination's traffic_percentage value; no configuration lives on this message itself.

fallbackobject

Fallback routing config, applied after primary destinations fail.

Show child attributesHide child attributes

destinationsarray of object

Ordered list of fallback destinations. Traversal is in list order; the attempt count is the length of the list. At most 5 are allowed.

first_token_timeoutstring

Timeout for the first token of a streaming response. If a destination does not return its first token within this duration, AI Gateway aborts the attempt and fails over to the next destination. Applies to streaming requests only. Leave unset for no first-token timeout.

rate_limitsarray of object

Rate limits applied to requests routed through this model service.

Show child attributesHide child attributes

keystring

Scope key. Determines whether principal is required.

Values: RATE_LIMIT_KEY_USER, RATE_LIMIT_KEY_USER_GROUP, RATE_LIMIT_KEY_SERVICE_PRINCIPAL, RATE_LIMIT_KEY_SERVICE, RATE_LIMIT_KEY_USER_DEFAULT, RATE_LIMIT_KEY_REQUEST_TAG

renewal_periodstring

Renewal period.

Values: RATE_LIMIT_RENEWAL_PERIOD_MINUTE, RATE_LIMIT_RENEWAL_PERIOD_HOUR

principalstring

Principal this limit applies to: user email, group name, or service principal application ID. Required unless key is RATE_LIMIT_KEY_SERVICE, RATE_LIMIT_KEY_USER_DEFAULT, or RATE_LIMIT_KEY_REQUEST_TAG (which must not set a principal).

requestsint64

Max requests allowed within a renewal period. Leave unset for no request limit.

tokensint64

Max tokens allowed within a renewal period. Leave unset for no token limit.

request_tag_keystring

Request tag key this limit applies to. Required when key is RATE_LIMIT_KEY_REQUEST_TAG, forbidden otherwise.

request_tag_valuestring

Request tag value this limit applies to. Only valid when key is RATE_LIMIT_KEY_REQUEST_TAG. Leave unset to apply the limit to every value of request_tag_key (an any-value default); a set value is a specific override for that value.

inference_tableobject

Inference table config for payload logging.

Show child attributesHide child attributes

parentstring

Parent UC schema where the inference table is created. Format: schemas/{catalog}.{schema}. Set at create time and immutable thereafter; changing it on an existing service is rejected.

table_name_prefixstring

Prefix for the inference-table's UC-registered name. The actual leaf name UC stores is <table_name_prefix>_payload; the _payload suffix is appended automatically. To find the actual UC table after Create, read the table field on the response. Defaults to <model_service_name>_payload when unset. Set at create time and immutable thereafter; changing it on an existing service is rejected.

disabledboolean

Indicates whether payload logging is disabled (opt-out). Unset means that payload logging is active (the on-by-default state coincides with the proto zero-value, so the server never fills this field for a client that leaves it unset). Set disabled = true to pause runtime logging while keeping the sub-message attached (preserving parent and table_name_prefix for a later flip back to active). parent remains required either way.

tablestring

Resolved UC table for payload logs. Format: tables/{catalog}.{schema}.{table}.

is_deletedboolean

True when the bound inference TABLE has been deleted but the parent service still references it. The dangling reference is surfaced (not silently dropped) so callers can see the broken dependency. AI Gateway payload logging fails closed in this state.

etagstring

Optimistic concurrency control token. Server-generated from the entity's state and returned on every read. To use it as an if-match precondition on a mutation, echo the last-read value back via the dedicated etag field on the Update / Delete request; the server rejects the mutation if the stored etag differs.

supported_api_typesarray of string

Unified API types this endpoint supports (e.g. "chat", "embeddings", "completions"). Derived from the destinations' backing models / providers at read time.

View as Markdown

The ModelService object

{
  "name": "string",
  "owner": "string",
  "effective_owner": "string",
  "metastore_id": "string",
  "create_time": "string",
  "created_by": "string",
  "update_time": "string",
  "updated_by": "string",
  "comment": "string",
  "config": {
    "routing": {},
    "rate_limits": [
      {}
    ],
    "inference_table": {}
  },
  "etag": "string",
  "supported_api_types": [
    "string"
  ]
}

Get

GET /api/2.1/unity-catalog/{name=model-services/*}

Returns the model service identified by its resource name.

You must be the owner of the model service or have EXECUTE, READ_METADATA, or MANAGE on it, plus USE_CATALOG on the parent catalog and USE_SCHEMA on the parent schema.

API scopes: unity-catalog

Parameters

namestringpath

Resource name of the model service. Format: model-services/{catalog}.{schema}.{model_service}. Each {...} component is capped at 255 characters individually.

Response

Returns the ModelService object.

View as Markdown

Response

{
  "name": "string",
  "owner": "string",
  "effective_owner": "string",
  "metastore_id": "string",
  "create_time": "string",
  "created_by": "string",
  "update_time": "string",
  "updated_by": "string",
  "comment": "string",
  "config": {
    "routing": {},
    "rate_limits": [
      {}
    ],
    "inference_table": {}
  },
  "etag": "string",
  "supported_api_types": [
    "string"
  ]
}

List

GET /api/2.1/unity-catalog/model-services

Lists the model services in a Unity Catalog schema. Provide parent as schemas/{catalog}.{schema}. Results are paginated; pass the returned next_page_token to fetch subsequent pages.

Requires USE_CATALOG on the parent catalog and USE_SCHEMA on the parent schema. Only model services the caller can access (as owner or through EXECUTE, READ_METADATA, or MANAGE) are returned.

API scopes: unity-catalog

Parameters

parentstringquery

Name of the parent schema to list within, as schemas/{catalog}.{schema}. Each {...} component is capped at 255 characters individually.

page_sizeint32query

Maximum number of model services to return. Defaults to 100 when unset or 0; the maximum is 100. Use page_token to retrieve additional pages.

Constraints: <= 100

page_tokenstringquery

Opaque pagination token from a previous request.

viewstringquery

View selector controlling which fields are populated per row. FULL returns the full representation of the service; BASIC returns a more compact version. Defaults to BASIC when unset.

Values: VIEW_UNSPECIFIED, FULL, BASIC

Response

Returns a list of ModelService objects.

View as Markdown

Response

{
  "model_services": [
    {
      "name": "string",
      "owner": "string",
      "effective_owner": "string",
      "metastore_id": "string",
      "create_time": "string",
      "created_by": "string",
      "update_time": "string",
      "updated_by": "string",
      "comment": "string",
      "config": {},
      "etag": "string",
      "supported_api_types": [
        "string"
      ]
    }
  ],
  "next_page_token": "string"
}

Create

POST /api/2.1/unity-catalog/model-services

Creates a model service in a Unity Catalog schema. A model service is a governed AI Gateway endpoint that routes inference requests to one or more model destinations. The caller supplies the leaf name in model_service_id.

You must be the owner of the parent schema or have the CREATE_SERVICE and USE_SCHEMA privileges on the parent schema and USE_CATALOG on the parent catalog.

API scopes: unity-catalog

Parameters

parentstringquery

Name of the parent schema. Format: schemas/{catalog}.{schema}. Each {...} component is capped at 255 characters individually.

model_service_idstringquery

Name for the model service, e.g. "my_model_service".

Request body

model_serviceobject

The model service to create. The server populates name from parent + model_service_id; clients should leave it unset.

Show child attributesHide child attributes

namestring

Resource name of the model service. Format: model-services/{catalog}.{schema}.{model_service}. Each {...} component is capped at 255 characters individually. Server-derived on Create from parent + model_service_id; required and immutable on Update/Get/Delete.

ownerstring

The owner of the model service. Write-only; read owner via effective_owner.

effective_ownerstring

The resolved owner of the ModelService. Falls back to the caller's identity when owner is not explicitly set on creation.

metastore_idstring

Metastore hosting the model service.

create_timestring

When the model service was created.

created_bystring

Creator identity.

update_timestring

When the model service was last modified.

updated_bystring

Identity of the last updater.

commentstring

User-provided description.

Constraints: <= 65536 characters

configobject

Operational configuration: destinations, routing, rate limits, inference table. Required on CreateModelService; on UpdateModelService it is required only when config (or a config.* subpath) appears in update_mask.

Show child attributesHide child attributes

routingobject

Routing configuration: destinations, routing strategy, and fallback.

Show child attributesHide child attributes

destinationsarray of object

Primary routing destinations. At most 10 are allowed. At least one is required on CreateModelService; on UpdateModelService it is required only when config.routing (or a config.routing.* subpath) appears in update_mask.

traffic_splittingobjectRequired

Marker message selecting request-based traffic splitting. Traffic is distributed according to each destination's traffic_percentage value; no configuration lives on this message itself.

fallbackobject

Fallback routing config, applied after primary destinations fail.

first_token_timeoutstring

Timeout for the first token of a streaming response. If a destination does not return its first token within this duration, AI Gateway aborts the attempt and fails over to the next destination. Applies to streaming requests only. Leave unset for no first-token timeout.

rate_limitsarray of object

Rate limits applied to requests routed through this model service.

Show child attributesHide child attributes

keystring

Scope key. Determines whether principal is required.

Values: RATE_LIMIT_KEY_USER, RATE_LIMIT_KEY_USER_GROUP, RATE_LIMIT_KEY_SERVICE_PRINCIPAL, RATE_LIMIT_KEY_SERVICE, RATE_LIMIT_KEY_USER_DEFAULT, RATE_LIMIT_KEY_REQUEST_TAG

renewal_periodstring

Renewal period.

Values: RATE_LIMIT_RENEWAL_PERIOD_MINUTE, RATE_LIMIT_RENEWAL_PERIOD_HOUR

principalstring

Principal this limit applies to: user email, group name, or service principal application ID. Required unless key is RATE_LIMIT_KEY_SERVICE, RATE_LIMIT_KEY_USER_DEFAULT, or RATE_LIMIT_KEY_REQUEST_TAG (which must not set a principal).

requestsint64

Max requests allowed within a renewal period. Leave unset for no request limit.

tokensint64

Max tokens allowed within a renewal period. Leave unset for no token limit.

request_tag_keystring

Request tag key this limit applies to. Required when key is RATE_LIMIT_KEY_REQUEST_TAG, forbidden otherwise.

request_tag_valuestring

Request tag value this limit applies to. Only valid when key is RATE_LIMIT_KEY_REQUEST_TAG. Leave unset to apply the limit to every value of request_tag_key (an any-value default); a set value is a specific override for that value.

inference_tableobject

Inference table config for payload logging.

Show child attributesHide child attributes

parentstring

Parent UC schema where the inference table is created. Format: schemas/{catalog}.{schema}. Set at create time and immutable thereafter; changing it on an existing service is rejected.

table_name_prefixstring

Prefix for the inference-table's UC-registered name. The actual leaf name UC stores is <table_name_prefix>_payload; the _payload suffix is appended automatically. To find the actual UC table after Create, read the table field on the response. Defaults to <model_service_name>_payload when unset. Set at create time and immutable thereafter; changing it on an existing service is rejected.

disabledboolean

Indicates whether payload logging is disabled (opt-out). Unset means that payload logging is active (the on-by-default state coincides with the proto zero-value, so the server never fills this field for a client that leaves it unset). Set disabled = true to pause runtime logging while keeping the sub-message attached (preserving parent and table_name_prefix for a later flip back to active). parent remains required either way.

tablestring

Resolved UC table for payload logs. Format: tables/{catalog}.{schema}.{table}.

is_deletedboolean

True when the bound inference TABLE has been deleted but the parent service still references it. The dangling reference is surfaced (not silently dropped) so callers can see the broken dependency. AI Gateway payload logging fails closed in this state.

etagstring

Optimistic concurrency control token. Server-generated from the entity's state and returned on every read. To use it as an if-match precondition on a mutation, echo the last-read value back via the dedicated etag field on the Update / Delete request; the server rejects the mutation if the stored etag differs.

supported_api_typesarray of string

Unified API types this endpoint supports (e.g. "chat", "embeddings", "completions"). Derived from the destinations' backing models / providers at read time.

Response

Returns the ModelService object.

View as Markdown

Request

{
  "model_service": {
    "name": "string",
    "owner": "string",
    "effective_owner": "string",
    "metastore_id": "string",
    "create_time": "string",
    "created_by": "string",
    "update_time": "string",
    "updated_by": "string",
    "comment": "string",
    "config": {},
    "etag": "string",
    "supported_api_types": [
      "string"
    ]
  }
}

Response

{
  "name": "string",
  "owner": "string",
  "effective_owner": "string",
  "metastore_id": "string",
  "create_time": "string",
  "created_by": "string",
  "update_time": "string",
  "updated_by": "string",
  "comment": "string",
  "config": {
    "routing": {},
    "rate_limits": [
      {}
    ],
    "inference_table": {}
  },
  "etag": "string",
  "supported_api_types": [
    "string"
  ]
}

Update

PATCH /api/2.1/unity-catalog/{model_service.name=model-services/*}

Updates a model service. Only the fields named in update_mask are changed; the resource name is immutable. Optionally supply an etag to make the update conditional on the model service not having changed since it was read.

You must be the owner of the model service or have MANAGE on it, plus USE_CATALOG on the parent catalog and USE_SCHEMA on the parent schema.

API scopes: unity-catalog

Parameters

model_serviceobjectpath

The model service with the updated field values. name identifies the resource (model-services/{catalog}.{schema}.{model_service}); only fields listed in update_mask are applied.

Show child attributesHide child attributes

namestring

Resource name of the model service. Format: model-services/{catalog}.{schema}.{model_service}. Each {...} component is capped at 255 characters individually. Server-derived on Create from parent + model_service_id; required and immutable on Update/Get/Delete.

ownerstring

The owner of the model service. Write-only; read owner via effective_owner.

effective_ownerstring

The resolved owner of the ModelService. Falls back to the caller's identity when owner is not explicitly set on creation.

metastore_idstring

Metastore hosting the model service.

create_timestring

When the model service was created.

created_bystring

Creator identity.

update_timestring

When the model service was last modified.

updated_bystring

Identity of the last updater.

commentstring

User-provided description.

Constraints: <= 65536 characters

configobject

Operational configuration: destinations, routing, rate limits, inference table. Required on CreateModelService; on UpdateModelService it is required only when config (or a config.* subpath) appears in update_mask.

Show child attributesHide child attributes

routingobject

Routing configuration: destinations, routing strategy, and fallback.

Show child attributesHide child attributes

destinationsarray of object

Primary routing destinations. At most 10 are allowed. At least one is required on CreateModelService; on UpdateModelService it is required only when config.routing (or a config.routing.* subpath) appears in update_mask.

traffic_splittingobjectRequired

Marker message selecting request-based traffic splitting. Traffic is distributed according to each destination's traffic_percentage value; no configuration lives on this message itself.

fallbackobject

Fallback routing config, applied after primary destinations fail.

first_token_timeoutstring

Timeout for the first token of a streaming response. If a destination does not return its first token within this duration, AI Gateway aborts the attempt and fails over to the next destination. Applies to streaming requests only. Leave unset for no first-token timeout.

rate_limitsarray of object

Rate limits applied to requests routed through this model service.

Show child attributesHide child attributes

keystring

Scope key. Determines whether principal is required.

Values: RATE_LIMIT_KEY_USER, RATE_LIMIT_KEY_USER_GROUP, RATE_LIMIT_KEY_SERVICE_PRINCIPAL, RATE_LIMIT_KEY_SERVICE, RATE_LIMIT_KEY_USER_DEFAULT, RATE_LIMIT_KEY_REQUEST_TAG

renewal_periodstring

Renewal period.

Values: RATE_LIMIT_RENEWAL_PERIOD_MINUTE, RATE_LIMIT_RENEWAL_PERIOD_HOUR

principalstring

Principal this limit applies to: user email, group name, or service principal application ID. Required unless key is RATE_LIMIT_KEY_SERVICE, RATE_LIMIT_KEY_USER_DEFAULT, or RATE_LIMIT_KEY_REQUEST_TAG (which must not set a principal).

requestsint64

Max requests allowed within a renewal period. Leave unset for no request limit.

tokensint64

Max tokens allowed within a renewal period. Leave unset for no token limit.

request_tag_keystring

Request tag key this limit applies to. Required when key is RATE_LIMIT_KEY_REQUEST_TAG, forbidden otherwise.

request_tag_valuestring

Request tag value this limit applies to. Only valid when key is RATE_LIMIT_KEY_REQUEST_TAG. Leave unset to apply the limit to every value of request_tag_key (an any-value default); a set value is a specific override for that value.

inference_tableobject

Inference table config for payload logging.

Show child attributesHide child attributes

parentstring

Parent UC schema where the inference table is created. Format: schemas/{catalog}.{schema}. Set at create time and immutable thereafter; changing it on an existing service is rejected.

table_name_prefixstring

Prefix for the inference-table's UC-registered name. The actual leaf name UC stores is <table_name_prefix>_payload; the _payload suffix is appended automatically. To find the actual UC table after Create, read the table field on the response. Defaults to <model_service_name>_payload when unset. Set at create time and immutable thereafter; changing it on an existing service is rejected.

disabledboolean

Indicates whether payload logging is disabled (opt-out). Unset means that payload logging is active (the on-by-default state coincides with the proto zero-value, so the server never fills this field for a client that leaves it unset). Set disabled = true to pause runtime logging while keeping the sub-message attached (preserving parent and table_name_prefix for a later flip back to active). parent remains required either way.

tablestring

Resolved UC table for payload logs. Format: tables/{catalog}.{schema}.{table}.

is_deletedboolean

True when the bound inference TABLE has been deleted but the parent service still references it. The dangling reference is surfaced (not silently dropped) so callers can see the broken dependency. AI Gateway payload logging fails closed in this state.

etagstring

Optimistic concurrency control token. Server-generated from the entity's state and returned on every read. To use it as an if-match precondition on a mutation, echo the last-read value back via the dedicated etag field on the Update / Delete request; the server rejects the mutation if the stored etag differs.

supported_api_typesarray of string

Unified API types this endpoint supports (e.g. "chat", "embeddings", "completions"). Derived from the destinations' backing models / providers at read time.

update_maskstringquery

The list of fields to update. The framework validates each path against the model_service field above. Wildcard paths (paths: ["*"]) are not supported; list each field path explicitly.

etagstringquery

If-match precondition: when set, the update proceeds only if the current server-side etag matches. Empty means an unconditional update.

Response

Returns the ModelService object.

View as Markdown

Response

{
  "name": "string",
  "owner": "string",
  "effective_owner": "string",
  "metastore_id": "string",
  "create_time": "string",
  "created_by": "string",
  "update_time": "string",
  "updated_by": "string",
  "comment": "string",
  "config": {
    "routing": {},
    "rate_limits": [
      {}
    ],
    "inference_table": {}
  },
  "etag": "string",
  "supported_api_types": [
    "string"
  ]
}

Delete

DELETE /api/2.1/unity-catalog/{name=model-services/*}

Deletes the model service identified by its resource name. Optionally supply an etag to make the delete conditional on the model service not having changed since it was read.

You must be the owner of the model service or have MANAGE on it, plus USE_CATALOG on the parent catalog and USE_SCHEMA on the parent schema.

API scopes: unity-catalog

Parameters

namestringpath

Resource name of the model service. Format: model-services/{catalog}.{schema}.{model_service}. Each {...} component is capped at 255 characters individually.

etagstringquery

If-match precondition: when set, the delete proceeds only if the current server-side etag matches. Empty means unconditional delete.

View as Markdown

Response

{}