# Put Inference Endpoint Ai Gateway

`PUT /api/2.0/serving-endpoints/{name}/ai-gateway`

Used to update the AI Gateway of a serving endpoint. NOTE: External model, provisioned throughput, and pay-per-token endpoints are fully supported; agent endpoints currently only support inference tables.

API scopes: model-serving

## Path parameters

- `name` (string, optional)
  The name of the serving endpoint whose AI Gateway is being updated. This field is required.
  Example: `feed-ads`
  Constraints: `[ 1 .. 63 ] characters`

## Request body

- `usage_tracking_config` (object, optional)
  Configuration to enable usage tracking using system tables.
   These tables allow you to monitor operational usage on endpoints and their associated costs.
  - `enabled` (boolean, optional)
    Whether to enable usage tracking.
    Example: `true`
- `inference_table_config` (object, optional)
  Configuration for payload logging using inference tables.
   Use these tables to monitor and audit data being sent to and received from model APIs and to improve model quality.
  - `catalog_name` (string, optional)
    The name of the catalog in Unity Catalog. Required when enabling inference tables.
     NOTE: On update, you have to disable inference table first in order to change the catalog name.
    Example: `my-catalog`
  - `schema_name` (string, optional)
    The name of the schema in Unity Catalog. Required when enabling inference tables.
     NOTE: On update, you have to disable inference table first in order to change the schema name.
    Example: `my-schema`
  - `table_name_prefix` (string, optional)
    The prefix of the table in Unity Catalog.
     NOTE: On update, you have to disable inference table first in order to change the prefix name.
    Example: `my-prefix`
  - `enabled` (boolean, optional)
    Indicates whether the inference table is enabled.
    Example: `true`
- `rate_limits` (array of object, optional)
  Configuration for rate limits which can be set to limit endpoint traffic.
  - `calls` (int64, optional)
    Used to specify how many calls are allowed for a key within the renewal_period.
    Example: `15`
  - `key` (string, optional)
    Key field for a rate limit. Currently, 'user', 'user_group, 'service_principal', and 'endpoint' are supported,
     with 'endpoint' being the default if not specified.
  - `renewal_period` (string, optional)
    Renewal period field for a rate limit. Currently, only 'minute' is supported.
  - `principal` (string, optional)
    Principal field for a user, user group, or service principal to apply rate limiting to. Accepts a user email, group name, or service principal application ID.
    Example: `user@test.com`
  - `tokens` (int64, optional)
    Used to specify how many tokens are allowed for a key within the renewal_period.
    Example: `10000`
- `guardrails` (object, optional)
  Configuration for AI Guardrails to prevent unwanted data and unsafe data in requests and responses.
  - `input` (object, optional)
    Configuration for input guardrail filters.
    - `safety` (boolean, optional)
      Indicates whether the safety filter is enabled.
      Example: `true`
    - `pii` (object, optional)
      Configuration for guardrail PII filter.
      - `behavior` (string, optional)
        Configuration for input guardrail filters.
        Possible values: `BEHAVIOR_UNSPECIFIED`, `NONE`, `BLOCK`, `MASK`
        Example: `BLOCK`
    - `valid_topics` (array of string, optional)
      The list of allowed topics.
       Given a chat request, this guardrail flags the request if its topic is not in the allowed topics.
    - `invalid_keywords` (array of string, optional)
      List of invalid keywords.
       AI guardrail uses keyword or string matching to decide if the keyword exists in the request or response content.
  - `output` (object, optional)
    Configuration for output guardrail filters.
    - `safety` (boolean, optional)
      Indicates whether the safety filter is enabled.
      Example: `true`
    - `pii` (object, optional)
      Configuration for guardrail PII filter.
      - `behavior` (string, optional)
        Configuration for input guardrail filters.
        Possible values: `BEHAVIOR_UNSPECIFIED`, `NONE`, `BLOCK`, `MASK`
        Example: `BLOCK`
    - `valid_topics` (array of string, optional)
      The list of allowed topics.
       Given a chat request, this guardrail flags the request if its topic is not in the allowed topics.
    - `invalid_keywords` (array of string, optional)
      List of invalid keywords.
       AI guardrail uses keyword or string matching to decide if the keyword exists in the request or response content.
- `fallback_config` (object, optional)
  Configuration for traffic fallback which auto fallbacks to other served entities if the request to a served
   entity fails with certain error codes, to increase availability.
  - `enabled` (boolean, optional)
    Whether to enable traffic fallback. When a served entity in the serving endpoint returns specific error
     codes (e.g. 500), the request will automatically be round-robin attempted with other served entities in the same
     endpoint, following the order of served entity list, until a successful response is returned.
     If all attempts fail, return the last response with the error code.
    Example: `true`

## Returns

- `usage_tracking_config` (object, optional)
  Configuration to enable usage tracking using system tables.
   These tables allow you to monitor operational usage on endpoints and their associated costs.
  - `enabled` (boolean, optional)
    Whether to enable usage tracking.
    Example: `true`
- `inference_table_config` (object, optional)
  Configuration for payload logging using inference tables.
   Use these tables to monitor and audit data being sent to and received from model APIs and to improve model quality.
  - `catalog_name` (string, optional)
    The name of the catalog in Unity Catalog. Required when enabling inference tables.
     NOTE: On update, you have to disable inference table first in order to change the catalog name.
    Example: `my-catalog`
  - `schema_name` (string, optional)
    The name of the schema in Unity Catalog. Required when enabling inference tables.
     NOTE: On update, you have to disable inference table first in order to change the schema name.
    Example: `my-schema`
  - `table_name_prefix` (string, optional)
    The prefix of the table in Unity Catalog.
     NOTE: On update, you have to disable inference table first in order to change the prefix name.
    Example: `my-prefix`
  - `enabled` (boolean, optional)
    Indicates whether the inference table is enabled.
    Example: `true`
- `rate_limits` (array of object, optional)
  Configuration for rate limits which can be set to limit endpoint traffic.
  - `calls` (int64, optional)
    Used to specify how many calls are allowed for a key within the renewal_period.
    Example: `15`
  - `key` (string, optional)
    Key field for a rate limit. Currently, 'user', 'user_group, 'service_principal', and 'endpoint' are supported,
     with 'endpoint' being the default if not specified.
  - `renewal_period` (string, optional)
    Renewal period field for a rate limit. Currently, only 'minute' is supported.
  - `principal` (string, optional)
    Principal field for a user, user group, or service principal to apply rate limiting to. Accepts a user email, group name, or service principal application ID.
    Example: `user@test.com`
  - `tokens` (int64, optional)
    Used to specify how many tokens are allowed for a key within the renewal_period.
    Example: `10000`
- `guardrails` (object, optional)
  Configuration for AI Guardrails to prevent unwanted data and unsafe data in requests and responses.
  - `input` (object, optional)
    Configuration for input guardrail filters.
    - `safety` (boolean, optional)
      Indicates whether the safety filter is enabled.
      Example: `true`
    - `pii` (object, optional)
      Configuration for guardrail PII filter.
      - `behavior` (string, optional)
        Configuration for input guardrail filters.
        Possible values: `BEHAVIOR_UNSPECIFIED`, `NONE`, `BLOCK`, `MASK`
        Example: `BLOCK`
    - `valid_topics` (array of string, optional)
      The list of allowed topics.
       Given a chat request, this guardrail flags the request if its topic is not in the allowed topics.
    - `invalid_keywords` (array of string, optional)
      List of invalid keywords.
       AI guardrail uses keyword or string matching to decide if the keyword exists in the request or response content.
  - `output` (object, optional)
    Configuration for output guardrail filters.
    - `safety` (boolean, optional)
      Indicates whether the safety filter is enabled.
      Example: `true`
    - `pii` (object, optional)
      Configuration for guardrail PII filter.
      - `behavior` (string, optional)
        Configuration for input guardrail filters.
        Possible values: `BEHAVIOR_UNSPECIFIED`, `NONE`, `BLOCK`, `MASK`
        Example: `BLOCK`
    - `valid_topics` (array of string, optional)
      The list of allowed topics.
       Given a chat request, this guardrail flags the request if its topic is not in the allowed topics.
    - `invalid_keywords` (array of string, optional)
      List of invalid keywords.
       AI guardrail uses keyword or string matching to decide if the keyword exists in the request or response content.
- `fallback_config` (object, optional)
  Configuration for traffic fallback which auto fallbacks to other served entities if the request to a served
   entity fails with certain error codes, to increase availability.
  - `enabled` (boolean, optional)
    Whether to enable traffic fallback. When a served entity in the serving endpoint returns specific error
     codes (e.g. 500), the request will automatically be round-robin attempted with other served entities in the same
     endpoint, following the order of served entity list, until a successful response is returned.
     If all attempts fail, return the last response with the error code.
    Example: `true`

## Request

```json
{
  "guardrails": {
    "input": {
      "invalid_keywords": [
        "keyword1"
      ],
      "pii": {
        "behavior": "BLOCK"
      },
      "safety": true,
      "valid_topics": [
        "topic1"
      ]
    },
    "output": {
      "invalid_keywords": [
        "keyword1"
      ],
      "pii": {
        "behavior": "BLOCK"
      },
      "safety": true,
      "valid_topics": [
        "topic1"
      ]
    }
  },
  "inference_table_config": {
    "catalog_name": "my-catalog",
    "enabled": true,
    "schema_name": "my-schema",
    "table_name_prefix": "my-prefix"
  },
  "rate_limits": [
    {
      "calls": 15,
      "key": "user",
      "renewal_period": "minute"
    }
  ],
  "usage_tracking_config": {
    "enabled": true
  }
}
```

## Response

```json
{
  "guardrails": {
    "input": {
      "invalid_keywords": [
        "keyword1"
      ],
      "pii": {
        "behavior": "BLOCK"
      },
      "safety": true,
      "valid_topics": [
        "topic1"
      ]
    },
    "output": {
      "invalid_keywords": [
        "keyword1"
      ],
      "pii": {
        "behavior": "BLOCK"
      },
      "safety": true,
      "valid_topics": [
        "topic1"
      ]
    }
  },
  "inference_table_config": {
    "catalog_name": "my-catalog",
    "enabled": true,
    "schema_name": "my-schema",
    "table_name_prefix": "my-prefix"
  },
  "rate_limits": [
    {
      "calls": 15,
      "key": "user",
      "renewal_period": "minute"
    }
  ],
  "usage_tracking_config": {
    "enabled": true
  }
}
```