# Create a new serving endpoint

Launch stage: GA

`POST /api/2.0/serving-endpoints`

Create a new serving endpoint.

API scopes: model-serving

## Request body

- `name` (string, required)
  The name of the serving endpoint. This field is required and must be unique across a <Workspace>.
   An endpoint name can consist of alphanumeric characters, dashes, and underscores.
  Example: `feed-ads`
- `config` (object, optional)
  The core config of the serving endpoint.
  - `served_entities` (array of object, optional)
    The list of served entities under the serving endpoint config.
    - `name` (string, optional)
      The name of a served entity. It must be unique across an endpoint. A served entity name can consist of alphanumeric characters, dashes, and underscores. If not specified for an external model, this field defaults to external_model.name, with '.' and ':' replaced with '-', and if not specified for other entities, it defaults to entity_name-entity_version.
      Example: `ads-model-3`
    - `external_model` (object, optional)
      The external model to be served. NOTE: Only one of external_model and (entity_name, entity_version, workload_size, workload_type, and scale_to_zero_enabled) can be specified with the latter set being used for custom model serving for a Databricks registered model. For an existing endpoint with external_model, it cannot be updated to an endpoint without external_model. If the endpoint is created without external_model, users cannot update it to add external_model later. The task type of all external models within an endpoint must be the same.
      - `provider` (string, required)
        The name of the provider for the external model. Currently, the supported providers are 'ai21labs', 'anthropic', 'amazon-bedrock', 'cohere', 'databricks-model-serving', 'google-cloud-vertex-ai', 'openai', 'palm', and 'custom'.
      - `name` (string, required)
        The name of the external model.
        Example: `gpt-4`
      - `task` (string, required)
        The task type of the external model.
        Example: `llm/v1/chat`
      - `ai21labs_config` (object, optional)
        AI21Labs Config. Only required if the provider is 'ai21labs'.
        - `ai21labs_api_key` (string, optional)
          The Databricks secret key reference for an AI21 Labs API key. If you
           prefer to paste your API key directly, see `ai21labs_api_key_plaintext`.
           You must provide an API key using one of the following fields:
           `ai21labs_api_key` or `ai21labs_api_key_plaintext`.
          Example: `{{secrets/my_scope/my_ai21labs_api_key}}`
        - `ai21labs_api_key_plaintext` (string, optional, Input only)
          An AI21 Labs API key provided as a plaintext string. If you prefer to
           reference your key using Databricks Secrets, see `ai21labs_api_key`. You
           must provide an API key using one of the following fields:
           `ai21labs_api_key` or `ai21labs_api_key_plaintext`.
          Example: `Your API Key`
      - `anthropic_config` (object, optional)
        Anthropic Config. Only required if the provider is 'anthropic'.
        - `anthropic_api_key` (string, optional)
          The Databricks secret key reference for an Anthropic API key. If you
           prefer to paste your API key directly, see `anthropic_api_key_plaintext`.
           You must provide an API key using one of the following fields:
           `anthropic_api_key` or `anthropic_api_key_plaintext`.
          Example: `{{secrets/my_scope/my_anthropic_api_key}}`
        - `anthropic_api_key_plaintext` (string, optional, Input only)
          The Anthropic API key provided as a plaintext string. If you prefer to
           reference your key using Databricks Secrets, see `anthropic_api_key`. You
           must provide an API key using one of the following fields:
           `anthropic_api_key` or `anthropic_api_key_plaintext`.
          Example: `Your API Key`
      - `amazon_bedrock_config` (object, optional)
        Amazon Bedrock Config. Only required if the provider is 'amazon-bedrock'.
        - `aws_region` (string, required)
          The AWS region to use. Bedrock has to be enabled there.
          Example: `myAwsRegion`
        - `aws_access_key_id` (string, optional)
          The Databricks secret key reference for an AWS access key ID with
           permissions to interact with Bedrock services. If you prefer to paste
           your API key directly, see `aws_access_key_id_plaintext`. You must provide an API
           key using one of the following fields: `aws_access_key_id` or
           `aws_access_key_id_plaintext`.
          Example: `{{secrets/my_scope/my_aws_access_key_id}}`
        - `aws_secret_access_key` (string, optional)
          The Databricks secret key reference for an AWS secret access key paired
           with the access key ID, with permissions to interact with Bedrock
           services. If you prefer to paste your API key directly, see
           `aws_secret_access_key_plaintext`. You must provide an API key using one
           of the following fields: `aws_secret_access_key` or
           `aws_secret_access_key_plaintext`.
          Example: `{{secrets/my_scope/my_aws_secret_access_key}}`
        - `bedrock_provider` (string, required)
          The underlying provider in Amazon Bedrock. Supported values (case
           insensitive) include: Anthropic, Cohere, AI21Labs, Amazon.
        - `aws_access_key_id_plaintext` (string, optional, Input only)
          An AWS access key ID with permissions to interact with Bedrock services
           provided as a plaintext string. If you prefer to reference your key using
           Databricks Secrets, see `aws_access_key_id`. You must provide an API key
           using one of the following fields: `aws_access_key_id` or
           `aws_access_key_id_plaintext`.
          Example: `Your AWS Access Key ID`
        - `aws_secret_access_key_plaintext` (string, optional, Input only)
          An AWS secret access key paired with the access key ID, with permissions
           to interact with Bedrock services provided as a plaintext string. If you
           prefer to reference your key using Databricks Secrets, see
           `aws_secret_access_key`. You must provide an API key using one of the
           following fields: `aws_secret_access_key` or
           `aws_secret_access_key_plaintext`.
          Example: `Your AWS Secret Access Key`
        - `instance_profile_arn` (string, optional)
          ARN of the instance profile that the external model will use to access AWS resources.
           You must authenticate using an instance profile or access keys.
           If you prefer to authenticate using access keys, see `aws_access_key_id`,
           `aws_access_key_id_plaintext`, `aws_secret_access_key` and `aws_secret_access_key_plaintext`.
          Example: `arn:aws:iam::123456789012:instance-profile/my-instance-profile`
        - `uc_service_credential_name` (string, optional, Public Preview)
          The name of the Unity Catalog service credential that the external model uses to access AWS
           resources.
          Example: `my-uc-service-credential`
      - `cohere_config` (object, optional)
        Cohere Config. Only required if the provider is 'cohere'.
        - `cohere_api_key` (string, optional)
          The Databricks secret key reference for a Cohere API key. If you prefer
           to paste your API key directly, see `cohere_api_key_plaintext`. You must
           provide an API key using one of the following fields: `cohere_api_key` or
           `cohere_api_key_plaintext`.
          Example: `{{secrets/my_scope/my_cohere_api_key}}`
        - `cohere_api_key_plaintext` (string, optional, Input only)
          The Cohere API key provided as a plaintext string. If you prefer to
           reference your key using Databricks Secrets, see `cohere_api_key`. You
           must provide an API key using one of the following fields:
           `cohere_api_key` or `cohere_api_key_plaintext`.
          Example: `Your API Key`
        - `cohere_api_base` (string, optional)
          This is an optional field to provide a customized base URL for the Cohere
           API. If left unspecified, the standard Cohere base URL is used.
          Example: `https://api.cohere.ai/v1`
      - `google_cloud_vertex_ai_config` (object, optional)
        Google Cloud Vertex AI Config. Only required if the provider is 'google-cloud-vertex-ai'.
        - `private_key` (string, optional)
          The Databricks secret key reference for a private key for the service
           account which has access to the Google Cloud Vertex AI Service. See [Best
           practices for managing service account keys]. If you prefer to paste your
           API key directly, see `private_key_plaintext`. You must provide an API
           key using one of the following fields: `private_key` or
           `private_key_plaintext`
          
           [Best practices for managing service account keys]: https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys
          Example: `{{secrets/my_scope/my_google_cloud_vertex_ai_api_key}}`
        - `project_id` (string, required)
          This is the Google Cloud project id that the service account is
           associated with.
          Example: `your-project-id`
        - `region` (string, required)
          This is the region for the Google Cloud Vertex AI Service. See [supported
           regions] for more details. Some models are only available in specific
           regions.
          
           [supported regions]: https://cloud.google.com/vertex-ai/docs/general/locations
          Example: `us-central1`
        - `private_key_plaintext` (string, optional, Input only)
          The private key for the service account which has access to the Google
           Cloud Vertex AI Service provided as a plaintext secret. See [Best
           practices for managing service account keys]. If you prefer to reference
           your key using Databricks Secrets, see `private_key`. You must provide an
           API key using one of the following fields: `private_key` or
           `private_key_plaintext`.
          
           [Best practices for managing service account keys]: https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys
          Example: `Your API Key`
      - `databricks_model_serving_config` (object, optional)
        Databricks Model Serving Config. Only required if the provider is 'databricks-model-serving'.
        - `databricks_api_token` (string, optional)
          The Databricks secret key reference for a Databricks API token that
           corresponds to a user or service principal with Can Query access to the
           model serving endpoint pointed to by this external model. If you prefer
           to paste your API key directly, see `databricks_api_token_plaintext`. You
           must provide an API key using one of the following fields:
           `databricks_api_token` or `databricks_api_token_plaintext`.
          Example: `{{secrets/my_scope/my_databricks_api_token}}`
        - `databricks_workspace_url` (string, required)
          The URL of the Databricks workspace containing the model serving endpoint
           pointed to by this external model.
          Example: `https://my-databricks-workspace.com`
        - `databricks_api_token_plaintext` (string, optional, Input only)
          The Databricks API token that corresponds to a user or service principal
           with Can Query access to the model serving endpoint pointed to by this
           external model provided as a plaintext string. If you prefer to reference
           your key using Databricks Secrets, see `databricks_api_token`. You must
           provide an API key using one of the following fields:
           `databricks_api_token` or `databricks_api_token_plaintext`.
          Example: `Your Databricks API Token`
      - `openai_config` (object, optional)
        OpenAI Config. Only required if the provider is 'openai'.
        - `openai_api_key` (string, optional)
          The Databricks secret key reference for an OpenAI API key using the
           OpenAI or Azure service. If you prefer to paste your API key directly,
           see `openai_api_key_plaintext`. You must provide an API key using one of
           the following fields: `openai_api_key` or `openai_api_key_plaintext`.
          Example: `{{secrets/my_scope/my_openai_api_key}}`
        - `openai_api_type` (string, optional)
          This is an optional field to specify the type of OpenAI API to use. For
           Azure OpenAI, this field is required, and adjust this parameter to
           represent the preferred security access validation protocol. For access
           token validation, use azure. For authentication using Azure Active
           Directory (Azure AD) use, azuread.
          Example: `azure`
        - `openai_api_base` (string, optional)
          This is a field to provide a customized base URl for the OpenAI API. For
           Azure OpenAI, this field is required, and is the base URL for the Azure
           OpenAI API service provided by Azure. For other OpenAI API types, this
           field is optional, and if left unspecified, the standard OpenAI base URL
           is used.
          Example: `https://api.openai.com/v1`
        - `openai_api_version` (string, optional)
          This is an optional field to specify the OpenAI API version. For Azure
           OpenAI, this field is required, and is the version of the Azure OpenAI
           service to utilize, specified by a date.
          Example: `2023-11-01`
        - `openai_deployment_name` (string, optional)
          This field is only required for Azure OpenAI and is the name of the
           deployment resource for the Azure OpenAI service.
          Example: `my_deployment_resource`
        - `openai_organization` (string, optional)
          This is an optional field to specify the organization in OpenAI or Azure
           OpenAI.
          Example: `Databricks`
        - `microsoft_entra_tenant_id` (string, optional)
          This field is only required for Azure AD OpenAI and is the Microsoft
           Entra Tenant ID.
          Example: `12345678-abcd-1234-5678-12345678abcd`
        - `microsoft_entra_client_id` (string, optional)
          This field is only required for Azure AD OpenAI and is the Microsoft
           Entra Client ID.
          Example: `12345678-abcd-1234-5678-12345678abcd`
        - `microsoft_entra_client_secret` (string, optional)
          The Databricks secret key reference for a client secret used for
           Microsoft Entra ID authentication. If you prefer to paste your client
           secret directly, see `microsoft_entra_client_secret_plaintext`. You must
           provide an API key using one of the following fields:
           `microsoft_entra_client_secret` or
           `microsoft_entra_client_secret_plaintext`.
          Example: `{{secrets/my_scope/my_microsoft_entra_client_secret}}`
        - `openai_api_key_plaintext` (string, optional, Input only)
          The OpenAI API key using the OpenAI or Azure service provided as a
           plaintext string. If you prefer to reference your key using Databricks
           Secrets, see `openai_api_key`. You must provide an API key using one of
           the following fields: `openai_api_key` or `openai_api_key_plaintext`.
          Example: `Your API Key`
        - `microsoft_entra_client_secret_plaintext` (string, optional, Input only)
          The client secret used for Microsoft Entra ID authentication provided as
           a plaintext string. If you prefer to reference your key using Databricks
           Secrets, see `microsoft_entra_client_secret`. You must provide an API key
           using one of the following fields: `microsoft_entra_client_secret` or
           `microsoft_entra_client_secret_plaintext`.
          Example: `Your Microsoft Entra Client Secret`
      - `palm_config` (object, optional)
        PaLM Config. Only required if the provider is 'palm'.
        - `palm_api_key` (string, optional)
          The Databricks secret key reference for a PaLM API key. If you prefer to
           paste your API key directly, see `palm_api_key_plaintext`. You must
           provide an API key using one of the following fields: `palm_api_key` or
           `palm_api_key_plaintext`.
          Example: `{{secrets/my_scope/my_palm_api_key}}`
        - `palm_api_key_plaintext` (string, optional, Input only)
          The PaLM API key provided as a plaintext string. If you prefer to
           reference your key using Databricks Secrets, see `palm_api_key`. You must
           provide an API key using one of the following fields: `palm_api_key` or
           `palm_api_key_plaintext`.
          Example: `Your API Key`
      - `custom_provider_config` (object, optional)
        Custom Provider Config. Only required if the provider is 'custom'.
        - `custom_provider_url` (string, required)
          This is a field to provide the URL of the custom provider API.
          Example: `https://custom-provider.com`
        - `bearer_token_auth` (object, optional)
          This is a field to provide bearer token authentication for the custom provider API.
           You can only specify one authentication method.
          - `token` (string, optional)
            The Databricks secret key reference for a token.
             If you prefer to paste your token directly, see `token_plaintext`.
            Example: `{{secrets/my_scope/my_openai_api_key}}`
          - `token_plaintext` (string, optional, Input only)
            The token provided as a plaintext string. If you prefer to reference your
             token using Databricks Secrets, see `token`.
            Example: `Your API Key`
        - `api_key_auth` (object, optional)
          This is a field to provide API key authentication for the custom provider API.
           You can only specify one authentication method.
          - `key` (string, required)
            The name of the API key parameter used for authentication.
            Example: `api-key`
          - `value` (string, optional)
            The Databricks secret key reference for an API Key.
             If you prefer to paste your token directly, see `value_plaintext`.
            Example: `{{secrets/my_scope/my_api_key}}`
          - `value_plaintext` (string, optional, Input only)
            The API Key provided as a plaintext string. If you prefer to reference your
             token using Databricks Secrets, see `value`.
            Example: `Your API Key`
    - `entity_name` (string, optional)
      The name of the entity to be served. The entity may be a model in the Databricks Model Registry, a model in the Unity Catalog (UC), or a function of type FEATURE_SPEC in the UC. If it is a UC object, the full name of the object should be given in the form of **catalog_name.schema_name.model_name**.
      Example: `ads-model`
    - `entity_version` (string, optional)
      Example: `3`
    - `min_provisioned_throughput` (int32, optional)
      The minimum tokens per second that the endpoint can scale down to.
      Example: `970`
    - `max_provisioned_throughput` (int32, optional)
      The maximum tokens per second that the endpoint can scale up to.
      Example: `1960`
    - `min_provisioned_concurrency` (int32, optional)
      The minimum provisioned concurrency that the endpoint can scale down to. Do not use if workload_size is specified.
      Example: `8`
    - `max_provisioned_concurrency` (int32, optional)
      The maximum provisioned concurrency that the endpoint can scale up to. Do not use if workload_size is specified.
      Example: `32`
    - `workload_size` (string, optional)
      The workload size of the served entity. The workload size corresponds to a range of provisioned concurrency that the compute autoscales between. A single unit of provisioned concurrency can process one request at a time. Valid workload sizes are "Small" (4 - 4 provisioned concurrency), "Medium" (8 - 16 provisioned concurrency), and "Large" (16 - 64 provisioned concurrency). Additional custom workload sizes can also be used when available in the workspace. If scale-to-zero is enabled, the lower bound of the provisioned concurrency for each workload size is 0. Do not use if min_provisioned_concurrency and max_provisioned_concurrency are specified.
    - `provisioned_model_units` (int64, optional, Public Preview)
      The number of model units provisioned.
      Example: `100`
    - `burst_scaling_enabled` (boolean, optional, Public Preview, Input only)
      Whether burst scaling is enabled. When enabled (default), the endpoint can automatically
       scale up beyond provisioned capacity to handle traffic spikes. When disabled, the endpoint
       maintains fixed capacity at provisioned_model_units.
      Example: `true`
    - `scale_to_zero_enabled` (boolean, optional)
      Whether the compute resources for the served entity should scale down to zero.
      Example: `false`
    - `model_name` (string, required)
    - `model_version` (string, required)
    - `environment_vars` (object, optional)
      An object containing a set of optional, user-specified environment variable key-value pairs used for serving this entity. Note: this is an experimental feature and subject to change. Example entity environment variables that refer to Databricks secrets: `{"OPENAI_API_KEY": "{{secrets/my_scope/my_key}}", "DATABRICKS_TOKEN": "{{secrets/my_scope2/my_key2}}"}`
    - `instance_profile_arn` (string, optional, Public Preview)
      ARN of the instance profile that the served entity uses to access AWS resources.
    - `foundation_model` (object, optional, Public Preview)
      - `name` (string, optional, Public Preview)
      - `display_name` (string, optional, Public Preview)
      - `docs` (string, optional, Public Preview)
      - `description` (string, optional, Public Preview)
    - `state` (object, optional)
      - `deployment` (string, optional)
        Possible values:
        - `DEPLOYMENT_UNKNOWN`
        - `DEPLOYMENT_CREATING`
        - `DEPLOYMENT_RECOVERING`
        - `DEPLOYMENT_READY`
        - `DEPLOYMENT_FAILED`
        - `DEPLOYMENT_ABORTED`
        - `DEPLOYMENT_STOPPED`
      - `deployment_state_message` (string, optional)
    - `creator` (string, optional)
    - `creation_timestamp` (int64, optional)
  - `served_models` (array of object, optional)
    (Deprecated, use served_entities instead) The list of served models under the serving endpoint config.
    - `name` (string, optional)
      The name of a served entity. It must be unique across an endpoint. A served entity name can consist of alphanumeric characters, dashes, and underscores. If not specified for an external model, this field defaults to external_model.name, with '.' and ':' replaced with '-', and if not specified for other entities, it defaults to entity_name-entity_version.
      Example: `ads-model-3`
    - `external_model` (object, optional)
      The external model to be served. NOTE: Only one of external_model and (entity_name, entity_version, workload_size, workload_type, and scale_to_zero_enabled) can be specified with the latter set being used for custom model serving for a Databricks registered model. For an existing endpoint with external_model, it cannot be updated to an endpoint without external_model. If the endpoint is created without external_model, users cannot update it to add external_model later. The task type of all external models within an endpoint must be the same.
      - `provider` (string, required)
        The name of the provider for the external model. Currently, the supported providers are 'ai21labs', 'anthropic', 'amazon-bedrock', 'cohere', 'databricks-model-serving', 'google-cloud-vertex-ai', 'openai', 'palm', and 'custom'.
      - `name` (string, required)
        The name of the external model.
        Example: `gpt-4`
      - `task` (string, required)
        The task type of the external model.
        Example: `llm/v1/chat`
      - `ai21labs_config` (object, optional)
        AI21Labs Config. Only required if the provider is 'ai21labs'.
        - `ai21labs_api_key` (string, optional)
          The Databricks secret key reference for an AI21 Labs API key. If you
           prefer to paste your API key directly, see `ai21labs_api_key_plaintext`.
           You must provide an API key using one of the following fields:
           `ai21labs_api_key` or `ai21labs_api_key_plaintext`.
          Example: `{{secrets/my_scope/my_ai21labs_api_key}}`
        - `ai21labs_api_key_plaintext` (string, optional, Input only)
          An AI21 Labs API key provided as a plaintext string. If you prefer to
           reference your key using Databricks Secrets, see `ai21labs_api_key`. You
           must provide an API key using one of the following fields:
           `ai21labs_api_key` or `ai21labs_api_key_plaintext`.
          Example: `Your API Key`
      - `anthropic_config` (object, optional)
        Anthropic Config. Only required if the provider is 'anthropic'.
        - `anthropic_api_key` (string, optional)
          The Databricks secret key reference for an Anthropic API key. If you
           prefer to paste your API key directly, see `anthropic_api_key_plaintext`.
           You must provide an API key using one of the following fields:
           `anthropic_api_key` or `anthropic_api_key_plaintext`.
          Example: `{{secrets/my_scope/my_anthropic_api_key}}`
        - `anthropic_api_key_plaintext` (string, optional, Input only)
          The Anthropic API key provided as a plaintext string. If you prefer to
           reference your key using Databricks Secrets, see `anthropic_api_key`. You
           must provide an API key using one of the following fields:
           `anthropic_api_key` or `anthropic_api_key_plaintext`.
          Example: `Your API Key`
      - `amazon_bedrock_config` (object, optional)
        Amazon Bedrock Config. Only required if the provider is 'amazon-bedrock'.
        - `aws_region` (string, required)
          The AWS region to use. Bedrock has to be enabled there.
          Example: `myAwsRegion`
        - `aws_access_key_id` (string, optional)
          The Databricks secret key reference for an AWS access key ID with
           permissions to interact with Bedrock services. If you prefer to paste
           your API key directly, see `aws_access_key_id_plaintext`. You must provide an API
           key using one of the following fields: `aws_access_key_id` or
           `aws_access_key_id_plaintext`.
          Example: `{{secrets/my_scope/my_aws_access_key_id}}`
        - `aws_secret_access_key` (string, optional)
          The Databricks secret key reference for an AWS secret access key paired
           with the access key ID, with permissions to interact with Bedrock
           services. If you prefer to paste your API key directly, see
           `aws_secret_access_key_plaintext`. You must provide an API key using one
           of the following fields: `aws_secret_access_key` or
           `aws_secret_access_key_plaintext`.
          Example: `{{secrets/my_scope/my_aws_secret_access_key}}`
        - `bedrock_provider` (string, required)
          The underlying provider in Amazon Bedrock. Supported values (case
           insensitive) include: Anthropic, Cohere, AI21Labs, Amazon.
        - `aws_access_key_id_plaintext` (string, optional, Input only)
          An AWS access key ID with permissions to interact with Bedrock services
           provided as a plaintext string. If you prefer to reference your key using
           Databricks Secrets, see `aws_access_key_id`. You must provide an API key
           using one of the following fields: `aws_access_key_id` or
           `aws_access_key_id_plaintext`.
          Example: `Your AWS Access Key ID`
        - `aws_secret_access_key_plaintext` (string, optional, Input only)
          An AWS secret access key paired with the access key ID, with permissions
           to interact with Bedrock services provided as a plaintext string. If you
           prefer to reference your key using Databricks Secrets, see
           `aws_secret_access_key`. You must provide an API key using one of the
           following fields: `aws_secret_access_key` or
           `aws_secret_access_key_plaintext`.
          Example: `Your AWS Secret Access Key`
        - `instance_profile_arn` (string, optional)
          ARN of the instance profile that the external model will use to access AWS resources.
           You must authenticate using an instance profile or access keys.
           If you prefer to authenticate using access keys, see `aws_access_key_id`,
           `aws_access_key_id_plaintext`, `aws_secret_access_key` and `aws_secret_access_key_plaintext`.
          Example: `arn:aws:iam::123456789012:instance-profile/my-instance-profile`
        - `uc_service_credential_name` (string, optional, Public Preview)
          The name of the Unity Catalog service credential that the external model uses to access AWS
           resources.
          Example: `my-uc-service-credential`
      - `cohere_config` (object, optional)
        Cohere Config. Only required if the provider is 'cohere'.
        - `cohere_api_key` (string, optional)
          The Databricks secret key reference for a Cohere API key. If you prefer
           to paste your API key directly, see `cohere_api_key_plaintext`. You must
           provide an API key using one of the following fields: `cohere_api_key` or
           `cohere_api_key_plaintext`.
          Example: `{{secrets/my_scope/my_cohere_api_key}}`
        - `cohere_api_key_plaintext` (string, optional, Input only)
          The Cohere API key provided as a plaintext string. If you prefer to
           reference your key using Databricks Secrets, see `cohere_api_key`. You
           must provide an API key using one of the following fields:
           `cohere_api_key` or `cohere_api_key_plaintext`.
          Example: `Your API Key`
        - `cohere_api_base` (string, optional)
          This is an optional field to provide a customized base URL for the Cohere
           API. If left unspecified, the standard Cohere base URL is used.
          Example: `https://api.cohere.ai/v1`
      - `google_cloud_vertex_ai_config` (object, optional)
        Google Cloud Vertex AI Config. Only required if the provider is 'google-cloud-vertex-ai'.
        - `private_key` (string, optional)
          The Databricks secret key reference for a private key for the service
           account which has access to the Google Cloud Vertex AI Service. See [Best
           practices for managing service account keys]. If you prefer to paste your
           API key directly, see `private_key_plaintext`. You must provide an API
           key using one of the following fields: `private_key` or
           `private_key_plaintext`
          
           [Best practices for managing service account keys]: https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys
          Example: `{{secrets/my_scope/my_google_cloud_vertex_ai_api_key}}`
        - `project_id` (string, required)
          This is the Google Cloud project id that the service account is
           associated with.
          Example: `your-project-id`
        - `region` (string, required)
          This is the region for the Google Cloud Vertex AI Service. See [supported
           regions] for more details. Some models are only available in specific
           regions.
          
           [supported regions]: https://cloud.google.com/vertex-ai/docs/general/locations
          Example: `us-central1`
        - `private_key_plaintext` (string, optional, Input only)
          The private key for the service account which has access to the Google
           Cloud Vertex AI Service provided as a plaintext secret. See [Best
           practices for managing service account keys]. If you prefer to reference
           your key using Databricks Secrets, see `private_key`. You must provide an
           API key using one of the following fields: `private_key` or
           `private_key_plaintext`.
          
           [Best practices for managing service account keys]: https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys
          Example: `Your API Key`
      - `databricks_model_serving_config` (object, optional)
        Databricks Model Serving Config. Only required if the provider is 'databricks-model-serving'.
        - `databricks_api_token` (string, optional)
          The Databricks secret key reference for a Databricks API token that
           corresponds to a user or service principal with Can Query access to the
           model serving endpoint pointed to by this external model. If you prefer
           to paste your API key directly, see `databricks_api_token_plaintext`. You
           must provide an API key using one of the following fields:
           `databricks_api_token` or `databricks_api_token_plaintext`.
          Example: `{{secrets/my_scope/my_databricks_api_token}}`
        - `databricks_workspace_url` (string, required)
          The URL of the Databricks workspace containing the model serving endpoint
           pointed to by this external model.
          Example: `https://my-databricks-workspace.com`
        - `databricks_api_token_plaintext` (string, optional, Input only)
          The Databricks API token that corresponds to a user or service principal
           with Can Query access to the model serving endpoint pointed to by this
           external model provided as a plaintext string. If you prefer to reference
           your key using Databricks Secrets, see `databricks_api_token`. You must
           provide an API key using one of the following fields:
           `databricks_api_token` or `databricks_api_token_plaintext`.
          Example: `Your Databricks API Token`
      - `openai_config` (object, optional)
        OpenAI Config. Only required if the provider is 'openai'.
        - `openai_api_key` (string, optional)
          The Databricks secret key reference for an OpenAI API key using the
           OpenAI or Azure service. If you prefer to paste your API key directly,
           see `openai_api_key_plaintext`. You must provide an API key using one of
           the following fields: `openai_api_key` or `openai_api_key_plaintext`.
          Example: `{{secrets/my_scope/my_openai_api_key}}`
        - `openai_api_type` (string, optional)
          This is an optional field to specify the type of OpenAI API to use. For
           Azure OpenAI, this field is required, and adjust this parameter to
           represent the preferred security access validation protocol. For access
           token validation, use azure. For authentication using Azure Active
           Directory (Azure AD) use, azuread.
          Example: `azure`
        - `openai_api_base` (string, optional)
          This is a field to provide a customized base URl for the OpenAI API. For
           Azure OpenAI, this field is required, and is the base URL for the Azure
           OpenAI API service provided by Azure. For other OpenAI API types, this
           field is optional, and if left unspecified, the standard OpenAI base URL
           is used.
          Example: `https://api.openai.com/v1`
        - `openai_api_version` (string, optional)
          This is an optional field to specify the OpenAI API version. For Azure
           OpenAI, this field is required, and is the version of the Azure OpenAI
           service to utilize, specified by a date.
          Example: `2023-11-01`
        - `openai_deployment_name` (string, optional)
          This field is only required for Azure OpenAI and is the name of the
           deployment resource for the Azure OpenAI service.
          Example: `my_deployment_resource`
        - `openai_organization` (string, optional)
          This is an optional field to specify the organization in OpenAI or Azure
           OpenAI.
          Example: `Databricks`
        - `microsoft_entra_tenant_id` (string, optional)
          This field is only required for Azure AD OpenAI and is the Microsoft
           Entra Tenant ID.
          Example: `12345678-abcd-1234-5678-12345678abcd`
        - `microsoft_entra_client_id` (string, optional)
          This field is only required for Azure AD OpenAI and is the Microsoft
           Entra Client ID.
          Example: `12345678-abcd-1234-5678-12345678abcd`
        - `microsoft_entra_client_secret` (string, optional)
          The Databricks secret key reference for a client secret used for
           Microsoft Entra ID authentication. If you prefer to paste your client
           secret directly, see `microsoft_entra_client_secret_plaintext`. You must
           provide an API key using one of the following fields:
           `microsoft_entra_client_secret` or
           `microsoft_entra_client_secret_plaintext`.
          Example: `{{secrets/my_scope/my_microsoft_entra_client_secret}}`
        - `openai_api_key_plaintext` (string, optional, Input only)
          The OpenAI API key using the OpenAI or Azure service provided as a
           plaintext string. If you prefer to reference your key using Databricks
           Secrets, see `openai_api_key`. You must provide an API key using one of
           the following fields: `openai_api_key` or `openai_api_key_plaintext`.
          Example: `Your API Key`
        - `microsoft_entra_client_secret_plaintext` (string, optional, Input only)
          The client secret used for Microsoft Entra ID authentication provided as
           a plaintext string. If you prefer to reference your key using Databricks
           Secrets, see `microsoft_entra_client_secret`. You must provide an API key
           using one of the following fields: `microsoft_entra_client_secret` or
           `microsoft_entra_client_secret_plaintext`.
          Example: `Your Microsoft Entra Client Secret`
      - `palm_config` (object, optional)
        PaLM Config. Only required if the provider is 'palm'.
        - `palm_api_key` (string, optional)
          The Databricks secret key reference for a PaLM API key. If you prefer to
           paste your API key directly, see `palm_api_key_plaintext`. You must
           provide an API key using one of the following fields: `palm_api_key` or
           `palm_api_key_plaintext`.
          Example: `{{secrets/my_scope/my_palm_api_key}}`
        - `palm_api_key_plaintext` (string, optional, Input only)
          The PaLM API key provided as a plaintext string. If you prefer to
           reference your key using Databricks Secrets, see `palm_api_key`. You must
           provide an API key using one of the following fields: `palm_api_key` or
           `palm_api_key_plaintext`.
          Example: `Your API Key`
      - `custom_provider_config` (object, optional)
        Custom Provider Config. Only required if the provider is 'custom'.
        - `custom_provider_url` (string, required)
          This is a field to provide the URL of the custom provider API.
          Example: `https://custom-provider.com`
        - `bearer_token_auth` (object, optional)
          This is a field to provide bearer token authentication for the custom provider API.
           You can only specify one authentication method.
          - `token` (string, optional)
            The Databricks secret key reference for a token.
             If you prefer to paste your token directly, see `token_plaintext`.
            Example: `{{secrets/my_scope/my_openai_api_key}}`
          - `token_plaintext` (string, optional, Input only)
            The token provided as a plaintext string. If you prefer to reference your
             token using Databricks Secrets, see `token`.
            Example: `Your API Key`
        - `api_key_auth` (object, optional)
          This is a field to provide API key authentication for the custom provider API.
           You can only specify one authentication method.
          - `key` (string, required)
            The name of the API key parameter used for authentication.
            Example: `api-key`
          - `value` (string, optional)
            The Databricks secret key reference for an API Key.
             If you prefer to paste your token directly, see `value_plaintext`.
            Example: `{{secrets/my_scope/my_api_key}}`
          - `value_plaintext` (string, optional, Input only)
            The API Key provided as a plaintext string. If you prefer to reference your
             token using Databricks Secrets, see `value`.
            Example: `Your API Key`
    - `entity_name` (string, optional)
      The name of the entity to be served. The entity may be a model in the Databricks Model Registry, a model in the Unity Catalog (UC), or a function of type FEATURE_SPEC in the UC. If it is a UC object, the full name of the object should be given in the form of **catalog_name.schema_name.model_name**.
      Example: `ads-model`
    - `entity_version` (string, optional)
      Example: `3`
    - `min_provisioned_throughput` (int32, optional)
      The minimum tokens per second that the endpoint can scale down to.
      Example: `970`
    - `max_provisioned_throughput` (int32, optional)
      The maximum tokens per second that the endpoint can scale up to.
      Example: `1960`
    - `min_provisioned_concurrency` (int32, optional)
      The minimum provisioned concurrency that the endpoint can scale down to. Do not use if workload_size is specified.
      Example: `8`
    - `max_provisioned_concurrency` (int32, optional)
      The maximum provisioned concurrency that the endpoint can scale up to. Do not use if workload_size is specified.
      Example: `32`
    - `workload_size` (string, optional)
      The workload size of the served entity. The workload size corresponds to a range of provisioned concurrency that the compute autoscales between. A single unit of provisioned concurrency can process one request at a time. Valid workload sizes are "Small" (4 - 4 provisioned concurrency), "Medium" (8 - 16 provisioned concurrency), and "Large" (16 - 64 provisioned concurrency). Additional custom workload sizes can also be used when available in the workspace. If scale-to-zero is enabled, the lower bound of the provisioned concurrency for each workload size is 0. Do not use if min_provisioned_concurrency and max_provisioned_concurrency are specified.
    - `provisioned_model_units` (int64, optional, Public Preview)
      The number of model units provisioned.
      Example: `100`
    - `burst_scaling_enabled` (boolean, optional, Public Preview, Input only)
      Whether burst scaling is enabled. When enabled (default), the endpoint can automatically
       scale up beyond provisioned capacity to handle traffic spikes. When disabled, the endpoint
       maintains fixed capacity at provisioned_model_units.
      Example: `true`
    - `scale_to_zero_enabled` (boolean, optional)
      Whether the compute resources for the served entity should scale down to zero.
      Example: `false`
    - `model_name` (string, required)
    - `model_version` (string, required)
    - `environment_vars` (object, optional)
      An object containing a set of optional, user-specified environment variable key-value pairs used for serving this entity. Note: this is an experimental feature and subject to change. Example entity environment variables that refer to Databricks secrets: `{"OPENAI_API_KEY": "{{secrets/my_scope/my_key}}", "DATABRICKS_TOKEN": "{{secrets/my_scope2/my_key2}}"}`
    - `instance_profile_arn` (string, optional, Public Preview)
      ARN of the instance profile that the served entity uses to access AWS resources.
    - `foundation_model` (object, optional, Public Preview)
      - `name` (string, optional, Public Preview)
      - `display_name` (string, optional, Public Preview)
      - `docs` (string, optional, Public Preview)
      - `description` (string, optional, Public Preview)
    - `state` (object, optional)
      - `deployment` (string, optional)
        Possible values:
        - `DEPLOYMENT_UNKNOWN`
        - `DEPLOYMENT_CREATING`
        - `DEPLOYMENT_RECOVERING`
        - `DEPLOYMENT_READY`
        - `DEPLOYMENT_FAILED`
        - `DEPLOYMENT_ABORTED`
        - `DEPLOYMENT_STOPPED`
      - `deployment_state_message` (string, optional)
    - `creator` (string, optional)
    - `creation_timestamp` (int64, optional)
  - `traffic_config` (object, optional)
    The traffic configuration associated with the serving endpoint config.
    - `routes` (array of object, optional)
      The list of routes that define traffic to each served entity.
      - `served_model_name` (string, optional)
        The name of the served model this route configures traffic for.
        Example: `ads-model-3`
      - `traffic_percentage` (int32, required)
        The percentage of endpoint traffic to send to this route. It must be an integer between 0 and 100 inclusive.
        Example: `100`
      - `served_entity_name` (string, optional)
        Example: `ads-model-3`
  - `auto_capture_config` (object, optional, Deprecated)
    Configuration for legacy Inference Tables which automatically log requests and responses to Unity
     Catalog.
     Deprecated: please use AI Gateway inference tables instead. See
     https://docs.databricks.com/aws/en/ai-gateway/inference-tables.
    - `catalog_name` (string, optional)
      The name of the catalog in Unity Catalog. NOTE: On update, you cannot change the catalog name if the inference table is already enabled.
      Example: `my-catalog`
    - `schema_name` (string, optional)
      The name of the schema in Unity Catalog. NOTE: On update, you cannot change the schema name if the inference table is already enabled.
      Example: `my-schema`
    - `table_name_prefix` (string, optional)
      The prefix of the table in Unity Catalog. NOTE: On update, you cannot change the prefix name if the inference table is already enabled.
      Example: `my-prefix-`
    - `state` (object, optional)
      - `payload_table` (object, optional)
        - `name` (string, optional)
        - `status` (string, optional)
        - `status_message` (string, optional)
    - `enabled` (boolean, optional)
      Indicates whether the inference table is enabled.
      Example: `true`
- `tags` (array of object, optional)
  Tags to be attached to the serving endpoint and automatically propagated to billing logs.
  - `key` (string, required)
    Key field for a serving endpoint tag.
    Example: `team`
  - `value` (string, optional)
    Optional value field for a serving endpoint tag.
    Example: `data science`
- `route_optimized` (boolean, optional)
  Enable route optimization for the serving endpoint.
  Example: `true`
- `rate_limits` (array of object, optional, Deprecated)
  Rate limits to be applied to the serving endpoint. NOTE: this field is deprecated, please use AI Gateway to manage rate limits.
  - `calls` (int64, required)
    Used to specify how many calls are allowed for a key within the renewal_period.
    Example: `15`
  - `key` (string, optional)
    Key field for a serving endpoint rate limit. Currently, only 'user' and 'endpoint' are supported, with 'endpoint' being the default if not specified.
  - `renewal_period` (string, required)
    Renewal period field for a serving endpoint rate limit. Currently, only 'minute' is supported.
- `ai_gateway` (object, optional)
  The AI Gateway configuration for the serving endpoint. NOTE: External model, provisioned throughput, and pay-per-token endpoints are fully supported; agent endpoints currently only support inference tables.
  - `usage_tracking_config` (object, optional)
    Configuration to enable usage tracking using system tables.
     These tables allow you to monitor operational usage on endpoints and their associated costs.
    - `enabled` (boolean, optional)
      Whether to enable usage tracking.
      Example: `true`
  - `inference_table_config` (object, optional)
    Configuration for payload logging using inference tables.
     Use these tables to monitor and audit data being sent to and received from model APIs and to improve model quality.
    - `catalog_name` (string, optional)
      The name of the catalog in Unity Catalog. Required when enabling inference tables.
       NOTE: On update, you have to disable inference table first in order to change the catalog name.
      Example: `my-catalog`
    - `schema_name` (string, optional)
      The name of the schema in Unity Catalog. Required when enabling inference tables.
       NOTE: On update, you have to disable inference table first in order to change the schema name.
      Example: `my-schema`
    - `table_name_prefix` (string, optional)
      The prefix of the table in Unity Catalog.
       NOTE: On update, you have to disable inference table first in order to change the prefix name.
      Example: `my-prefix`
    - `enabled` (boolean, optional)
      Indicates whether the inference table is enabled.
      Example: `true`
  - `rate_limits` (array of object, optional)
    Configuration for rate limits which can be set to limit endpoint traffic.
    - `calls` (int64, optional)
      Used to specify how many calls are allowed for a key within the renewal_period.
      Example: `15`
    - `key` (string, optional)
      Key field for a rate limit. Currently, 'user', 'user_group, 'service_principal', and 'endpoint' are supported,
       with 'endpoint' being the default if not specified.
    - `renewal_period` (string, required)
      Renewal period field for a rate limit. Currently, only 'minute' is supported.
    - `principal` (string, optional)
      Principal field for a user, user group, or service principal to apply rate limiting to. Accepts a user email, group name, or service principal application ID.
      Example: `user@test.com`
    - `tokens` (int64, optional)
      Used to specify how many tokens are allowed for a key within the renewal_period.
      Example: `10000`
  - `guardrails` (object, optional, Public Preview)
    Configuration for AI Guardrails to prevent unwanted data and unsafe data in requests and responses.
    - `input` (object, optional, Public Preview)
      Configuration for input guardrail filters.
      - `safety` (boolean, optional, Public Preview)
        Indicates whether the safety filter is enabled.
        Example: `true`
      - `pii` (object, optional, Public Preview)
        Configuration for guardrail PII filter.
        - `behavior` (string, optional, Public Preview)
          Configuration for input guardrail filters.
          Possible values:
          - `BEHAVIOR_UNSPECIFIED`
          - `NONE`
          - `BLOCK`
          - `MASK`
          Example: `BLOCK`
      - `valid_topics` (array of string, optional, Deprecated, Public Preview)
        The list of allowed topics.
         Given a chat request, this guardrail flags the request if its topic is not in the allowed topics.
      - `invalid_keywords` (array of string, optional, Deprecated, Public Preview)
        List of invalid keywords.
         AI guardrail uses keyword or string matching to decide if the keyword exists in the request or response content.
    - `output` (object, optional, Public Preview)
      Configuration for output guardrail filters.
      - `safety` (boolean, optional, Public Preview)
        Indicates whether the safety filter is enabled.
        Example: `true`
      - `pii` (object, optional, Public Preview)
        Configuration for guardrail PII filter.
        - `behavior` (string, optional, Public Preview)
          Configuration for input guardrail filters.
          Possible values:
          - `BEHAVIOR_UNSPECIFIED`
          - `NONE`
          - `BLOCK`
          - `MASK`
          Example: `BLOCK`
      - `valid_topics` (array of string, optional, Deprecated, Public Preview)
        The list of allowed topics.
         Given a chat request, this guardrail flags the request if its topic is not in the allowed topics.
      - `invalid_keywords` (array of string, optional, Deprecated, Public Preview)
        List of invalid keywords.
         AI guardrail uses keyword or string matching to decide if the keyword exists in the request or response content.
  - `fallback_config` (object, optional)
    Configuration for traffic fallback which auto fallbacks to other served entities if the request to a served
     entity fails with certain error codes, to increase availability.
    - `enabled` (boolean, required)
      Whether to enable traffic fallback. When a served entity in the serving endpoint returns specific error
       codes (e.g. 500), the request will automatically be round-robin attempted with other served entities in the same
       endpoint, following the order of served entity list, until a successful response is returned.
       If all attempts fail, return the last response with the error code.
      Example: `true`
- `budget_policy_id` (string, optional)
  The budget policy to be applied to the serving endpoint.
- `email_notifications` (object, optional)
  Email notification settings.
  - `on_update_success` (array of string, optional)
    A list of email addresses to be notified when an endpoint successfully updates its configuration or state.
    Example: `user.name@databricks.com`
  - `on_update_failure` (array of string, optional)
    A list of email addresses to be notified when an endpoint fails to update its configuration or state.
    Example: `user.name@databricks.com`
- `description` (string, optional)
  Constraints: `<= 1000 characters`
- `telemetry_config` (object, optional, Public Preview)
  Configuration for persisting endpoint telemetry (logs, traces, and metrics) to Unity Catalog tables.
  - `telemetry_profile_id` (string, optional, Public Preview)
    The ID of an existing telemetry profile to apply to this endpoint. Provide this to reuse a
     telemetry profile that has already been created, instead of specifying table_names.
  - `table_names` (object, optional, Public Preview)
    The Unity Catalog tables to which endpoint telemetry (logs, traces, and metrics) is exported.
     Provide this to create a new telemetry profile for the endpoint from the given tables.
    - `logs_table` (string, optional, Public Preview)
      The full three-level Unity Catalog name (catalog.schema.table) of the table that receives
       exported logs.
    - `metrics_table` (string, optional, Public Preview)
      The full three-level Unity Catalog name (catalog.schema.table) of the table that receives
       exported metrics.
    - `traces_table` (string, optional, Public Preview)
      The full three-level Unity Catalog name (catalog.schema.table) of the table that receives
       exported traces (spans).
    - `annotations_table` (string, optional, Public Preview)
      The full three-level Unity Catalog name (catalog.schema.table) of the table that receives
       exported annotations.
  - `inference_table_config` (object, optional, Public Preview)
    Configuration for inference table payload logging, including sampling.
    - `sampling_fraction` (double, optional, Public Preview)
      Fraction of requests sampled for payload logging, in the range [0.0, 1.0], where 1.0 logs all requests.
  - `enabled_telemetry_features` (array of string, optional, Public Preview)
    The telemetry signals to enable for this endpoint. If empty or omitted, all signals are
     enabled; otherwise only the listed signals are enabled.
    Possible values:
    - `TELEMETRY_FEATURE_LOGS`
    - `TELEMETRY_FEATURE_TRACES`
    - `TELEMETRY_FEATURE_METRICS`
    - `TELEMETRY_FEATURE_INFERENCE_TABLE`

## Returns

Returns the InferenceEndpointDetailed object.

## Request

### External model endpoint

```json
{
  "ai_gateway": {
    "guardrails": {
      "input": {
        "invalid_keywords": [
          "keyword1",
          "keyword2"
        ],
        "pii": {
          "behavior": "BLOCK"
        },
        "safety": true,
        "valid_topics": [
          "topic1",
          "topic2"
        ]
      },
      "output": {
        "invalid_keywords": [
          "keyword1",
          "keyword2"
        ],
        "pii": {
          "behavior": "BLOCK"
        },
        "safety": true,
        "valid_topics": [
          "topic1",
          "topic2"
        ]
      }
    },
    "inference_table_config": {
      "catalog_name": "my-catalog",
      "enabled": true,
      "schema_name": "my-schema",
      "table_name_prefix": "my-prefix"
    },
    "rate_limits": [
      {
        "calls": 100,
        "key": "user",
        "renewal_period": "minute"
      }
    ],
    "usage_tracking_config": {
      "enabled": true
    }
  },
  "config": {
    "served_entities": [
      {
        "external_model": {
          "name": "text-embedding-ada-002",
          "openai_config": {
            "openai_api_key": "{{secrets/my_scope/my_openai_api_key}}"
          },
          "provider": "openai",
          "task": "llm/v1/embeddings"
        },
        "name": "openai_embeddings"
      }
    ]
  },
  "name": "openai_endpoint",
  "tags": [
    {
      "key": "team",
      "value": "gen-ai"
    }
  ]
}
```

### Feature Serving endpoint

```json
{
  "config": {
    "served_entities": [
      {
        "entity_name": "feature_store.me.travel_spec",
        "name": "feature-spec-travel",
        "scale_to_zero_enabled": true,
        "workload_size": "Small"
      }
    ]
  },
  "name": "feature-spec-endpoint",
  "tags": [
    {
      "key": "team",
      "value": "data science"
    }
  ]
}
```

### Models in Unity Catalog endpoint

```json
{
  "config": {
    "served_entities": [
      {
        "entity_name": "myCatalog.mySchema.my-ads-model",
        "entity_version": "10",
        "scale_to_zero_enabled": true,
        "workload_size": "Small"
      }
    ],
    "traffic_config": {
      "routes": [
        {
          "served_model_name": "my-ads-model-10",
          "traffic_percentage": 100
        }
      ]
    }
  },
  "name": "uc-model-endpoint",
  "tags": [
    {
      "key": "team",
      "value": "data science"
    }
  ]
}
```

### Workspace model endpoint

```json
{
  "config": {
    "served_entities": [
      {
        "entity_name": "my-ads-model",
        "entity_version": "3",
        "scale_to_zero_enabled": true,
        "workload_size": "Small"
      },
      {
        "entity_name": "my-ads-model",
        "entity_version": "4",
        "scale_to_zero_enabled": true,
        "workload_size": "Small"
      }
    ],
    "traffic_config": {
      "routes": [
        {
          "served_model_name": "my-ads-model-3",
          "traffic_percentage": 80
        },
        {
          "served_model_name": "my-ads-model-4",
          "traffic_percentage": 20
        }
      ]
    }
  },
  "name": "workspace-model-endpoint",
  "tags": [
    {
      "key": "team",
      "value": "data science"
    }
  ]
}
```

## Response

### External model endpoint

```json
{
  "ai_gateway": {
    "guardrails": {
      "input": {
        "invalid_keywords": [
          "keyword1",
          "keyword2"
        ],
        "pii": {
          "behavior": "BLOCK"
        },
        "safety": true,
        "valid_topics": [
          "topic1",
          "topic2"
        ]
      },
      "output": {
        "invalid_keywords": [
          "keyword1",
          "keyword2"
        ],
        "pii": {
          "behavior": "BLOCK"
        },
        "safety": true,
        "valid_topics": [
          "topic1",
          "topic2"
        ]
      }
    },
    "inference_table_config": {
      "catalog_name": "my-catalog",
      "enabled": true,
      "schema_name": "my-schema",
      "table_name_prefix": "my-prefix"
    },
    "rate_limits": [
      {
        "calls": 100,
        "key": "user",
        "renewal_period": "minute"
      }
    ],
    "usage_tracking_config": {
      "enabled": true
    }
  },
  "config": {
    "config_version": 1,
    "served_entities": [
      {
        "creation_timestamp": 1699617587000,
        "creator": "user@email.com",
        "external_model": {
          "name": "text-embedding-ada-002",
          "openai_config": {
            "openai_api_key": "{{secrets/my_scope/my_openai_api_key}}"
          },
          "provider": "openai",
          "task": "llm/v1/embeddings"
        },
        "name": "openai_embeddings",
        "state": {
          "deployment": "DEPLOYMENT_READY",
          "deployment_state_message": ""
        }
      }
    ],
    "traffic_config": {
      "routes": [
        {
          "served_model_name": "openai_embeddings",
          "traffic_percentage": 100
        }
      ]
    }
  },
  "creation_timestamp": 1699617587000,
  "creator": "user@email.com",
  "id": "69962db6b9db47c4a8a222d2ac79d7f8",
  "last_updated_timestamp": 1699617587000,
  "name": "openai_endpoint",
  "permission_level": "CAN_MANAGE",
  "route_optimized": false,
  "state": {
    "ready": "READY"
  },
  "tags": {
    "key": "team",
    "value": "gen-ai"
  }
}
```

### Feature Serving endpoint

```json
{
  "config": {
    "config_version": 1,
    "served_entities": [
      {
        "creation_timestamp": 1700089760000,
        "creator": "user@email.com",
        "entity_name": "feature_store.me.travel_spec",
        "name": "feature-spec-travel",
        "scale_to_zero_enabled": true,
        "state": {
          "deployment": "DEPLOYMENT_READY",
          "deployment_state_message": ""
        },
        "workload_size": "Small",
        "workload_type": "CPU"
      }
    ],
    "traffic_config": {
      "routes": [
        {
          "served_model_name": "feature-spec-travel",
          "traffic_percentage": 100
        }
      ]
    }
  },
  "creation_timestamp": 1700089637000,
  "creator": "user@email.com",
  "id": "e3bd3e471d6045d6b75f384279e4b6ab",
  "last_updated_timestamp": 1700089760000,
  "name": "feature-spec-endpoint",
  "permission_level": "CAN_MANAGE",
  "route_optimized": false,
  "state": {
    "config_update": "NOT_UPDATING",
    "ready": "READY"
  },
  "tags": [
    {
      "key": "team",
      "value": "data science"
    }
  ]
}
```

### Models in Unity Catalog endpoint

```json
{
  "config": {
    "config_version": 1,
    "served_entities": [
      {
        "creation_timestamp": 1700089760000,
        "creator": "user@email.com",
        "entity_name": "myCatalog.mySchema.my-ads-model",
        "entity_version": "10",
        "name": "my-ads-model-10",
        "scale_to_zero_enabled": true,
        "state": {
          "deployment": "DEPLOYMENT_READY",
          "deployment_state_message": ""
        },
        "workload_size": "Small",
        "workload_type": "CPU"
      }
    ],
    "traffic_config": {
      "routes": [
        {
          "served_model_name": "my-ads-model-10",
          "traffic_percentage": 100
        }
      ]
    }
  },
  "creation_timestamp": 1700089637000,
  "creator": "user@email.com",
  "id": "e3bd3e471d6045d6b75f384279e4b6ab",
  "last_updated_timestamp": 1700089760000,
  "name": "uc-model-endpoint",
  "permission_level": "CAN_MANAGE",
  "route_optimized": false,
  "state": {
    "config_update": "NOT_UPDATING",
    "ready": "READY"
  },
  "tags": [
    {
      "key": "team",
      "value": "data science"
    }
  ]
}
```

### Workspace model endpoint

```json
{
  "config": {
    "config_version": 1,
    "served_entities": [
      {
        "creation_timestamp": 1700089760000,
        "creator": "user@email.com",
        "entity_name": "my-ads-model",
        "entity_version": "3",
        "name": "my-ads-model-3",
        "scale_to_zero_enabled": true,
        "state": {
          "deployment": "DEPLOYMENT_READY",
          "deployment_state_message": ""
        },
        "workload_size": "Small",
        "workload_type": "CPU"
      },
      {
        "creation_timestamp": 1700089760000,
        "creator": "user@email.com",
        "entity_name": "my-ads-model",
        "entity_version": "4",
        "name": "my-ads-model-4",
        "scale_to_zero_enabled": true,
        "state": {
          "deployment": "DEPLOYMENT_READY",
          "deployment_state_message": ""
        },
        "workload_size": "Small",
        "workload_type": "CPU"
      }
    ],
    "traffic_config": {
      "routes": [
        {
          "served_model_name": "my-ads-model-3",
          "traffic_percentage": 80
        },
        {
          "served_model_name": "my-ads-model-4",
          "traffic_percentage": 20
        }
      ]
    }
  },
  "creation_timestamp": 1700089637000,
  "creator": "user@email.com",
  "id": "e3bd3e471d6045d6b75f384279e4b6ab",
  "last_updated_timestamp": 1700089760000,
  "name": "workspace-model-endpoint",
  "permission_level": "CAN_MANAGE",
  "route_optimized": false,
  "state": {
    "config_update": "NOT_UPDATING",
    "ready": "READY"
  },
  "tags": [
    {
      "key": "team",
      "value": "data science"
    }
  ]
}
```