# Get all serving endpoints Launch stage: GA `GET /api/2.0/serving-endpoints` Get all serving endpoints. API scopes: model-serving ## Returns - `endpoints` (array of object, optional) The list of endpoints. - `name` (string, optional) The name of the serving endpoint. Example: `feed-ads` - `creator` (string, optional) The email of the user who created the serving endpoint. Example: `alice@company.com` - `creation_timestamp` (int64, optional) The timestamp when the endpoint was created in Unix time. - `last_updated_timestamp` (int64, optional) The timestamp when the endpoint was last updated by a user in Unix time. - `state` (object, optional) Information corresponding to the state of the serving endpoint. - `ready` (string, optional) The state of an endpoint, indicating whether or not the endpoint is queryable. An endpoint is READY if all of the served entities in its active configuration are ready. If any of the actively served entities are in a non-ready state, the endpoint state will be NOT_READY. Possible values: - `READY_STATE_UNSPECIFIED` - `READY` - `NOT_READY` - `config_update` (string, optional) The state of an endpoint's config update. This informs the user if the pending_config is in progress, if the update failed, or if there is no update in progress. Note that if the endpoint's config_update state value is IN_PROGRESS, another update can not be made until the update completes or fails. Possible values: - `CONFIG_UPDATE_STATE_UNSPECIFIED` - `NOT_UPDATING` - `IN_PROGRESS` - `UPDATE_FAILED` - `UPDATE_CANCELED` - `config` (object, optional) The config that is currently being served by the endpoint. - `served_entities` (array of object, optional) The list of served entities under the serving endpoint config. - `name` (string, optional) - `model_name` (string, optional) Only one of model_name and entity_name should be populated - `entity_name` (string, optional) - `model_version` (string, optional) Only one of model_version and entity_version should be populated - `entity_version` (string, optional) - `external_model` (object, optional) - `provider` (string, required) The name of the provider for the external model. Currently, the supported providers are 'ai21labs', 'anthropic', 'amazon-bedrock', 'cohere', 'databricks-model-serving', 'google-cloud-vertex-ai', 'openai', 'palm', and 'custom'. - `name` (string, required) The name of the external model. Example: `gpt-4` - `task` (string, required) The task type of the external model. Example: `llm/v1/chat` - `ai21labs_config` (object, optional) AI21Labs Config. Only required if the provider is 'ai21labs'. - `ai21labs_api_key` (string, optional) The Databricks secret key reference for an AI21 Labs API key. If you prefer to paste your API key directly, see `ai21labs_api_key_plaintext`. You must provide an API key using one of the following fields: `ai21labs_api_key` or `ai21labs_api_key_plaintext`. Example: `{{secrets/my_scope/my_ai21labs_api_key}}` - `anthropic_config` (object, optional) Anthropic Config. Only required if the provider is 'anthropic'. - `anthropic_api_key` (string, optional) The Databricks secret key reference for an Anthropic API key. If you prefer to paste your API key directly, see `anthropic_api_key_plaintext`. You must provide an API key using one of the following fields: `anthropic_api_key` or `anthropic_api_key_plaintext`. Example: `{{secrets/my_scope/my_anthropic_api_key}}` - `amazon_bedrock_config` (object, optional) Amazon Bedrock Config. Only required if the provider is 'amazon-bedrock'. - `aws_region` (string, required) The AWS region to use. Bedrock has to be enabled there. Example: `myAwsRegion` - `aws_access_key_id` (string, optional) The Databricks secret key reference for an AWS access key ID with permissions to interact with Bedrock services. If you prefer to paste your API key directly, see `aws_access_key_id_plaintext`. You must provide an API key using one of the following fields: `aws_access_key_id` or `aws_access_key_id_plaintext`. Example: `{{secrets/my_scope/my_aws_access_key_id}}` - `aws_secret_access_key` (string, optional) The Databricks secret key reference for an AWS secret access key paired with the access key ID, with permissions to interact with Bedrock services. If you prefer to paste your API key directly, see `aws_secret_access_key_plaintext`. You must provide an API key using one of the following fields: `aws_secret_access_key` or `aws_secret_access_key_plaintext`. Example: `{{secrets/my_scope/my_aws_secret_access_key}}` - `bedrock_provider` (string, required) The underlying provider in Amazon Bedrock. Supported values (case insensitive) include: Anthropic, Cohere, AI21Labs, Amazon. - `instance_profile_arn` (string, optional) ARN of the instance profile that the external model will use to access AWS resources. You must authenticate using an instance profile or access keys. If you prefer to authenticate using access keys, see `aws_access_key_id`, `aws_access_key_id_plaintext`, `aws_secret_access_key` and `aws_secret_access_key_plaintext`. Example: `arn:aws:iam::123456789012:instance-profile/my-instance-profile` - `uc_service_credential_name` (string, optional, Public Preview) The name of the Unity Catalog service credential that the external model uses to access AWS resources. Example: `my-uc-service-credential` - `cohere_config` (object, optional) Cohere Config. Only required if the provider is 'cohere'. - `cohere_api_key` (string, optional) The Databricks secret key reference for a Cohere API key. If you prefer to paste your API key directly, see `cohere_api_key_plaintext`. You must provide an API key using one of the following fields: `cohere_api_key` or `cohere_api_key_plaintext`. Example: `{{secrets/my_scope/my_cohere_api_key}}` - `cohere_api_base` (string, optional) This is an optional field to provide a customized base URL for the Cohere API. If left unspecified, the standard Cohere base URL is used. Example: `https://api.cohere.ai/v1` - `google_cloud_vertex_ai_config` (object, optional) Google Cloud Vertex AI Config. Only required if the provider is 'google-cloud-vertex-ai'. - `private_key` (string, optional) The Databricks secret key reference for a private key for the service account which has access to the Google Cloud Vertex AI Service. See [Best practices for managing service account keys]. If you prefer to paste your API key directly, see `private_key_plaintext`. You must provide an API key using one of the following fields: `private_key` or `private_key_plaintext` [Best practices for managing service account keys]: https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys Example: `{{secrets/my_scope/my_google_cloud_vertex_ai_api_key}}` - `project_id` (string, required) This is the Google Cloud project id that the service account is associated with. Example: `your-project-id` - `region` (string, required) This is the region for the Google Cloud Vertex AI Service. See [supported regions] for more details. Some models are only available in specific regions. [supported regions]: https://cloud.google.com/vertex-ai/docs/general/locations Example: `us-central1` - `databricks_model_serving_config` (object, optional) Databricks Model Serving Config. Only required if the provider is 'databricks-model-serving'. - `databricks_api_token` (string, optional) The Databricks secret key reference for a Databricks API token that corresponds to a user or service principal with Can Query access to the model serving endpoint pointed to by this external model. If you prefer to paste your API key directly, see `databricks_api_token_plaintext`. You must provide an API key using one of the following fields: `databricks_api_token` or `databricks_api_token_plaintext`. Example: `{{secrets/my_scope/my_databricks_api_token}}` - `databricks_workspace_url` (string, required) The URL of the Databricks workspace containing the model serving endpoint pointed to by this external model. Example: `https://my-databricks-workspace.com` - `openai_config` (object, optional) OpenAI Config. Only required if the provider is 'openai'. - `openai_api_key` (string, optional) The Databricks secret key reference for an OpenAI API key using the OpenAI or Azure service. If you prefer to paste your API key directly, see `openai_api_key_plaintext`. You must provide an API key using one of the following fields: `openai_api_key` or `openai_api_key_plaintext`. Example: `{{secrets/my_scope/my_openai_api_key}}` - `openai_api_type` (string, optional) This is an optional field to specify the type of OpenAI API to use. For Azure OpenAI, this field is required, and adjust this parameter to represent the preferred security access validation protocol. For access token validation, use azure. For authentication using Azure Active Directory (Azure AD) use, azuread. Example: `azure` - `openai_api_base` (string, optional) This is a field to provide a customized base URl for the OpenAI API. For Azure OpenAI, this field is required, and is the base URL for the Azure OpenAI API service provided by Azure. For other OpenAI API types, this field is optional, and if left unspecified, the standard OpenAI base URL is used. Example: `https://api.openai.com/v1` - `openai_api_version` (string, optional) This is an optional field to specify the OpenAI API version. For Azure OpenAI, this field is required, and is the version of the Azure OpenAI service to utilize, specified by a date. Example: `2023-11-01` - `openai_deployment_name` (string, optional) This field is only required for Azure OpenAI and is the name of the deployment resource for the Azure OpenAI service. Example: `my_deployment_resource` - `openai_organization` (string, optional) This is an optional field to specify the organization in OpenAI or Azure OpenAI. Example: `Databricks` - `microsoft_entra_tenant_id` (string, optional) This field is only required for Azure AD OpenAI and is the Microsoft Entra Tenant ID. Example: `12345678-abcd-1234-5678-12345678abcd` - `microsoft_entra_client_id` (string, optional) This field is only required for Azure AD OpenAI and is the Microsoft Entra Client ID. Example: `12345678-abcd-1234-5678-12345678abcd` - `microsoft_entra_client_secret` (string, optional) The Databricks secret key reference for a client secret used for Microsoft Entra ID authentication. If you prefer to paste your client secret directly, see `microsoft_entra_client_secret_plaintext`. You must provide an API key using one of the following fields: `microsoft_entra_client_secret` or `microsoft_entra_client_secret_plaintext`. Example: `{{secrets/my_scope/my_microsoft_entra_client_secret}}` - `palm_config` (object, optional) PaLM Config. Only required if the provider is 'palm'. - `palm_api_key` (string, optional) The Databricks secret key reference for a PaLM API key. If you prefer to paste your API key directly, see `palm_api_key_plaintext`. You must provide an API key using one of the following fields: `palm_api_key` or `palm_api_key_plaintext`. Example: `{{secrets/my_scope/my_palm_api_key}}` - `custom_provider_config` (object, optional) Custom Provider Config. Only required if the provider is 'custom'. - `custom_provider_url` (string, required) This is a field to provide the URL of the custom provider API. Example: `https://custom-provider.com` - `bearer_token_auth` (object, optional) This is a field to provide bearer token authentication for the custom provider API. You can only specify one authentication method. - `token` (string, optional) The Databricks secret key reference for a token. If you prefer to paste your token directly, see `token_plaintext`. Example: `{{secrets/my_scope/my_openai_api_key}}` - `api_key_auth` (object, optional) This is a field to provide API key authentication for the custom provider API. You can only specify one authentication method. - `key` (string, required) The name of the API key parameter used for authentication. Example: `api-key` - `value` (string, optional) The Databricks secret key reference for an API Key. If you prefer to paste your token directly, see `value_plaintext`. Example: `{{secrets/my_scope/my_api_key}}` - `foundation_model` (object, optional, Public Preview) - `name` (string, optional, Public Preview) - `display_name` (string, optional, Public Preview) - `docs` (string, optional, Public Preview) - `description` (string, optional, Public Preview) - `served_models` (array of object, optional) (Deprecated, use served_entities instead) The list of served models under the serving endpoint config. - `name` (string, optional) - `model_name` (string, optional) Only one of model_name and entity_name should be populated - `entity_name` (string, optional) - `model_version` (string, optional) Only one of model_version and entity_version should be populated - `entity_version` (string, optional) - `external_model` (object, optional) - `provider` (string, required) The name of the provider for the external model. Currently, the supported providers are 'ai21labs', 'anthropic', 'amazon-bedrock', 'cohere', 'databricks-model-serving', 'google-cloud-vertex-ai', 'openai', 'palm', and 'custom'. - `name` (string, required) The name of the external model. Example: `gpt-4` - `task` (string, required) The task type of the external model. Example: `llm/v1/chat` - `ai21labs_config` (object, optional) AI21Labs Config. Only required if the provider is 'ai21labs'. - `ai21labs_api_key` (string, optional) The Databricks secret key reference for an AI21 Labs API key. If you prefer to paste your API key directly, see `ai21labs_api_key_plaintext`. You must provide an API key using one of the following fields: `ai21labs_api_key` or `ai21labs_api_key_plaintext`. Example: `{{secrets/my_scope/my_ai21labs_api_key}}` - `anthropic_config` (object, optional) Anthropic Config. Only required if the provider is 'anthropic'. - `anthropic_api_key` (string, optional) The Databricks secret key reference for an Anthropic API key. If you prefer to paste your API key directly, see `anthropic_api_key_plaintext`. You must provide an API key using one of the following fields: `anthropic_api_key` or `anthropic_api_key_plaintext`. Example: `{{secrets/my_scope/my_anthropic_api_key}}` - `amazon_bedrock_config` (object, optional) Amazon Bedrock Config. Only required if the provider is 'amazon-bedrock'. - `aws_region` (string, required) The AWS region to use. Bedrock has to be enabled there. Example: `myAwsRegion` - `aws_access_key_id` (string, optional) The Databricks secret key reference for an AWS access key ID with permissions to interact with Bedrock services. If you prefer to paste your API key directly, see `aws_access_key_id_plaintext`. You must provide an API key using one of the following fields: `aws_access_key_id` or `aws_access_key_id_plaintext`. Example: `{{secrets/my_scope/my_aws_access_key_id}}` - `aws_secret_access_key` (string, optional) The Databricks secret key reference for an AWS secret access key paired with the access key ID, with permissions to interact with Bedrock services. If you prefer to paste your API key directly, see `aws_secret_access_key_plaintext`. You must provide an API key using one of the following fields: `aws_secret_access_key` or `aws_secret_access_key_plaintext`. Example: `{{secrets/my_scope/my_aws_secret_access_key}}` - `bedrock_provider` (string, required) The underlying provider in Amazon Bedrock. Supported values (case insensitive) include: Anthropic, Cohere, AI21Labs, Amazon. - `instance_profile_arn` (string, optional) ARN of the instance profile that the external model will use to access AWS resources. You must authenticate using an instance profile or access keys. If you prefer to authenticate using access keys, see `aws_access_key_id`, `aws_access_key_id_plaintext`, `aws_secret_access_key` and `aws_secret_access_key_plaintext`. Example: `arn:aws:iam::123456789012:instance-profile/my-instance-profile` - `uc_service_credential_name` (string, optional, Public Preview) The name of the Unity Catalog service credential that the external model uses to access AWS resources. Example: `my-uc-service-credential` - `cohere_config` (object, optional) Cohere Config. Only required if the provider is 'cohere'. - `cohere_api_key` (string, optional) The Databricks secret key reference for a Cohere API key. If you prefer to paste your API key directly, see `cohere_api_key_plaintext`. You must provide an API key using one of the following fields: `cohere_api_key` or `cohere_api_key_plaintext`. Example: `{{secrets/my_scope/my_cohere_api_key}}` - `cohere_api_base` (string, optional) This is an optional field to provide a customized base URL for the Cohere API. If left unspecified, the standard Cohere base URL is used. Example: `https://api.cohere.ai/v1` - `google_cloud_vertex_ai_config` (object, optional) Google Cloud Vertex AI Config. Only required if the provider is 'google-cloud-vertex-ai'. - `private_key` (string, optional) The Databricks secret key reference for a private key for the service account which has access to the Google Cloud Vertex AI Service. See [Best practices for managing service account keys]. If you prefer to paste your API key directly, see `private_key_plaintext`. You must provide an API key using one of the following fields: `private_key` or `private_key_plaintext` [Best practices for managing service account keys]: https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys Example: `{{secrets/my_scope/my_google_cloud_vertex_ai_api_key}}` - `project_id` (string, required) This is the Google Cloud project id that the service account is associated with. Example: `your-project-id` - `region` (string, required) This is the region for the Google Cloud Vertex AI Service. See [supported regions] for more details. Some models are only available in specific regions. [supported regions]: https://cloud.google.com/vertex-ai/docs/general/locations Example: `us-central1` - `databricks_model_serving_config` (object, optional) Databricks Model Serving Config. Only required if the provider is 'databricks-model-serving'. - `databricks_api_token` (string, optional) The Databricks secret key reference for a Databricks API token that corresponds to a user or service principal with Can Query access to the model serving endpoint pointed to by this external model. If you prefer to paste your API key directly, see `databricks_api_token_plaintext`. You must provide an API key using one of the following fields: `databricks_api_token` or `databricks_api_token_plaintext`. Example: `{{secrets/my_scope/my_databricks_api_token}}` - `databricks_workspace_url` (string, required) The URL of the Databricks workspace containing the model serving endpoint pointed to by this external model. Example: `https://my-databricks-workspace.com` - `openai_config` (object, optional) OpenAI Config. Only required if the provider is 'openai'. - `openai_api_key` (string, optional) The Databricks secret key reference for an OpenAI API key using the OpenAI or Azure service. If you prefer to paste your API key directly, see `openai_api_key_plaintext`. You must provide an API key using one of the following fields: `openai_api_key` or `openai_api_key_plaintext`. Example: `{{secrets/my_scope/my_openai_api_key}}` - `openai_api_type` (string, optional) This is an optional field to specify the type of OpenAI API to use. For Azure OpenAI, this field is required, and adjust this parameter to represent the preferred security access validation protocol. For access token validation, use azure. For authentication using Azure Active Directory (Azure AD) use, azuread. Example: `azure` - `openai_api_base` (string, optional) This is a field to provide a customized base URl for the OpenAI API. For Azure OpenAI, this field is required, and is the base URL for the Azure OpenAI API service provided by Azure. For other OpenAI API types, this field is optional, and if left unspecified, the standard OpenAI base URL is used. Example: `https://api.openai.com/v1` - `openai_api_version` (string, optional) This is an optional field to specify the OpenAI API version. For Azure OpenAI, this field is required, and is the version of the Azure OpenAI service to utilize, specified by a date. Example: `2023-11-01` - `openai_deployment_name` (string, optional) This field is only required for Azure OpenAI and is the name of the deployment resource for the Azure OpenAI service. Example: `my_deployment_resource` - `openai_organization` (string, optional) This is an optional field to specify the organization in OpenAI or Azure OpenAI. Example: `Databricks` - `microsoft_entra_tenant_id` (string, optional) This field is only required for Azure AD OpenAI and is the Microsoft Entra Tenant ID. Example: `12345678-abcd-1234-5678-12345678abcd` - `microsoft_entra_client_id` (string, optional) This field is only required for Azure AD OpenAI and is the Microsoft Entra Client ID. Example: `12345678-abcd-1234-5678-12345678abcd` - `microsoft_entra_client_secret` (string, optional) The Databricks secret key reference for a client secret used for Microsoft Entra ID authentication. If you prefer to paste your client secret directly, see `microsoft_entra_client_secret_plaintext`. You must provide an API key using one of the following fields: `microsoft_entra_client_secret` or `microsoft_entra_client_secret_plaintext`. Example: `{{secrets/my_scope/my_microsoft_entra_client_secret}}` - `palm_config` (object, optional) PaLM Config. Only required if the provider is 'palm'. - `palm_api_key` (string, optional) The Databricks secret key reference for a PaLM API key. If you prefer to paste your API key directly, see `palm_api_key_plaintext`. You must provide an API key using one of the following fields: `palm_api_key` or `palm_api_key_plaintext`. Example: `{{secrets/my_scope/my_palm_api_key}}` - `custom_provider_config` (object, optional) Custom Provider Config. Only required if the provider is 'custom'. - `custom_provider_url` (string, required) This is a field to provide the URL of the custom provider API. Example: `https://custom-provider.com` - `bearer_token_auth` (object, optional) This is a field to provide bearer token authentication for the custom provider API. You can only specify one authentication method. - `token` (string, optional) The Databricks secret key reference for a token. If you prefer to paste your token directly, see `token_plaintext`. Example: `{{secrets/my_scope/my_openai_api_key}}` - `api_key_auth` (object, optional) This is a field to provide API key authentication for the custom provider API. You can only specify one authentication method. - `key` (string, required) The name of the API key parameter used for authentication. Example: `api-key` - `value` (string, optional) The Databricks secret key reference for an API Key. If you prefer to paste your token directly, see `value_plaintext`. Example: `{{secrets/my_scope/my_api_key}}` - `foundation_model` (object, optional, Public Preview) - `name` (string, optional, Public Preview) - `display_name` (string, optional, Public Preview) - `docs` (string, optional, Public Preview) - `description` (string, optional, Public Preview) - `tags` (array of object, optional) Tags attached to the serving endpoint. - `key` (string, required) Key field for a serving endpoint tag. Example: `team` - `value` (string, optional) Optional value field for a serving endpoint tag. Example: `data science` - `id` (string, optional) System-generated ID of the endpoint, included to be used by the Permissions API. Example: `88fd3f75a0d24b0380ddc40484d7a31b` - `task` (string, optional) The task type of the serving endpoint. Example: `model-serving-task` - `ai_gateway` (object, optional) The AI Gateway configuration for the serving endpoint. NOTE: External model, provisioned throughput, and pay-per-token endpoints are fully supported; agent endpoints currently only support inference tables. - `usage_tracking_config` (object, optional) Configuration to enable usage tracking using system tables. These tables allow you to monitor operational usage on endpoints and their associated costs. - `enabled` (boolean, optional) Whether to enable usage tracking. Example: `true` - `inference_table_config` (object, optional) Configuration for payload logging using inference tables. Use these tables to monitor and audit data being sent to and received from model APIs and to improve model quality. - `catalog_name` (string, optional) The name of the catalog in Unity Catalog. Required when enabling inference tables. NOTE: On update, you have to disable inference table first in order to change the catalog name. Example: `my-catalog` - `schema_name` (string, optional) The name of the schema in Unity Catalog. Required when enabling inference tables. NOTE: On update, you have to disable inference table first in order to change the schema name. Example: `my-schema` - `table_name_prefix` (string, optional) The prefix of the table in Unity Catalog. NOTE: On update, you have to disable inference table first in order to change the prefix name. Example: `my-prefix` - `enabled` (boolean, optional) Indicates whether the inference table is enabled. Example: `true` - `rate_limits` (array of object, optional) Configuration for rate limits which can be set to limit endpoint traffic. - `calls` (int64, optional) Used to specify how many calls are allowed for a key within the renewal_period. Example: `15` - `key` (string, optional) Key field for a rate limit. Currently, 'user', 'user_group, 'service_principal', and 'endpoint' are supported, with 'endpoint' being the default if not specified. - `renewal_period` (string, required) Renewal period field for a rate limit. Currently, only 'minute' is supported. - `principal` (string, optional) Principal field for a user, user group, or service principal to apply rate limiting to. Accepts a user email, group name, or service principal application ID. Example: `user@test.com` - `tokens` (int64, optional) Used to specify how many tokens are allowed for a key within the renewal_period. Example: `10000` - `guardrails` (object, optional, Public Preview) Configuration for AI Guardrails to prevent unwanted data and unsafe data in requests and responses. - `input` (object, optional, Public Preview) Configuration for input guardrail filters. - `safety` (boolean, optional, Public Preview) Indicates whether the safety filter is enabled. Example: `true` - `pii` (object, optional, Public Preview) Configuration for guardrail PII filter. - `behavior` (string, optional, Public Preview) Configuration for input guardrail filters. Possible values: - `BEHAVIOR_UNSPECIFIED` - `NONE` - `BLOCK` - `MASK` Example: `BLOCK` - `valid_topics` (array of string, optional, Deprecated, Public Preview) The list of allowed topics. Given a chat request, this guardrail flags the request if its topic is not in the allowed topics. - `invalid_keywords` (array of string, optional, Deprecated, Public Preview) List of invalid keywords. AI guardrail uses keyword or string matching to decide if the keyword exists in the request or response content. - `output` (object, optional, Public Preview) Configuration for output guardrail filters. - `safety` (boolean, optional, Public Preview) Indicates whether the safety filter is enabled. Example: `true` - `pii` (object, optional, Public Preview) Configuration for guardrail PII filter. - `behavior` (string, optional, Public Preview) Configuration for input guardrail filters. Possible values: - `BEHAVIOR_UNSPECIFIED` - `NONE` - `BLOCK` - `MASK` Example: `BLOCK` - `valid_topics` (array of string, optional, Deprecated, Public Preview) The list of allowed topics. Given a chat request, this guardrail flags the request if its topic is not in the allowed topics. - `invalid_keywords` (array of string, optional, Deprecated, Public Preview) List of invalid keywords. AI guardrail uses keyword or string matching to decide if the keyword exists in the request or response content. - `fallback_config` (object, optional) Configuration for traffic fallback which auto fallbacks to other served entities if the request to a served entity fails with certain error codes, to increase availability. - `enabled` (boolean, required) Whether to enable traffic fallback. When a served entity in the serving endpoint returns specific error codes (e.g. 500), the request will automatically be round-robin attempted with other served entities in the same endpoint, following the order of served entity list, until a successful response is returned. If all attempts fail, return the last response with the error code. Example: `true` - `budget_policy_id` (string, optional) The budget policy associated with the endpoint. - `description` (string, optional) Description of the endpoint - `usage_policy_id` (string, optional) The usage policy associated with serving endpoint. - `telemetry_config` (object, optional, Public Preview) Telemetry configuration for the endpoint, including inference-table payload logging. - `telemetry_profile_id` (string, optional, Public Preview) The ID of an existing telemetry profile to apply to this endpoint. Provide this to reuse a telemetry profile that has already been created, instead of specifying table_names. - `table_names` (object, optional, Public Preview) The Unity Catalog tables to which endpoint telemetry (logs, traces, and metrics) is exported. Provide this to create a new telemetry profile for the endpoint from the given tables. - `logs_table` (string, optional, Public Preview) The full three-level Unity Catalog name (catalog.schema.table) of the table that receives exported logs. - `metrics_table` (string, optional, Public Preview) The full three-level Unity Catalog name (catalog.schema.table) of the table that receives exported metrics. - `traces_table` (string, optional, Public Preview) The full three-level Unity Catalog name (catalog.schema.table) of the table that receives exported traces (spans). - `annotations_table` (string, optional, Public Preview) The full three-level Unity Catalog name (catalog.schema.table) of the table that receives exported annotations. - `inference_table_config` (object, optional, Public Preview) Configuration for inference table payload logging, including sampling. - `sampling_fraction` (double, optional, Public Preview) Fraction of requests sampled for payload logging, in the range [0.0, 1.0], where 1.0 logs all requests. - `name` (string, optional, Public Preview, Output only) The full name of the inference table created for this endpoint. - `enabled_telemetry_features` (array of string, optional, Public Preview) The telemetry signals to enable for this endpoint. If empty or omitted, all signals are enabled; otherwise only the listed signals are enabled. Possible values: - `TELEMETRY_FEATURE_LOGS` - `TELEMETRY_FEATURE_TRACES` - `TELEMETRY_FEATURE_METRICS` - `TELEMETRY_FEATURE_INFERENCE_TABLE` ## Response ```json { "endpoints": [ { "ai_gateway": { "guardrails": { "input": { "invalid_keywords": [ "keyword1" ], "pii": { "behavior": "BLOCK" }, "safety": true, "valid_topics": [ "topic1" ] }, "output": { "invalid_keywords": [ "keyword1" ], "pii": { "behavior": "BLOCK" }, "safety": true, "valid_topics": [ "topic1" ] } }, "inference_table_config": { "catalog_name": "my-catalog", "enabled": true, "schema_name": "my-schema", "table_name_prefix": "my-prefix" }, "rate_limits": [ { "calls": 15, "key": "user", "renewal_period": "minute" } ], "usage_tracking_config": { "enabled": true } }, "config": { "served_entities": [ { "entity_name": "ads-model", "entity_version": "3", "external_model": { "ai21labs_config": { "ai21labs_api_key": "{{secrets/my_scope/my_ai21labs_api_key}}" }, "amazon_bedrock_config": { "aws_access_key_id": "{{secrets/my_scope/my_aws_access_key_id}}", "aws_region": "myAwsRegion", "aws_secret_access_key": "{{secrets/my_scope/my_aws_secret_access_key}}", "bedrock_provider": "anthropic" }, "anthropic_config": { "anthropic_api_key": "{{secrets/my_scope/my_anthropic_api_key}}" }, "cohere_config": { "cohere_api_base": "https://api.cohere.ai/v1", "cohere_api_key": "{{secrets/my_scope/my_cohere_api_key}}" }, "databricks_model_serving_config": { "databricks_api_token": "{{secrets/my_scope/my_databricks_api_token}}", "databricks_workspace_url": "https://my-databricks-workspace.com" }, "google_cloud_vertex_ai_config": { "private_key": "{{secrets/my_scope/my_google_cloud_vertex_ai_api_key}}", "project_id": "your-project-id", "region": "us-central1" }, "name": "gpt-4", "openai_config": { "microsoft_entra_client_id": "12345678-abcd-1234-5678-12345678abcd", "microsoft_entra_client_secret": "{{secrets/my_scope/my_microsoft_entra_client_secret}}", "microsoft_entra_tenant_id": "12345678-abcd-1234-5678-12345678abcd", "openai_api_base": "https://api.openai.com/v1", "openai_api_key": "{{secrets/my_scope/my_openai_api_key}}", "openai_api_type": "azure", "openai_api_version": "2023-11-01", "openai_deployment_name": "my_deployment_resource", "openai_organization": "Databricks" }, "palm_config": { "palm_api_key": "{{secrets/my_scope/my_palm_api_key}}" }, "provider": "openai", "task": "llm/v1/chat" }, "foundation_model": { "description": "Llama 3.1 is a state-of-the-art 70B parameter dense language model trained and released by Meta. The model supports a context length of 128K tokens. The model is optimized for multilingual dialogue use cases and aligned with human preferences for helpfulness and safety. It is not intended for use in languages other than English. Meta Llama 3.1 is licensed under the Meta Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. Customers are responsible for ensuring compliance with applicable model licenses.", "display_name": "Meta Llama 3.1 70B Instruct", "docs": "https://docs.databricks.com/machine-learning/foundation-models/supported-models.html#meta-llama-31-70b-instruct", "name": "databricks-meta-llama-3-1-70b-instruct" }, "name": "ads-model-3" } ], "served_models": [ { "model_name": "ads-model", "model_version": "3", "name": "ads-model-3" } ] }, "creation_timestamp": 0, "creator": "alice@company.com", "id": "88fd3f75a0d24b0380ddc40484d7a31b", "last_updated_timestamp": 0, "name": "feed-ads", "state": { "config_update": "NOT_UPDATING", "ready": "READY" }, "tags": [ { "key": "team", "value": "data science" } ], "task": "model-serving-task" } ] } ```