Query external model providers (model provider services)

Query models through a Model Provider Service in Unity Gateway, which supplies stored credentials and routes the request to the external provider, so callers don't handle the provider secret.

Requirements

• Workspace entitlement to query: Workspace access, or Consumer access with the Consumer access to Unity Gateway preview enabled for your account (Public Preview). See Manage entitlements and Manage Databricks previews.
• The following permissions are needed. See Discover and govern access to external model providers (model provider services).
  • EXECUTE on the model provider service
  • USE CATALOG on the catalog
  • USE SCHEMA on the schema

Identify a model provider service

You select a model provider service for a request with the Databricks-Model-Provider-Service header, set to the service's three-part name:

Text

Databricks-Model-Provider-Service: main.default.openai_prod

Authenticate with your Databricks token, not the provider's credential. The base URL is your workspace URL followed by /ai-gateway.

Query Supported APIs

Managed paths make available each provider's API under a stable Unity Gateway path. Unity Gateway translates between the request and the provider, applies governance such as guardrails and rate limits, and records usage. This is the recommended way to query a model provider service.

The following example sends a chat completion through an OpenAI model provider service using the managed OpenAI path. Because the request uses the OpenAI Chat Completions API, you can point the OpenAI client at the Unity Gateway base URL.

Python

Python

from openai import OpenAI

client = OpenAI(
    api_key="<databricks-token>",
    base_url="https://<workspace-url>/ai-gateway/openai/v1",
    default_headers={"Databricks-Model-Provider-Service": "main.default.openai_prod"},
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Say hello in exactly 3 words."}],
)
print(response.choices[0].message.content)

REST

Bash

curl https://<workspace-url>/ai-gateway/openai/v1/chat/completions \
  -H "Authorization: Bearer $DATABRICKS_TOKEN" \
  -H "Databricks-Model-Provider-Service: main.default.openai_prod" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Say hello in exactly 3 words."}]
  }'

The managed path you call depends on the provider's API:

Provider API	Managed path
OpenAI (chat completions)	/ai-gateway/openai/v1/chat/completions
OpenAI (responses)	/ai-gateway/openai/v1/responses
OpenAI (embeddings)	/ai-gateway/openai/v1/embeddings
Anthropic (messages)	/ai-gateway/anthropic/v1/messages
Gemini (generate content)	/ai-gateway/gemini/v1beta/models/<model>:generateContent
Gemini (generate content with streaming responses)	/ai-gateway/gemini/v1beta/models/<model>:streamGenerateContent

Provider API	Managed path
OpenAI (chat completions)	/ai-gateway/openai/v1/chat/completions
OpenAI (responses)	/ai-gateway/openai/v1/responses
OpenAI (embeddings)	/ai-gateway/openai/v1/embeddings
Anthropic (messages)	/ai-gateway/anthropic/v1/messages
Gemini (generate content)	/ai-gateway/gemini/v1beta/models/<model>:generateContent
Gemini (generate content with streaming responses)	/ai-gateway/gemini/v1beta/models/<model>:streamGenerateContent

The model in the request body (or the Gemini path segment) must be a model the model provider service allows.

Query Other APIs (Passthrough)

If a managed path does not cover a provider endpoint, such as an OpenAI file or batch endpoint, you can pass the request through to the provider unchanged. Unity Gateway strips the /ai-gateway prefix, attaches the stored credential, and forwards the remaining path to the provider.

To enable unmanaged passthrough, select Forward all URL paths under Advanced options when you create or update the model provider service in the UI.

note

Usage token and cost tracking, token-based rate limits, model access control, and service policies do not apply to passthrough requests.

After you enable passthrough, call the provider's native path under /ai-gateway. For example, list files on the OpenAI files endpoint:

Bash

curl https://<workspace-url>/ai-gateway/files \
  -H "Authorization: Bearer $DATABRICKS_TOKEN" \
  -H "Databricks-Model-Provider-Service: main.default.openai_prod"

Headers and query parameters forwarding

By default, Unity Gateway does not pass the client's request headers or query parameters to the upstream provider. Two service-configuration flags change this, and they apply to both managed and unmanaged paths:

• forward_headers: When true, Unity Gateway forwards client request headers to the provider. Enable this when a provider requires a header that Unity Gateway does not set for you, such as OpenAI-Organization.
• forward_query_parameters: When true, Unity Gateway forwards client query parameters to the provider.

Set them on the model provider service like any other configuration field:

Bash

curl https://<workspace-url>/api/2.1/unity-catalog/model-provider-services/main.default.openai_prod \
  -X PATCH \
  -H "Authorization: Bearer $DATABRICKS_TOKEN" \
  -H "Content-Type: application/json" \
  -G \
  --data-urlencode "update_mask=config.forward_headers,config.forward_query_parameters" \
  --data '{ "config": { "forward_headers": true, "forward_query_parameters": true } }'

Next steps

• Discover and govern access to external model providers (model provider services)
• Track model usage
• Request tagging
• Model providers external to Databricks