Skip to main content
Last updated on

Query model services in AI Playground

AI Playground is a chat interface in your workspace where you can query Unity Gateway model services without writing code. You can query system-provided model services in system.ai and custom model services that you or your team create in Unity Catalog.

Requests that you send from the playground go through Unity Gateway, so the permissions, rate limits, guardrails, and usage tracking on the model service apply to them. For other playground features, such as tools and agent prototyping, see Chat with LLMs and prototype AI apps using AI Playground.

Requirements​

note

Unity Gateway is not supported on AWS GovCloud.

Permissions on an underlying model don't grant access to the model service. See Migrate to Unity Gateway.

Open a model service in AI Playground​

Open the playground in one of the following ways:

  • From the playground: Select Playground from the left navigation pane under AI/ML, then select the model service from the dropdown list at the top.
  • From the model service: In the workspace sidebar, click Unity Gateway. On the Models tab, select the model service, then click Chat in playground. If the playground doesn't select the model service for you, select it from the dropdown list at the top.

The dropdown list shows only the model services that you have access to.

Send prompts and compare model services​

Send prompts and compare responses the same way you do with any model in the playground. See Chat with LLMs and prototype AI apps using AI Playground.

Note the fully qualified name of the model service, such as system.ai.claude-sonnet-4-5 or main.default.team_chat. You use the same name when you call the model service from code.

To compare model services side by side, select + to add a pane, then select a different model service. Comparing helps you choose a model service for your application, or compare providers before you set up traffic splitting. For example, compare a system-provided model service with a custom model service.

How Unity Gateway features apply to playground requests​

The playground sends each request to the model service that you select. The features configured on that model service apply to the request:

Feature

Behavior in AI Playground

Permissions

Only users with EXECUTE on the model service can query it from the playground. Model services use definer's privileges, so you don't need access to the models that the service routes to.

Rate limits

Service and per-user rate limits on the model service apply to playground requests. When you exceed a limit, the request fails until capacity recovers.

Guardrails

Service policies attached to the model service evaluate your prompts and the model's responses. When a policy blocks content, the response reports that the content was blocked by policy. Service policies are in Beta. A workspace admin turns them on with the Unity Gateway Service Policies preview on the Previews page. See Manage workspace-level previews and Tutorial: Add service policy guardrails to a model service.

Usage tracking

The system.ai_gateway.usage system table records playground requests with invocation_metadata.source set to AI_PLAYGROUND.

Inference tables

If the model service logs to an inference table, the table logs playground requests and responses.

Feature

Behavior in AI Playground

Permissions

Only users with EXECUTE on the model service can query it from the playground. Model services use definer's privileges, so you don't need access to the models that the service routes to.

Rate limits

Service and per-user rate limits on the model service apply to playground requests. When you exceed a limit, the request fails until capacity recovers.

Guardrails

Service policies attached to the model service evaluate your prompts and the model's responses. When a policy blocks content, the response reports that the content was blocked by policy. Service policies are in Beta. A workspace admin turns them on with the Unity Gateway Service Policies preview on the Previews page. See Manage workspace-level previews and Tutorial: Add service policy guardrails to a model service.

Usage tracking

The system.ai_gateway.usage system table records playground requests with invocation_metadata.source set to AI_PLAYGROUND.

Inference tables

If the model service logs to an inference table, the table logs playground requests and responses.

note

Privilege and service policy changes can take a few minutes to propagate. During that time, playground requests might still use the previous privileges or policies.

To review your playground usage, run the following query. Querying the usage table requires access to system tables. See Grant access to system tables.

SQL
SELECT
  service_name,
  requester,
  COUNT(*) AS request_count,
  SUM(total_tokens) AS total_tokens
FROM system.ai_gateway.usage
WHERE invocation_metadata.source = 'AI_PLAYGROUND'
GROUP BY service_name, requester
ORDER BY total_tokens DESC;

Move from the playground to code​

To call the same model service from your application, pass its fully qualified name as the model. You can query a model service with the unified APIs, the provider-native APIs, or, for system-provided model services, ai_query.

For example, to query a model service with the Unified Responses API and the OpenAI Python SDK, run the following:

Python
from openai import OpenAI
import os

DATABRICKS_TOKEN = os.environ.get('DATABRICKS_TOKEN')

client = OpenAI(
  api_key=DATABRICKS_TOKEN,
  base_url="https://<workspace-url>/ai-gateway/mlflow/v1"
)

response = client.responses.create(
  model="<model-service>",
  input=[{"role": "user", "content": "What is Databricks?"}]
)

print(response.output_text)

Replace <workspace-url> with your Databricks workspace URL and <model-service> with the fully qualified name of your model service.

To send your first request and see its usage and cost metrics, see Unity Gateway quickstart. If you prototyped an agent in the playground, see Step 3: Export your agent.

Additional resources​

On this page