Foundation Model APIs limits and quotas
This page describes the limits and quotas for Databricks Foundation Model APIs workloads.
Databricks Foundation Model APIs enforce rate limits to ensure reliable performance and fair resource allocation across all users. These limits vary based on the workspace platform tier, foundation model type and how you deploy your foundation model.
Pay-per-token endpoint rate limits
Pay-per-token endpoints are governed by token-based and query-based rate limits. Token-based rate limits control the maximum number of tokens that can be processed per minute and are enforced separately for input and output tokens.
- Input tokens per minute (ITPM): The maximum number of input tokens (from your prompts) that can be processed within a 60-second window. An ITPM rate limit controls the input token throughput of an endpoint.
- Output tokens per minute (OTPM): The maximum number of output tokens (from the model's responses) that can be generated within a 60-second window. An OTPM rate limit controls the output token throughput of an endpoint.
- Queries per hour: The maximum number of queries or requests that can be processed within a 60 minute window. For production applications with sustained usage patterns, Databricks recommends provisioned throughput endpoints, which provide guaranteed capacity.
How limits are tracked and enforced
The most restrictive rate limit (ITPM, OTPM, QPH) applies at any given time. For example, even if you haven't reached your ITPM limit, you might still be rate-limited if you exceed the QPH or OTPM limit. When either ITPM or OTPM limit is reached, subsequent requests receive a 429 error that indicates too many requests were received. This message persists until the rate limit window resets.
Databricks tracks and enforces tokens per minute (TPM) rate limits using the following features:
Feature | Details |
|---|---|
Token accounting and pre-admission checks |
|
Burst capacity and smoothing |
|
The following is an example of how pre-admission checking and the credit-back behavior work.
# Request with max_tokens specified
request = {
"prompt": "Write a story about...", # 10 input tokens
"max_tokens": 500 # System reserves 500 output tokens
}
# Pre-admission check:
# - Verifies 10 tokens against ITPM limit
# - Reserves 500 tokens against OTPM limit
# - If either would exceed limits, returns 429 immediately
# If admitted, actual response uses only 350 tokens
# The system credits back 150 tokens (500 - 350) to your OTPM allowance
# These 150 tokens are immediately available for other requests
Rate limits by model
The following tables summarize the ITPM, OTPM, and QPH rate limits for pay-per-token Foundation Model API endpoints for workspaces on the Enterprise workspace platform tier:
For models that support provisioned throughput, create a provisioned throughput endpoint if the pay-per-token limits don't meet your use case requirements.
Partner model token quota levels
The partner-model token quota ladder has three numbered tiers. Partner models default to Tier 1 unless a model-specific provider-capacity constraint requires a lower limit. These quota tiers are separate from your workspace platform tier.
Quota level | ITPM limit | OTPM limit | Availability |
|---|---|---|---|
Tier 1 | 1,000,000 | 100,000 | Applied automatically to partner models unless a model-specific provider-capacity exception applies |
Tier 2 | 5,000,000 | 500,000 | Available through the quota request flow |
Tier 3 | 10,000,000 | 1,000,000 | Available through the quota request flow |
Custom token limits require a capacity review by Databricks and are outside these three tiers. Token quota tiers change only ITPM and OTPM limits; RPS, QPS, RPH, and QPH limits remain model-specific. A partner model retains a lower default only for a model-specific provider-capacity constraint. Open models retain the defaults shown in the following tables.
Large language models | ITPM limit | OTPM limit | QPH limit |
|---|---|---|---|
GPT-6.1 Sol | 1,000,000 | 100,000 | 360,000 |
GPT-6 Sol | 1,000,000 | 100,000 | 360,000 |
GPT-6 Luna | 1,000,000 | 100,000 | 360,000 |
GPT-6 Astra | 1,000,000 | 100,000 | 360,000 |
GPT-5.6 Sol | 1,000,000 | 100,000 | 360,000 |
GPT-5.6 Terra | 1,000,000 | 100,000 | 360,000 |
GPT-5.6 Luna | 1,000,000 | 100,000 | 360,000 |
GPT-5.5 Pro | 1,000,000 | 100,000 | 360,000 |
GPT-5.5 | 1,000,000 | 100,000 | 360,000 |
GPT-5.4 | 1,000,000 | 100,000 | 360,000 |
GPT-5.4 mini | 1,000,000 | 100,000 | 360,000 |
GPT-5.4 nano | 1,000,000 | 100,000 | 360,000 |
GPT-5.3 Codex | 1,000,000 | 100,000 | 360,000 |
GPT-5.2 | 1,000,000 | 100,000 | 360,000 |
GPT-5.1 | 1,000,000 | 100,000 | 360,000 |
GPT-5 | 1,000,000 | 100,000 | 360,000 |
GPT-5 mini | 1,000,000 | 100,000 | 360,000 |
GPT-5 nano | 1,000,000 | 100,000 | 360,000 |
Gemini 3.1 Flash Image | 1,000,000 | 100,000 | 360,000 |
Gemini 3 Pro Image | 1,000,000 | 100,000 | 360,000 |
Gemini 3.5 Flash Lite | 1,000,000 | 100,000 | 360,000 |
Gemini 3.8 Flash | 1,000,000 | 100,000 | 360,000 |
Gemini 3.7 Flash | 1,000,000 | 100,000 | 360,000 |
Gemini 3.6 Flash | 1,000,000 | 100,000 | 360,000 |
Gemini 3.5 Flash | 1,000,000 | 100,000 | 360,000 |
Gemini 3.1 Pro Preview | 1,000,000 | 100,000 | 360,000 |
Gemini 3.1 Flash Lite | 1,000,000 | 100,000 | 360,000 |
Gemini 3 Flash | 1,000,000 | 100,000 | 360,000 |
Qwen3.5 122B A10B (Public Preview) | 1,000,000 | 100,000 | 360,000 |
Qwen3-Next 80B A3B Instruct (Beta) | 1,000,000 | 100,000 | 360,000 |
GLM 5.3 | 2,000,000 | 40,000 | 7,200 |
GLM 5.3 Flash | 2,000,000 | 40,000 | 7,200 |
GLM 5.2 (legacy) | 200,000 | 20,000 | 7,200 |
OpenJev (Qwen3.5 4B) | 2,000,000 | 40,000 | 7,200 |
DeepSeek V4.1 Flash | 2,000,000 | 40,000 | 7,200 |
DeepSeek V4 Pro (0813) | 200,000 | 4,000 | 7,200 |
DeepSeek V4 Flash (0731) | 200,000 | 10,000 | |
Kimi K3 | 2,000,000 | 40,000 | |
Grok 4.6 | 1,000,000 | 100,000 | 360,000 |
Grok 4.7 | 1,000,000 | 100,000 | 360,000 |
Inkling (Public Preview) | 200,000 | 10,000 | 7,200 |
GPT OSS 120B | 1,000,000 | 100,000 | 360,000 |
GPT OSS 20B | 1,000,000 | 100,000 | 360,000 |
Gemma 3 12B | 1,000,000 | 100,000 | 360,000 |
Llama 4 Maverick | 1,000,000 | 100,000 | 360,000 |
Llama 3.3 70B Instruct | 1,000,000 | 100,000 | 360,000 |
Llama 3.1 8B Instruct | 1,000,000 | 100,000 | 360,000 |
Anthropic Claude models | ITPM limit | OTPM limit | QPH limit |
|---|---|---|---|
Claude Fable 5.1 | 1,000,000 | 100,000 | 360,000 |
Claude Fable 5 | 1,000,000 | 100,000 | 360,000 |
Claude Haiku 5.5 | 200,000 | 20,000 | 360,000 |
Claude Haiku 4.5 | 1,000,000 | 100,000 | 360,000 |
Claude Opus 5 | 1,000,000 | 100,000 | 360,000 |
Claude Opus 5.5 | 1,000,000 | 100,000 | 360,000 |
Claude Opus 4.8 | 1,000,000 | 100,000 | 360,000 |
Claude Opus 4.7 | 1,000,000 | 100,000 | 360,000 |
Claude Opus 4.6 | 1,000,000 | 100,000 | 360,000 |
Claude Opus 4.5 | 1,000,000 | 100,000 | 360,000 |
Claude Opus 4.1 | 1,000,000 | 100,000 | 360,000 |
Claude Sonnet 5 | 1,000,000 | 100,000 | 360,000 |
Claude Sonnet 5.5 | 1,000,000 | 100,000 | 360,000 |
Claude Sonnet 4.6 | 1,000,000 | 100,000 | 360,000 |
Claude Sonnet 4.5 | 1,000,000 | 100,000 | 360,000 |
Embedding models | ITPM limit | OTPM limit | QPH limit |
|---|---|---|---|
Qwen3-Embedding-0.6B | N/A | N/A | 2,160,000 |
GTE Large (En) | N/A | N/A | 540,000 |
BGE Large (En) | N/A | N/A | 2,160,000 |
Best practices for managing TPM rate limits
Step 1. Monitor token usage
Track both input and output token counts separately in your applications:
# Example: Track token usage
response = model.generate(prompt)
input_tokens = response.usage.prompt_tokens
output_tokens = response.usage.completion_tokens
total_tokens = response.usage.total_tokens
# Check against limits
if input_tokens > ITPM_LIMIT or output_tokens > OTPM_LIMIT:
# Implement backoff strategy
pass
Step 2. Implement retry logic
Add exponential backoff when you encounter rate limit errors:
import time
import random
def retry_with_exponential_backoff(
func,
initial_delay: float = 1,
exponential_base: float = 2,
jitter: bool = True,
max_retries: int = 10,
):
"""Retry a function with exponential backoff."""
num_retries = 0
delay = initial_delay
while num_retries < max_retries:
try:
return func()
except Exception as e:
if "rate_limit" in str(e) or "429" in str(e):
num_retries += 1
if jitter:
delay *= exponential_base * (1 + random.random())
else:
delay *= exponential_base
time.sleep(delay)
else:
raise e
raise Exception(f"Maximum retries {max_retries} exceeded")
Step 3. Optimize token usage
- Minimize prompt length: Use concise, well-structured prompts
- Control output length: Use
max_tokensparameter to limit response size - Batch efficiently: Group related requests when possible while staying within limits
Step 4. Consider model selection
- Smaller models for high-volume tasks: Use models like GPT OSS 20B for tasks that require higher throughput.
- Large models for complex tasks: Reserve GPT OSS 120B for tasks that require maximum capability.
Monitoring and troubleshooting
Monitor your token usage patterns to optimize performance:
# Example: Log token usage for monitoring
import logging
logger = logging.getLogger(__name__)
def log_token_usage(response):
usage = response.usage
logger.info(f"Input tokens: {usage.prompt_tokens}")
logger.info(f"Output tokens: {usage.completion_tokens}")
logger.info(f"Total tokens: {usage.total_tokens}")
# Alert if approaching limits
if usage.prompt_tokens > ITPM_LIMIT * 0.8:
logger.warning("Approaching ITPM limit")
if usage.completion_tokens > OTPM_LIMIT * 0.8:
logger.warning("Approaching OTPM limit")
Handle rate limit errors
When you exceed rate limits, the API returns a 429 Too Many Requests error:
{
"error": {
"message": "Rate limit exceeded: ITPM limit of 1,000,000 tokens reached",
"type": "rate_limit_exceeded",
"code": 429,
"limit_type": "input_tokens_per_minute",
"limit": 1000000,
"current": 1000150,
"retry_after": 15
}
}
The error response includes:
limit_type: Which specific limit was exceeded (ITPM, OTPM, QPS, or QPH)limit: The configured limit valuecurrent: Your current usageretry_after: Suggested wait time in seconds
Handle service capacity errors
A 429 Too Many Requests error means your workload exceeded a rate limit under your control. A 503 Service Unavailable error is different: it means Foundation Model APIs are temporarily at capacity and could not serve your request. A 503 does not count against or indicate that you reached your own rate limits.
Treat 503 responses as transient and retry with exponential backoff, following the same pattern as in Step 2. Implement retry logic. For production workloads that need consistent capacity, use provisioned throughput, which provides dedicated capacity instead of shared pay-per-token capacity. See documentation on reserved provisioned throughput and on-demand provisioned throughput for supported foundation models.
Common issues and solutions
Issue | Solution |
|---|---|
Frequent 429 errors | Implement exponential backoff, reduce request rate, and request higher rate limits |
ITPM limit reached | Optimize prompt length |
OTPM limit reached | Use |
QPH limit reached | Distribute requests more evenly over time |
503 Service Unavailable | Retry with exponential backoff; for consistent production capacity, use provisioned throughput |
Provisioned throughput limits
For production workloads that require higher limits, provisioned throughput endpoints offer:
- No TPM restrictions: Processing capacity based on provisioned resources
- Higher rate limits: Up to 200 queries per second per workspace
- Predictable performance: Dedicated resources ensure consistent latency
Output token limits
The following table summarizes the output token limits for each supported model:
Model | Output token limit |
|---|---|
GPT OSS 120B | 25,000 |
GPT OSS 20B | 25,000 |
Gemma 3 12B | 8,192 |
Llama 4 Maverick | 8,192 |
Llama 3.1 70B | 8,192 |
Llama 3.1 8B | 8,192 |
Additional limits
The following are limitations for provisioned throughput workloads:
- For provisioned throughput workloads that use Llama 4 Maverick:
- Support for this model on provisioned throughput workloads is in Public Preview.
- Autoscaling is not supported.
- Metrics panels are not supported.
- Traffic splitting is not supported on an endpoint that serves Llama 4 Maverick. You cannot serve multiple models on an endpoint that serves Llama 4 Maverick.
- To deploy a Meta Llama model from
system.aiin Unity Catalog, you must choose the applicable Instruct version. Base versions of the Meta Llama models are not supported for deployment from Unity Catalog. See Deploy provisioned throughput endpoints.
Regional availability and data processing
For Databricks-hosted foundation model region availability, see Foundation Model overview.
For data processing and residency details, see Data processing and residency.
Resource and payload limits for Foundation models and external models
The following tables summarize resource and payload limits for endpoints serving foundation models and external models.
Feature | Granularity | Limit |
|---|---|---|
Payload size | Per request | 4 MB |
Request/response size | Per request | Any request/response over 1 MB will not be logged. |
Queries per second (QPS) | Per workspace | 200 |
Model execution duration | Per request | 597 seconds |
Overhead latency | Per request | Less than 50 milliseconds |