Foundation model REST API reference

This article provides general API information for Databricks Foundation Model APIs and the models they support. The Foundation Model APIs are designed to be similar to OpenAI's REST API to make migrating existing projects easier. Both the pay-per-token and provisioned throughput endpoints accept the same REST API request format.

Endpoints

Foundation Model APIs supports pay-per-token endpoints and provisioned throughput endpoints.

A preconfigured endpoint is available in your workspace for each pay-per-token supported model, and users can interact with these endpoints using HTTP POST requests. See Pay-per-token for supported models.

Provisioned throughput endpoints can be created using the API or the Serving UI. These endpoints support multiple models per endpoint for A/B testing, as long as both served models expose the same API format. For example, both models are chat models. See POST /api/2.0/serving-endpoints for endpoint configuration parameters.

Requests and responses use JSON, the exact JSON structure depends on an endpoint's task type. Chat and completion endpoints support streaming responses.

Usage

Responses include a usage sub-message which reports the number of tokens in the request and response. The format of this sub-message is the same across all task types.

Field	Type	Description
completion_tokens	Integer	Number of generated tokens. Not included in embedding responses.
prompt_tokens	Integer	Number of tokens from the input prompt(s).
total_tokens	Integer	Number of total tokens.

For models like databricks-meta-llama-3-3-70b-instruct a user prompt is transformed using a prompt template before being passed into the model. For pay-per-token endpoints, a system prompt might also be added. prompt_tokens includes all text added by our server.

Chat task

Chat tasks are optimized for multi-turn conversations with a model. The model response provides the next assistant message in the conversation. See POST /serving-endpoints/{name}/invocations for querying endpoint parameters.

Chat request

Field	Default	Type	Description
messages		ChatMessage list	Required. A list of messages representing the current conversation.
max_tokens	null	null, which means no limit, or an integer greater than zero	The maximum number of tokens to generate.
stream	true	Boolean	Stream responses back to a client in order to allow partial results for requests. If this parameter is included in the request, responses are sent using the Server-sent events standard.
temperature	1.0	Float in [0,2]	The sampling temperature. 0 is deterministic and higher values introduce more randomness.
top_p	1.0	Float in (0,1]	The probability threshold used for nucleus sampling.
top_k	null	null, which means no limit, or an integer greater than zero	Defines the number of k most likely tokens to use for top-k-filtering. Set this value to 1 to make outputs deterministic.
stop	[]	String or List[String]	Model stops generating further tokens when any one of the sequences in stop is encountered.
n	1	Integer greater than zero	The API returns n independent chat completions when n is specified. Recommended for workloads that generate multiple completions on the same input for additional inference efficiency and cost savings. Only available for provisioned throughput endpoints.
tool_choice	none	String or ToolChoiceObject	Used only in conjunction with the tools field. tool_choice supports a variety of keyword strings such as auto, required, and none. auto means that you are letting the model decide which (if any) tool is relevant to use. With auto if the model doesn't believe any of the tools in tools are relevant, the model generates a standard assistant message instead of a tool call. required means that the model picks the most relevant tool in tools and must generate a tool call. none means that the model does not generate any tool calls and instead must generate a standard assistant message. To force a tool call with a specific tool defined in tools, use a ToolChoiceObject. By default, if the tools field is populated tool_choice = "auto". Else, the tools field defaults to tool_choice = "none"
tools	null	ToolObject	A list of tools that the model can call. Currently, function is the only supported tool type and a max of 32 functions are supported.
response_format	null	ResponseFormatObject	An object specifying the format that the model must output. Accepted types are text, json_schema or json_object Setting to { "type": "json_schema", "json_schema": {...} } enables structured outputs which ensures the model follows your supplied JSON schema. Setting to { "type": "json_object" } ensures the responses the model generates is valid JSON, but does not ensure that responses follow a specific schema.
logprobs	false	Boolean	This parameter indicates whether to provide the log probability of a token being sampled.
top_logprobs	null	Integer	This parameter controls the number of most likely token candidates to return log probabilities for at each sampling step. Can be 0-20. logprobs must be true if using this field.

ChatMessage

Field	Type	Description
role	String	Required. The role of the author of the message. Can be "system", "user", "assistant" or "tool".
content	String or List[ContentItem]	Required for chat tasks that do not involve tool calls. The content can be either a string or an array that contains a series of multimodal elements in a single chat interaction. These elements follow the sequence in which they are processed as inputs or outputs by the models. This array input is specifically designed for use with proprietary models accessible only through external model providers. Currently, only Claude models are supported. Use string-typed content for other external model providers, open source models (Llama), or models hosted by customers on Databricks. list[ContentItem] is not compatible with OpenAI's specifications.
tool_calls	ToolCall list	The list of tool_calls that the model generated. Must have role as "assistant" and no specification for the content field.
tool_call_id	String	When role is "tool", the ID associated with the ToolCall that the message is responding to. Must be empty for other role options.

The system role can only be used once, as the first message in a conversation. It overrides the model's default system prompt.

ContentItem

ContentItem is one of the following content types: TextContent, ReasoningContent, DocumentContent, or ImageContent

TextContent

Field	Type	Description
type	String	Required. Must be text.
text	String	Required text content.
citation	List[Citation]	Optional citation information. See table below.

Citation fields are as follows:

Field	Type	Description
type	String	Required. Must be char_location.
cited_text	String	The text cited from the document.
document_index	Integer	The index of the cited document.
document_title	String	The title of the cited document.
start_char_index	Integer	The starting index of the cited text in the document.
end_char_index	Integer	The ending index of the cited text in the document.

ImageContent

Field	Type	Description
type	String	Required. Must be an image_url.
image_url	ImageURL	Equivalent to the OpenAI image_url object.

ImageURL fields are below:

Field	Type	Description
url	String	Either a URL of the image or the base64 encoded image data.
detail	String	Specifies the detail level of the image.

ReasoningContent

Field	Type	Description
type	String	Required. Must be an reasoning.
summary	List[Summary]	Reasoning text contents. Summary can be either TextSummary or EncryptedTextSummary

TextSummary

Field	Type	Description
type	String	Required. Must be an summary_text.
text	String	A short summary of the reasoning used by the model when generating the response.
signature	String	Optional cryptographic tokens to verify the authenticity of the data.

EncryptedTextSummary

Field	Type	Description
type	String	Required. Must be a summary_encrypted_text.
data	String	Encrypted text content which is not human-readable due to safety reasons.

DocumentContent

DocumentContent is only for requests.

Field	Type	Description
type	String	Required. Must be document.
title	String	Title for the document.
context	String	Description of the document.
source	Source	Required. Specifies more information about the document, including format and contents.
citations	Map[string, bool]	Map with single field “enabled” that maps to a bool indicating whether to enable citations for the document.

Source

Field	Type	Description
type	String	Required. Must be one of base64 (PDF), text, content, or url (URLPDFSource).
media_type	String	Required for PDF and text type. Must be application or pdf for PDF. Must be text or plain for text.
data	String	Required for PDF and text. The data containing the document source.
content	String or List[TextContent] or List[ImageContent]	Required for content type. The content of the document.
url	String	Required for URLPDFSource type. The URL of the PDF document.

ToolCall

A tool call action suggestion by the model. See Function calling on Databricks.

Field	Type	Description
id	String	Required. A unique identifier for this tool call suggestion.
type	String	Required. Only "function" is supported.
function	FunctionCallCompletion	Required. A function call suggested by the model.

FunctionCallCompletion

Field	Type	Description
name	String	Required. The name of the function the model recommended.
arguments	Object	Required. Arguments to the function as a serialized JSON dictionary.

ToolChoiceObject

See Function calling on Databricks.

Field	Type	Description
type	String	Required. The type of the tool. Currently, only "function" is supported.
function	Object	Required. An object defining which tool to call of the form {"type": "function", "function": {"name": "my_function"}} where "my_function is the name of a FunctionObject in the tools field.

ToolObject

See Function calling on Databricks.

Field	Type	Description
type	String	Required. The type of the tool. Currently, only function is supported.
function	FunctionObject	Required. The function definition associated with the tool.

FunctionObject

Field	Type	Description
name	String	Required. The name of the function to be called.
description	Object	Required. The detailed description of the function. The model uses this description to understand the relevance of the function to the prompt and generate the tool calls with higher accuracy.
parameters	Object	The parameters the function accepts, described as a valid JSON schema object. If the tool is called, then the tool call is fit to the JSON schema provided. Omitting parameters defines a function without any parameters. The number of properties is limited to 15 keys.
strict	Boolean	Whether to enable strict schema adherence when generating the function call. If set to true, the model follows the exact schema defined in the schema field. Only a subset of JSON schema is supported when strict is true

ResponseFormatObject

See Structured outputs on Databricks.

Field	Type	Description
type	String	Required. The type of response format being defined. Either text for unstructured text, json_object for unstructured JSON objects, or json_schema for JSON objects adhering to a specific schema.
json_schema	JsonSchemaObject	Required. The JSON schema to adhere to if type is set to json_schema

JsonSchemaObject

See Structured outputs on Databricks.

Field	Type	Description
name	String	Required. The name of the response format.
description	String	A description of what the response format is for, used by the model to determine how to respond in the format.
schema	Object	Required. The schema for the response format, described as a JSON schema object.
strict	Boolean	Whether to enable strict schema adherence when generating the output. If set to true, the model follows the exact schema defined in the schema field. Only a subset of JSON schema is supported when strict is true

Chat response

For non-streaming requests, the response is a single chat completion object. For streaming requests, the response is a text/event-stream where each event is a completion chunk object. The top-level structure of completion and chunk objects is almost identical: only choices has a different type.

Field	Type	Description
id	String	Unique identifier for the chat completion.
choices	List[ChatCompletionChoice] or List[ChatCompletionChunk] (streaming)	List of chat completion texts. n choices are returned if the n parameter is specified.
object	String	The object type. Equal to either "chat.completions" for non-streaming or "chat.completion.chunk" for streaming.
created	Integer	The time the chat completion was generated in seconds.
model	String	The model version used to generate the response.
usage	Usage	Token usage metadata. Might not be present on streaming responses.

ChatCompletionChoice

Field	Type	Description
index	Integer	The index of the choice in the list of generated choices.
message	ChatMessage	A chat completion message returned by the model. The role will be assistant.
finish_reason	String	The reason the model stopped generating tokens.
extra_fields	String	When using proprietary models from external model providers, the provider's APIs might include additional metadata in responses. Databricks filters these responses and returns only a subset of the provider's original fields. The safetyRating is the only extra field supported at this time, see the Gemini documentation for more details.

ChatCompletionChunk

Field	Type	Description
index	Integer	The index of the choice in the list of generated choices.
delta	ChatMessage	A chat completion message part of generated streamed responses from the model. Only the first chunk is guaranteed to have role populated.
finish_reason	String	The reason the model stopped generating tokens. Only the last chunk will have this populated.

Completion task

Text completion tasks are for generating responses to a single prompt. Unlike Chat, this task supports batched inputs: multiple independent prompts can be sent in one request. See POST /serving-endpoints/{name}/invocations for querying endpoint parameters.

Completion request

Field	Default	Type	Description
prompt		String or List[String]	Required. The prompts for the model.
max_tokens	null	null, which means no limit, or an integer greater than zero	The maximum number of tokens to generate.
stream	true	Boolean	Stream responses back to a client in order to allow partial results for requests. If this parameter is included in the request, responses are sent using the Server-sent events standard.
temperature	1.0	Float in [0,2]	The sampling temperature. 0 is deterministic and higher values introduce more randomness.
top_p	1.0	Float in (0,1]	The probability threshold used for nucleus sampling.
top_k	null	null, which means no limit, or an integer greater than zero	Defines the number of k most likely tokens to use for top-k-filtering. Set this value to 1 to make outputs deterministic.
error_behavior	"error"	"truncate" or "error"	For timeouts and context-length-exceeded errors. One of: "truncate" (return as many tokens as possible) and "error" (return an error). This parameter is only accepted by pay per token endpoints.
n	1	Integer greater than zero	The API returns n independent chat completions when n is specified. Recommended for workloads that generate multiple completions on the same input for additional inference efficiency and cost savings. Only available for provisioned throughput endpoints.
stop	[]	String or List[String]	Model stops generating further tokens when any one of the sequences in stop is encountered.
suffix	""	String	A string that is appended to the end of every completion.
echo	false	Boolean	Returns the prompt along with the completion.
use_raw_prompt	false	Boolean	If true, pass the prompt directly into the model without any transformation.

Completion response

Field	Type	Description
id	String	Unique identifier for the text completion.
choices	CompletionChoice	A list of text completions. For every prompt passed in, n choices are generated if n is specified. Default n is 1.
object	String	The object type. Equal to "text_completion"
created	Integer	The time the completion was generated in seconds.
usage	Usage	Token usage metadata.

CompletionChoice

Field	Type	Description
index	Integer	The index of the prompt in request.
text	String	The generated completion.
finish_reason	String	The reason the model stopped generating tokens.

Embedding task

Embedding tasks map input strings into embedding vectors. Many inputs can be batched together in each request. See POST /serving-endpoints/{name}/invocations for querying endpoint parameters.

Embedding request

Field	Type	Description
input	String or List[String]	Required. The input text to embed. Can be a string or a list of strings.
instruction	String	An optional instruction to pass to the embedding model.

Instructions are optional and highly model specific. For instance the The BGE authors recommend no instruction when indexing chunks and recommend using the instruction "Represent this sentence for searching relevant passages:" for retrieval queries. Other models like Instructor-XL support a wide range of instruction strings.

Embeddings response

Field	Type	Description
id	String	Unique identifier for the embedding.
object	String	The object type. Equal to "list".
model	String	The name of the embedding model used to create the embedding.
data	EmbeddingObject	The embedding object.
usage	Usage	Token usage metadata.

EmbeddingObject

Field	Type	Description
object	String	The object type. Equal to "embedding".
index	Integer	The index of the embedding in the list of embeddings generated by the model.
embedding	List[Float]	The embedding vector. Each model will return a fixed size vector (1024 for BGE-Large)

Additional resources

• Databricks Foundation Model APIs
• Use foundation models

• Supported models for Databricks Foundation Models APIs