Foundation model serving endpoints (legacy)
This page describes the legacy endpoint-based Model Serving workflow. To find and query models through Unity Gateway, see available models and query model services.
Access foundation models hosted outside of Databricks
Foundation models created by LLM providers, such as OpenAI and Anthropic, are also accessible on Databricks using External models. These models are hosted outside of Databricks and you can create an endpoint to query them. These endpoints can be centrally governed from Databricks, which streamlines the use and management of various LLM providers within your organization.
The following table presents a non-exhaustive list of supported models and corresponding endpoint types. You can use the listed model associations to help you configure an endpoint for any newly released model types as they become available with a given provider. Customers are responsible for ensuring compliance with applicable model licenses.
With the rapid development of LLMs, there is no guarantee that this list is up to date at all times. New model versions from the same provider are typically supported even if they are not on the list.
Model provider | llm/v1/completions | llm/v1/chat | llm/v1/embeddings |
|---|---|---|---|
OpenAI** |
|
|
|
Azure OpenAI** |
|
|
|
Anthropic |
|
| |
Cohere** |
|
|
|
Model Serving | Databricks serving endpoint | Databricks serving endpoint | Databricks serving endpoint |
Amazon Bedrock | Anthropic:
Cohere:
AI21 Labs:
| Anthropic:
Cohere:
Amazon:
| Amazon:
Cohere:
|
AI21 Labs |
| ||
Google Cloud Vertex AI | text-bison |
|
|
** Model provider supports fine-tuned completion and chat models. To query a fine-tuned model, populate the name field of the external model configuration with the name of your fine-tuned model.
† Model provider supports custom completion models.
Create foundation model serving endpoints
To query and use foundation models in your AI applications, you must first create a model serving endpoint. Model Serving uses a unified API and UI for creating and updating foundation model serving endpoints.
- To create an endpoint that serves fine-tuned variants of foundation models made available using Foundation Model APIs provisioned throughput, see Create your provisioned throughput endpoint using the REST API.
- For creating serving endpoints that access foundation models made available using the External models offering, see Create an external model serving endpoint.
Query foundation model serving endpoints
After you create your serving endpoint you are able to query your foundation model. Model Serving uses a unified OpenAI-compatible API and SDK for querying foundation models. This unified experience simplifies how you experiment with and customize foundation models for production across supported clouds and providers.