Deploy generative AI agents and models
This article describes Mosaic AI Model Serving support for deploying generative AI agents and models for your generative AI applications.
What are generative AI agents?
Generative AI agents are compound AI systems that depend on large language models and user input to determine which steps to take to perform a task. See Create and log AI agents.
What are generative AI models?
Generative AI models create new content from inputs like text, images, and code. These models are trained on large datasets and use deep learning to identify patterns and structures in existing data, and then generate new content based on what they’ve learned.
Foundation models are a type of generative AI model. These models are pre-trained with the intention that they are to be fine-tuned for more specific language understanding and generation tasks.
Deploy a generative AI agent
Databricks supports deploying a generative AI agent using the deploy()
method in the Mosaic AI Agent Framework. This method automatically creates:
A CPU endpoint for deployment and testing using Model Serving.
A URL to the Agent Evaluation review app where stakeholders can interact with the agent and record feedback.
See Deploy an agent for generative AI application for additional detail about deploying agents.
Deploy a generative AI model
Mosaic AI Model Serving supports serving and querying generative AI models using the following capabilities:
Foundation Model APIs. This functionality makes state-of-the-art open models and fine-tuned model variants available to your model serving endpoint. These models are curated foundation model architectures that support optimized inference. Base models, like DBRX Instruct, Llama-2-70B-chat, BGE-Large, and Mistral-7B are available for immediate use with pay-per-token pricing, and workloads that require performance guarantees, like fine-tuned model variants, can be deployed with provisioned throughput.
External models. These are generative AI models that are hosted outside of Databricks. Endpoints that serve external models can be centrally governed and customers can establish rate limits and access control for them. Examples include foundation models like OpenAI’s GPT-4, Anthropic’s Claude, and others.
For a getting started tutorial on how to query a foundation model on Databricks, see Get started querying LLMs on Databricks.