Introduction to building gen AI apps on Databricks
Mosaic AI provides a comprehensive platform for building, deploying, and managing generative AI applications. This page guides you through the essential tools and workflows for developing gen AI apps on Databricks, from serving models to deploying advanced AI agents.
Serve and query gen AI models
For simple use cases, you can directly serve gen AI models. Model Serving gives you access to a curated set of open source and third-party gen AI models from LLM providers such as OpenAI and Anthropic.
Feature | Description |
|---|---|
Serve Databricks-hosted gen AI models, including open source and fine-tuned model variants such as DBRX Instruct, Meta-Llama-3.1-70B-Instruct, GTE-Large, and Mistral-7B. | |
Serve gen AI models hosted outside of Databricks. Databricks can centrally govern external model endpoints to manage rate limits and access rights to models like OpenAI GPT-4 and Anthropic Claude. |
Build enterprise-grade AI agents
For more advanced use cases, Mosaic AI lets you prototype, build, and deploy AI agents - from simple tool-calling agents to complex Retrieval Augmented Generation (RAG) and multi-agent systems.
You can start with no-code tools or use robust, code-first frameworks that integrate with popular libraries like LangChain and LlamaIndex.
Feature | Description |
|---|---|
Prototype and test AI agents in a no-code environment. Quickly experiment with agent behaviors and tool integrations before generating code for deployment. | |
Author, deploy, and evaluate agents in Python. Supports agents written with any authoring library, including LangChain, LangGraph, and pure Python code agents. Supports Unity Catalog for governance and MLflow for tracking. | |
Build and optimize domain-specific AI agent systems with a simple, no-code interface. Focus on your data and metrics while AI Builder streamlines implementation. |
Extend AI agents using tools
AI agent tools let your agents do more than generate text. They can retrieve data, call APIs, or run custom code. Tools can be managed through Unity Catalog for governance and reuse.
Feature | Description |
|---|---|
Create agent tools that query structured and unstructured data, run code, or connect to external service APIs. |
Evaluate, debug, and optimize agents
Continuous monitoring and evaluation are essential to ensuring high-quality agents. Mosaic AI has built-in tools for tracking agent performance, collecting feedback, and driving improvements.
Task | Description |
|---|---|
Use Agent Evaluation and MLflow to measure quality, cost, and latency. Collect feedback from stakeholders and subject matter experts through built-in review apps and use LLM judges to identify and resolve quality issues. | |
Use MLflow Tracing for end-to-end observability. Log every step your agent takes, making it easy to debug, monitor, and audit agent behavior in development and production. |
Productionize AI agents
To productionize AI agents, you need observability, governance, and reliable deployment. Mosaic AI provides managed, scalable, and secure features to productionize agents.
Task | Description |
|---|---|
Log your agent’s code, configuration, and artifacts in Unity Catalog for unified governance and lifecycle management. | |
Deploy agents as managed, scalable endpoints using Mosaic AI Model Serving. Get autoscaling, low-latency inference, and integrated monitoring out of the box. | |
Lakehouse Monitoring is tightly integrated with Agent Evaluation so that you can use the same evaluation configuration (LLM judges and custom metrics) in offline evaluation and online monitoring. |