Mosaic AI capabilities for GenAI

Mosaic AI provides a platform for building, evaluating, deploying, and monitoring generative AI applications (GenAI apps). It brings together a suite of tools that tackle the challenges of developing enterprise-grade GenAI apps. Mosaic AI integrates with popular open source frameworks, adding enterprise-grade governance, observability, and operational tooling, collectively known as LLMOps.

This page lists major features for GenAI, organized by GenAI workflow stages.

Query GenAI

Databricks makes state-of-the-art GenAI models from top model providers readily available through Databricks-hosted Foundation Models. You can also query models from external providers and your custom models. All of these models can be queried through UI, API/SDK, and SQL interfaces. This optionality lets you use GenAI for all use cases---general chat, complex agents, automated data pipelines, interactive data analytics, and more.

Method	Features
UI	For development, AI Playground provides a UI for querying available GenAI models and agents. For testing, your agents and apps can be queried and evaluated by domain experts using the Review App. For production, your apps can be hosted with a UI using Databricks Apps for use inside your organization. For external apps, you can power user-facing apps using Databricks APIs.
API and SDK	Model Serving provides REST API endpoints for querying models and agents. See more query options including the OpenAI client and the Databricks Python SDK.
SQL	AI functions provide task-specific and general-purpose SQL functions for querying models and agents.

Build GenAI

Databricks provides a flexible set of tools for building GenAI apps, agents, tools, and models. These include UI and code-based frameworks, all of which can optimize GenAI systems based on your data. You can leverage any open-source GenAI framework and can integrate custom tools and MCP servers.

Category	Features
Apps	Databricks Apps provide flexible app development and deployment to authenticated users. Custom apps can be powered by Databricks APIs.
Agents	AI Playground provides a UI for prototyping tool-calling agents. Custom code and open-source frameworks can be developed and deployed using Agent Framework or Databricks Apps.
Tools	Tool support includes MCP servers and Unity Catalog Functions, both of which provide Unity Catalog-based governance.
Models and prompts	Prompt engineering can be done interactively using AI Playground, or through data-driven optimization using MLflow Prompt Optimization. Fine-tune models using GPU-enabled Databricks Runtime for Machine Learning.

Prepare and serve data

Databricks simplifies data for AI by unifying governance of traditional data and AI workloads. With all data managed under Unity Catalog with fine-grained access controls, it is easy to adjust data engineering and AI boundaries to fit your organization. Data can be prepared for GenAI using any data engineering tools such as Lakeflow Spark Declarative Pipelines. A table in Unity Catalog can be served for GenAI, using a vector index for unstructured data or a feature table for structured data.

Type of data	Features
Unstructured (text, images, etc.)	Vector Search automatically indexes your knowledge base at scale for semantic or hybrid search.
Structured (tables)	Serverless SQL lets you integrate tables and SQL queries into your GenAI app for analytics or transformations. Genie agents can be used in multi-agent systems to answer natural language queries about your structured data.

Deploy and serve GenAI

Databricks provides production-ready serving systems for GenAI apps, agents, and models backed by Databricks Apps and Model Serving. These scalable deployments can be used for both real-time serving and batch inference. All deployments integrate with observability and evaluation and monitoring tooling.

The framework you use to develop your GenAI system will guide the deployment path:

Development method	Deployment and serving
Databricks Apps	Apps and agents can be deployed using Databricks Apps, which provides UI- and API-based deployment.
Custom agent code and open-source frameworks	If you develop an agent using custom code or frameworks, you can use Agent Framework to simplify deployment to Model Serving.
Fine-tuned models	Deploy fine-tuned models to Foundation Model APIs.

Trace, evaluate, and monitor GenAI

Databricks provides managed MLflow for GenAI observability, evaluation, and monitoring. Open-source APIs make integration and portability simple, while the managed service provides production-ready endpoints. Databricks-managed MLflow can be used for GenAI apps and agents hosted on Databricks and hosted elsewhere.

Category	Features
Tracing and observability	MLflow Tracing allows you to instrument your GenAI agents and apps to collect telemetry and observability data for evaluation, production monitoring, and auditing.
Evaluation	MLflow Evaluation lets you measure the quality of GenAI systems. Built-in and custom LLM judges and scorers and evaluation datasets let you tailor evaluation to your use case.
Monitoring	Production monitoring lets you measure quality on production traces, using the same judges and scorers from development-time evaluation.
Human feedback	During development, the Review App lets you run quick vibe checks using a Chat UI and collect expert feedback in labeling sessions. During production, APIs let you annotate traces with user feedback.

LLMOps

Databricks provides a full suite of tools for GenAI operations, or "LLMOps." Unified governance of data and AI assets under Unity Catalog simplifies and secures deployment of AI across an organization. AI Gateway simplifies managing models from many GenAI model providers. MLflow and Databricks Asset Bundles provide versioning and infrastructure-as-code for implementing robust LLMOps processes.

Category	Features
Data and AI asset governance	Unity Catalog provides unified governance for data and AI assets. Data assets include files, tables, vector indexes, and feature stores. AI assets include models, tools, and connections for MCP servers and other APIs.
Model endpoint governance	AI Gateway provides central governance and monitoring for GenAI model endpoints.
App, agent, and prompt versioning	MLflow supports app and agent versioning and provides a Prompt Registry, as well as experiment tracking.
Infrastructure as code	MLOps Stacks, which is built on top of Databricks Assets Bundles, provides code-based management and deployment of infrastructure and workflows.

Open source support in Mosaic AI

Mosaic AI provides full support for the rapidly growing open-source ecosystem for GenAI.

For development, you can use any open-source framework and deploy it with Agent Framework to Model Serving. Databricks services can be used by third-party GenAI tools and apps by using MCP servers or the REST API or SDKs.

For observability, evaluation, and monitoring, MLflow Tracing provides native autologging for 20+ open-source GenAI frameworks, and you can add custom tracing to any other frameworks or code. The traces can then be used with MLflow Evaluation and production monitoring. The traces follow OpenTelemetry trace specs and can be exported to third-party tools.

Learn more

• Introduction to generative AI apps on Databricks
• AI and machine learning tutorials
• Open source vs. managed MLflow on Databricks