What are compound AI systems and AI agents?

Mosaic AI Agent Framework helps developers overcome the unique challenges of developing AI agents and compound AI systems. Learn what makes an AI application a compound AI system and an AI agent.

Compound AI systems

Compound AI systems are systems that tackle AI tasks by combining multiple interacting components. In contrast, an AI model is simply a statistical model, e.g., a Transformer that predicts the next token in text. Compound AI systems are an increasingly common design pattern for AI applications due to their performance and flexibility.

For more information, see The Shift from Models to Compound AI Systems .

What are AI agents?

The industry is still defining AI agents, however it generally understood as an AI system where the model makes some or all of the planning decisions in contrast to hard-coded logic. These agents use large language models (LLMs) to make decisions and accomplish their objectives.

Many AI agents applications are made of multiple systems, thus qualifying them as compound AI systems.

Agency is a continuum, the more freedom we provide models to control the behavior of the system, the more agent-like the application becomes.

AI agentic applications are a subset of compound AI systems

What are tools?

AI agents use tools to perform actions besides language generation, for example to retrieve structured or unstructured data, run code, or talk to remote services like sending an email or Slack message.

On Databricks, you can use Unity Catalog functions as tools, enabling easy discovery, governance, and sharing of tools. You can also define tools using open source agent authoring libraries like LangChain.

In typical agentic workflows, the agent LLM is given metadata about tools, which it uses to determine when and how to use the tool. So when defining tools, you must ensure that the tool, its parameters, and its return value are well-documented, so that the agent LLM can best use the tool.

From LLMs to AI agents

To understand AI agents, it’s helpful to consider the evolution of AI systems.

  1. LLMs: Initially, large language models simply responded to prompts based on knowledge from a vast training dataset.

LLM respond to users

  1. LLMs + tool chains: Then, developers added hardcoded tools to expand the LLM’s capabilities. For example, retrieval augmented generation (RAG) expanded an LLM’s knowledge base with custom documentation sets, while API tools allowed LLMs to perform tasks like create support tickets or send emails.

predetermined toolchains

  1. AI agents: Now, AI agents autonomously create plans and execute tasks based on their understanding of the problem. AI agents still use tools but it’s up to them to decide which tool to use and when. The key distinction is in the level of autonomy and decision-making capabilities compared to compound AI systems.

AI agents rationalize a plan and execute it with tools

From a development standpoint, AI applications, whether they are individual LLMs, LLMs with toolchains, or full AI agents face similar challenges. Mosaic AI Agent Framework helps developers manage the unique challenges of building and AI applications at all levels of complexity.

Examples of AI agents

Here are some examples of AI agents across industries:

AI/BI: AI-powered chatbots and dashboards accept natural language prompts to perform analysis on a businesses’ data, drawing insights from the full lifecycle of their data. AI/BI agents parse requests, decide which data sources to, and how to communicate findings. AI/BI agents can improve over time through human feedback, offering tools to verify and refine its outputs.

Customer service: AI-powered chatbots, such as those used by customer service platforms, interact with users, understand natural language, and provide relevant responses or perform tasks. Companies use AI chatbots for customer service by answering queries, providing product information, and assisting with troubleshooting.

Manufacturing predictive maintenance: AI agents can go beyond simply predicting equipment failures, autonomously acting on them by ordering replacements, or scheduling maintenance to reduce downtime and increase productivity.