Agent observability recipes

End-to-end recipes for common agent observability and quality tasks — connect your coding agent, then evaluate and improve it.

Connect your coding agent

Give your coding agent knowledge of MLflow's APIs and live access to your traces, so it writes accurate instrumentation and answers questions about your agent from your IDE.

• • Set up a coding agent
  • Install the MLflow skills so Claude Code, Cursor, VS Code, or OpenCode generate accurate tracing and evaluation code.
• • Set up the MLflow MCP server
  • Configure the MLflow MCP server so your coding agent can reach your traces and experiments.
• • Query traces from a coding agent
  • Use the MLflow MCP server to search, inspect, and annotate traces without leaving your IDE.

Evaluate and improve

Score your agent against your own criteria, from a quick end-to-end walkthrough to purpose-built LLM judges.

• • Evaluate and improve an agent
  • Run mlflow.genai.evaluate() on an evaluation dataset to score an agent end to end and compare versions.
• • Create a guidelines LLM judge
  • Turn pass/fail natural-language criteria into a scorer — no evaluation code required.
• • Create a custom judge
  • Build a purpose-built judge with make_judge(), including trace-based judges that analyze execution.