databricks-logo

    dspy-pyfunc-simple-agent

    (Python)

    Mosaic AI Agent Framework: Author and deploy a simple DSPy agent

    This notebook demonstrates how to author a DSPy agent that's compatible with Mosaic AI Agent Framework features. You learn how to:

    • Author a DSPy agent with ChatAgent.
    • Manually test the agent's output.
    • Log and deploy the agent.

    To learn more about authoring an agent using Mosaic AI Agent Framework, see Databricks documentation (AWS | Azure).

    Prerequisites

    • Address all TODOs in this notebook.

    Limitations

    • The agent in this example does not support streaming output.
    2
    %pip install -U -qqqq mlflow-skinny[databricks] dspy databricks-agents uv matplotlib aiohttp
dbutils.library.restartPython()

    Define the agent in code

    Define the agent code in a single cell below. Then, you can write the agent code to a local Python file, using the %%writefile magic command, for subsequent logging and deployment.

    %%writefile agent.py

from typing import Any, Generator, Optional
import mlflow
from databricks.sdk import WorkspaceClient
from mlflow.entities import SpanType
from mlflow.pyfunc.model import ChatAgent
from mlflow.types.agent import (
    ChatAgentMessage,
    ChatAgentResponse,
    ChatContext,
)
import dspy
import uuid

# Autolog DSPy traces to MLflow
mlflow.dspy.autolog()

# Set up DSPy with a Databricks-hosted LLM
LLM_ENDPOINT_NAME = "databricks-meta-llama-3-3-70b-instruct"
lm = dspy.LM(model=f"databricks/{LLM_ENDPOINT_NAME}", max_tokens=250)
dspy.settings.configure(lm=lm)


class DSPyChatAgent(ChatAgent):     
    def __init__(self):
        self.agent = dspy.ChainOfThought("question,history -> answer")


    def prepare_message_history(self, messages: list[ChatAgentMessage]):
        history_entries = []
        # Assume the last message in the input is the most recent user question.
        for i in range(0, len(messages) - 1, 2):
            history_entries.append({"question": messages[i].content, "answer": messages[i + 1].content})
        return dspy.History(messages=history_entries)

    @mlflow.trace(span_type=SpanType.AGENT)
    def predict(
        self,
        messages: list[ChatAgentMessage],
        context: Optional[ChatContext] = None,
        custom_inputs: Optional[dict[str, Any]] = None,
    ) -> ChatAgentResponse:
        latest_question = messages[-1].content
        response = self.agent(question=latest_question, history=self.prepare_message_history(messages)).answer
        return ChatAgentResponse(
            messages=[ChatAgentMessage(role="assistant", content=response, id=uuid.uuid4().hex)]
        )

# Set model for logging or interactive testing
from mlflow.models import set_model
AGENT = DSPyChatAgent()
set_model(AGENT)

    Test the agent

    Interact with the agent to test its output.

    Since you manually traced methods within ChatAgent, you can view the trace for each step the agent takes, with any LLM calls made via DSPy APIs automatically traced by autologging.

    Replace this placeholder input with an appropriate domain-specific example for your agent.

    dbutils.library.restartPython()
    from agent import AGENT

AGENT.predict({"messages": [{"role": "user", "content": "What is 5+5?"}, {"role": "assistant", "content": "5+5=10"}, {"role":"user", "content": "What is the square root of that?"}]})

    Log the agent as an MLflow model

    Log the agent as code from the agent.py file. See MLflow - Models from Code.

    import mlflow
from agent import LLM_ENDPOINT_NAME
from mlflow.models.resources import DatabricksServingEndpoint
from pkg_resources import get_distribution

with mlflow.start_run():
    logged_agent_info = mlflow.pyfunc.log_model(
        name="agent",
        python_model="agent.py",
        pip_requirements=[
            f"mlflow=={get_distribution('mlflow').version}",
            f"dspy=={get_distribution('dspy').version}",
            f"databricks-sdk=={get_distribution('databricks-sdk').version}",
        ],
        resources=[DatabricksServingEndpoint(endpoint_name=LLM_ENDPOINT_NAME)],
    )

    Pre-deployment agent validation

    Before registering and deploying the agent, perform pre-deployment checks using the mlflow.models.predict() API. See Databricks documentation (AWS | Azure).

    mlflow.models.predict(
    model_uri=f"runs:/{logged_agent_info.run_id}/agent",
    input_data={"messages": [{"role": "user", "content": "Hello!"}]},
    env_manager="uv",
)

    Register the model to Unity Catalog

    Before you deploy the agent, you must register the agent to Unity Catalog.

    • TODO Update the catalog, schema, and model_name below to register the MLflow model to Unity Catalog.
    mlflow.set_registry_uri("databricks-uc")

# TODO: define the catalog, schema, and model name for your UC model.
catalog = ""
schema = ""
model_name = ""
UC_MODEL_NAME = f"{catalog}.{schema}.{model_name}"

# register the model to UC
uc_registered_model_info = mlflow.register_model(model_uri=logged_agent_info.model_uri, name=UC_MODEL_NAME)

    Deploy the agent

    from databricks import agents

agents.deploy(UC_MODEL_NAME, uc_registered_model_info.version, tags={"endpointSource": "docs"})

    Next steps

    After your agent is deployed, you can chat with it in AI playground to perform additional checks, share it with SMEs in your organization for feedback, or embed it in a production application. See Databricks documentation (AWS | Azure).

    ;