dspy-create-rag-program(Python)

Import Notebook

Part 2: Create and optimize a DSPy program for RAG

This notebook shows how to:

  • Create a basic RAG DSPy program.
  • Run the DSPy program from a notebook.
  • Optimize prompts using DSPy BootstrapFewShot optimizer.
  • Run the optimized DSPy program.

This notebook is part 2 of 2 notebooks for creating a DSPy program for RAG.

Requirements

This notebook assumes:

  • You have completed and run the Part 1: Prepare data and vector search index for a RAG DSPy program notebook.
  • You have specified the following information in the notebook widgets:
    • vs_index: Databricks Vector Search index to be used in the RAG program.
    • source_catalog: UC catalog of the schema where the index is located.
    • source_schema: UC schema containing the Vector Search index.

Install dependencies

3

Define notebook widgets

5

Define configurations

The following example shows how to obtain a personal access token from the session using the specified notebook widget values. However, this method is not recommended for production; instead use a Databricks secret (AWS| Azure)

7

Define the DSPy program

A DSPy program consists of a Python class inherited from dspy.Module that implements the forward() method, which runs the following steps:

  • Query a Databricks Vector Search index to retrieve document chunks (context) related to the request.
  • Generate an response by sending the context containing the document chunks and the request to an LLM.

The __init__ function initializes the resources the forward function uses. In this example, the resources are:

  • retrieve: Databricks Vector Search retriever
  • lm: Databricks Foundation Model pay-per-token Llama3-1-70B-instruct
  • response_generator: The prediction technique, in this case DSPy.predict, that uses an LLM to process retrieved documents and instructions to generate a response. Additional prediction techniques include dspy.ChainOfThought and dspy.ReAct.
9

Run the program

To run the DSPy program, instantiate it and pass in the request.

11

Not bad for such a simple program!!

Try another query:

13

This response is unxpected, since the program should have responded with Yes. When this happens, you can inspect the prompt generated by DSPy.

Inspecting generated prompt

16

    You can see it is a simple prompt with minimal instructions. Try optimizing it by providing few-shot examples. DSPy selects which few-shot examples are most effective based on an evaluation criteria.

    Optimizing prompts

    Define training set

    First, define eight examples of request and expected_response pairs.

    20

    Define a prompt optimization evaluation function

    The following defines and implements a function to evaluate if the responses from the program are correct. Mosaic Agent Evaluation (AWS | Azure) is an ideal tool for this purpose.

    22

    Run optimization

    Now, the final step is to run the optimization. DSPy offers several optimizers, this example uses the BootstrapFewShot optimizer. The BootstrapFewShot optimizer selects the best few-shot examples for all the stages of the DSPy program, but in this notebook you only use one stage. The examples are obtained from the training set labels (expected_response) and the evaluation executions. For more information about this and other optimizers, see the DSPy documentaion.

    24

    Run the optimized DSPy module

    Try the tricky question again:

    26

    Inspect the prompt used by the optimized program

    When inspecting the prompt generated from the optimized program, the few-shot examples are added by DSPy:

    28