RAG Studio

Preview

This feature is in Private Preview. To try it, reach out to your Databricks contact.

Important

To navigate RAG Studio documentation, return to this index page. Private preview documentation doesn’t include navigation links in the side panel.

Overview

RAG Studio provides tools and an opinionated workflow for developing, evaluating, and iterating on Retrieval-Augmented Generation (RAG) applications in order to build apps that deliver consistent, accurate answers. RAG Studio is built on top of MLflow and is tightly integrated with Databricks tools and infrastructure.

Read more about RAG Studio’s product philosophy about developing RAG Applications.

Development workflow

The RAG Studio approach to improving quality is to make it easy for developers to quickly:

  1. Adjust various knobs throughout the RAG application’s 📥 Data Ingestor, 🗃️ Data Processor, 🔍 Retriever, and 🔗 Chain to create a new Version

  2. Test the Version offline with a 📖 Evaluation Set and 🤖 LLM Judges

  3. Deploy the Version in the 💬 Review UI to collect feedback from 🧠 Expert Users

  4. Review 📈 Evaluation Results to determine if the changes had a positive impact of quality, cost, and/or latency

  5. Investigate the details in the 🗂️ Request Log and 👍 Assessment & Evaluation Results Log to identify hypotheses for how to improve quality, cost, and/or latency

  6. If needed, collect additional feedback on specific 🗂️ Request Logs from 🧠 Expert Users using the 💬 Review UI

  7. Repeat until you reach your quality/cost/latency targets!

  8. Deploy the application to production

Note

Importantly, the same development workflow above applies to production traffic! The RAG Studio data model for logs, assessments, and metrics is fully unified between development and production.

Tutorials

Tutorials demonstrate how to do the key developer workflows mentioned above, based on the fully featured sample RAG Application included with RAG Studio - a Documentation Q&A bot on the Databricks documentation.

Important

Databricks suggests getting started by going through these tutorials. Following tutorials #1 and #2 will deploy a fully functioning chat UI for the sample application. While you can do these tutorials in any order, they are designed to be done sequentially.

  1. Initialize a RAG Application

  2. Ingest or connect raw data

  3. Deploy a version of a RAG Application

  4. View logs & assessments

  5. Run offline evaluation with a 📖 Evaluation Set

  6. Collect feedback from 🧠 Expert Users

  7. Create an 📖 Evaluation Set

  8. Create versions of your RAG application to iterate on the app’s quality

    • Create a version of the 🗃️ Data Processor

    • Create a version of the 🔍 Retriever

    • Create a version of the 🔗 Chain

  9. Collect feedback on 🗂️ Request Logs from expert users

  10. Deploy a RAG application to production

Concept guides

For a deep dive of RAG Studio concepts and architecture, review these guides.

Additional reference

These documents provide additional reference material that is linked from the above guides.