MLflow Prompt Optimization (beta)
This feature is currently in Beta.
MLflow's prompt optimization feature mlflow.genai.optimize_prompt() enables you to automatically improve your prompts using data-driven approaches.
This API integrates your existing prompts with advanced optimization algorithms. Currently, the API supports DSPy's MIPROv2 algorithm.
This powerful feature works seamlessly with MLflow's prompt registry, tracing, and evaluation capabilities to enhance prompt performance for your generative AI applications.
Key Benefits
-
Unified Interface
Access powerful prompt optimization methods through a standardized, framework-agnostic API. -
Prompt Management
Seamless integration with the MLflow Prompt Registry enables version control, lineage tracking, and reusability. -
Dataset Creation
Combined with MLflow tracing and evaluation features, enables efficient dataset creation from your application traces. -
Comprehensive Evaluation
Evaluate and improve prompts using Databricks's evaluation infrastructure and custom scorers.
Workflow Overview
The prompt optimization workflow consists of four main steps:
- Trace Collection: Gather execution traces from your GenAI application
- Dataset Creation: Create evaluation datasets from MLflow traces
- Data Labeling: Review and refine labels in the dataset
- Prompt Optimization: Run the optimization process
Example Notebook
The example demonstrates the end-to-end workflow for optimizing your prompts using the mlflow.genai.optimize_prompt() API, including trace collection, dataset creation, and evaluation metric definition.
Prompt Optimization notebook
Next Steps
To learn more about the API, see Optimize Prompts (Experimental).
To learn more about tracing and evaluation for GenAI applications, see the following articles: