DSPy (Declarative Self-Improving Python)

What is DSPy?

DSPy (Declarative Self-Improving Python) is a framework for programming, rather than prompt engineering, applications built on large language models. Created by researchers at Stanford, DSPy lets developers define the desired behavior of an AI pipeline declaratively, while the framework optimizes the underlying prompts, and optionally the model weights, against a chosen metric.

In traditional LLM development, practitioners spend hours tweaking instructions, hand-crafting few-shot examples, and adjusting prompt templates. This process is brittle. A prompt that works well with one model, say GPT-5, may perform differently when switching to another, such as Llama 4 or Claude Sonnet 4.5, or when the data distribution shifts.

DSPy treats prompts as compiled artifacts rather than hand-maintained source text. Developers write Pythonic modules, similar in spirit to PyTorch layers, that specify the inputs and outputs of their system. DSPy then uses an optimizer (also called a teleprompter) to compile these declarative programs. During compilation, the framework runs the pipeline, evaluates its outputs, and searches for instructions and few-shot examples that score well on a given metric. If you change the underlying LLM, update your data, or modify the pipeline architecture, you recompile, and DSPy searches again for prompts suited to the new configuration.

By borrowing abstractions from machine learning frameworks like PyTorch, DSPy brings more structure and reproducibility to generative AI development, letting developers build multi-step LLM applications that can be re-optimized as models and data change. The project, maintained under the stanfordnlp organization, is on its 3.x line (version 3.3.0 published to PyPI on August 3, 2026), with dspy.LM as the provider-neutral model interface and GEPA as its headline optimizer.

Core Architectural Abstractions: Signatures, Modules, and Teleprompters (Optimizers)

DSPy's architecture is built around three foundational concepts that abstract away the messy details of interacting with language models.

Signatures: The "Type System" for Prompts

A Signature in DSPy is a declarative specification of what an LLM needs to do, without specifying how to do it. It defines the inputs and outputs of a task in a clean, concise format.

Instead of writing a long prompt like, "You are an expert summarizer. Read this text and write a short, one-sentence summary," a DSPy Signature simply defines the transformation: "document -> summary". For more complex tasks, you can define custom classes with detailed input and output fields, including descriptions.

Signatures act as the interface, separating the goal of the task from the string manipulation required to achieve it. This allows the DSPy compiler to experiment with different phrasings and instructions under the hood to find the best way to prompt the model.

Modules: Building Blocks of Pipelines

Modules are the building blocks that execute Signatures. They are analogous to layers in PyTorch (nn.Module). DSPy provides built-in modules that implement different prompting techniques.

• dspy.Predict: The most basic module; it simply executes a Signature as a direct prompt.
• dspy.ChainOfThought: Automatically adds chain-of-thought reasoning steps before producing the final output, improving performance on complex tasks.
• dspy.ReAct: Implements the ReAct (Reasoning and Acting) paradigm, allowing the LLM to use tools and iterate based on observations.
• dspy.Retrieve: Interfaces with retrieval systems, including retrieval-augmented generation setups backed by a vector database, to fetch relevant context.

Modules can be composed together into custom pipelines. You define an __init__ method to declare the modules you need and a forward method to dictate how data flows between them, exactly as you would in a neural network framework.

Teleprompters (Optimizers): The Compilation Engine

Teleprompters, now referred to more broadly as optimizers, are the algorithms that compile and optimize DSPy programs. They take your pipeline (Modules + Signatures), a training dataset, and an evaluation metric, and they return an optimized version of your pipeline.

Optimizers work by tuning the internal parameters of your modules. In the context of LLMs, these "parameters" include the textual instructions, the few-shot examples, and the specific combinations of retrieved context, rather than neural network weights alone. The optimizer searches the space of possible prompts to find configurations that score well on your evaluation metric.

How Prompt Compilation Works

When you compile a DSPy program, the framework does not just format strings. It runs an automated search to maximize performance against your metric. This process generally involves three mechanisms, depending on the optimizer used.

1. Few-Shot Bootstrapping

One of DSPy's most powerful capabilities is bootstrapping few-shot examples. Instead of manually writing examples (which is tedious and often introduces human bias), DSPy generates them automatically.

During compilation, DSPy runs your pipeline on the training data. For pipelines with multiple steps (e.g., retrieving context, extracting facts, generating an answer), DSPy traces the execution. If the final output is correct (according to your metric), it saves the intermediate inputs and outputs of every module in the pipeline. These successful traces are then stored as highly effective, verified few-shot examples. This ensures that the examples used in the prompt are not only accurate but also perfectly aligned with the specific quirks of your pipeline and data.

2. Instruction Optimization and Iterative Search

Advanced optimizers like MIPROv2 (Multiprompt Instruction PRoposal Optimizer, version 2) go beyond few-shot examples and optimize the actual instructions in the prompt.

DSPy uses another LLM (often a capable model like GPT-5 or Claude Opus 4.5) as an "instructor" to propose variations of the task instructions based on the data and previous failures. The optimizer then tests these variations, along with different combinations of bootstrapped few-shot examples, using Bayesian optimization or evolutionary algorithms to find prompts that score well on your metric. This iterative search can explore combinations of instructions and examples at a scale that would be impractical by hand.

3. Reflective Prompt Evolution with GEPA

The current flagship optimizer is dspy.GEPA (Genetic-Pareto), introduced in the July 2025 paper "GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning" (Agrawal et al., arXiv:2507.19457). Where MIPROv2 searches over candidate instructions with Bayesian optimization, GEPA has the model read its own failed execution traces in natural language, reflect on what went wrong, and propose targeted prompt revisions, keeping a Pareto frontier of candidates that each excel on different examples. Because each mutation is informed by an explanation of the failure rather than blind sampling, GEPA is sample-efficient: across six tasks, the paper's abstract reports it beating the GRPO reinforcement-learning baseline by 6% on average and up to 20%, while using up to 35x fewer rollouts, and beating MIPROv2 by over 10% (including +12% accuracy on AIME-2025). GEPA also accepts textual feedback from your metric (compiler errors, rubric notes, failed assertions), not just a scalar score, which gives the optimizer visibility into why an attempt scored poorly.

4. Weight Fine-Tuning

For further performance and cost reduction, DSPy can also compile pipelines down to fine-tuned weights. Once a pipeline is optimized with prompts, DSPy can generate high-quality traces and use them to fine-tune a smaller, cheaper model, such as Llama 4 or Claude Haiku 4.5. This allows developers to prototype with large, expensive models and deploy with smaller, efficient, fine-tuned models, without changing the pipeline code.

DSPy vs. Traditional Prompt Engineering & Frameworks

Understanding DSPy requires contrasting it with existing approaches to building AI applications.

LangChain and LlamaIndex are excellent orchestration frameworks. They provide tools for connecting LLMs to APIs, databases, and memory. However, when it comes to the actual prompts, they largely rely on static templates. If a LangChain pipeline fails, a human must manually debug and tweak the strings inside the templates.

DSPy differs by treating prompts as tunable parameters. It does not replace the orchestration capabilities of other tools, and you can use DSPy within LangChain, but it replaces manual prompt tuning with algorithmic compilation. The optimizer searches for prompts that score well against your metric on the training data; it does not guarantee a mathematically optimal result, and quality still depends on how representative that data and metric are.

Example DSPy Pipeline Implementation

Here is a simplified example of how to build and compile a DSPy pipeline for a complex task: answering questions based on retrieved context, complete with reasoning steps.

import dspy
from dspy.teleprompt import BootstrapFewShot

# 1. Configure the Language Model and Retrieval Model
lm = dspy.LM('openai/gpt-5-mini')
dspy.configure(lm=lm)
# Configure any retrieval model you have set up, e.g. a hosted
# vector search endpoint or a self-hosted ColBERTv2 index.
# dspy.configure(rm=your_configured_retriever)

# 2. Define the Signature (The Type System)
class GenerateAnswer(dspy.Signature):
  """Answer questions with short factoid answers."""
  context = dspy.InputField(desc='may contain relevant facts')
  question = dspy.InputField()
  answer = dspy.OutputField(desc='often between 1 and 5 words')

# 3. Define the Module (The Pipeline Architecture)
class RAG(dspy.Module):
  def __init__(self, num_passages=3):
    super().__init__()
    # Use built-in Retrieve module; works with any retrieval model
    # configured above via dspy.configure(rm=...)
    self.retrieve = dspy.Retrieve(k=num_passages)
    # Use ChainOfThought module with our custom Signature
    self.generate_answer = dspy.ChainOfThought(GenerateAnswer)

  def forward(self, question):
    # Define how data flows
    context = self.retrieve(question).passages
    prediction = self.generate_answer(context=context, question=question)
    return dspy.Prediction(context=context, answer=prediction.answer)

# 4. Define an Evaluation Metric
def exact_match_metric(example, pred, trace=None):
  return example.answer.lower() == pred.answer.lower()

# 5. Compile the Pipeline with an Optimizer
# We use BootstrapFewShot to automatically generate few-shot examples
optimizer = BootstrapFewShot(metric=exact_match_metric, max_bootstrapped_demos=4, max_labeled_demos=4)

# Assume `trainset` is a list of dspy.Example objects
compiled_rag = optimizer.compile(RAG(), trainset=trainset)

# 6. Execute the compiled, optimized pipeline
response = compiled_rag(question='What is the capital of France?')
print(response.answer)

In this example, the developer never writes a prompt. They simply declare that they want to retrieve 3 passages and generate an answer using Chain of Thought. The BootstrapFewShot compiler handles the hard work of finding the best internal prompts and examples to make the RAG module succeed based on the exact_match_metric.

Production Benefits and Limitations of Automated Prompt Optimization

Adopting DSPy changes how engineering teams operate, bringing both benefits and new challenges.

Production Benefits

1. Model Agnosticism: DSPy offers model portability. When a new, better, or cheaper model is released, for example moving from GPT-5 to a newer Gemini or Llama release, you do not need to spend weeks rewriting prompts. You change the dspy.configure call and recompile. The framework searches for a prompting strategy suited to the new model.
2. Data-Driven Performance: Performance is tied explicitly to data and metrics rather than manual intuition. As your dataset grows and improves, recompiling your DSPy programs can systematically increase pipeline accuracy.
3. Modularity and Maintenance: Complex AI workflows are broken down into testable, composable modules. This makes large codebases easier to maintain, debug, and scale across engineering teams compared to files filled with prompt strings.
4. Automated Distillation: DSPy makes it straightforward to distill complex behaviors from larger models into smaller, cheaper models, reducing inference costs and latency in production.

Limitations and Challenges

1. Learning Curve: DSPy introduces a different paradigm. Developers accustomed to string manipulation often struggle initially with the concepts of Signatures, Modules, and compilation, since it requires thinking more like a machine learning engineer than a traditional software developer.
2. Need for High-Quality Data and Metrics: DSPy's optimizers are only as good as the evaluation metrics and training data provided. If your metric is flawed or your training set is too small or unrepresentative, the compiler will optimize for the wrong behavior, a form of overfitting.
3. Compilation Time and Cost: Optimizing a complex pipeline can be time-consuming and expensive. Running algorithms like MIPROv2 involves making many API calls to LLMs to test different prompt variations. While this saves engineering time, it requires upfront compute investment during the build phase.

Frequently Asked Questions

Do I need a large dataset to use DSPy? No. While more data allows for better optimization, DSPy can bootstrap effective few-shot examples with as few as 20 to 50 labeled examples. Even with zero labeled examples, DSPy can provide structural benefits through its declarative Signatures and Modules, though compilation will be limited.

Can DSPy be used alongside LangChain or LlamaIndex? Yes. DSPy focuses on prompt optimization and execution logic, while frameworks like LangChain excel at orchestration and integrations. You can wrap a compiled DSPy module inside a LangChain tool, or use LlamaIndex to handle document ingestion and retrieval while DSPy handles the reasoning steps.

Should I use GEPA or MIPROv2? GEPA is the better default when your metric can produce useful textual feedback (error messages, rubric explanations, test failures) or when your evaluation budget is tight, since its reflective mutations need far fewer rollouts. MIPROv2 remains a solid choice for tasks with a purely scalar metric and a budget large enough for its Bayesian search over instruction and few-shot combinations. Both compile the same DSPy programs, so switching optimizers is a one-line change.

What is the difference between DSPy and fine-tuning? DSPy primarily optimizes the inputs to the model, meaning prompts and few-shot examples, rather than updating the model's internal weights. This is generally faster, cheaper, and more accessible than traditional fine-tuning. DSPy can also be used to orchestrate fine-tuning, by generating the datasets required to train smaller models.

Why is it called DSPy? DSPy stands for Declarative Self-Improving Python. It evolved from an earlier framework called DSP (Demonstrate-Search-Predict). The "Py" was added to reflect its Pythonic, PyTorch-like API, which brought modules and programmatic compilation to the original concept.