DSPy — Programming Foundation Models Declaratively

What it is

DSPy is a framework for programming (not prompting) language models. Instead of tweaking prompt strings, you define what your AI pipeline should do declaratively using signatures and modules, then DSPy optimizes the prompts automatically through compilation. It treats LLM calls as optimizable modules, similar to how PyTorch treats neural network layers.

DSPy is created at Stanford NLP and targets AI engineers building reliable LLM pipelines who want systematic optimization over manual prompt engineering.

How it saves time or tokens

Manual prompt engineering is trial and error. DSPy replaces this with automatic compilation that optimizes prompts based on evaluation metrics. The optimizer tries different prompt strategies and selects the one that scores highest on your test set. This produces better prompts in less time, and the resulting prompts are often more token-efficient because the optimizer removes unnecessary instructions.

How to use

1. Install DSPy:

pip install dspy

1. Configure a language model:

import dspy

lm = dspy.LM('openai/gpt-4o-mini')
dspy.configure(lm=lm)

1. Define a signature and module:

qa = dspy.ChainOfThought('question -> answer')
result = qa(question='What is the capital of France?')
print(result.answer)

Example

Building a fact-checking pipeline with compilation:

import dspy

class FactCheck(dspy.Signature):
    claim = dspy.InputField(desc='A factual claim to verify')
    evidence = dspy.OutputField(desc='Supporting or refuting evidence')
    verdict = dspy.OutputField(desc='True, False, or Uncertain')

class FactChecker(dspy.Module):
    def __init__(self):
        self.check = dspy.ChainOfThought(FactCheck)

    def forward(self, claim):
        return self.check(claim=claim)

# Compile with training examples
from dspy.teleprompt import BootstrapFewShot

optimizer = BootstrapFewShot(metric=your_metric)
compiled_checker = optimizer.compile(FactChecker(), trainset=train_data)

# Use the optimized pipeline
result = compiled_checker(claim='The Earth is flat')
print(result.verdict)

Related on TokRepo

• Prompt library — curated prompts and prompt patterns
• AI tools for research — research and analysis tools

Common pitfalls

• Compilation requires a training set with labeled examples; without good examples, the optimizer cannot improve over the default prompt
• DSPy adds abstraction overhead; for simple one-shot tasks, direct prompting is simpler and equally effective
• The optimizer runs multiple LLM calls during compilation, which can be expensive; start with a small training set and a cheaper model