DSPy — Program LLMs Instead of Prompting

What it is

DSPy is a Python framework that replaces manual prompt engineering with programmatic modules. Instead of writing fragile prompt strings, you define typed signatures (input/output fields), compose modules into pipelines, and let DSPy optimize the underlying prompts or fine-tuning weights automatically against a metric you specify.

DSPy targets ML engineers and AI application developers building classifiers, RAG pipelines, agent loops, and multi-step reasoning systems. It decouples your application logic from the specific prompt text, making systems portable across models and more robust to model updates.

Why it saves time or tokens

Manual prompt engineering is trial-and-error. DSPy's optimizers (formerly 'teleprompters') search over prompt variations, few-shot example selections, and even fine-tuning data to find the configuration that maximizes your metric. This automated search replaces hours of manual tweaking. Because the optimizer finds efficient prompts, the final system often uses fewer tokens per call than hand-written prompts that include unnecessary instructions.

How to use

1. Install DSPy: pip install dspy
2. Define a signature: class QA(dspy.Signature): question = dspy.InputField(); answer = dspy.OutputField()
3. Create a module, compile it with an optimizer, and run inference

Example

import dspy

lm = dspy.LM('openai/gpt-4o-mini')
dspy.configure(lm=lm)

class Summarize(dspy.Signature):
    '''Summarize the document in one paragraph.'''
    document = dspy.InputField()
    summary = dspy.OutputField()

summarizer = dspy.ChainOfThought(Summarize)
result = summarizer(document='DSPy is a framework for...')
print(result.summary)

This defines a typed summarization module with chain-of-thought reasoning. DSPy generates the actual prompt from the signature and docstring.

Related on TokRepo

• AI tools for coding — developer tools for building AI applications
• Prompt library — curated prompts and prompt engineering resources on TokRepo

Common pitfalls

• DSPy optimizers need labeled examples or a reliable metric function; without good evaluation data, optimization produces inconsistent results
• The abstraction hides prompt text, which makes debugging harder; use dspy.inspect_history() to see what prompts DSPy actually sends
• Compiling with a large optimizer on an expensive model can burn significant API credits; start with a small training set and a cheap model