Promptfoo — Test & Red-Team LLM Apps

What it is

Promptfoo is a CLI tool for evaluating prompts, comparing LLM model outputs side-by-side, and red-teaming AI applications for security vulnerabilities. It runs test suites against your prompts with configurable assertions, generates comparison tables across models, and scans for prompt injection, jailbreaks, and other vulnerabilities.

Promptfoo is designed for AI engineers and teams building LLM-powered applications who need systematic testing and security evaluation before production deployment.

How it saves time or tokens

Manually testing prompts across models is tedious and inconsistent. Promptfoo automates the process: define test cases once, run them across multiple models, and compare results in a structured view. The red-teaming feature automatically generates adversarial inputs to find vulnerabilities you would not think to test manually. CI/CD integration means prompt quality is validated on every code change.

How to use

1. Install Promptfoo:

npm install -g promptfoo

1. Initialize an evaluation config:

promptfoo init

1. Run evaluation and view results:

promptfoo eval
promptfoo view

1. Red-team scan for vulnerabilities:

promptfoo redteam run

Example

A promptfoo configuration for comparing models:

# promptfooconfig.yaml
prompts:
  - 'Summarize this text in 2 sentences: {{text}}'

providers:
  - openai:gpt-4
  - anthropic:claude-sonnet-4-20250514
  - ollama:llama3

tests:
  - vars:
      text: 'The quick brown fox jumps over the lazy dog. The fox was very quick.'
    assert:
      - type: contains
        value: 'fox'
      - type: llm-rubric
        value: 'The summary should be exactly 2 sentences'
      - type: max-tokens
        value: 50

Run promptfoo eval to see a side-by-side comparison table of all three models with pass/fail assertions.

Related on TokRepo

• Testing tools — Browse AI testing and evaluation tools
• Security tools — Explore AI security tools

Common pitfalls

• Writing assertions that are too strict. LLM outputs are non-deterministic. Use llm-rubric for semantic evaluation instead of exact string matching.
• Not running red-team scans before production. Prompt injection and jailbreak vulnerabilities are common in LLM applications. Run promptfoo redteam to discover them before attackers do.
• Testing only happy paths. Include edge cases, long inputs, multilingual text, and adversarial inputs in your test suite for comprehensive coverage.
• Starting with an overly complex configuration instead of defaults. Begin with the minimal setup, verify it works, then customize incrementally. This approach catches configuration errors early and keeps troubleshooting straightforward.

For teams evaluating this tool, the time saved on initial setup alone justifies the adoption. The well-documented API and active community mean most common questions have already been answered, reducing the learning curve and the number of tokens spent explaining basic usage to AI assistants.