ScriptsApr 1, 2026·2 min read

Promptfoo — Test & Red-Team LLM Apps

Promptfoo is a CLI for evaluating prompts, comparing models, and red-teaming AI apps. 18.9K+ GitHub stars. Side-by-side comparison, vulnerability scanning, CI/CD. MIT.

TO
TokRepo精选 · Community
Quick Use

Use it first, then decide how deep to go

This block should tell both the user and the agent what to copy, install, and apply first.

# Install
npm install -g promptfoo
# or: brew install promptfoo / pip install promptfoo

# Initialize evaluation config
promptfoo init

# Run evaluation
promptfoo eval

# View results in browser
promptfoo view

# Red-team scan for vulnerabilities
promptfoo redteam init
promptfoo redteam run
Intro

Promptfoo is a CLI and library for evaluating and red-teaming LLM applications. With 18,900+ GitHub stars and MIT license, it enables testing prompts across multiple LLM providers, side-by-side model comparison (OpenAI, Claude, Gemini, Llama), automated red teaming and vulnerability scanning, CI/CD integration for continuous testing, and privacy-first local execution where prompts never leave your machine. Results are viewable in a web-based UI or command line.

Best for: Teams shipping AI products who need systematic prompt testing and security scanning Works with: Claude Code, OpenAI Codex, Cursor, Gemini CLI, Windsurf Providers: OpenAI, Claude, Gemini, Llama, Mistral, and 50+ more

Key Features

  • Prompt evaluation: Test prompts against assertions and expected outputs
  • Model comparison: Side-by-side across 50+ LLM providers
  • Red teaming: Automated vulnerability scanning for AI apps
  • CI/CD integration: Run evaluations in your deployment pipeline
  • Privacy-first: All prompts stay on your machine
  • Web UI: Visual results viewer and comparison dashboard
  • Code scanning: Detect prompt injection and security issues

FAQ

Q: What is Promptfoo? A: Promptfoo is an LLM evaluation and red-teaming CLI with 18.9K+ stars. Test prompts, compare models, scan for vulnerabilities. Local execution, CI/CD ready. MIT.

Q: How do I install Promptfoo? A: npm install -g promptfoo or brew install promptfoo. Then promptfoo init and promptfoo eval.

🙏

Source & Thanks

Created by Promptfoo. Licensed under MIT. promptfoo/promptfoo — 18,900+ GitHub stars

Related Assets

Void — Open-Source Cursor Alternative

Void is an open-source AI code editor alternative to Cursor. 28.5K+ stars. Checkpoints, custom models, local hosting, built on VS Code. Apache 2.0.

TokRepo精选

AI Shell — Natural Language to Shell Commands

AI Shell converts natural language into shell commands using AI. 5.2K+ stars. 13+ languages, chat mode, custom endpoints. MIT. By Builder.io.

TokRepo精选

LLM — CLI Tool for 100+ Language Models

LLM is a CLI and Python library for accessing 100+ LLMs via APIs or locally. 11.5K+ stars. SQLite logging, embeddings, structured data. Apache 2.0.

TokRepo精选
◈Home🏆Trending👤Me