Prompt Optimizer — Improve LLM Prompts with AI Feedback

Introduction

Prompt Optimizer is an open-source web application that helps developers and prompt engineers write better prompts for large language models. Instead of manual trial-and-error, it uses AI models themselves to analyze, critique, and rewrite prompts through iterative optimization loops.

What Prompt Optimizer Does

• Takes a raw prompt and runs it through multiple optimization strategies (rewrite, expand, compress, role-play)
• Provides side-by-side comparison of original vs. optimized prompt outputs
• Supports multiple LLM backends including OpenAI, Anthropic, and local models via OpenAI-compatible APIs
• Offers batch optimization to process many prompts at once
• Exports optimized prompts in various formats for integration into your workflows

Architecture Overview

The application is built with Vue 3 and TypeScript on the frontend, with a lightweight backend proxy for API calls. The optimization pipeline sends the user's prompt to a configurable LLM with a meta-prompt that instructs the model to analyze weaknesses and produce an improved version. Multiple rounds of optimization can be chained, and each iteration's results are stored for comparison. The UI provides a clean editor with syntax highlighting and diff views.

Self-Hosting & Configuration

• Requires Node.js 18+ for local development; Docker deployment is also supported
• Set your LLM API keys via environment variables or the in-app settings panel
• Configure optimization strategies and iteration count in the settings
• Supports custom meta-prompts for specialized optimization goals
• Deploy behind a reverse proxy for team use with shared API key management

Key Features

• Iterative optimization: each round builds on the previous result for compounding improvements
• Multi-model support: optimize with one model and test with another to avoid overfitting
• Prompt version history with diff tracking to see exactly what changed
• Batch mode for optimizing prompt libraries at scale
• Fully self-hosted with no data leaving your infrastructure

Comparison with Similar Tools

• Promptfoo — focuses on testing and evaluation; Prompt Optimizer emphasizes automated rewriting
• DSPy — programmatic prompt optimization via compilation; Prompt Optimizer offers a visual, interactive approach
• LangSmith — commercial prompt playground with tracing; Prompt Optimizer is fully open-source and self-hosted
• Pezzo — prompt management platform; Prompt Optimizer specializes in automated improvement rather than versioning
• PromptPerfect — commercial SaaS optimizer; Prompt Optimizer runs entirely on your own infrastructure

FAQ

Q: Which LLM providers are supported? A: OpenAI, Anthropic, and any provider with an OpenAI-compatible API endpoint, including locally hosted models.

Q: Can I define my own optimization strategies? A: Yes. Custom meta-prompts can be configured to tailor the optimization approach to your specific use case.

Q: Does it work for non-English prompts? A: Yes. The optimization is language-agnostic since it relies on the underlying LLM's multilingual capabilities.

Q: Is there a CLI version? A: The primary interface is a web UI, but the optimization logic can be called programmatically through the exported modules.

Sources

• https://github.com/linshenkx/prompt-optimizer
• https://github.com/linshenkx/prompt-optimizer/blob/develop/README.md