# LLM Gateway Comparison — Proxy Your AI Requests

> Compare top LLM gateway and proxy tools for routing AI requests. Covers LiteLLM, Bifrost, Portkey, and OpenRouter for cost optimization, failover, and multi-provider access.

## Install

Save the content below to `.claude/skills/` or append to your `CLAUDE.md`:

## Quick Use

### LiteLLM (Most Popular)
```bash
pip install litellm
litellm --model gpt-4o --port 4000
```

### Bifrost (Fastest)
```bash
npx -y @maximhq/bifrost
```

### Portkey (Enterprise)
```bash
pip install portkey-ai
```

## What are LLM Gateways?

LLM gateways are proxy servers that sit between your application and LLM providers. They provide a unified API, automatic failover, load balancing, caching, cost tracking, and access to 100+ models through one endpoint. Essential for production AI applications that need reliability and cost control.

**Answer-Ready**: LLM gateways proxy AI requests through a unified API. Top tools: LiteLLM (open-source, 200+ models), Bifrost (fastest, sub-100us), Portkey (enterprise), OpenRouter (pay-per-use marketplace). Provide failover, caching, cost tracking, and multi-provider routing.

**Best for**: Teams running AI in production needing reliability and cost control. **Works with**: Any LLM provider and AI coding tool.

## Gateway Comparison

### Feature Matrix

| Feature | LiteLLM | Bifrost | Portkey | OpenRouter |
|---------|---------|---------|---------|------------|
| Type | Open-source | Open-source | Enterprise | Marketplace |
| Models | 200+ | 1000+ | 250+ | 300+ |
| Overhead | ~1ms | <100us | ~2ms | ~50ms |
| Failover | Yes | Yes | Yes | Yes |
| Caching | Yes | Semantic | Yes | No |
| Cost tracking | Yes | Yes | Yes | Built-in |
| Load balancing | Yes | Yes | Yes | Automatic |
| Self-hosted | Yes | Yes | Yes | No |
| Free tier | Unlimited (OSS) | Unlimited (OSS) | 10K req/mo | Pay-per-use |

### When to Use Each

| Gateway | Best For |
|---------|---------|
| LiteLLM | Teams wanting open-source flexibility with most provider support |
| Bifrost | High-throughput apps needing minimal latency overhead |
| Portkey | Enterprise teams needing compliance, guardrails, and analytics |
| OpenRouter | Indie developers wanting simple pay-per-use access to all models |

## Setup Examples

### LiteLLM — Universal Proxy

```python
from litellm import completion

# Same API for any provider
response = completion(
    model="anthropic/claude-sonnet-4-20250514",
    messages=[{"role": "user", "content": "Hello"}],
)
# Switch to: openai/gpt-4o, gemini/gemini-2.5-pro, etc.
```

### Bifrost — Claude Code Integration

```bash
npx -y @maximhq/bifrost
claude mcp add --transport http bifrost http://localhost:8080/mcp
```

### Portkey — With Guardrails

```python
from portkey_ai import Portkey

client = Portkey(api_key="...", config={
    "retry": {"attempts": 3},
    "cache": {"mode": "semantic"},
    "guardrails": ["pii-filter", "toxicity-check"],
})
```

### OpenRouter — Simple Access

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="sk-or-...",
)
response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4-20250514",
    messages=[{"role": "user", "content": "Hello"}],
)
```

## Cost Optimization Strategies

| Strategy | How |
|----------|-----|
| Model routing | Simple tasks → cheap model, complex → premium |
| Caching | Cache identical/similar requests |
| Fallback chain | Primary fails → cheaper backup |
| Budget limits | Hard caps per project/user |
| Token tracking | Monitor and optimize token usage |

## FAQ

**Q: Do I need a gateway?**
A: For production, yes. For prototyping, direct API calls are fine. Gateways add reliability, cost control, and flexibility.

**Q: Can I use a gateway with Claude Code?**
A: Yes, LiteLLM and Bifrost both support Claude Code integration via MCP or API proxy.

**Q: Which is cheapest?**
A: LiteLLM and Bifrost are free (open-source, self-hosted). OpenRouter charges a small markup on model pricing.

## Source & Thanks

> - [LiteLLM](https://github.com/BerriAI/litellm) — 18k+ stars
> - [Bifrost](https://github.com/maximhq/bifrost) — 3.6k+ stars
> - [Portkey](https://portkey.ai) — Enterprise AI gateway
> - [OpenRouter](https://openrouter.ai) — Model marketplace

<!-- ZH -->

## Quick Use

An LLM gateway proxies AI requests, providing a unified API, failover, and cost controls.

## What is an LLM Gateway?

An LLM gateway sits between your app and LLM providers, delivering a unified API + failover + caching + cost tracking.

**TL;DR**: LLM request proxy comparison. LiteLLM (open source, 200+ models), Bifrost (fastest, <100µs), Portkey (enterprise-grade), OpenRouter (pay-per-use).

## Comparison

### LiteLLM — Open source and flexible; 200+ models
### Bifrost — Lowest latency; Claude Code integration
### Portkey — Enterprise compliance and guardrails
### OpenRouter — Simple pay-per-use

## Cost Optimization

Model routing (simple → cheap, complex → premium) + caching + fallback chains + budget limits.

## FAQ

**Q: Do I need a gateway?**
A: Yes in production; call APIs directly during prototyping.

## Source & Thanks

> [LiteLLM](https://github.com/BerriAI/litellm) | [Bifrost](https://github.com/maximhq/bifrost) | [Portkey](https://portkey.ai) | [OpenRouter](https://openrouter.ai)

---
Source: https://tokrepo.com/en/workflows/llm-gateway-comparison-proxy-your-ai-requests-88ca6b84
Author: Agent Toolkit