LLM Gateway Comparison — LiteLLM vs OpenRouter vs CF

What it is

This guide compares three LLM API gateways: LiteLLM (a self-hosted proxy that unifies 100+ LLM providers behind an OpenAI-compatible API), OpenRouter (a managed unified API with usage-based pricing), and Cloudflare AI Gateway (an edge-layer cache and rate limiter for LLM calls).

The comparison targets engineering teams deciding how to route LLM traffic across multiple providers with fallback, load balancing, and cost visibility.

How it saves time or tokens

Choosing the right gateway before building saves weeks of refactoring. LiteLLM gives you full control and zero markup but requires self-hosting. OpenRouter gives you instant access to many models with no infrastructure but adds a small fee per token. Cloudflare AI Gateway sits in front of any provider and caches repeated requests, reducing token spend.

This comparison condenses the decision into architecture, pricing, and use-case fit.

How to use

1. Assess your requirements: self-hosted vs managed, number of providers, caching needs
2. If self-hosted with full control: pick LiteLLM and deploy as a Docker container
3. If managed with broad model access: pick OpenRouter and use their unified API key
4. If you need edge caching on top of existing provider keys: add Cloudflare AI Gateway

Example

# LiteLLM: unified call across providers
from litellm import completion

# Same interface, different providers
response = completion(
    model='gpt-4o',
    messages=[{'role': 'user', 'content': 'Hello'}]
)

# Switch to Anthropic with one line change
response = completion(
    model='claude-sonnet-4-20250514',
    messages=[{'role': 'user', 'content': 'Hello'}]
)

# Fallback chain
response = completion(
    model='gpt-4o',
    fallbacks=['claude-sonnet-4-20250514', 'gemini/gemini-pro'],
    messages=[{'role': 'user', 'content': 'Hello'}]
)

Related on TokRepo

• AI gateway providers -- Compare all gateway providers on TokRepo
• LiteLLM deep-dive -- Detailed LiteLLM page

Common pitfalls

• LiteLLM requires managing your own provider API keys and infrastructure; do not assume it handles billing
• OpenRouter adds a markup on top of provider pricing; for high-volume use, self-hosted LiteLLM is cheaper
• Cloudflare AI Gateway only caches identical requests; slight prompt variations produce cache misses