LiteLLM — Unified Proxy for 100+ LLM APIs

Key Features

Unified API

One format for all providers — just change the model string:

# OpenAI
completion(model="gpt-4o", messages=messages)
# Anthropic
completion(model="anthropic/claude-sonnet-4-20250514", messages=messages)
# Bedrock
completion(model="bedrock/anthropic.claude-3", messages=messages)

Proxy Server (AI Gateway)

Deploy as a centralized gateway for your team:

• Cost tracking per user, team, and API key
• Rate limiting and budget caps
• Load balancing across providers
• Fallbacks — auto-retry with backup models
• Guardrails — content filtering, PII detection

100+ Providers

OpenAI, Anthropic, Azure, AWS Bedrock, Google Vertex, Cohere, Mistral, Ollama, vLLM, Together, Replicate, HuggingFace, and many more.

Observability

Built-in logging to Langfuse, Helicone, Lunary, and custom callbacks.

FAQ

Q: What is LiteLLM? A: A Python SDK and proxy server to call 100+ LLM APIs using the OpenAI format with cost tracking, load balancing, and guardrails. 42K+ GitHub stars.

Q: How is LiteLLM different from OpenRouter? A: LiteLLM is self-hosted (you control the proxy and API keys), while OpenRouter is a managed service. LiteLLM gives you cost tracking, rate limiting, and team management.