LiteLLM — Proxy LLM open source para más de 100 proveedores

Quick Start — SDK or Proxy

El SDK es el camino más rápido al soporte multi-proveedor — sin servidor que correr. El Proxy es un pequeño servidor FastAPI que expone endpoints compatibles con OpenAI; apunta cualquier SDK de OpenAI hacia él. El enrutamiento dirigido por configuración significa que cambias de proveedor o de estrategia de load-balancing sin tocar el código de la app.

# Option A: SDK only (no server needed)
# pip install litellm
from litellm import completion

resp = completion(
    model="anthropic/claude-3-5-sonnet-20241022",
    messages=[{"role": "user", "content": "Hello from LiteLLM"}],
)
print(resp.choices[0].message.content)

# Option B: Run the Proxy for team use
# pip install 'litellm[proxy]'
# litellm --config config.yaml --port 4000
#
# config.yaml:
# model_list:
#   - model_name: fast
#     litellm_params:
#       model: gpt-4o-mini
#       api_key: os.environ/OPENAI_KEY
#   - model_name: fast
#     litellm_params:
#       model: claude-3-5-haiku-20241022
#       api_key: os.environ/ANTHROPIC_KEY
# router_settings:
#   routing_strategy: usage-based-routing-v2

# Now call the proxy as if it were OpenAI
from openai import OpenAI
client = OpenAI(base_url="http://localhost:4000", api_key="sk-proxy-token")
r = client.chat.completions.create(model="fast", messages=[{"role":"user","content":"hi"}])
# Proxy load-balances between gpt-4o-mini and claude-3-5-haiku based on usage.

Características clave

Comparación

Casos de uso

Precios y licencia

Activos relacionados en TokRepo

LiteLLM Cost Tracking — Per-Project LLM Spend Dashboard

LiteLLM ships a built-in cost dashboard. Track LLM spend by project, user, model, tag. Hard budgets that block at the proxy. SOC2 / SSO via Pro tier.

LiteLLM Proxy — Unified Gateway for 100+ LLM APIs

LiteLLM Proxy maps 100+ LLM providers (Anthropic, OpenAI, Bedrock, Vertex) to one OpenAI-compatible endpoint. Auth, rate limit, cost track, fallbacks.

LiteLLM Router — Smart Failover & Load Balancing in Python

LiteLLM Router routes LLM endpoints with retry, fallback, latency-based, weighted A/B. Pure Python — drop into any codebase, no separate proxy needed.

LiteLLM — Unified Proxy for 100+ LLM APIs

Python SDK and proxy server to call 100+ LLM APIs in OpenAI format. Cost tracking, guardrails, load balancing, logging. Supports Bedrock, Azure, Anthropic, Vertex, and more. 42K+ stars.

Preguntas frecuentes