Scripts2026年3月31日·1 分钟阅读

LiteLLM — Unified Proxy for 100+ LLM APIs

Python SDK and proxy server to call 100+ LLM APIs in OpenAI format. Cost tracking, guardrails, load balancing, logging. Supports Bedrock, Azure, Anthropic, Vertex, and more. 42K+ stars.

TO
TokRepo精选 · Community
快速使用

先拿来用，再决定要不要深挖

这里应该同时让用户和 Agent 知道第一步该复制什么、安装什么、落到哪里。

pip install litellm

# Use as SDK
python -c "
from litellm import completion
resp = completion(model='anthropic/claude-sonnet-4-20250514', messages=[{'role':'user','content':'Hi'}])
print(resp.choices[0].message.content)
"

Or run as a proxy server:

litellm --model anthropic/claude-sonnet-4-20250514
# Now call http://localhost:4000 with OpenAI format
介绍

LiteLLM is a Python SDK and AI Gateway proxy to call 100+ LLM APIs using the OpenAI format. Write your code once, switch providers by changing one string. Includes cost tracking, rate limiting, guardrails, load balancing, fallbacks, and logging. Supports OpenAI, Anthropic, Azure, AWS Bedrock, Google Vertex, Cohere, HuggingFace, Ollama, and 90+ more. 42,000+ GitHub stars.

Best for: Teams managing multiple LLM providers with unified API, cost control, and observability Works with: OpenAI, Anthropic, Google, Azure, AWS Bedrock, Ollama, 100+ providers

Key Features

Unified API

One format for all providers — just change the model string:

# OpenAI
completion(model="gpt-4o", messages=messages)
# Anthropic
completion(model="anthropic/claude-sonnet-4-20250514", messages=messages)
# Bedrock
completion(model="bedrock/anthropic.claude-3", messages=messages)

Proxy Server (AI Gateway)

Deploy as a centralized gateway for your team:

  • Cost tracking per user, team, and API key
  • Rate limiting and budget caps
  • Load balancing across providers
  • Fallbacks — auto-retry with backup models
  • Guardrails — content filtering, PII detection

100+ Providers

OpenAI, Anthropic, Azure, AWS Bedrock, Google Vertex, Cohere, Mistral, Ollama, vLLM, Together, Replicate, HuggingFace, and many more.

Observability

Built-in logging to Langfuse, Helicone, Lunary, and custom callbacks.

FAQ

Q: What is LiteLLM? A: A Python SDK and proxy server to call 100+ LLM APIs using the OpenAI format with cost tracking, load balancing, and guardrails. 42K+ GitHub stars.

Q: How is LiteLLM different from OpenRouter? A: LiteLLM is self-hosted (you control the proxy and API keys), while OpenRouter is a managed service. LiteLLM gives you cost tracking, rate limiting, and team management.

🙏

来源与感谢

Created by BerriAI. Licensed under MIT. BerriAI/litellm — 42,000+ GitHub stars

相关资产

LiveKit Agents — Build Real-Time Voice AI Agents

Framework for building real-time voice AI agents. STT, LLM, TTS pipeline with sub-second latency. Supports OpenAI, Anthropic, Deepgram, ElevenLabs. 9.9K+ stars.

TokRepo精选

Notion MCP Server — Official Notion Integration

Official Notion MCP server. Search, read, create, and update Notion pages, databases, and blocks from Claude Code, Cursor, or any MCP client. By Notion. 4.1K+ stars.

TokRepo精选

Cline — Autonomous AI Coding Agent for VS Code

Autonomous coding agent in VS Code. Creates/edits files, runs commands, browses the web, and uses MCP tools — with human approval at every step. 60K+ stars.

TokRepo精选
◈首页🏆Trending👤我的