# Portkey AI Gateway — Unified API for 200+ LLMs

> Route, load-balance, and fallback across 200+ LLMs with a single API. Built-in caching, guardrails, observability, and budget controls for production AI apps.

## Install

Save as a script file and run:

## Quick Use

```bash
pip install portkey-ai
```

```python
from portkey_ai import Portkey

client = Portkey(
    api_key="YOUR_PORTKEY_KEY",
    virtual_key="openai-xxx",  # Your OpenAI key stored in Portkey
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)
```

## What is Portkey?

Portkey is an AI gateway that sits between your app and LLM providers. It provides a unified OpenAI-compatible API to route requests across 200+ models with automatic fallbacks, load balancing, caching, and cost tracking.

**Answer-Ready**: Portkey AI Gateway is a unified API layer for 200+ LLMs that provides routing, fallbacks, caching, guardrails, and cost tracking for production AI applications.

## Core Features

### 1. Automatic Fallbacks

```python
from portkey_ai import Portkey, createHeaders

client = Portkey(
    api_key="YOUR_KEY",
    config={
        "strategy": {"mode": "fallback"},
        "targets": [
            {"virtual_key": "openai-key", "override_params": {"model": "gpt-4o"}},
            {"virtual_key": "anthropic-key", "override_params": {"model": "claude-sonnet-4-20250514"}},
        ],
    },
)
```

### 2. Load Balancing

```python
config = {
    "strategy": {"mode": "loadbalance"},
    "targets": [
        {"virtual_key": "openai-1", "weight": 0.7},
        {"virtual_key": "openai-2", "weight": 0.3},
    ],
}
```

### 3. Semantic Caching
Cache similar requests to reduce costs and latency:

```python
config = {
    "cache": {"mode": "semantic", "max_age": 3600}
}
```

### 4. Guardrails
Add input/output checks:

```python
config = {
    "input_guardrails": ["pii-detection", "prompt-injection"],
    "output_guardrails": ["toxicity-check"],
}
```

### 5. Cost & Usage Tracking
Real-time dashboard showing spend per model, per user, per feature.

## Supported Providers

OpenAI, Anthropic, Google Gemini, Mistral, Cohere, Azure OpenAI, AWS Bedrock, Groq, Together AI, Fireworks, Ollama, and 190+ more.

## FAQ

**Q: Is it open source?**
A: Yes, the gateway is open source. Managed cloud version available.

**Q: Latency overhead?**
A: < 5ms for direct routing. Semantic caching adds ~10ms but saves full LLM call.

**Q: OpenAI SDK compatible?**
A: Yes, drop-in replacement — change base URL and add Portkey headers.

## Source & Thanks

- GitHub: [Portkey-AI/gateway](https://github.com/Portkey-AI/gateway) (7k+ stars)
- Docs: [docs.portkey.ai](https://docs.portkey.ai)

<!-- ZH -->

## 快速使用

```bash
pip install portkey-ai
```

一行代码切换 LLM 供应商，内置故障转移和负载均衡。

## 什么是 Portkey？

Portkey 是 AI 网关，提供统一的 OpenAI 兼容 API，跨 200+ 模型路由请求，支持自动故障转移、负载均衡、缓存和成本追踪。

**一句话总结**：Portkey AI Gateway 为 200+ LLM 提供统一 API 层，包含路由、故障转移、缓存、护栏和成本追踪。

## 核心功能

### 1. 自动故障转移
主模型失败自动切换到备用模型。

### 2. 负载均衡
按权重分配请求到多个 API Key 或模型。

### 3. 语义缓存
相似请求复用缓存，降低成本和延迟。

### 4. 护栏
输入输出检查：PII 检测、注入防护、毒性过滤。

### 5. 成本追踪
实时仪表盘显示按模型、用户、功能的花费。

## 常见问题

**Q: 开源吗？**
A: 网关开源，也有托管云版本。

**Q: 延迟开销？**
A: 直接路由 < 5ms，语义缓存约 10ms 但省去完整 LLM 调用。

## 来源与致谢

- GitHub: [Portkey-AI/gateway](https://github.com/Portkey-AI/gateway) (7k+ stars)

---
Source: https://tokrepo.com/en/workflows/978ebe61-0db8-4591-95f4-f9af0ee37888
Author: Script Depot