Guide AI Gateway — contrôle des coûts LLM, fallback routing et observabilité (2026)

Cloudflare AI Gateway — Edge Proxy for LLM Traffic

Cloudflare AI Gateway is a free edge proxy that sits between your app and LLM providers — caching responses, rate-limiting abuse, failover across models, and emitting analytics without changing your SDK code.

Portkey — AI Gateway with Prompt Management & Observability

Portkey is an end-to-end LLM control plane: gateway for routing and fallback, prompt manager for versioning, and an observability suite with cost tracking and guardrails — all behind a single API.

LiteLLM — Open-source LLM Proxy for 100+ Providers

LiteLLM is an open-source proxy that normalizes 100+ LLM APIs behind the OpenAI SDK. Drop it in front of Claude, Gemini, Ollama, Bedrock, Vertex, Azure — one client, unified calls.

OpenRouter — Unified API for 300+ Models, One Invoice

OpenRouter is a managed router that exposes 300+ LLMs (OpenAI, Claude, Gemini, open-source via Groq/Together/Fireworks) behind a single OpenAI-compatible API and one consolidated bill.

Helicone — Zero-Code LLM Observability Platform

Helicone is an open-source observability platform that gives you LLM request logs, cost tracking, user analytics, and prompt experiments — by changing only the base URL of your OpenAI or Anthropic client.

Langfuse — Open-source LLM Engineering Platform

Langfuse is the dominant open-source platform for LLM traces, prompts, evaluations, and datasets. Instrument your agent with the SDK or OpenTelemetry and get production-grade debugging and eval.

Kong AI Gateway — Enterprise-grade LLM Proxy

Kong AI Gateway adds LLM-specific plugins (prompt transforms, semantic caching, cost limits, guardrails) to the Kong API gateway — ideal for teams already running Kong who want AI controls on the same plane.

Arize Phoenix — Open-source LLM Observability & Evals

Arize Phoenix is the open-source observability and evaluation library from Arize AI. OpenTelemetry-native, with strong eval primitives — built for data scientists and ML engineers who want notebooks + production in one stack.

Traceloop — OpenTelemetry-first LLM Observability

Traceloop ships OpenLLMetry, the popular OSS library for instrumenting LLM apps with OpenTelemetry. Backend-agnostic traces: send to Traceloop Cloud, Grafana, Datadog, or your existing OTEL stack.