Kong — Cloud-Native API and AI Gateway

Introduction

Kong is a high-performance API gateway used to front microservices, public APIs, and now LLM endpoints. It ships as a single NGINX-based binary extensible via Lua plugins and is commonly deployed in Kubernetes as an ingress controller.

What Kong Does

• Terminates client traffic and routes it to upstream services
• Enforces auth (JWT, OAuth2, key-auth, OIDC) before requests hit upstreams
• Applies rate limiting, caching, request/response transforms
• Collects Prometheus/OTel telemetry and access logs
• Proxies LLM providers via AI plugins (prompt guarding, token counting)

Architecture Overview

Kong runs a customized OpenResty (NGINX + LuaJIT). A control plane stores declarative config in PostgreSQL or a YAML file; data plane nodes pull/watch config and serve traffic. Plugins hook into phases (access, header_filter, log) and run per-route or globally.

Self-Hosting & Configuration

• Install via Docker image, Helm chart kong/kong, or .deb/.rpm
• Pick DB-less (YAML) for GitOps or Postgres for dynamic updates
• Admin API on port 8001 — firewall it off the public internet
• Use decK to sync declarative config from CI
• Tune worker_processes and upstream_keepalive_pool_size for throughput

Key Features

• 50+ first-party plugins including AI Proxy and AI Prompt Guard
• Kubernetes Ingress Controller with Gateway API support
• Clustering via control-plane/data-plane split
• Hybrid mode for multi-region deployments
• mTLS upstreams and automatic Let''s Encrypt via ACME plugin

Comparison with Similar Tools

• Traefik — simpler K8s ingress, fewer enterprise plugins
• Envoy — lower-level proxy, needs xDS control plane like Istio
• APISIX — also NGINX/Lua based, Apache project, etcd backed
• Tyk — Go-based gateway with built-in dashboard
• AWS API Gateway — managed only, vendor lock-in

FAQ

Q: DB-less or Postgres mode? A: DB-less for immutable GitOps deploys; Postgres if you need runtime admin API writes.

Q: Can Kong proxy OpenAI/Anthropic? A: Yes — the AI Proxy plugin normalizes requests across providers and tracks tokens.

Q: Is the Kubernetes controller separate? A: Yes — kong/kubernetes-ingress-controller translates Ingress/Gateway resources into Kong config.

Q: Is it CNCF? A: Kong Gateway is open source (Apache 2.0) but not CNCF. Kong Mesh is built on Kuma, which is a CNCF sandbox project.

Sources

• https://github.com/Kong/kong
• https://docs.konghq.com/gateway/