Higress — AI-Native API Gateway Built on Envoy

Introduction

Higress is a cloud-native API gateway built on Envoy and Istio that provides AI-native capabilities alongside traditional gateway features. Originally developed at Alibaba to handle large-scale production traffic, it was open-sourced to serve as a unified gateway for microservices, AI model routing, and API management.

What Higress Does

• Routes HTTP, gRPC, and WebSocket traffic with advanced matching rules
• Proxies requests to LLM providers with load balancing and fallback
• Enforces rate limiting, authentication, and WAF protection at the edge
• Manages API versions and provides a built-in console UI
• Integrates with Kubernetes Ingress and Gateway API standards

Architecture Overview

Higress extends Envoy Proxy with a control plane built on Istio Pilot for configuration management. It uses a plugin system based on Wasm (WebAssembly), allowing developers to write custom logic in Go, Rust, or JS that runs inside the data plane without recompiling Envoy. The console provides a web UI for route and plugin management backed by a Nacos-compatible config store.

Self-Hosting & Configuration

• Deploy via Helm on any Kubernetes cluster with a single command
• Standalone Docker mode available for non-K8s environments
• Configure routes, services, and plugins through the built-in web console
• Supports Nacos, Consul, and Kubernetes for service discovery
• Wasm plugins can be loaded from OCI registries or local files

Key Features

• AI gateway capabilities with LLM proxy, token-based rate limiting, and prompt caching
• Wasm-based plugin system supporting Go, Rust, and JavaScript
• Full Kubernetes Gateway API and Ingress compatibility
• Built-in observability with Prometheus metrics and access logging
• Canary releases and traffic splitting for progressive delivery

Comparison with Similar Tools

• Kong — Lua-based plugin system; Higress uses Wasm for safer, sandboxed extensibility
• APISIX — Similar Envoy alternative; Higress has deeper AI gateway features
• Envoy Gateway — Reference K8s Gateway API impl; Higress adds console UI and AI routing
• Traefik — Auto-discovery focused; Higress offers richer policy and Wasm plugins

FAQ

Q: Does Higress require Kubernetes? A: No. It supports standalone Docker deployment for environments without K8s.

Q: Can I use Higress as an LLM proxy? A: Yes. It can route to multiple LLM providers with fallback, rate limiting per token, and response caching.

Q: How does the Wasm plugin system work? A: Plugins compile to Wasm modules and run inside Envoy's sandbox, providing isolation without the overhead of external processes.

Q: Is Higress production-ready? A: It powers internal traffic at Alibaba and is actively maintained with regular releases.

Sources

• https://github.com/higress-group/higress
• https://higress.io/docs