SGLang — Fast LLM Serving with RadixAttention

SGLang is a high-performance serving framework for large language models and multimodal models, delivering low-latency and high-throughput inference. With 25,300+ GitHub stars and Apache 2.0 license, SGLang features RadixAttention for efficient prefix caching, zero-overhead scheduling, prefill-decode disaggregation, speculative decoding, and structured output generation. It supports NVIDIA, AMD, Intel, Google TPU, and Ascend NPU hardware, with broad model compatibility including Llama, Qwen, DeepSeek, and diffusion models.

Best for: Teams deploying LLMs in production needing maximum throughput and lowest latency Works with: Claude Code, OpenAI Codex, Cursor, Gemini CLI, Windsurf Hardware: NVIDIA, AMD, Intel, Google TPU, Ascend NPU