Pygments — Syntax Highlighting Engine for 500+ Languages

Introduction

Pygments is a generic syntax highlighting library written in Python. It supports over 500 programming and markup languages through modular lexers and can output highlighted code in HTML, LaTeX, RTF, ANSI terminal sequences, SVG, and other formats. It is the default highlighter for Sphinx documentation, Jupyter notebooks, and many static site generators.

What Pygments Does

• Tokenizes source code using language-specific lexers for 500+ languages and formats
• Renders highlighted output via formatters for HTML, LaTeX, RTF, terminal ANSI, SVG, and images
• Ships a pygmentize CLI for one-line highlighting of files and piped input
• Provides a Python API for embedding highlighting into applications and build pipelines
• Supports custom lexers, formatters, styles, and filters via a plugin architecture

Architecture Overview

Pygments processes source code in two stages. First, a lexer tokenizes the input into a stream of (token-type, value) pairs using regular-expression-based state machines. Then, a formatter consumes that token stream and produces styled output in the target format. Styles define color and font mappings for each token type, and filters can transform the token stream between the lexer and formatter stages.

Self-Hosting & Configuration

• Install from PyPI: pip install Pygments
• Use the CLI: pygmentize -l python -f html -o out.html input.py
• In Sphinx, Pygments is used automatically for code blocks via highlight_language config
• Custom styles can be installed as Python packages and selected with -O style=monokai
• For web apps, use pygments.highlight() with HtmlFormatter and include the generated CSS

Key Features

• 500+ built-in lexers covering mainstream and niche languages, configs, logs, and data formats
• Multiple output formats: HTML with CSS classes, inline styles, LaTeX, RTF, terminal 256-color, SVG
• 30+ built-in color themes including Monokai, Dracula, Solarized, and Nord
• Extensible plugin system for adding custom lexers, formatters, and styles
• Line numbering, line highlighting, and anchor generation for documentation use cases

Comparison with Similar Tools

• highlight.js — browser-side JavaScript highlighter; Pygments runs server-side in Python with more language support
• Prism.js — lightweight browser highlighter with plugin ecosystem; Pygments covers more languages and outputs to non-HTML formats
• Shiki — uses VS Code TextMate grammars for accurate highlighting; Pygments uses its own regex lexers and supports non-web output
• Rouge — Ruby highlighter compatible with Jekyll; Pygments is the Python equivalent with a larger lexer library
• Tree-sitter — full parser for structural editing; Pygments is lighter, focused purely on tokenizing for display

FAQ

Q: How do I list all available lexers? A: Run pygmentize -L lexers to see every supported language and its file patterns.

Q: Can I use Pygments to highlight code in a Markdown pipeline? A: Yes. Libraries like Python-Markdown and MkDocs use Pygments as their default code-block highlighter via the codehilite extension.

Q: How do I create a custom color theme? A: Subclass pygments.style.Style, define token-to-color mappings, install the package, and reference the style by name.

Q: Is Pygments fast enough for large files? A: It handles typical source files in milliseconds. For very large files (10 MB+), streaming or chunking may help avoid memory spikes.

Sources

• https://github.com/pygments/pygments
• https://pygments.org/