Nokogiri — Fast HTML and XML Parser for Ruby

Introduction

Nokogiri wraps the battle-tested libxml2 and libgumbo C libraries to give Ruby developers fast, standards-compliant parsing of HTML and XML documents. It handles malformed markup gracefully and provides both CSS selector and XPath interfaces for querying document trees.

What Nokogiri Does

• Parses HTML4, HTML5, XML, and XSLT documents into traversable node trees
• Supports both CSS selector and XPath query syntax for searching nodes
• Handles broken and malformed HTML gracefully with automatic error recovery
• Provides a builder DSL for generating XML and HTML programmatically
• Detects and converts character encodings transparently

Architecture Overview

Nokogiri wraps two C libraries: libxml2 for XML and HTML4 parsing, and libgumbo for HTML5 parsing. Ruby objects mirror the underlying C document tree through a thin binding layer. When you call css() or xpath(), the query is dispatched to the C library's native search implementation, which is significantly faster than a pure-Ruby approach. Precompiled native gems for major platforms eliminate the need to compile C extensions on most systems.

Self-Hosting & Configuration

• Install via gem install nokogiri; precompiled binaries cover Linux, macOS, and Windows
• Use Nokogiri::HTML() for HTML4 or Nokogiri::HTML5() for spec-compliant HTML5 parsing
• Pass encoding options explicitly when dealing with non-UTF-8 documents
• Configure parse options like NOBLANKS, NOERROR, or RECOVER for strict or lenient parsing
• Use Bundler's platform constraints to ensure the correct native gem is selected in CI

Key Features

• Native C extensions backed by libxml2 and libgumbo for high throughput
• Dual query API with CSS selectors and full XPath 1.0 support
• HTML5 parsing that follows the WHATWG specification
• SAX and Reader streaming parsers for processing large documents without loading them fully into memory
• Precompiled native gems that install without requiring a C toolchain

Comparison with Similar Tools

• Oga — Pure-Ruby XML/HTML parser; no native dependencies but significantly slower on large documents
• REXML — Ruby standard library XML parser; always available but limited features and slow
• Ox — Fast Ruby XML parser optimized for speed; lacks CSS selector support
• Hpricot — Legacy Ruby HTML parser; unmaintained, superseded by Nokogiri
• Beautiful Soup (Python) — Equivalent role in Python; Nokogiri is the Ruby counterpart

FAQ

Q: Do I need to install libxml2 separately? A: Usually no. Precompiled native gems bundle the required C libraries. Building from source requires libxml2 and libxslt development headers.

Q: How do I parse malformed HTML without errors? A: Use Nokogiri::HTML() which enables error recovery by default. Pass Nokogiri::XML::ParseOptions::RECOVER for XML documents.

Q: Can Nokogiri handle very large files? A: Yes. Use the SAX parser or Reader interface to process documents as a stream without loading the entire tree into memory.

Q: Is Nokogiri thread-safe? A: Individual document objects are not thread-safe. Parse and query documents within a single thread, or protect shared documents with a mutex.

Sources

• https://github.com/sparklemotion/nokogiri
• https://nokogiri.org/