gojq — Pure Go Implementation of jq for JSON Processing

Introduction

gojq reimplements the jq JSON processor in pure Go, providing the same filter language and capabilities while adding YAML and TOML input support, improved error messages, and easy embedding in Go applications as a library. It aims for full compatibility with jq while extending its reach.

What gojq Does

• Parses and filters JSON data using the standard jq filter language
• Reads YAML and TOML input in addition to JSON
• Provides clear error messages with position information for debugging filters
• Works as both a standalone CLI tool and an embeddable Go library
• Supports streaming mode for processing large files incrementally

Architecture Overview

gojq implements a jq-compatible lexer, parser, and compiler in Go. Filters are compiled to an internal bytecode representation and executed by a virtual machine that processes JSON values. The Go implementation avoids CGo dependencies, making cross-compilation straightforward and enabling use as a library in other Go programs without external linking.

Self-Hosting & Configuration

• Install via go install, Homebrew, or download prebuilt binaries
• No configuration file; all behavior is controlled by command-line flags
• Use --yaml-input to parse YAML files with jq filters
• Use --slurp to read all input into a single array before filtering
• Embed in Go programs by importing github.com/itchyny/gojq as a module

Key Features

• Full jq language compatibility including advanced features like reduce and limit
• YAML and TOML input support without external converters
• Embeddable as a Go library for programmatic JSON processing
• Improved error messages showing exact position of filter errors
• No CGo dependencies, enabling easy cross-platform compilation

Comparison with Similar Tools

• jq — The original C implementation, slightly faster but no YAML support or library embedding
• yq — Focused on YAML processing with jq-like syntax, different filter language
• Dasel — Multi-format selector tool with its own query syntax, not jq-compatible
• Miller — Tabular data processor for CSV/JSON, different paradigm from jq filters

FAQ

Q: Is gojq fully compatible with jq? A: gojq aims for complete compatibility. A few edge cases around floating-point behavior and some undocumented jq features may differ.

Q: Can I use gojq as a Go library? A: Yes. Import the gojq package and use the compiler and VM APIs to run jq filters programmatically in your Go applications.

Q: How does performance compare to jq? A: gojq is generally close to jq in performance. For very large files, jq's C implementation may have an edge, but gojq compensates with streaming mode.

Q: Does gojq support jq modules? A: Yes. gojq supports jq's module system including import and include statements.

Sources

• https://github.com/itchyny/gojq