fq — jq for Binary Formats

Introduction

fq is a tool inspired by jq that decodes, inspects, and queries binary file formats from the command line. Where jq parses JSON, fq parses MP4, PCAP, ELF, ZIP, FLAC, Matroska, and over 200 other binary formats into a tree structure you can query with jq-compatible expressions. It is invaluable for debugging media pipelines, reverse engineering file formats, and inspecting network captures.

What fq Does

• Decodes 200+ binary formats into a queryable tree structure
• Uses a jq-compatible query language for filtering and transforming decoded data
• Provides a REPL for interactive exploration of binary files
• Supports nested formats (e.g., a ZIP containing PNGs, each fully decoded)
• Outputs results as JSON, text, hexdump, or raw binary

Architecture Overview

fq is written in Go and embeds a jq interpreter extended with binary-aware types. Each format has a decoder that reads raw bytes and produces a tree of typed values with bit-level precision. Decoders can recursively invoke other decoders when nested formats are detected. The query engine operates on this tree using standard jq syntax plus binary-specific functions like tobits and toactual.

Self-Hosting & Configuration

• Install as a single static binary from GitHub releases or via package managers
• No configuration file is needed for basic usage
• Set default output format with -o flag (json, text, hex)
• Use -d flag to force a specific decoder when auto-detection is ambiguous
• Integrate into shell scripts by piping binary data through stdin

Key Features

• 200+ format decoders covering media, archives, network, executables, and filesystems
• Bit-level precision for formats where fields are not byte-aligned
• Interactive REPL with tab completion for exploring large files
• Recursive decoding of nested formats (containers within containers)
• jq-compatible syntax so existing jq knowledge transfers directly

Comparison with Similar Tools

• jq — works only on JSON text; fq extends the same language to binary formats
• hexdump/xxd — raw hex output without format awareness; fq provides structured decoding
• Wireshark — GUI-based packet analyzer; fq is a CLI tool scriptable in pipelines
• binwalk — scans for embedded formats; fq fully decodes and queries the structure
• MediaInfo — shows media metadata; fq exposes every atom and box in the container

FAQ

Q: What formats does fq support? A: Over 200 including MP4, MKV, FLAC, MP3, PNG, JPEG, ELF, PE, ZIP, TAR, PCAP, DNS, TLS, and many more.

Q: Can I use fq in a shell pipeline? A: Yes. fq reads from stdin and outputs JSON or raw binary, making it composable with other CLI tools.

Q: How does fq handle very large files? A: fq uses lazy decoding and only decodes parts of the file that are accessed by the query.

Q: Is the query language exactly jq? A: It is jq-compatible with extensions for binary operations like tobits, tobytes, and format-specific functions.

Sources

• https://github.com/wader/fq