Hyperfine — Command-Line Benchmarking Tool

What it is

Hyperfine is a command-line benchmarking tool written in Rust. It runs shell commands multiple times, performs statistical analysis on the timing results, and reports mean, standard deviation, min, and max execution times. It supports warmup runs, parameterized benchmarks, and exporting results to CSV, JSON, or Markdown.

The target audience includes developers optimizing CLI tools, DevOps engineers comparing script performance, and anyone who needs reproducible timing measurements for shell commands.

How it saves time or tokens

Manual benchmarking with time gives a single noisy measurement. Hyperfine automates multiple runs, discards outliers, and provides confidence intervals -- turning a 5-minute manual process into a one-liner. The parameterized benchmark feature lets you compare multiple implementations in a single command.

For AI-assisted optimization workflows, hyperfine provides structured JSON output that an LLM can parse to recommend which implementation is faster, with statistical confidence.

How to use

1. Install hyperfine:

# macOS
brew install hyperfine

# Ubuntu/Debian
sudo apt install hyperfine

# Cargo
cargo install hyperfine

1. Benchmark a single command:

hyperfine 'sleep 0.3'

1. Compare two commands:

hyperfine 'fd . /usr/lib' 'find /usr/lib'

Example

Parameterized benchmark comparing grep implementations:

# Compare ripgrep vs grep vs ag on a large codebase
hyperfine \
  --warmup 3 \
  --min-runs 10 \
  'rg TODO src/' \
  'grep -r TODO src/' \
  'ag TODO src/' \
  --export-json results.json

# Parameterized: test different thread counts
hyperfine \
  --parameter-scan threads 1 8 \
  'sort --parallel={threads} large-file.txt'

Related on TokRepo

• Automation tools -- CLI productivity and developer tools
• AI tools for testing -- Performance testing and benchmarking tools

Common pitfalls

• Not using --warmup for commands that benefit from filesystem caching. The first run is often significantly slower due to cold cache.
• Benchmarking commands that modify state (creating files, writing databases) without cleanup. Use --prepare 'rm -f output.txt' to reset between runs.
• Running benchmarks on a loaded system. Background processes introduce variance. Close other applications or use --min-runs 20 for more stable results.
• Comparing commands with different output volumes without redirecting to /dev/null. I/O time dominates when terminal rendering is involved.
• Forgetting --shell=none for commands that do not need shell interpretation. This avoids shell startup overhead in measurements.