bpftrace — High-Level Tracing Language for Linux eBPF

What it is

bpftrace is a high-level tracing language for Linux that uses eBPF to instrument the kernel and user-space programs. It provides an awk-like syntax for writing powerful one-liner performance analysis scripts that compile directly to eBPF bytecode.

System administrators, SREs, and performance engineers who need to diagnose production issues without restarting services or adding custom instrumentation will find bpftrace indispensable.

How it saves time or tokens

bpftrace replaces complex custom C-based eBPF programs with concise one-liners. What previously required writing BPF C code, compiling with clang, and loading with libbpf can now be expressed in a single command. This reduces debugging time from hours to minutes for common performance investigations.

How to use

1. Install bpftrace from your distribution's package manager.
2. Write a one-liner or script file targeting a kernel probe, tracepoint, or USDT probe.
3. Run with root privileges to attach the eBPF program.

# Install on Ubuntu/Debian
sudo apt-get install bpftrace

# Trace all open() syscalls with the filename argument
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_openat { printf("%s %s\n", comm, str(args->filename)); }'

# Histogram of read() sizes by process
sudo bpftrace -e 'tracepoint:syscalls:sys_exit_read /args->ret > 0/ { @bytes[comm] = hist(args->ret); }'

Example

Count syscalls by process name in real time:

sudo bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @[comm] = count(); }'

This attaches to every syscall entry, groups by the calling process name, and prints a sorted count table when you press Ctrl-C.

Related on TokRepo

• DevOps tools — Infrastructure and operations tooling
• Monitoring tools — Observability and performance monitoring

Common pitfalls

• bpftrace requires root (or CAP_BPF) and a kernel version 4.9+ with BTF support for best results.
• Some probes generate high overhead when attached to very hot paths like scheduler functions.
• The awk-like syntax has limits; for complex multi-probe programs, consider BCC or libbpf-based tools.