Buf — Modern Protocol Buffers Toolchain

Introduction

Buf replaces the fragmented protoc workflow with a single CLI that handles linting, formatting, breaking change detection, and code generation for Protocol Buffers. It aims to make protobuf development as reliable and ergonomic as modern programming language toolchains.

What Buf Does

• Lints .proto files against configurable rule sets for style and correctness
• Detects breaking changes between schema versions automatically
• Formats protobuf files with a consistent canonical style
• Generates code in any language using a plugin system that replaces protoc
• Publishes and resolves protobuf dependencies via the Buf Schema Registry

Architecture Overview

Buf reads a buf.yaml configuration file at the module root to discover .proto files and their dependencies. The CLI compiles schemas internally using a custom parser rather than shelling out to protoc. Breaking change detection works by comparing the current schema against a reference (a Git branch, tag, or BSR module). Code generation is configured in buf.gen.yaml and can invoke any protoc plugin or Buf-native plugin.

Self-Hosting & Configuration

• Install via Homebrew, npm, Go install, or download prebuilt binaries
• Create buf.yaml to define the module name, lint rules, and breaking change policy
• Declare dependencies on BSR modules or local paths in buf.yaml
• Configure code generation targets and plugins in buf.gen.yaml
• Run buf push to publish modules to the Buf Schema Registry

Key Features

• Over 60 lint rules covering style, naming, and package conventions
• Breaking change detection for wire and JSON compatibility
• Deterministic code generation without relying on protoc installation
• Dependency management with lockfiles similar to Go modules
• Editor integration via a language server for .proto files

Comparison with Similar Tools

• protoc — the reference compiler from Google; Buf wraps and extends it with linting, formatting, and breaking change detection
• prototool — Uber's protobuf linter (archived); Buf is its actively maintained successor
• grpc-gateway — HTTP/JSON gateway for gRPC; Buf focuses on schema management, not runtime proxying
• Connect — Buf's own RPC framework; Buf CLI handles schema tooling while Connect handles runtime
• Protolock — breaking change detection only; Buf provides a complete toolchain beyond just compatibility checks

FAQ

Q: Does Buf replace protoc entirely? A: For most workflows, yes. Buf handles compilation and code generation internally and can invoke protoc plugins without requiring a protoc installation.

Q: Is the Buf Schema Registry required? A: No. You can use Buf purely as a local CLI for linting, formatting, and generation without publishing to BSR.

Q: Can I use Buf in CI pipelines? A: Yes. Buf is commonly used in CI to enforce lint rules and block merges that introduce breaking schema changes.

Q: What languages does Buf generate code for? A: Any language supported by protoc plugins, including Go, Java, Python, TypeScript, Rust, C++, and more.

Sources

• https://github.com/bufbuild/buf
• https://buf.build/docs