Cap'n Proto — Zero-Copy Serialization and RPC System

Introduction

Cap'n Proto is a serialization format and RPC system that eliminates the encode/decode step found in Protocol Buffers and similar systems. The wire format is the same as the in-memory format, enabling zero-copy reads that can significantly reduce latency and CPU overhead in performance-sensitive applications.

What Cap'n Proto Does

• Serializes structured data with zero encoding or decoding overhead
• Provides a built-in RPC system with promise pipelining to reduce round trips
• Generates type-safe bindings for C++, Rust, Go, Python, Java, and other languages
• Supports schema evolution with backwards and forwards compatibility guarantees
• Handles mmap-friendly data files that can be read directly without parsing

Architecture Overview

Cap'n Proto stores data in a pointer-based format that matches its in-memory layout. Messages are composed of segments containing structs, lists, and capabilities. The RPC layer uses an object-capability model where remote references are passed as capabilities, enabling promise pipelining: a caller can chain method calls on a not-yet-returned result without waiting for each round trip.

Self-Hosting & Configuration

• Install the compiler and runtime from system packages or build from source
• Write .capnp schema files to define message types and RPC interfaces
• Run the capnp compiler with language-specific code generator plugins
• Link against the Cap'n Proto runtime library in your build system
• Use the KJ async framework (bundled) for non-blocking RPC servers in C++

Key Features

• Zero-copy deserialization means reading a field is a pointer dereference, not a parse
• Promise pipelining lets you call methods on future results before they arrive
• Time-travel RPC reduces multiple sequential calls into a single round trip
• Schema evolution supports adding fields, renaming types, and reordering without breaking wire compatibility
• Canonical encoding mode ensures deterministic output for hashing and signing

Comparison with Similar Tools

• Protocol Buffers — widely adopted but requires encode/decode steps; Cap'n Proto was created as its successor by the same author
• FlatBuffers — also zero-copy but lacks a built-in RPC system and promise pipelining
• MessagePack — schemaless binary format, compact but without type safety or zero-copy reads
• Apache Avro — schema-based with strong Hadoop integration, but requires full deserialization

FAQ

Q: Who created Cap'n Proto? A: Kenton Varda, who previously designed Protocol Buffers v2 at Google. He now works on Cloudflare Workers.

Q: Is Cap'n Proto compatible with Protocol Buffers? A: No. They are separate formats with different wire encodings. However, schema concepts are similar, and migration tools exist.

Q: What languages are supported? A: The reference implementation is C++. Community-maintained bindings exist for Rust, Go, Java, Python, TypeScript, C#, and others.

Q: When should I use Cap'n Proto over Protobuf? A: When you need minimal serialization overhead, promise-pipelined RPC, or want to mmap data files directly without parsing.

Sources

• https://github.com/capnproto/capnproto
• https://capnproto.org