Apache Thrift — Cross-Language RPC and Serialization Framework

Introduction

Apache Thrift was originally developed at Facebook and donated to the Apache Software Foundation. It provides a complete RPC framework: an IDL for defining data types and services, a compiler that generates client and server stubs, and runtime libraries with pluggable transports and protocols. It is used in production at scale by companies processing billions of requests per day.

What Apache Thrift Does

• Defines data structures and service interfaces in a language-neutral .thrift IDL
• Generates client and server code for 20+ languages from a single schema
• Supports multiple wire protocols: binary, compact, JSON, and multiplexed
• Provides pluggable transport layers: sockets, HTTP, TLS, framed, and buffered
• Handles connection pooling, async I/O, and thread management in server runtimes

Architecture Overview

Thrift uses a layered architecture. The IDL compiler reads .thrift files and outputs language-specific stubs. At runtime, data flows through a Transport layer (socket, HTTP, file), a Protocol layer (binary encoding, compact encoding, JSON), and a Processor layer that dispatches RPC calls to user-implemented handlers. Servers can be threaded, non-blocking, or event-driven depending on the language runtime.

Self-Hosting & Configuration

• Install the Thrift compiler from Apache releases, Homebrew, or build from source
• Write .thrift IDL files defining structs, enums, exceptions, and services
• Generate code: thrift --gen cpp --gen go --gen py service.thrift
• Implement the generated service interface in your server language
• Choose transport (TSocket, THttpClient) and protocol (TBinaryProtocol, TCompactProtocol) per deployment needs

Key Features

• Single IDL produces clients and servers for 20+ languages simultaneously
• Compact binary protocol minimizes payload size for high-throughput systems
• Multiplexed protocol serves multiple services on a single port
• Built-in async and non-blocking server implementations for high concurrency
• Supports forward and backward compatibility via optional fields and field numbering

Comparison with Similar Tools

• gRPC — HTTP/2-based RPC with protobuf; stronger ecosystem and streaming support but heavier runtime
• Protocol Buffers — Serialization-only (no built-in RPC); often paired with gRPC for services
• Avro — Schema-evolution-focused with dynamic typing; popular in Hadoop but no built-in RPC server
• Cap'n Proto — Zero-copy serialization with RPC; faster reads but smaller community and language support
• JSON-RPC — Simple text-based RPC; easy to debug but significantly less efficient at scale

FAQ

Q: How does Thrift compare to gRPC for new projects? A: gRPC has a larger ecosystem and HTTP/2 streaming support. Thrift offers broader language coverage and a lighter runtime. Choose gRPC for greenfield microservices; Thrift when you need niche language support or already have Thrift infrastructure.

Q: Can Thrift work over HTTP? A: Yes. Use THttpTransport on the client and a Thrift HTTP server handler to tunnel Thrift calls over HTTP for firewall-friendly deployments.

Q: Does Thrift support streaming? A: Not natively in the way gRPC does. You can simulate streaming with repeated RPC calls or by using oneway methods for fire-and-forget patterns.

Q: Is schema evolution safe with Thrift? A: Yes, as long as you use field IDs and mark new fields as optional. Never reuse or renumber field IDs.

Sources

• https://github.com/apache/thrift
• https://thrift.apache.org/