BigCache — Fast Concurrent Cache for Gigabytes of Data in Go

Introduction

BigCache is a concurrent cache library developed by Allegro (a Polish e-commerce company) to solve the problem of caching gigabytes of HTTP response data in Go without causing long GC pauses. It stores entries as byte slices in a pre-allocated buffer, keeping them invisible to the garbage collector.

What BigCache Does

• Stores millions of cache entries with sub-millisecond read and write latency
• Avoids GC overhead by keeping data in contiguous byte slices, not pointer-heavy maps
• Uses sharding (256 segments by default) for lock-free concurrent reads across goroutines
• Supports automatic TTL-based eviction without external timers
• Provides a simple key-value API with string keys and byte-slice values

Architecture Overview

BigCache divides its storage into shards, each protected by its own lock. Within each shard, entries are appended to a byte-slice queue (ring buffer). A hash map maps keys to offsets within the queue. Because the map stores only uint64 hashes as keys and uint32 offsets as values (no pointers), the GC does not scan the map contents. Eviction walks the queue in FIFO order, removing entries older than the configured TTL.

Self-Hosting & Configuration

• Install with go get github.com/allegro/bigcache/v3
• Configure TTL, shard count, max entry size, and initial capacity via bigcache.Config
• Set HardMaxCacheSize (in MB) to cap total memory usage
• Use OnRemove callback to react to evictions
• For HTTP caching, store serialized response bodies as byte slices

Key Features

• Zero GC overhead — designed to hold millions of entries without increasing GC pause times
• Sharded concurrency — reads across different shards proceed in parallel without contention
• FIFO eviction with TTL — expired entries are cleaned up automatically
• Simple API — just Get, Set, Delete, and Reset
• Configurable callbacks on entry removal for cleanup or metrics

Comparison with Similar Tools

• sync.Map — good for few writes, many reads; BigCache is faster for high-throughput mixed workloads
• FreeCache — similar zero-GC design with LRU; BigCache uses FIFO and supports larger datasets
• Ristretto — admission-based cache with TinyLFU; BigCache is simpler but lacks frequency-based eviction
• Groupcache — distributed cache; BigCache is single-node and focuses on local performance

FAQ

Q: Why not just use a Go map with a mutex? A: A standard map storing pointers causes the GC to scan all entries. BigCache avoids this by storing data as raw bytes.

Q: Can I store structured data? A: Values are byte slices. Serialize structs with JSON, protobuf, or msgpack before storing.

Q: What happens when the cache is full? A: If HardMaxCacheSize is set, the oldest entries are evicted. Otherwise, the cache grows.

Q: Is BigCache thread-safe? A: Yes, it is safe for concurrent use from multiple goroutines.

Sources

• https://github.com/allegro/bigcache
• https://pkg.go.dev/github.com/allegro/bigcache/v3