FreeCache — Zero-GC Overhead In-Memory Cache for Go

Introduction

FreeCache is a Go in-memory cache library that eliminates garbage collection overhead by storing all entries in a pre-allocated byte buffer. It was designed for scenarios where caching millions of small entries in a standard Go map would cause unacceptable GC pause times.

What FreeCache Does

• Stores cache entries in a fixed-size pre-allocated memory region invisible to the GC
• Evicts entries using an approximate LRU algorithm when the cache is full
• Supports per-entry TTL for automatic time-based expiration
• Uses segment-level locking for safe concurrent access from multiple goroutines
• Provides an iterator for walking all entries without blocking writers

Architecture Overview

FreeCache divides its memory into 256 segments, each with its own lock. Each segment contains a ring buffer for storing serialized entries and a hash table for lookups. Keys and values are stored as contiguous byte sequences in the ring buffer. When a segment fills up, the oldest entries are overwritten (LRU-like). The hash table maps key hashes to ring buffer offsets. Because neither the hash table nor the ring buffer uses Go pointers, the GC never scans cache contents.

Self-Hosting & Configuration

• Install with go get github.com/coocood/freecache
• Allocate cache with freecache.NewCache(sizeInBytes) — minimum 512 KB
• Set entries with cache.Set(key, value, expireSeconds) — pass 0 for no expiration
• Use cache.GetOrSet() for atomic check-and-fill operations
• Monitor hit rate with cache.HitRate() and entry count with cache.EntryCount()

Key Features

• Zero GC overhead — stores data in flat byte arrays, not pointer-heavy structures
• Fixed memory footprint — cache never grows beyond the allocated size
• Approximate LRU eviction keeps frequently accessed entries in cache
• Per-entry TTL allows mixing short-lived and long-lived data in one cache
• Thread-safe with per-segment locking for high concurrency

Comparison with Similar Tools

• BigCache — also zero-GC, but uses FIFO eviction; FreeCache offers LRU which keeps hot data longer
• Ristretto — TinyLFU admission policy for better hit rates; FreeCache is simpler with lower overhead
• sync.Map — no size limit and causes GC pressure; FreeCache has bounded memory and no GC impact
• Groupcache — distributed caching with peer replication; FreeCache is single-node only

FAQ

Q: What is the minimum cache size? A: 512 KB. The library enforces this minimum because each of the 256 segments needs at least 2 KB.

Q: Can I store structured data? A: Keys and values are byte slices. Serialize structs before storing and deserialize after retrieval.

Q: How accurate is the LRU eviction? A: It is approximate. FreeCache uses a ring buffer per segment, so eviction order is per-segment FIFO with access-time reordering for recently read entries.

Q: What happens when the cache is full and I add a new entry? A: The oldest entries in the target segment are evicted to make room.

Sources

• https://github.com/coocood/freecache
• https://pkg.go.dev/github.com/coocood/freecache