Ristretto — High-Performance Concurrent Cache for Go

Introduction

Ristretto is a high-performance, concurrent cache library for Go developed by the Dgraph team. It uses a TinyLFU admission policy to decide which items enter the cache and a sampled LFU eviction policy to remove the least valuable items when the cache is full. This combination delivers hit ratios competitive with full LFU implementations while maintaining low memory overhead.

What Ristretto Does

• Provides a concurrent, thread-safe cache with near-optimal hit ratios
• Uses TinyLFU admission to prevent one-hit-wonder items from evicting frequently accessed ones
• Enforces a cost-based size limit so the cache respects memory budgets
• Supports TTL-based expiration for time-sensitive cached data
• Handles contention-free reads using a striped lock-free architecture

Architecture Overview

Ristretto separates the hot path (reads) from the cold path (writes). Reads are lock-free and return immediately from a concurrent hash map. Writes go through a lossy ring buffer that batches admissions; a background goroutine processes the buffer and applies the TinyLFU admission check. The admission policy uses a Count-Min Sketch to estimate item frequency, and the eviction policy samples random entries to find low-frequency victims.

Self-Hosting & Configuration

• Set NumCounters to roughly 10x the expected number of unique keys for accurate frequency estimation
• Set MaxCost to the memory budget in bytes (each item's cost is user-defined or defaults to 1)
• Adjust BufferItems to tune the write buffer size (higher values improve throughput at the cost of latency)
• Use SetWithTTL for items that should expire after a fixed duration
• Call cache.Wait() after Set to ensure the item is processed before reading it back

Key Features

• TinyLFU admission: filters out low-value items before they enter the cache
• Cost-based eviction: each item can declare its memory cost for accurate capacity management
• Metrics collection: track hit rate, eviction count, and cost added via built-in metrics
• Generics support: v2 uses Go generics for type-safe keys and values
• Battle-tested: powers Dgraph, Badger, and other high-throughput systems

Comparison with Similar Tools

• go-cache — simpler API with expiration but no eviction policy; Ristretto is better when memory bounds and hit ratios matter
• BigCache — optimized for reduced GC pressure with byte storage; Ristretto offers better hit ratios through smart admission
• FreeCache — pre-allocated zero-GC cache; Ristretto trades some GC friendliness for higher hit ratios
• Caffeine (Java) — Window TinyLFU cache for Java; Ristretto brings similar algorithms to Go

FAQ

Q: Why do I need to call cache.Wait() after Set? A: Sets are batched through an async buffer for performance. Wait ensures the buffer is flushed and the item is available for Get.

Q: What is the cost parameter in Set? A: Cost represents the size of the value (e.g., in bytes). Ristretto evicts items to keep total cost under MaxCost. Set cost to 1 if all items are equal size.

Q: How do I monitor cache performance? A: Enable metrics in the config and call cache.Metrics to get hit count, miss count, and eviction statistics.

Q: Can Ristretto be used as a distributed cache? A: No. Ristretto is an in-process cache. For distributed caching, pair it with a remote store like Redis and use Ristretto as a local L1 cache.

Sources

• https://github.com/dgraph-io/ristretto
• https://dgraph.io/blog/post/introducing-ristretto-high-perf-go-cache/