Apache Kvrocks — Redis-Compatible Store Built on RocksDB

Introduction

Apache Kvrocks is a distributed key-value store that implements the Redis wire protocol on top of RocksDB. It stores data on disk rather than entirely in memory, making it a cost-effective alternative for workloads that need Redis compatibility but have data sets larger than available RAM.

What Apache Kvrocks Does

• Speaks the Redis RESP protocol so existing Redis clients and libraries work without changes
• Stores data on SSD or HDD via RocksDB, reducing memory requirements by orders of magnitude
• Supports most Redis data structures: strings, hashes, lists, sets, sorted sets, and streams
• Provides cluster mode with hash-slot-based sharding compatible with Redis Cluster clients
• Offers namespace-based multi-tenancy for isolating workloads on a single instance

Architecture Overview

Kvrocks uses a multi-threaded architecture with an event-driven network layer that handles Redis protocol parsing. Commands are translated into RocksDB operations through an encoding layer that maps Redis data structures to key-value pairs on disk. A WAL-based replication mechanism supports leader-follower setups, and cluster mode distributes hash slots across multiple Kvrocks nodes using a Raft-based metadata store.

Self-Hosting & Configuration

• Build from source with CMake or download pre-built binaries for Linux
• Edit kvrocks.conf to set the listening port, data directory, and RocksDB tuning parameters
• Enable cluster mode by setting cluster-enabled to yes and configuring node topology
• Set up replication with the slaveof command, same as Redis
• Use namespaces to partition data between different applications on one instance

Key Features

• Drop-in Redis replacement for most commands and data structures
• Disk-based storage that handles data sets far exceeding available memory
• Built-in cluster mode with automatic slot migration and rebalancing
• Namespace isolation for multi-tenant deployments
• Lua scripting and pub/sub support compatible with Redis semantics

Comparison with Similar Tools

• Redis — in-memory, fast but expensive at scale; Kvrocks trades some latency for much lower memory cost
• KeyDB — multi-threaded Redis fork, still in-memory; Kvrocks is disk-based
• DragonflyDB — high-performance in-memory store; Kvrocks targets large data on disk
• Valkey — Redis-compatible fork with community governance; Kvrocks differs architecturally by using RocksDB
• Garnet — Microsoft's cache-store in C#; Kvrocks is C++ on RocksDB with Apache governance

FAQ

Q: Is Kvrocks slower than Redis? A: Read and write latency is higher than in-memory Redis because data goes through RocksDB on disk. For many workloads, the latency difference is acceptable, especially on SSDs.

Q: Which Redis commands are not supported? A: Most commands are supported. Some advanced features like Redis modules and certain cluster management commands may not be available. The documentation lists specific gaps.

Q: Can I migrate from Redis to Kvrocks without downtime? A: Yes. You can set Kvrocks as a replica of a Redis instance to sync data, then promote Kvrocks and redirect clients.

Q: Does Kvrocks support persistence and backups? A: Data is always persisted via RocksDB. Backups can be taken using RocksDB checkpoints or filesystem snapshots.

Sources

• https://github.com/apache/kvrocks
• https://kvrocks.apache.org/