OpenEBS — Leading Open Source Container-Attached Storage

Introduction

OpenEBS pioneered the Container-Attached Storage (CAS) pattern: each volume is a small, user-space storage controller that runs next to the workload, instead of a monolithic cluster-wide storage daemon. That keeps the blast radius tight, makes upgrades painless, and matches Kubernetes'' Pod-per-concern ethos.

What OpenEBS Does

• Provisions PersistentVolumes via Kubernetes CSI drivers across multiple engines.
• Local PV (Hostpath, Device, LVM, ZFS) for high-performance node-local storage.
• Mayastor for NVMe-over-Fabric replicated block storage with µs-level latency.
• cStor / Jiva for replicated iSCSI-backed volumes with snapshots and clones.
• Exposes metrics, snapshots, clones, and volume expansion through standard Kubernetes APIs.

Architecture Overview

OpenEBS is deployed as a set of operators and per-engine controllers. For Local PV it is effectively a thin CSI shim around Hostpath, LVM, or ZFS on each node. For Mayastor, a user-space data plane (MIO + SPDK) runs as DaemonSets connecting NVMe devices to replicated “nexus” volumes. Control lives in Kubernetes CRDs (StoragePoolClaim, CStorVolume, MayastorPool) so GitOps works end-to-end.

Self-Hosting & Configuration

• Minimum: a few nodes with spare block devices or a directory for Hostpath volumes.
• Enable only the engines you need by setting Helm values (localpv-provisioner.enabled, mayastor.enabled, …).
• Label nodes with openebs.io/storage-pool=<pool> to control where volumes land.
• Use openebs-zfspv or openebs-lvmlocalpv for snapshot- and clone-capable local storage.
• Integrate with Prometheus: a ServiceMonitor for each engine exposes IO, latency, and rebuild metrics.

Key Features

• Engine choice per workload — ultra-fast local NVMe for Kafka, replicated block for Postgres, Hostpath for logs.
• CSI-native: snapshots, clones, volume resize, topology-aware scheduling.
• Non-disruptive upgrades of the storage control plane.
• Mayastor delivers NVMe-oF performance with optional 3-way replication.
• Strong CNCF credentials (Sandbox → Incubating) and an active commercial ecosystem (MayaData, DataCore).

Comparison with Similar Tools

• Rook-Ceph — full POSIX + object + block; heavier cluster dependency. OpenEBS is simpler and per-volume.
• Longhorn — single-engine replicated block storage with great UI; OpenEBS offers more engine options.
• Portworx — commercial and polished; OpenEBS is free and avoids licensing negotiations.
• TopoLVM — pure Local PV LVM; OpenEBS Local PV LVM gives you the same plus a broader ecosystem.
• Piraeus (LINSTOR / DRBD) — great replicated block; OpenEBS adds local engines for workloads that don''t need replication.

FAQ

Q: Which engine should I pick? A: Local PV Hostpath for lab work, LVM/ZFS Local PV for production single-node performance, Mayastor for replicated NVMe.

Q: Does OpenEBS work on ARM / Raspberry Pi clusters? A: Yes — Local PV engines run everywhere; Mayastor requires modern x86_64 with HugePages enabled.

Q: Can I use it alongside cloud block storage (EBS, PD)? A: Absolutely — many teams use EBS for databases and OpenEBS Local PV for caches and ephemeral state.

Q: How mature is Mayastor? A: Production-ready since 2.0; widely used for Kafka, Cassandra, and Postgres at scale.

Sources

• https://github.com/openebs/openebs
• https://openebs.io/docs