RisingWave — Cloud-Native Streaming Database in Rust

Introduction

RisingWave is a distributed SQL streaming database written in Rust that treats streams as tables and materialized views as continuously-updated query results. It speaks the Postgres wire protocol, so BI tools, ORMs, and psql just work — while under the hood it runs a fully incremental stream processor designed for the cloud.

What RisingWave Does

• Ingests streams from Kafka, Pulsar, Redpanda, Kinesis, MySQL/Postgres CDC, and more.
• Computes incremental materialized views using SQL: joins, windows, aggregations, time travel.
• Stores state in tiered storage (S3/GCS/Azure) for cheap, elastic scaling.
• Sinks results to Kafka, Iceberg, Postgres, MySQL, ClickHouse, Redis, and more.
• Offers full Postgres-compatible query endpoint for ad-hoc analytics on live streams.

Architecture Overview

A RisingWave cluster separates compute (compute nodes running streaming operators), meta (catalog + cluster coordination), and storage (Hummock — an LSM-backed shared-storage engine on object storage). Streaming jobs are compiled to dataflow graphs; state is checkpointed to object storage, so compute nodes are stateless and elastic. Frontend nodes handle SQL parsing and the Postgres wire protocol.

Self-Hosting & Configuration

• Run locally with risingwave playground, on Docker Compose, or on Kubernetes via the RisingWave Operator.
• Point Hummock storage to S3/GCS/Azure and MinIO for on-prem deployments.
• Configure auth and TLS via risingwave.toml; integrate with OIDC through a proxy or bouncer.
• Scale compute and frontend pods independently; add compute nodes for throughput during bursts.
• Observability via Prometheus metrics, Grafana dashboards, and structured logs.

Key Features

• Incremental SQL — stream analytics expressed as CREATE MATERIALIZED VIEW.
• Postgres-wire compatible; existing clients and drivers "just work".
• Separation of compute and storage for elastic, cost-efficient scaling.
• Built-in CDC sources for Postgres and MySQL without external Debezium deployment.
• UDFs in Python, JavaScript, Java, and Rust for custom transformations.

Comparison with Similar Tools

• Materialize — Closest peer; RisingWave emphasizes cloud-native storage + horizontal scale-out.
• Apache Flink — DataStream/SQL APIs; Flink is a toolkit, RisingWave is a full database product.
• ksqlDB — Kafka-centric; RisingWave supports broader sources and standard Postgres SQL.
• Apache Pinot / Druid — Real-time OLAP; great for aggregations, less expressive joins than RisingWave.
• Materialized views in Snowflake/BigQuery — Batch-refreshed; RisingWave updates continuously.

FAQ

Q: Is RisingWave Postgres-compatible? A: It implements the Postgres wire protocol and a large subset of SQL; it is not a drop-in OLTP Postgres.

Q: How is state stored? A: State lives in Hummock on S3/GCS/Azure; compute nodes are stateless and can be replaced freely.

Q: Can it replace a Kafka Streams app? A: Often yes — you get SQL semantics, auto-scaling, and a SQL surface instead of imperative JVM code.

Q: What is the license? A: Apache 2.0 for RisingWave; a paid cloud service is offered by RisingWave Labs for managed use.

Sources

• https://github.com/risingwavelabs/risingwave
• https://docs.risingwave.com