Maxwell — MySQL Binlog Change Data Capture Daemon

Introduction

Maxwell is a CDC (Change Data Capture) application that reads MySQL binary logs and outputs row-level data changes as JSON documents. It provides a lightweight alternative to more complex CDC setups for streaming database changes to downstream consumers.

What Maxwell Does

• Reads MySQL binary logs and emits row-level INSERT, UPDATE, and DELETE events as JSON
• Outputs to Kafka, Amazon Kinesis, Google Pub/Sub, RabbitMQ, Redis, or stdout
• Tracks its own binlog position in a MySQL schema for crash recovery
• Supports initial bootstrapping to capture the full current state of a table
• Handles DDL changes and schema evolution automatically

Architecture Overview

Maxwell connects to a MySQL server as a replication client, reading binary log events in real time. It maintains an internal representation of the database schema by tracking DDL statements. Each row-level change is transformed into a self-contained JSON document that includes the database, table, type of change, and the affected column values. Binlog position is persisted in a dedicated Maxwell schema within MySQL itself, enabling exactly-once delivery semantics with idempotent consumers.

Self-Hosting & Configuration

• Requires MySQL with binlog_format=ROW and binlog_row_image=FULL
• Create a Maxwell-specific MySQL user with replication and schema privileges
• Run via Docker or as a standalone Java application with a config.properties file
• Configure the output producer (Kafka, Kinesis, etc.) and topic routing
• Use the bootstrapping feature to capture the initial state of existing tables

Key Features

• Lightweight single-process daemon with minimal resource requirements
• JSON output format is easy to parse and integrate with downstream systems
• Schema-aware: includes column names and types in change events
• Supports filtering by database, table, or column to reduce event volume
• Built-in bootstrapping for initial data loads without stopping writes

Comparison with Similar Tools

• Debezium — JVM-based with broader database support; Maxwell is MySQL-specific and simpler to operate
• Canal — Alibaba CDC tool; Maxwell outputs standard JSON and supports more producers
• MySQL Streamer — Yelp tool; Maxwell is actively maintained with broader community support
• AWS DMS — managed service with vendor lock-in; Maxwell is open-source and self-hosted
• Tungsten Replicator — complex clustering tool; Maxwell focuses on CDC-to-streaming simplicity

FAQ

Q: Which MySQL versions does Maxwell support? A: Maxwell supports MySQL 5.1 through 8.x and MariaDB. It requires row-based binary logging to be enabled.

Q: How does Maxwell handle schema changes? A: Maxwell tracks DDL statements in the binary log and updates its internal schema representation. Schema changes are emitted as separate events.

Q: Can Maxwell replay historical changes? A: Maxwell can rewind to any saved binlog position. The bootstrap feature can also re-emit the current state of a table as INSERT events.

Q: What happens if Maxwell crashes? A: Maxwell stores its binlog position in MySQL. On restart, it resumes from the last committed position, ensuring no events are skipped.

Sources

• https://github.com/zendesk/maxwell
• https://maxwells-daemon.io/