XTDB — Bitemporal Database with Immutable History

Introduction

XTDB (formerly Crux) is a bitemporal database that tracks two independent time axes for every record: valid time (when a fact is true in the real world) and transaction time (when the database learned about it). This enables regulatory compliance queries, point-in-time auditing, and temporal corrections without data loss.

What XTDB Does

• Stores every version of every record with both valid-time and transaction-time metadata
• Supports SQL queries with temporal predicates (AS OF, BETWEEN, ALL TIME)
• Provides immutable append-only storage where no data is ever deleted
• Enables retroactive corrections by inserting facts with past valid-time ranges
• Offers both document and columnar storage engines for different workload profiles

Architecture Overview

XTDB v2 uses Apache Arrow as its in-memory format and stores data in columnar pages on object storage (S3, local filesystem). The transaction log is append-only, and each transaction produces new immutable pages. Temporal indexes enable efficient range queries across both time dimensions. The SQL engine compiles queries into vectorized execution plans that push temporal predicates into the scan operators for early filtering.

Self-Hosting & Configuration

• Run via Docker or embed as a JVM library in Clojure/Java/Kotlin applications
• Configure storage backend: local filesystem, S3, or Azure Blob Storage
• Set transaction log storage (Kafka or local) for durability guarantees
• HTTP server exposes SQL endpoint on port 3000 by default
• Tune memory allocation for the Arrow buffer pool based on query workload

Key Features

• Bitemporal by default: every record automatically tracks two time dimensions
• Immutable history enables full audit trails without custom event-sourcing
• SQL interface with temporal extensions makes adoption straightforward
• Columnar storage (Arrow-based) provides efficient analytical queries
• Schema flexibility: define columns per table without rigid migrations

Comparison with Similar Tools

• PostgreSQL + temporal tables — Manual setup with triggers; XTDB has native bitemporality
• Datomic — Similar immutable model but closed-source and Clojure-only deployment
• TimescaleDB — Optimized for time-series metrics; XTDB handles arbitrary bitemporal facts
• EventStoreDB — Event sourcing without temporal queries; XTDB provides SQL over history
• TerminusDB — Git-like versioning; XTDB models time as first-class temporal dimensions

FAQ

Q: What is bitemporality and why does it matter? A: It tracks when something was true (valid time) separately from when the database recorded it (transaction time). This is required for audit compliance and enables corrections to historical data without losing the record of what was previously believed.

Q: Can I delete data for GDPR compliance? A: XTDB supports eviction operations that physically remove specific records while preserving the rest of the timeline.

Q: Is XTDB suitable for OLTP workloads? A: It handles transactional writes well but is optimized for analytical and temporal queries. High-throughput OLTP may benefit from a complementary system.

Q: What happened to Crux? A: XTDB v2 is the successor to Crux, rebuilt with a columnar engine and SQL support while keeping the same bitemporal principles.

Sources

• https://github.com/xtdb/xtdb
• https://docs.xtdb.com