TDengine — High-Performance Time-Series Database for IoT

Introduction

TDengine is an open-source time-series database built in C, designed from scratch for IoT, telemetry, and industrial monitoring. It combines a database engine, caching, stream processing, and message queuing in a single package, eliminating the need to deploy Kafka, Redis, and a TSDB separately.

What TDengine Does

• Ingests billions of data points per day with microsecond timestamps
• Compresses time-series data 10x or more using column-oriented storage
• Runs SQL queries with built-in time-window functions (INTERVAL, SLIDING, FILL)
• Supports super-tables that group millions of devices under one schema with tags
• Includes built-in stream processing for real-time aggregation

Architecture Overview

TDengine uses a vnode-based architecture where each virtual node owns a subset of time-series data. Data is written append-only in column format with aggressive compression (delta-of-delta, simple-8b, and dictionary encoding). A management node coordinates metadata. Clustering is native — add nodes and TDengine redistributes vnodes automatically. The query engine pushes computation to data nodes for parallel execution.

Self-Hosting & Configuration

• Install via Docker, apt/yum packages, or compile from source on Linux/macOS
• Configure taosd.cfg for memory limits, WAL, and replication factor
• Use taosAdapter (port 6041) for RESTful and WebSocket client access
• Enable authentication with ALTER USER and per-database privileges
• Scale horizontally by adding data nodes to the cluster with CREATE DNODE

Key Features

• Super-table model groups millions of devices under a single schema with tags
• Built-in stream processing replaces external Flink/Spark for common aggregations
• Data retention policies automatically drop expired partitions
• Schemaless writing accepts InfluxDB Line Protocol, OpenTSDB, and JSON
• Native connectors for Python, Java, Go, Rust, Node.js, and C#

Comparison with Similar Tools

• InfluxDB — single-node OSS; TDengine scales natively with clustering
• TimescaleDB — Postgres extension; TDengine is purpose-built with higher ingest speed
• QuestDB — fast for analytics; TDengine adds built-in stream processing and caching
• ClickHouse — general OLAP; TDengine is optimized for time-series with super-table semantics
• Apache IoTDB — Java-based TSDBfor IoT; TDengine is written in C for lower resource usage

FAQ

Q: Can TDengine replace Kafka for IoT data? A: For many IoT pipelines yes. TDengine includes built-in message queuing and stream processing, so a separate Kafka layer is not needed.

Q: Does TDengine support SQL? A: Yes. TDengine uses a SQL dialect with extensions for time-series operations like INTERVAL, FILL, and PARTITION BY.

Q: How does the super-table model work? A: You define one super-table with a schema and tags. Each device gets a child table inheriting that schema. Queries across millions of child tables use tag-based filtering for speed.

Q: What are the hardware requirements? A: A single node can handle 100K+ inserts per second on modest hardware (4 cores, 8 GB RAM). Larger workloads scale linearly by adding nodes.

Sources

• https://github.com/taosdata/TDengine
• https://docs.tdengine.com