# Fluent Bit — Lightweight High-Performance Log and Metrics Processor

> Fluent Bit is a fast, lightweight telemetry agent from the Fluentd family. It collects logs, metrics and traces from any source, processes them with filters, and forwards them to dozens of backends.

## Install

Save in your project root:

# Fluent Bit — Lightweight Log and Metrics Processor

## Quick Use
```bash
# Run a container that tails stdin and writes to stdout
docker run --rm -it fluent/fluent-bit:latest \
  /fluent-bit/bin/fluent-bit -i stdin -o stdout

# Minimal config file
cat > fluent-bit.conf <<EOF
[INPUT]
  Name tail
  Path /var/log/containers/*.log
  Parser cri
[OUTPUT]
  Name loki
  Match *
  Host loki.example.com
  Port 3100
EOF

fluent-bit -c fluent-bit.conf
```

## Introduction
Fluent Bit is the successor to Fluentd for environments where a small footprint matters most: Kubernetes DaemonSets, embedded devices and edge gateways. Written in C with a tiny runtime footprint (roughly 1 MB binary, a few MB RAM), it speaks the Fluent protocol, OpenTelemetry, Prometheus and dozens of native outputs.

## What Fluent Bit Does
- Tails files, listens on TCP/UDP, scrapes Prometheus endpoints or reads from systemd
- Parses logs with built-in and user-defined parsers (JSON, regex, multiline, CRI)
- Enriches records with Kubernetes metadata, GeoIP, Lua scripts and WASM filters
- Forwards to Fluentd, Loki, Elastic, OpenSearch, Kafka, S3, Datadog, Splunk, NATS, etc.
- Exposes OpenTelemetry traces and metrics over OTLP

## Architecture Overview
A Fluent Bit pipeline has four stages: Inputs → Parsers → Filters → Outputs. Records flow through an event engine backed by Chunk I/O, which uses memory and disk buffers to guarantee durability under backpressure. The single-process runtime uses an async I/O loop with minimal locking, so a single Pod can handle hundreds of thousands of events per second. Plugins live in the same process; you can also compile community plugins or write your own in Go/C.

## Self-Hosting & Configuration
- Deploy as a DaemonSet on Kubernetes using the official Helm chart
- Mount `/var/log`, `/var/lib/docker/containers` read-only to tail container logs
- Use the `kubernetes` filter to attach Pod labels, annotations and container metadata
- Configure `Mem_Buf_Limit` and filesystem storage to prevent unbounded growth
- Use `Hot Reload` (SIGHUP) to update the config without restarts

## Key Features
- Tiny footprint compared with JVM-based agents
- Native Kubernetes metadata filter and CRI parser
- 100+ plugins spanning log, metric and trace backends
- Disk-backed buffers for at-least-once delivery
- Stream processing queries over the event stream

## Comparison with Similar Tools
- **Fluentd** — Ruby-based predecessor; heavier, richer plugin ecosystem
- **Logstash** — JVM-based, powerful filters, but memory-hungry
- **Vector** — Rust agent with similar goals; strong observability story
- **Promtail** — Loki-focused only; Fluent Bit ships to more backends
- **rsyslog / syslog-ng** — great for classic Unix logging, weaker on cloud outputs

## FAQ
**Q:** Is Fluent Bit a replacement for Fluentd?
A: For edge/agent scenarios yes. Many deployments still use Fluent Bit as a forwarder and Fluentd as an aggregator for complex pipelines.

**Q:** Can Fluent Bit handle metrics and traces?
A: Yes, via OpenTelemetry OTLP inputs/outputs and Prometheus scrape support. Full OTel parity is ongoing.

**Q:** How do multiline logs work?
A: Use the built-in `multiline` filter or `Multiline.Parser`, with presets for Java, Python, Go, Ruby and Kubernetes CRI logs.

**Q:** What about backpressure?
A: Configure filesystem buffering and `storage.type filesystem` so events persist on disk until the output catches up.

## Sources
- https://github.com/fluent/fluent-bit
- https://docs.fluentbit.io

---
Source: https://tokrepo.com/en/workflows/18438936-38e7-11f1-9bc6-00163e2b0d79
Author: AI Open Source