# Quickwit — Cloud-Native Sub-Second Search Engine

> Quickwit is a cloud-native search engine built in Rust for log management and distributed search on object storage. It indexes data directly to S3-compatible stores, enabling cost-efficient search at petabyte scale.

## Install

Save as a script file and run:

# Quickwit — Cloud-Native Sub-Second Search Engine

## Quick Use
```bash
curl -L https://install.quickwit.io | sh
quickwit run
# Index and search via REST API on port 7280
```

## Introduction
Quickwit is a distributed search engine designed for logs, traces, and other append-only data. It decouples compute from storage by indexing directly to object storage like S3, making it significantly cheaper than Elasticsearch for large-scale observability workloads.

## What Quickwit Does
- Indexes structured and unstructured data to S3, MinIO, or local disk
- Provides sub-second full-text search over terabytes of log data
- Ingests data via REST API, Kafka, Kinesis, or Pulsar sources
- Supports native OpenTelemetry for logs and traces
- Offers a Jaeger-compatible gRPC endpoint for distributed tracing

## Architecture Overview
Quickwit uses a stateless indexer-searcher architecture where indexers create splits (immutable chunks of indexed data) and upload them to object storage. Searchers download only the metadata catalog and fetch relevant splits on demand. A control plane coordinates cluster membership, index scheduling, and shard routing. This design lets you scale compute independently from storage.

## Self-Hosting & Configuration
- Run a single binary or deploy via Docker / Helm chart on Kubernetes
- Configure data sources (Kafka, Kinesis, Pulsar, file) in YAML index configs
- Store indexes on S3, GCS, Azure Blob, or MinIO with a single storage URI
- Set retention policies to automatically delete old splits
- Enable the Jaeger gRPC endpoint for trace search integration with Grafana

## Key Features
- True compute-storage separation reduces cost for large datasets
- Tantivy-based indexing engine delivers fast full-text and columnar search
- Native OpenTelemetry support for logs and traces out of the box
- Schemaless or strict schema modes for flexible data modeling
- Multi-tenant with per-index access control

## Comparison with Similar Tools
- **Elasticsearch** — feature-rich but expensive at scale; Quickwit is 5-10x cheaper on object storage
- **Grafana Loki** — log aggregation focused on labels, not full-text; Quickwit offers richer search
- **OpenObserve** — Rust-based observability platform; Quickwit focuses on search with deeper Tantivy integration
- **ClickHouse** — columnar analytics DB; Quickwit is purpose-built for full-text log search
- **Zinc** — lightweight single-node search; Quickwit is distributed and cloud-native

## FAQ
**Q: Can Quickwit replace Elasticsearch?**
A: For log and trace search workloads, yes. It is not designed for application search with real-time updates.

**Q: What object storage backends are supported?**
A: Amazon S3, Google Cloud Storage, Azure Blob Storage, MinIO, and any S3-compatible service.

**Q: Does Quickwit support SQL queries?**
A: It supports a subset of SQL via its query language and is adding broader SQL support over time.

**Q: How does Quickwit handle high-cardinality fields?**
A: It uses columnar storage for fast aggregations on high-cardinality fields within splits.

## Sources
- https://github.com/quickwit-oss/quickwit
- https://quickwit.io/docs

---
Source: https://tokrepo.com/en/workflows/cd2966a8-3b63-11f1-9bc6-00163e2b0d79
Author: Script Depot