# LEANN — Privacy-First RAG with 97% Storage Savings

> A local-first retrieval-augmented generation system that runs entirely on personal devices with dramatically reduced storage requirements and no cloud dependency.

## Install

Save in your project root:

# LEANN — Privacy-First RAG with 97% Storage Savings

## Quick Use
```bash
pip install leann
# Index your documents
leann index --dir ./my-docs
# Query locally
leann query "What does the Q3 report say about revenue?"
```

## Introduction
LEANN is a retrieval-augmented generation framework designed to run fast, accurate, and fully private RAG applications on personal devices. By rethinking how document embeddings are stored, it achieves up to 97% storage savings compared to traditional vector databases while maintaining high retrieval accuracy.

## What LEANN Does
- Indexes documents into a compressed local store optimized for consumer hardware
- Performs semantic search and retrieval without sending data to external servers
- Integrates with local LLMs via Ollama, llama.cpp, and other inference backends
- Supports PDF, Markdown, plain text, and structured data formats for ingestion
- Provides a Python API and CLI for building custom RAG pipelines

## Architecture Overview
LEANN replaces dense float-vector storage with a learned approximate nearest neighbor index that compresses embeddings into compact binary representations. During query time, it performs a two-stage retrieval: a fast approximate search over compressed codes followed by a precise re-ranking step. This lets it maintain recall comparable to FAISS or Chroma while using a fraction of the disk space.

## Self-Hosting & Configuration
- Install via pip; no external database or service dependencies required
- Configure embedding model, chunk size, and overlap in a YAML config file
- Point the indexer at any local directory or glob pattern to ingest documents
- Adjust compression ratio to trade off between storage savings and retrieval precision
- Deploy as a local HTTP server for multi-application access on the same machine

## Key Features
- 97% storage reduction compared to traditional vector databases like FAISS or Chroma
- Runs entirely offline with no data leaving the user's device
- Sub-100ms query latency on consumer laptops without GPU acceleration
- Incremental indexing that updates the store without full re-indexing
- Compatible with LangChain, LlamaIndex, and other orchestration frameworks

## Comparison with Similar Tools
- **FAISS** — high-performance vector search from Meta; requires significantly more storage than LEANN for equivalent recall
- **Chroma** — developer-friendly vector DB; cloud-optional but no built-in compression at LEANN's level
- **Qdrant** — full-featured vector database; better for multi-tenant production but heavier to self-host
- **txtai** — all-in-one embeddings database; broader feature set but larger resource footprint

## FAQ
**Q: How does LEANN achieve 97% storage savings?**
A: It uses learned approximate nearest neighbor techniques that compress dense float vectors into compact binary codes while preserving retrieval quality through a re-ranking step.

**Q: Does compression hurt retrieval accuracy?**
A: Minimally. Benchmarks show recall within 1-2% of uncompressed FAISS indices for most document types and query patterns.

**Q: What hardware do I need?**
A: Any modern laptop or desktop. LEANN is designed for consumer devices and does not require a GPU for indexing or retrieval.

**Q: Can I use LEANN with cloud LLMs?**
A: Yes. While retrieval stays local, you can route the retrieved context to any LLM API for generation if desired.

## Sources
- https://github.com/StarTrail-org/LEANN
- https://arxiv.org/abs/2505.12345

---
Source: https://tokrepo.com/en/workflows/asset-65fb2e96
Author: AI Open Source