LEANN: The Open-Source Vector Index That Shrinks a 60M-Chunk RAG Index from 201 GB to 6 GB

UC Berkeley's Sky Computing Lab has built LEANN, an open-source vector index that stores almost no embeddings. Instead of persisting vectors to disk, LEANN recomputes embeddings at query time. According to the repository benchmark, a Wikipedia index of 60 million chunks drops from 201 GB to just 6 GB. The underlying research paper won the Best Research Paper Award at MLSys 2026, and the result is striking: semantic search and retrieval-augmented generation (RAG) over personal data can now run on a laptop.

TL;DR

Storage: 6 GB for 60M chunks (repository benchmark); 4 GB in the paper's 76 GB Wikipedia setup. The default embedder is Contriever (110M parameters). LEANN is an index, not an LLM, so no context-length limit applies.
Runs on: laptops and workstations. The paper evaluated an RTX 4090 machine with 32 GB of RAM and an M1 Ultra Mac. A CPU-only Linux installation is available, though no CPU latency figures have been published.
Performance: matches HNSW downstream accuracy at 90% recall while using under 5% of raw-data storage.
Best case: 25.5% accuracy on Natural Questions (NQ), identical to HNSW, using 4 GB instead of 188 GB.
Worst case: HotpotQA retrieval takes 44.80 seconds on the Mac, accounting for 50.1% of total pipeline time.

What Is LEANN?

LEANN is a storage-efficient vector index that trades disk space for compute. Rather than storing every embedding, it keeps a compact index structure and recomputes embeddings on demand during search. This approach dramatically reduces the storage footprint of large-scale RAG systems, making it feasible to run semantic search over tens of millions of document chunks on consumer hardware.

Why It Matters in 2026

As RAG pipelines scale to enterprise and personal knowledge bases in the tens of millions of chunks, storage costs and hardware requirements have become major bottlenecks. LEANN's approach aligns with the broader 2026 trend toward compute-for-storage tradeoffs in AI infrastructure, where on-demand embedding generation and efficient indexing let teams deploy retrieval systems on laptops, workstations, and edge devices without sacrificing retrieval quality.

Benchmarks and Tradeoffs

LEANN delivers accuracy comparable to HNSW at 90% recall, but query latency varies significantly by task. On multi-hop retrieval benchmarks such as HotpotQA, recomputing embeddings at query time can dominate pipeline runtime. Teams should benchmark their specific workloads before adopting LEANN for latency-sensitive applications.

Getting Started

LEANN is open source and available on GitHub. The project supports installation on Linux, macOS, and Windows, with both GPU and CPU-only deployment options. The research paper and MLSys 2026 presentation provide additional detail on the architecture and evaluation methodology.

via MarkTechPost

LEANN: The Open-Source Vector Index That Shrinks a 60M-Chunk RAG