Large Language Models
Noisy Text in RAG: Typos, OCR, and the Gap Classical Spell-Check Leaves
Enterprise Document Intelligence [Vol.1 #B1] — Three sources of one problem. User typos, fast-typing transcription noise, OCR character errors. Classical spell-check handles one of them. Embeddings carry the rest.
By Kezhan Shi | August 30, 2026 | 28 min read
The user types “assurance décénale” and the document says “décennale.” One missing letter, and a literal search finds nothing. Real questions arrive with typos, and real documents have their own; before retrieval can match anything, someone has to fix the spelling on both sides.
This article is a bonus in Enterprise Document Intelligence, a series that builds an enterprise RAG system from four bricks. It tackles noisy text across the pipeline: user typos, fast-typing transcription noise, OCR character errors, what classical spell-check fixes, and what embeddings have to carry.
🧭 New to the series? Every article in this series sits on our two Towards Data Science author pages, Angela Shi and Kezhan Shi. That is the shortest way to see what is covered and where this one sits.
📓 Runnable companion notebooks are on GitHub: doc-intel/notebooks-vol1.
!The public companion-code repo at doc-intel/notebooks-vol1 - Image by author
The same problem shows up on both sides of a pipeline. On the question side, type “wat is teh covarge for fyre damge?” into a chatbot — and the intent is clear, but the spelling is a mess. On the document side, a scanned contract converted with OCR might render “indemnification” as “indemnificat10n” or “lien” as “1ien.” While a human can infer meaning from context, retrieval systems struggle.
Why Typos and OCR Errors Break Naive Retrieval
At its core, a RAG (Retrieval-Augmented Generation) system retrieves relevant document chunks based on a user query, then feeds those chunks to a language model to generate an answer. The retriever often relies on term-based or vector-based matching. Both are sensitive to noise:
- Term-based methods (e.g., BM25) rely on exact token matches. A typo like “décénale” instead of “décennale” breaks the match.
- Vector-based methods (embeddings) map text to high-dimensional semantic representations. While they are more robust to synonyms and paraphrases, they still suffer when character-level noise is severe, because word embeddings were typically trained on clean text.
In practice, noisy input can lead to:
- Lower retrieval recall: relevant documents are not fetched.
- Higher retrieval latency: system may need multiple retries.
- Wrong answers: the generation stage sees irrelevant context.
The Three Sources of Noise
1. User Typos
Human errors in typing: missing letters, swapped characters, extra characters, or phonetic substitutions. Examples: “wat” for “what,” “teh” for “the.” These are usually simple, context-independent, and often handled by classic spell-checkers.
2. Fast-Typing Transcription Noise
This appears when humans quickly transcribe spoken content or when automatic speech recognition (ASR) produces text. It may include:
- Homophones: “write” vs. “right”
- Dropped suffixes or incorrect tense
- Deleted or inserted small words
These mistakes can be trickier because they are often valid words in the wrong context.
3. OCR Character Errors
Optical character recognition (OCR) converts scanned documents or images into text. Common issues include:
- Confusing similar characters: ‘l’ vs. ‘1’, ‘O’ vs. ‘0’, ‘rn’ vs. ‘m’
- Reordering or splitting characters
- Noise artifacts from low-quality scans
OCR errors are systematic and can be modeled, but they are also document-specific — a font or scan quality difference changes the error pattern.
Classical Spell-Check: Strengths and Limits
Classical spell-checking (e.g., Levenshtein edit distance, Norvig’s probabilistic approach) works by comparing each word against a known dictionary and suggesting corrections based on edit distance or phonetic similarity.
What it handles well:
- Simple typos: “teh” → “the”
- Transposition or missing characters in isolated words
- Low-context mistakes where the intended word is unambiguous
What it misses:
- Context-dependent errors: “their” vs. “there” — both valid words, spell-check cannot choose.
- OCR errors that produce non-word but plausible substrings, like “rn” for “m.”
- Domain-specific jargon: technical terms may be flagged as misspelled only because they are not in a standard dictionary.
- Multilingual text: classical spell-checkers are typically monolingual.
Thus, for enterprise document processing where content is diverse and noisy, spell-check alone is not sufficient.
The Role of Embeddings and Dense Retrieval
Dense retrieval with embeddings (e.g., Sentence-BERT, BGE, OpenAI embeddings) maps queries and documents to a shared vector space. Even if the query has a typo, the embedding model may still produce a vector close to the correct query’s vector, because it uses subword tokenization and context.
However, embeddings have limits:
- If the typo changes the word into another real word (e.g., “form” for “from”), the embedding may drift.
- OCR errors that split or merge tokens can break subword tokenization assumptions.
- Very rare words with heavy noise may have unstable embeddings.
Therefore, a robust RAG pipeline should use a combination of:
- Preprocessing: spell-check and normalization (both query and documents)
- Hybrid retrieval: term-based + dense retrieval to maximize recall
- Fine-tuning embeddings on noisy data or using data augmentation
Practical Approaches to Noisy Text in RAG (2026 Update)
As of 2026, the ecosystem has shifted toward more context-aware and self-correcting solutions:
- LLM-based error correction: Large language models (like GPT-4 or Llama 3) can be prompted to fix typos and OCR errors, given context. This works well but adds latency and cost.
- On-the-fly query expansion: Instead of fixing the query, the system can generate multiple candidate corrected queries and retrieve for all.
- Character-aware embeddings: Some dense models are trained on noisy text or use character n-gram features, making them more robust.
- Hybrid retrieval with fuzzy matching: Use BM25 with fuzzy matching (e.g., allowing edit distance) alongside dense retrieval.
- Fine-tuning on domain noise: For enterprise domains (legal, insurance), collect typical OCR errors and typos, and fine-tune a small model or embeddings to be resilient.
Conclusion: Spell-Check Is a Start, Not the Finish
Noisy text in RAG is a multi-source problem. Classical spell-check fixes the easy cases. Embeddings cover some of the rest, but neither is sufficient alone. A production-grade enterprise RAG system in 2026 must adopt a layered approach: normalize both sides of the pipeline, use hybrid retrieval, and leverage LLMs as a fallback for ambiguous corrections. The gap left by classical spell-check is real — but it is exactly where modern AI techniques are closing the distance.
All images unless otherwise noted are by the author.
Published in Towards Data Science