R2VC: Modular Fact-Checking with Retrieval, Verification, and Confidence Calibration

Authors: Dhruv Dixit, Paritosh Pandey

Submitted: 6 Aug 2026

arXiv: 2609.11955 [cs.CL]

Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)

Comments: 21 pages, 15 figures

DOI: 10.48550/arXiv.2609.11955

Abstract

Large language models are increasingly used for automated fact-checking, but end-to-end prompting often entangles evidence retrieval, reasoning, and uncertainty estimation, making failures difficult to diagnose and confidence difficult to trust. We present R2VC, a modular retrieve–reason–verify–calibrate architecture for evidence-grounded fact-checking with citations and abstention.

R2VC combines:

Hybrid sparse + dense retrieval over Wikipedia
A supervised fine-tuned and DPO-aligned generator that produces diverse structured verdict candidates
An external NLI cross-encoder for evidence-based candidate selection
A lightweight sequence-level calibrator for confidence estimation and selective abstention

On FEVER, an 8B backbone with R2VC achieves 13.74% higher accuracy than the baseline. Ablation studies show that verifier-based candidate selection and confidence calibration are the largest contributors to performance:

Removing candidate selection drops FEVER accuracy to 76.24%
Removing calibration nearly doubles the Brier score to 0.161

A manual analysis of 250 errors further shows that retrieval failures—especially wrong-entity evidence—remain the dominant bottleneck. Together, these results show that modular fact-checking pipelines can substantially improve both predictive accuracy and confidence reliability in open-domain verification.

Key Contributions

Modular architecture. R2VC disentangles retrieval, reasoning, verification, and calibration into distinct stages, enabling targeted diagnosis of failure modes in automated fact-checking—a critical requirement as LLM-based verification systems are increasingly deployed in high-stakes information environments in 2026.

Evidence-grounded verdicts. By producing structured verdict candidates and selecting among them with a natural language inference (NLI) cross-encoder, the system grounds each decision in retrieved evidence and supports citation.

Calibrated confidence and abstention. A sequence-level calibrator enables the model to express reliable confidence and to abstain selectively when evidence is insufficient—an essential capability given growing regulatory and platform-level demand for transparency in AI-generated claims assessment.

Empirical validation on FEVER. The pipeline delivers substantial accuracy gains and markedly improved calibration, with ablations isolating the contribution of each module.

Why This Matters in 2026

As large language models are embedded in search, newsroom tooling, and content-moderation pipelines, the ability to verify claims with traceable evidence, calibrated confidence, and principled abstention has become a baseline expectation rather than a research nicety. R2VC’s modular design responds directly to this need by making each stage of the verification process inspectable and independently improvable—an important property as organizations move from monolithic prompting toward production-grade verification systems.

Limitations and Future Work

The manual error analysis identifies retrieval failures—particularly cases in which evidence concerns the wrong entity—as the dominant remaining bottleneck. Future work may focus on entity-aware retrieval, query reformulation, and multi-hop evidence aggregation to address these failure modes.

Citation

@article{dixit2026r2vc,
  title={R2VC: Modular Fact-Checking with Retrieval, Verification, and Confidence Calibration},
  author={Dixit, Dhruv and Pandey, Paritosh},
  journal={arXiv preprint arXiv:2609.11955},
  year={2026}
}

via ArXiv CL+LG

R2VC: Modular Fact-Checking with Retrieval, Verification, and