A Small Model Asks Better: Fine-Tuning Query Clarification for Retrieval-Augmented Generation

Authors: Lihong He, Lucian Popa, Marina Danilevsky

Submitted: 19 October 2026

arXiv: 2610.28686 [cs.AI]

DOI: https://doi.org/10.48550/arXiv.2610.28686

Abstract

Query clarification is essential for retrieval-augmented generation (RAG) systems that face ambiguous user queries. We consider multi-turn grounded conversational scenarios in which a user query is either ill-formed or ambiguous in the context of the prior dialog or—more interestingly—may be well-formed but, when posed against retrieved documents, admits multiple relevant interpretations. The task is to recognize both such cases and then generate an appropriate clarification request to the user.

We present ClariEdge, a new dataset of approximately 3,900 examples that include such ambiguous queries along with negative examples that do not require clarification. We also show that a lightweight fine-tuned LoRA achieves 96.6% classification accuracy and outperforms prompted baselines of many larger models on both classification and the quality of clarification generation.

Our study shows that hard negatives are an essential ingredient for separating queries that are truly ambiguous from queries that may look ambiguous but are not (i.e., can be resolved from context). The same LoRA achieves high performance in both pre-retrieval and post-retrieval scenarios; thus, it can be used in two places in a RAG pipeline as a check before answer generation.

The ClariEdge dataset is available at: https://huggingface.co/datasets/ibm-research/ClariEdge

Key Contributions

ClariEdge dataset: Approximately 3,900 examples covering ambiguous queries and negative examples, including hard negatives that require contextual disambiguation.
Lightweight fine-tuned LoRA: Achieves 96.6% classification accuracy and outperforms prompted baselines from many larger models on both classification and clarification-generation quality.
Dual-stage applicability: The same LoRA performs well in both pre-retrieval and post-retrieval settings, allowing it to serve as a clarification check at two points in a RAG pipeline before answer generation.
Role of hard negatives: Demonstrates that hard negatives are essential for distinguishing genuinely ambiguous queries from queries that only appear ambiguous but can be resolved from context.

Technical Details

Task: Recognize ambiguous or ill-formed queries in multi-turn grounded conversations and generate appropriate clarification requests.
Method: Lightweight fine-tuning with LoRA.
Primary results: 96.6% classification accuracy; stronger clarification quality than prompted larger-model baselines.
Pipeline placement: Usable as a pre-retrieval and post-retrieval check ahead of answer generation.

Subjects

Artificial Intelligence (cs.AI)

Citation

@article{he2026smallmodel,
  title   = {A Small Model Asks Better: Fine-Tuning Query Clarification for Retrieval-Augmented Generation},
  author  = {He, Lihong and Popa, Lucian and Danilevsky, Marina},
  journal = {arXiv preprint arXiv:2610.28686},
  year    = {2026},
  doi     = {10.48550/arXiv.2610.28686}
}

via ArXiv AI

A Small Model Asks Better: Fine-Tuning Query Clarification for