FinAbstain: Uncertainty-Calibrated Multimodal RAG for Selective Financial Forecasting

Large language models (LLMs) are capable of synthesizing financial narratives, yet they often express high confidence even when supporting evidence is sparse, stale, or contradictory. This issue is particularly critical in financial forecasting, where filings, news, prices, volume, and technical signals may conflict. Introduced in 2026, FinAbstain is a research framework that integrates uncertainty-calibrated multimodal retrieval-augmented generation (RAG) with selective prediction to address this challenge. The system employs a point-in-time retriever that only admits information publicly available at the forecast timestamp, supplying modality-specific evidence to a set of agents covering fundamentals, news, technicals, risk, and verification. Each agent produces probabilistic assessments, which are then aggregated using retrieval relevance, evidence contradiction, repeated-sample consistency, and historical calibration statistics. Four calibration techniques are evaluated under a unified chronological protocol: temperature scaling, isotonic regression, conformal prediction, and a novel hybrid uncertainty score. A controller component predicts bullish, bearish, or neutral outcomes only when uncertainty falls below a validated threshold; otherwise, it abstains, requests additional evidence, reduces exposure, or routes the case to human review. Evaluation spans one- and five-day abnormal-return direction, twenty-day volatility intervals, and abstention decisions, using metrics including accuracy, calibration, risk–coverage, citation, trading, latency, and cost. To ensure the design is auditable prior to full data collection, the paper reports explicitly labeled simulated results rather than empirical claims. These preliminary results illustrate the intended hypothesis: calibrated abstention can trade coverage for lower selective error and reduced drawdown. FinAbstain contributes a time-safe architecture, a composite uncertainty formulation, and a reproducible evaluation blueprint for evidence-grounded selective financial forecasting.

via ArXiv LG