I²-Med: A Training-Free Framework for Interpretable and Visually Grounded Medical Vision-Language Inference

arXiv:2610.23858 (cs.CV) — Submitted 25 June 2026

Authors: Md Sajid, Shreeyut Maheshwari, Akshat Mishra, Ritik Mishra, Mohammad Tanveer

Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

Abstract

Medical vision-language models (VLMs) have shown strong potential for medical image understanding and reasoning. However, their interpretability and reliability remain limited, because medical vision-language tasks demand precise visual grounding and clinically coherent explanations. These requirements are especially difficult to meet given complex imaging patterns and scarce expert annotations. As a result, conventional reasoning methods are less effective and are prone to visual hallucination—the generation of text that is not grounded in the given visual input—which further restricts their use in clinical practice.

To address these challenges, we propose I²-Med, an interpretable medical inference framework that enhances visual faithfulness during inference without additional training. During text generation, I²-Med uses a visual guidance module to check whether each candidate word is supported by the medical image and consistent with the current reasoning. Based on this guidance, the model's output scores are adjusted to favor visually grounded words, without changing the model parameters.

Extensive experiments on the OmniMedVQA benchmark, covering eight medical imaging modalities and five clinical question types, show that I²-Med generates more clinically meaningful and visually faithful reasoning chains and reduces visual hallucination under standard medical VQA evaluation metrics. The source code is publicly available at https://github.com/mtanveer1/l2-med.

Key Contributions

• Training-free inference-time calibration. I²-Med improves reasoning and explanation faithfulness without fine-tuning or modifying model weights.
• Visual-guided dynamic logits calibration. A visual guidance module evaluates each candidate word during generation for image support and reasoning consistency, then adjusts output scores to favor visually grounded tokens.
• Reduced visual hallucination. The framework mitigates the generation of text unsupported by the input image, a key barrier to clinical deployment.
• Strong empirical results. Evaluated across eight medical imaging modalities and five clinical question types on OmniMedVQA, using standard medical VQA metrics.

Why It Matters in 2026

As medical VLMs move from research prototypes toward clinical decision support, trustworthiness has become the central bottleneck. Regulatory frameworks and hospital deployment pipelines increasingly require auditable reasoning and verifiable image grounding—not just benchmark accuracy. Training-free, plug-in calibration methods like I²-Med are attractive because they can be layered onto existing models without costly retraining or re-annotation, and they align with the broader 2026 shift toward inference-time reliability engineering for multimodal foundation models.

Citation

@article{sajid2026i2med,
  title   = {I$^2$-Med: Interpretable Medical Inference through Visual-Guided Dynamic Logits Calibration},
  author  = {Sajid, Md and Maheshwari, Shreeyut and Mishra, Akshat and Mishra, Ritik and Tanveer, Mohammad},
  journal = {arXiv preprint arXiv:2610.23858},
  year    = {2026},
  doi     = {10.48550/arXiv.2610.23858}
}

via ArXiv CV