Batch-Conditioned Semantic Anchors Enable Robust Transductive

Paper Overview

Authors: Mohammed Rahman Sherif Khan Mohammad, Ardhendu Behera, Sandip Pradhan, Swagat Kumar, and Amr Ahmed

Venue: Accepted at NeurIPS 2026

Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

arXiv: 2611.06993 [cs.CV] — submitted 4 November 2026

Code: github.com/MR-Sherif/JPTA

Abstract

Transductive adaptation improves vision–language models at test time by refining predictions on an unlabeled batch, but current methods face a key trade-off. Aggressive approaches exploit batch structure but can drift when batches are sparse, exhibit prior shift, or cover only a small portion of the label space. Conservative methods better preserve CLIP text semantics, but fixed semantic references can under-adapt even when the batch provides strong corrective evidence.

To address this, the authors propose JPTA (Joint Prior and Transport Adaptation), a prior-aware framework that casts transductive VLM adaptation as batch-conditioned semantic reference estimation. Rather than treating CLIP text prototypes as fixed priors or as freely replaceable, JPTA views them as semantic references that shift toward image-side batch structure only under calibrated evidence. The method estimates batch priors and soft image-side prototypes to form transported semantic references, whose impact is scaled by batch reliability.

Across standard benchmarks, low effective-class regimes, all-class evaluation, and online streams, JPTA consistently outperforms TransCLIP and StatA, with the largest gains on sparse and prior-shifted batches. Anchor-drift, active-support, and absent-class-mass diagnostics show that JPTA resolves effective-class mismatch without inducing unconstrained transductive drift. The results indicate that robust transductive VLM adaptation should recalibrate semantic references using reliable batch evidence, rather than choosing between unrestricted transduction and fixed prototypes.

Key Contributions

• Batch-conditioned semantic references: CLIP text prototypes are treated as anchors that shift toward image-side batch structure only when calibrated evidence supports it.
• Reliability-scaled adaptation: The influence of transported references is weighted by estimated batch reliability, mitigating drift on sparse or prior-shifted batches.
• Strong empirical results: Consistent gains over TransCLIP and StatA across standard, low-effective-class, all-class, and online-stream settings.
• Diagnostic validation: Anchor-drift, active-support, and absent-class-mass analyses confirm that JPTA corrects effective-class mismatch without unconstrained transductive drift.

Citation

@inproceedings{mohammad2026jpta,
  title     = {Batch-Conditioned Semantic Anchors for Robust Transductive Adaptation of Vision--Language Models},
  author    = {Mohammad, Mohammed Rahman Sherif Khan and Behera, Ardhendu and Pradhan, Sandip and Kumar, Swagat and Ahmed, Amr},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026},
  note      = {arXiv:2611.06993 [cs.CV]}
}

via ArXiv CV