Overview
A new arXiv preprint (arXiv:2610.37678, submitted 12 October 2026) by Varun Kotte examines how to choose a partner model for selective span tagging. A span tagger that ranks its outputs by risk can defer the riskiest cases to a human, and a second model can sharpen that ranking. The central question is which second model to choose.
Key Insight: Partner Selection Is Conditional, Not Global
The paper argues that the best second opinion is the model that is right precisely where the primary model is wrong. Across 576 evaluated primary–partner pairings—covering eight tagger types, three seeds, and three English benchmarks—selecting the partner by held-out accuracy on the primary's errors recovered 82% of the best partner's gain. By contrast, selecting by span F1 or a random pick recovered only 69–70%.
Replication and Directionality
- Replication: The selection rule was frozen before a fourth dataset was chosen and replicated there (Spearman correlation of 0.40, versus 0.04 for span F1).
- Directionality: Partner value is also directional. The predictive distribution serves as the second opinion: hard labels discard roughly half of a partner's value, and scoring the served label improved 94% of pairings. This scoring method was fixed before the fourth dataset was selected and also replicated.
- LoRA-tuned decoders: For LoRA-tuned decoders, independently trained partners outperform every readout of the primary network.
- Cross-locale transfer: Applied zero-shot to 51 MASSIVE locales, every primary model had a helpful trained partner in every locale, and the preferred partner tracked competence.
Metadata
- Authors: Varun Kotte
- Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
- Cite as: arXiv:2610.37678 [cs.CL]
- DOI: https://doi.org/10.48550/arXiv.2610.37678
- Submission History: v1 — Mon, 12 Oct 2026, 03:01:35 UTC (602 KB)
2026 Context
As selective prediction and human-in-the-loop deferral become standard components of deployed NLP pipelines, the choice of a secondary model has grown into a distinct design decision. This work contributes a simple, empirically validated selection rule—measuring a partner on the primary's errors rather than on aggregate benchmark performance—alongside evidence that partner value is directional and transfers across locales.
via ArXiv CL+LG