Overview
This 2026 study investigates the mechanisms underlying interactive improvement in multi-turn language agents, specifically examining whether natural-language feedback yields gains beyond the benefits of repeated attempts alone. The research is motivated by the observation that higher final accuracy in multi-turn settings may result from resampling, format correction, additional test-time computation, or genuine feedback-driven improvement—and aims to disentangle these factors.
Methodology
The authors introduce a controlled student-teacher protocol that is applied across four diverse benchmarks:
- Omni-MATH
- Codeforces
- BBEH Linguini
- ARC-AGI1
Thirteen open-weight models are evaluated in both student and teacher roles. The study compares:
- External feedback (from a separate teacher model)
- Self-feedback (student generates its own feedback)
- Unguided self-refinement (student revises without any feedback)
Variables manipulated include interaction history, task difficulty, and teacher access to privileged task information.
Key Findings
- Multi-turn improvement ≠ evidence of feedback use: Self-generated feedback yields minimal additional gains over unguided self-refinement, implying that improvement often stems from generic retry mechanisms rather than feedback quality.
- External teachers matter: The strongest external teachers produce substantially larger feedback-specific improvements, suggesting that truly useful feedback must convey guidance beyond what a generic retry provides.
- Student ability to use feedback is the dominant bottleneck: Dense student-teacher interaction matrices reveal that interactive gains are more strongly correlated with the student's capacity to act on feedback than with the teacher's identity alone—though teacher choice remains important for a fixed student.
- Evaluation baselines: Feedback-based agents should be benchmarked against repeated-attempt baselines to isolate genuine feedback effects.
- Bottleneck insights: The central bottleneck in interactive improvement is not the availability of feedback but the agent's ability to internalize and act on it—pointing to a need for better student-side inference mechanisms.
- Framework release: The authors release their controlled student-teacher evaluation framework at https://j-lojek.github.io/feedback-generation-is-a-bottleneck/.
- Accepted to: RLxF Workshop at ICML 2026
- Format: 9 pages, 7 figures
- Subjects: Artificial Intelligence (cs.AI)
- arXiv ID: 2606.30774
- Authors: Bartłomiej Cupiał, Jan Łojek, Mikołaj Garstecki, Szymon Pobłocki, Alicja Ziarko, Piotr Miłoś
Implications for 2026 and Beyond
These results carry significant implications for the design and evaluation of interactive AI systems in 2026:
Publication Details
via ArXiv AI