What Do We Expect from LLMs? A Systematic Map of LLM Benchmark Design (2022–2026)

Author: Chao Wang (Independent Researcher)

Submitted: 15 September 2026

arXiv: 2609.19182 [cs.AI]

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

DOI: https://doi.org/10.48550/arXiv.2609.19182

Length: 15 pages, 5 figures, 7 tables

Data & Code: https://github.com/xxcg322/LLM-Bench-Map

Abstract

Benchmarks are central to how progress in large language models (LLMs) is assessed and communicated. Yet model rankings alone reveal little about how evaluation requirements themselves are changing. The expanding variety of benchmarks offers another perspective: what researchers expect LLMs to do, and what they count as successful performance.

We systematically map 14,767 papers introducing or updating evaluation resources from arXiv submissions between January 2022 and August 2026. Using staged screening and automated full-text coding, we examine changes in target systems and domains, evaluation materials and conditions, and scoring mechanisms.

The collection shows growing emphasis on action, interaction, and professional applications, while established and newer design elements frequently coexist. Model participation also develops unevenly: LLM-based scoring grows within both agent and non-agent groups, whereas model-generated materials show no comparable sustained increase in recent cohorts.

These findings illuminate how public research translates capability expectations into concrete tests and criteria for success. As AI participates in constructing tests, performing tasks, and judging responses, they also raise a question: does expanding evaluation provide more independent evidence, or risk reproducing the preferences and blind spots of its participating models?

Key Contributions

A large-scale, longitudinal map of 14,767 arXiv papers that introduce or update LLM evaluation resources, spanning January 2022 through August 2026.
A staged screening and automated full-text coding pipeline that tracks shifts across three design dimensions:
Target systems and domains — what is being evaluated and where.
Evaluation materials and conditions — what inputs and settings are used.
Scoring mechanisms — how success is defined and measured.
Empirical findings on the uneven growth of model participation in evaluation, distinguishing between LLM-based scoring and model-generated evaluation materials.

Findings at a Glance

Broadening scope over time. Benchmarks increasingly emphasize action, interaction, and professional applications, reflecting a shift toward evaluating agentic and real-world task performance rather than static knowledge recall alone.

Design elements accumulate rather than replace. Established conventions (e.g., fixed test sets, human annotation) and newer practices coexist, suggesting the field layers new requirements on top of prior ones instead of abandoning them.

Asymmetric model participation. LLM-based scoring (often referred to as LLM-as-a-judge) has grown steadily in both agent and non-agent benchmark groups. In contrast, model-generated evaluation materials — such as synthetic prompts, tasks, or datasets produced by LLMs — show no comparable sustained increase in recent cohorts.

Why This Matters in 2026

As of 2026, evaluation of frontier and open-weight LLMs has become a bottleneck for both research and deployment. Benchmarks increasingly serve as regulatory signals, procurement criteria, and public communication devices, not just internal research tools. This work interrogates a tension that has become central to the field: much of today's evaluation infrastructure already involves models at multiple points in the pipeline — generating items, performing tasks, and judging responses.

That raises a question with practical consequences for benchmark validity: does expanding evaluation provide more independent evidence about model capability, or does it risk reproducing the preferences and blind spots of the models participating in its construction? Answering this empirically, as this paper does, is a prerequisite for any trustworthy evaluation ecosystem going forward.

Citation

@misc{wang2026llmbenchmarks,
  title  = {What Do We Expect from LLMs? Mapping the Design of LLM Benchmarks},
  author = {Chao Wang},
  year   = {2026},
  eprint = {2609.19182},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  doi    = {10.48550/arXiv.2609.19182}
}

Resources

Paper: https://arxiv.org/abs/2609.19182
Data and code: https://github.com/xxcg322/LLM-Bench-Map

via ArXiv AI

What Do We Expect from LLMs? A Systematic Map of LLM Benchmark