ThinkJev: One Thought, Every Budget for Calibrated Typed Decisions
Authors: Xiaolu Ding, Kai Yang, Dongdong Liu
arXiv: 2610.37732 [cs.AI] · Submitted 27 Oct 2026
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
Comments: 31 pages, 7 figures, 10 tables
DOI: https://doi.org/10.48550/arXiv.2610.37732
Code and data generators: https://github.com/ding-xiaolu/ThinkJev
Overview
Many applications of language models culminate in a typed decision: a selection among a closed set of options whose output probabilities downstream code can act on directly. Extracting those probabilities from a single forward pass is fast—milliseconds per decision—but frequently wrong on multi-step problems such as arithmetic, counting, and date comparison. Allowing the model to "think" first corrects most of these errors, but at a cost of seconds rather than milliseconds per decision.
ThinkJev addresses this trade-off by paying for reasoning only where it demonstrably helps. The method builds a fast and a slow system from a single open hybrid-thinking model that can either answer directly or after a chain of thought.
Method
- System 1 reads the model's probabilities over the option letters when it answers directly.
- System 2 reads the same letters after the model has thought on the same cached prompt. Because escalation reuses the cache, no work is repeated, and both systems yield probabilities over an identical option set.
- Since the option letters can be read after any number of thought tokens, a single sampled thought simultaneously yields the answer under every thinking budget—hence "one thought, every budget."
Crucially, ThinkJev requires no training. System 1 is permitted to answer alone only where a statistical test on labeled development data guarantees that its accepted answers stay below a target error rate.
Results
At a 5% target error rate, Qwen3.5-27B answers 24.5% of 8,359 test decisions via System 1 alone, with a realized error of 3.9%. The smaller Qwen3.5-4B and 9B models answer 6.5% and 0.0% of decisions this way, respectively. For the remaining decisions, terminating each thought as soon as its answer becomes confident reaches 81.6% accuracy, compared with 82.7% for always thinking—while consuming only 72% of the thought tokens.
Selecting when to think from System 1's confidence alone saves less, because the decisions System 1 is least sure about are often not the ones that thinking actually fixes.
Benchmark Evaluation
On the public items of JevBench, an independent benchmark of typed-decision services, Qwen3.5-27B invokes thinking on 29.4% of decisions and is not statistically distinguishable in accuracy from the two thinking APIs listed (exploratory).
Availability
Code and data generators are publicly available at https://github.com/ding-xiaolu/ThinkJev.
via ArXiv AI