Baidu's ERNIE Evolve Automated Research System Outperforms NanoChat and NanoGPT Records

Baidu has released ERNIE Evolve, an automated research system designed to improve AI training and inference code. Given a repository, an objective, and evaluation criteria, the system identifies research directions, edits code, and runs experiments autonomously. Only changes that pass a hidden evaluator—along with a separate reviewer—are retained.

The Baidu team evaluated ERNIE Evolve across three public benchmarks, then applied it to optimize inference for ERNIE-5.5-Flash, a model still in development. The results position the system as a notable entrant in the emerging field of agentic, self-improving research pipelines—a space that has gained significant traction in 2026 as labs race to automate parts of the model development loop.

TL;DR

Size: ERNIE Evolve is an agent system; Baidu has not disclosed which LLMs power its agents.
Runs on: Benchmarks used 1× NVIDIA B200 (NanoChat, SOL-ExecBench) and 8× H100 (Speedrun Track 1). Production tests ran on Kunlunxin P800 and GPUs (model undisclosed).
Performance: Beat the best public result on 3 of 4 public metrics and cleared production checks on ERNIE-5.5-Flash.
Best result: Generation speed on Kunlunxin P800 reached 623.6% of baseline after 4 research rounds.
Worst result: SOL-ExecBench mean SOL of 0.819, behind the 0.827 leaderboard best.
Bottom line (best): Real production inference gains of +33.7% deterministic throughput and +14.9% decode speed.
Bottom line (worst): Results are self-reported, the system is not publicly released, and public-benchmark margins are small.

What Is ERNIE Evolve?

ERNIE Evolve operates as a closed-loop research agent. Rather than relying on a single LLM pass, it decomposes the task into a structured workflow: propose a research direction, modify the codebase, run experiments, and evaluate outcomes against a hidden evaluator. A separate reviewer stage acts as a second gate, filtering out changes that pass evaluation but fail on broader quality or robustness criteria.

This two-stage validation approach reflects a broader 2026 trend toward verified agentic workflows, where autonomous systems must satisfy independent checks before their outputs are trusted in production.

Benchmark Results

Baidu tested ERNIE Evolve on three public benchmarks:

NanoChat (1× NVIDIA B200)
SOL-ExecBench (1× NVIDIA B200)
Speedrun Track 1 (8× H100)

The system outperformed the best public result on three of four public metrics. Its weakest showing was on SOL-ExecBench, where it achieved a mean SOL of 0.819—just short of the 0.827 leaderboard best.

On NanoChat and NanoGPT-related records, ERNIE Evolve surpassed prior published results, though Baidu has not released full per-metric breakdowns or reproducibility artifacts.

Production Validation on ERNIE-5.5-Flash

Beyond public benchmarks, Baidu applied ERNIE Evolve to optimize inference for ERNIE-5.5-Flash, an in-development model. Production tests ran on Kunlunxin P800 hardware alongside GPUs.

The headline production outcomes:

Deterministic throughput: +33.7%
Decode speed: +14.9%
Generation speed on Kunlunxin P800: 623.6% of baseline after four research rounds

These are meaningful gains for a model still in development, and they suggest the system can deliver improvements that transfer beyond controlled benchmark settings.

Caveats and Open Questions

Several limitations temper the results:

Self-reported: All figures come from Baidu, with no independent replication.
Not released: The system itself is not publicly available, limiting external validation.
Narrow margins: Public-benchmark improvements over existing records are small (e.g., the SOL-ExecBench gap of 0.008).
Undisclosed agents: Baidu has not revealed which LLMs power the underlying agents, making it difficult to assess reproducibility or cost.

Why It Matters

ERNIE Evolve adds to a growing set of autonomous research systems—alongside efforts from other major labs—that aim to compress the human-in-the-loop cycle of model optimization. If such systems can reliably clear independent review and hidden evaluators, they could meaningfully accelerate training and inference engineering. For now, the ERNIE Evolve results are promising but require independent verification before broader conclusions can be drawn.

Source: Baidu ERNIE research release.

via MarkTechPost

Baidu's ERNIE Evolve Automated Research System Outperforms