Baidu Releases ERNIE 5.5 Flash Base: A 280B MoE Model With 14B Active Parameters

By Michal Sutter — November 5, 2026

Baidu has released ERNIE 5.5 Flash Base, the raw pre-training checkpoint of its new sparse Mixture-of-Experts (MoE) language model, published on Hugging Face. The model is a 280B-parameter text-only system that activates 14B parameters per token. Its weights stop at the end of pre-training, with no mid-training or post-training applied—positioning it as a clean starting point for teams that want to compare fine-tuning, instruction tuning, or reinforcement learning methods from a single, consistent checkpoint.

TL;DR

Size: 280B total parameters, 14B active per token; 128K training context, extendable to 1M for fine-tuning and inference.
Deployment: Runs on multi-GPU servers. Baidu's reference configurations use 4-way parallelism in SGLang and vLLM. GPU model and VRAM requirements are not disclosed. FP8 and BF16 weights are available.
Performance: Top score on 24 of 30 base-model benchmarks in Baidu's unified evaluation pipeline (self-reported).
Best result: MMLU-Pro at 78.4, ahead of DeepSeek-V4-Flash-Base at 69.2.
Weakest result: HELMET-Rerank at 56.2, behind DeepSeek-V4-Flash-Base at 60.0.

What Is ERNIE 5.5 Flash Base?

ERNIE 5.5 Flash Base is a text-only MoE base model built on 43 layers with a 4096 hidden size. As a base checkpoint, it is not instruction-tuned and is not intended for direct chat use out of the box. Instead, it is designed for researchers and engineering teams that want full control over the post-training pipeline—whether supervised fine-tuning, preference optimization, or RL—starting from a reproducible pre-training snapshot.

Architecture and Context Window

The model pairs a large total parameter count (280B) with sparse activation (14B per token), keeping inference costs closer to a mid-sized dense model while retaining the representational capacity of a much larger one. Training context is 128K tokens, with support for extension to 1M tokens during fine-tuning and inference.

Deployment and Precision

ERNIE 5.5 Flash Base targets multi-GPU serving. Baidu's reference configurations use 4-way parallelism in both SGLang and vLLM. Weights are published in FP8 and BF16. The company has not disclosed specific GPU models or minimum VRAM requirements.

Benchmark Performance

In Baidu's unified evaluation pipeline, ERNIE 5.5 Flash Base leads on 24 of 30 base-model benchmarks. The strongest showing is MMLU-Pro at 78.4, compared with 69.2 for DeepSeek-V4-Flash-Base. The weakest is HELMET-Rerank at 56.2, where DeepSeek-V4-Flash-Base scores 60.0. All results are self-reported by Baidu and should be validated independently for production use cases.

Why It Matters

Open base checkpoints remain relatively rare at this scale. By releasing the pre-training weights without post-training modifications, Baidu gives the research community a controlled baseline for comparing alignment and tuning techniques—an increasingly important need as teams evaluate which post-training stacks deliver the best returns on compute in 2026.

Getting Started

Model weights: Available on Hugging Face at baidu/ERNIE-5.5-Flash-Base.
Serving: Reference configurations provided for SGLang and vLLM with 4-way parallelism.
Formats: FP8 and BF16 checkpoints.

via MarkTechPost

Baidu Releases ERNIE 5.5 Flash Base: A 280B MoE Model With 14B