MoE Routing Compared: DeepSeek, Qwen, Mixtral, Llama 4, and gpt-oss in 2026
Open Mixture-of-Experts (MoE) models share a single core idea, yet they disagree on almost every routing detail. Expert counts range from 8 to 384, top-k values range from 1 to 8, and some models keep an always-on shared expert while others have dropped it entirely. As of 2026, the design space has matured β but it has also become harder to compare at a glance.
TL;DR
- Size: Ten models, from Qwen3.6-35B-A3B (35B total, 3B active) to DeepSeek-V4-Pro (1.6T total, 49B active).
- Deployment: gpt-oss-120b fits on a single 80 GB GPU with MXFP4; DeepSeek-V4-Pro and Kimi K2 require multi-GPU expert parallelism.
- Performance: Routed sparsity spans 4x in Mixtral (8 experts, top-2) to 128x in Llama 4 Maverick (128 experts, top-1).
- Best finding: Kimi K2 found that a sparsity of 48 required 1.69x fewer FLOPs than a sparsity of 8 at equal validation loss.
- Worst gap: Llama 4 Maverick and gpt-oss do not publish their gating or balancing method in
config.json. - Bottom line: Configs make routing fully auditable. The downside is that names like β17B activeβ obscure very different routers.
How do open MoE models route tokens?
Every model discussed below uses token-choice routing: each token selects its own experts. The models differ along five dimensions β expert count, experts per token (top-k), shared experts, the scoring function, and the load-balancing method. The table below lists each dimension per model.
MoE routing comparison table (November 2026)
| Model | Total / active params | Routed experts | Top-k | Shared experts | Gating | Load balancing |
| --- | --- | --- | --- | --- | --- | --- |
| (table data truncated in source) | | | | | | |
Note: The original comparison table was cut off in the source material. The editorial structure above is retained so the table can be completed with verified per-model data.
via MarkTechPost