#mixture-of-experts
Mixture-Of-Experts: 27 AI articles covering mixture-of-experts news, analysis, and research
Articles
Aleph Alpha Releases Kolibri: A 78.1B Open-Weight English-German⭐9
Aleph Alpha releases Kolibri, a 78.1B open-weight English-German MoE model with 3.46B active parameters, 1M-token context, Apache 2.0 license, and single-GPU de...
Qualcomm Unveils Snapdragon 8 Elite Gen 6 Chips, Doubling Down⭐9
Qualcomm's Snapdragon 8 Elite Gen 6 chips bring on-device AI agents, local 30B-parameter models, and pro-grade 8K camera features to flagship smartphones.
StepFun Launches Step 5 Preview: A 600B-Total, 27B-Active MoE⭐8
StepFun's Step 5 Preview is a 600B-parameter MoE model with 27B active parameters and 1M context, built for long-horizon agentic work like coding and finance.
DeepSeek-V4.1-Flash Launches with 1M Context, FP4 KV Cache, and⭐9
DeepSeek-V4.1-Flash launches with 1M context, FP4 KV cache, and cross-layer attention reuse—cutting KV memory to 890 bytes per token for input-heavy agent workl...
GLM-5.3-Flash vs Qwen3.8-Flash-Next: Two Chinese AI Labs⭐8
Two Chinese labs independently release similar frontier MoE models: GLM-5.3-Flash and Qwen3.8-Flash-Next, sharing a hybrid attention recipe.
From Mixtral to Kimi K3: The Evolution of Mixture-of-Experts Models⭐8
From Mixtral to Kimi K3, explore Mixture-of-Experts evolution from 8 to 896 experts per layer, compression, and stability for trillions of parameters.
Z.ai Unveils GLM-5.3-Flash: A 320B-A18B Multimodal MoE Model⭐7
GLM-5.3-Flash: Z.ai's 320B MoE model, MIT-licensed, multimodal, 1M-token context, rivals Claude at 1/10 the cost.
NVIDIA Unveils Nemotron 3.5 Lightning: A 30B Open MoE with 3B⭐8
NVIDIA launches Nemotron 3.5 Lightning, a 30B open MoE with 3B active params, plus NeMo Switchyard router for faster, efficient AI agents.
TEXAS: Task-Expert-Aware Supervision for Downstream⭐9
TEXAS method improves MoE LLM adaptation by identifying task-relevant experts and optimizing token supervision, boosting performance across benchmarks.
Cursor Open-Sources Mixture-of-Kittens (MoK): A Deterministic⭐7
Cursor open-sources MoK, a deterministic MoE training megakernel for NVIDIA GB300 NVL72 racks. It boosts throughput up to 2.37x for Composer models.