Alibaba's Qwen team has released Qwen3.8-Flash-Next, an open-weight multimodal Mixture-of-Experts (MoE) model optimized for cost per token. The checkpoint pairs a 125B backbone with a 51B N-gram embedding table and a 4B multi-token prediction module, all while activating only 6B parameters per token. The team positions this as an early preview of the architecture that will underpin Qwen4, mirroring the role Qwen3-Next played for Qwen3.5. This release introduces four key innovations: a hybrid Gated DeltaNet and Qwen Sparse Attention mechanism, Gated Residual connections, N-gram Embedding, and the Muon optimizer. According to the team, training costs are roughly one-ninth that of Qwen3.7-Plus.
Deployment Considerations
Yes, but not on a workstation. The FP8 checkpoint is 172.78 GiB, while the BF16 variant is 335.28 GiB. Per vLLM recipes, TP2 is the minimum validated FP8 configuration on GB300, with TP4 recommended. On an 8รH200 node, TEP8 is required; plain TP8 is incompatible with the checkpoint's 128-wide quantization blocks. Sparse activation reduces compute but not storage needs.
via MarkTechPost