#ant group inclusionai

NVIDIA's Three-Tier Optimization Cuts MoE Inference Latency onNEW⭐8

NVIDIA stacks a new MoE kernel, FP8 output layer, and speculative decoding to boost Ling-3.0-flash decode throughput 42.5% on DGX Spark with no accuracy loss.