Is MoE Routing a Huffman Code? Discovering the Frequency-Diversity Law in Chain-of-Thought⭐7

Mixture-of-Experts routing follows a Frequency-Diversity Law, acting like neural Huffman coding. New Subset Difference Pruning eliminates redundancy, pushing Mo...

Knowledge Injection in MoE? Expert-Aware Contrast Decoding for Hallucination Mitigation in LLMs⭐7

Expert-aware contrast decoding leverages distinct expert activation patterns to mitigate LLM hallucinations in MoE models, outperforming baselines across benchm...