#mixture-of-experts
Mixture-Of-Experts: 13 AI articles covering mixture-of-experts news, analysis, and research
Articles
Induction Labs Photon-1: A Single Pretraining Run Powers Desktop⭐7
Induction Labs debuts Photon-1, a 106B-parameter MoE model that learns from pure video observation. It powers desktop simulation, checkers, and billiard
Is MoE Routing a Huffman Code? Discovering the⭐7
Mixture-of-Experts routing follows a Frequency-Diversity Law, acting like neural Huffman coding. New Subset Difference Pruning eliminates redundancy,
Knowledge Injection in MoE? Expert-Aware Contrast Decoding for⭐7
Expert-aware contrast decoding leverages distinct expert activation patterns to mitigate LLM hallucinations in MoE models, outperforming baselines across benchm...