#knowledge distillation
Knowledge Distillation: 3 AI articles covering knowledge distillation news, analysis, and research
Articles
Zyphra AI Introduces Zyphra Hierarchical Attention (ZHA): A PathNEW⭐8
Zyphra's Hierarchical Attention (ZHA) adds a tree-structured indexer for true subquadratic sparse attention, matching dense performance while attending to ~1% o...
Improving OCR Faithfulness via Gated and Attenuated On-Policy⭐8
GAD-RL adaptively gates and attenuates teacher supervision during on-policy distillation, improving OCR faithfulness and achieving strong CHAOS-Bench and OmniDo...
Hands-On Evolution of Deep Learning: Geoffrey Hinton's AI Legacy⭐8
Explore Geoffrey Hinton's AI legacy in this 27-hour freeCodeCamp course, recreating deep learning breakthroughs like AlexNet, Dropout, and t-SNE in PyTorch.