#distribution shift
Distribution Shift: 3 AI articles covering distribution shift news, analysis, and research
Articles
Evaluating Model Capabilities After Benchmark Saturationβ8
Saturated benchmark transcripts still reveal model capabilities, with pairwise judging predicting performance 5.3x better, though results can fail under distrib...
What Does 99% Accuracy Actually Measure? A Reproducible Audit ofβ9
A reproducible audit of the ISOT fake news corpus shows 99% accuracy measures source and topic shortcuts, not veracity, with style signals failing to transfer.
Does Marginal Coverage Guarantee Class-Conditional Safety forβ9
Marginal conformal coverage fails to ensure class-conditional safety for zero-shot VLMs under shift, with worst-class coverage near zero on ImageNet-Sketch desp...
