Gradient-Based Latent Decomposition Reveals Mechanisms of Feature Degradation in Weakly Supervised Mammography
Authors: Vinceline Bertrand, Ionut Cardei
Submitted: 24 July 2026
Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
arXiv ID: 2607.24835
Abstract
Weakly supervised hierarchical models exhibit a persistent asymmetry: while coarse lesion-type features are preserved under reconstruction, fine-grained malignancy cues degrade—a pattern with direct consequences for the clinical reliability of breast cancer screening pipelines. This paper introduces a gradient-based orthogonal latent decomposition for hierarchical Variational Autoencoders (H-VAEs) to mechanistically explain this asymmetry.
The latent space is partitioned into a task-aligned component (\( z1 \)), shaped by coarse supervisory gradients, and an orthogonal residual (\( z{\text{res}} \)) that captures remaining representational capacity. Using 3,550 mammographic Regions of Interest (ROIs) from the CBIS-DDSM dataset, the results show that only approximately 4.4% of latent magnitude aligns with supervisory gradients, leaving about 95.6% in the orthogonal residual. Fine-grained pathology prediction primarily depends on this residual subspace.
Key performance metrics include:
- Stage 1 AUC: 0.866
- Stage 2 AUC: 0.552
- Reconstruction stability gap: \( \Delta_{\text{diag}} = 5\% \) (p = 0.005)
- Classification gap: \( \Delta_{\text{AUC}} = 0.314 \) (p < 0.001)
Latent ablation studies confirm that features for both tasks reside heavily in \( z_{\text{res}} \), structurally explaining why reconstruction degrades pathology stability disproportionately. Comparisons with Multi-Instance Learning (MIL) and Multi-Task Learning (MTL) confirm generalization across architectures and modalities.
Findings
These findings reveal that in high-dimensional spaces, a single coarse supervisory signal isolates only a sparse 1D latent direction. This forces critical fine-grained features into a vulnerable residual subspace, leading to the observed feature degradation. The work provides a mechanistic explanation for a known weakness in weakly supervised mammography models and suggests pathways toward more robust clinical screening systems.
Subjects
- Primary: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Machine Learning (cs.LG)
Citation
Bertrand, V., & Cardei, I. (2026). Gradient-Based Latent Decomposition Reveals Mechanisms of Feature Degradation in Weakly Supervised Mammography. arXiv:2607.24835 [cs.CV]. https://doi.org/10.48550/arXiv.2607.24835
via ArXiv CV