Subliminal Learning in Vision-Language Models
Hidden preferences transfer through unrelated training data, with implications for how VLMs are trained in 2026 and beyond.
Overview
Subliminal learning is a phenomenon in large language models (LLMs) in which a teacher model passes a hidden preference to a student model through training data that is unrelated to that preference. In this work, researchers investigate whether subliminal learning also occurs in vision-language models (VLMs).
Methodology
The researchers fine-tune a teacher VLM to prefer a specific animal or facial attribute, using a control teacher drawn from the same base model. Both teachers generate numbers for an unrelated visual task involving geometric shapes. Two students, both initialized from the same base VLM, are then fine-tuned on the same images—one using numbers generated by the biased teacher and the other using numbers from the control teacher.
Key Findings
Across VLMs ranging from 2B to 7B parameters, the study finds that both animal and facial preferences can transfer through this unrelated task. Students trained on biased-teacher outputs select the target more often than control students, with a difference of up to approximately 10 percentage points in the animal preference task.
Transfer can become stronger with more training data and varies according to the target preference, the model, and how the numbers are generated. Beyond preferences, the biased student's explanations use more of the same words and phrases as the biased teacher's explanations than those of the control student—even when all three models select the same image.
Implications
These results matter for training VLMs. They show that training on unrelated data may change both a VLM's image preferences and how it explains its choices, and that this transfer can become stronger with more training samples.
Publication Details
- Authors: Arman Hatami, Romina Aalishah, Ilya E. Monosov
- Submitted: 4 November 2026
- Comments: Accepted to WACV 2027
- Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Image and Video Processing (eess.IV)
- Cite as: arXiv:2611.08538 [cs.CV]
- DOI: https://doi.org/10.48550/arXiv.2611.08538
via ArXiv CV