Position: Unlabeled Data Does Not Mean Unsupervised Visual Learning
Abstract
This position paper argues that the absence of labels does not imply the absence of human supervision in visual learning, and urges the research community to identify sources of supervision more explicitly. Many recent methods in computer vision build upon representations learned from large-scale unlabeled data, and are therefore grouped under the umbrella term “unsupervised.” However, different data curation schemes and training objectives embed substantially different human priors on which models rely, and we argue that one “unsupervised” umbrella term no longer captures these distinctions. This ambiguity makes it harder to compare unsupervised learning research conducted under different assumptions, coinciding with a sharp decline in papers titled with “unsupervised” in flagship computer vision conferences since 2021, despite continued growth of the field. While we fully embrace pre-training as a strong foundation for modern computer vision, we advocate for a community-level effort toward greater conceptual clarity: authors are encouraged to disclose priors in data selection and learning objectives, and to specify which components of a learning pipeline depend on which assumptions. Standardized disclosure practices can improve academic communication, ensure fairer comparisons, and preserve methodological diversity in unsupervised learning.
Introduction
The term “unsupervised learning” has long been used to describe methods that do not rely on explicit labels. In recent years, however, the field has shifted toward large-scale pre-training on unlabeled data, and this has blurred the lines between truly unsupervised approaches and those that are implicitly guided by human decisions. As the scope of computer vision expands, the reliance on unlabeled data has grown, yet the human involvement in shaping the data and the learning process remains significant—albeit subtle. This position paper contends that the presence of unlabeled data does not equate to the absence of human supervision, and that the current use of a single umbrella term “unsupervised” oversimplifies the landscape and hinders accurate comparison and communication among researchers.
Throughout this paper, we examine how different data curation schemes and training objectives introduce distinct human priors, which can significantly affect model behavior and performance. We also analyze the recent decline in papers explicitly using the term “unsupervised” in top computer vision venues, and we propose that this decline is a symptom of the term’s growing inadequacy. Our goal is to encourage the community to embrace a more nuanced view of supervision, and to adopt transparent disclosure practices that will improve the rigor and reproducibility of research in this area.
The Misleading Equation: Unlabeled = Unsupervised
A common assumption in visual learning is that if a dataset is not annotated with labels, then the training process is entirely free of human supervision. However, this overlooks the many ways in which human decisions shape the learning pipeline. For example, the selection of the dataset itself is a human choice—curators decide what content is included, what is excluded, and how the data are organized. Even in seemingly generic image collections, these curation choices reflect assumptions about which visual concepts matter and how they should be represented.
Moreover, the design of self-supervised learning objectives often incorporates human-defined transformation rules, augmentation strategies, and pretext tasks. For instance, contrastive learning relies on the assumption that different augmented views of the same image should produce similar representations—a prior that is not neutral. Similarly, the use of spatial, temporal, or color-based transformations embeds human knowledge about invariances that are important for the target task.
Thus, the lack of explicit labels does not mean that the learning process is free from human guidance. Instead, the supervision is simply encoded in the data selection and in the design of the learning objective. This reality calls for a more careful conceptualization of what we mean by “unsupervised,” and discourages oversimplified terminology.
The Rise and Decline of “Unsupervised” in Computer Vision
Despite the growing popularity of pre-training and self-supervised learning, the explicit use of the term “unsupervised” in titles of papers at major computer vision conferences has declined since 2021. We observe that this drop is not due to a decrease in the number of works that could be categorized as unsupervised; rather, it reflects the growing ambiguity of the term. Researchers increasingly adopt more specific descriptors, such as “self-supervised” or “pre-training,” to better communicate their methods, yet the umbrella term persists in many discussions.
This decline highlights the need for a richer terminology that can differentiate between various forms of implicit supervision. By adopting a more explicit framework, we can avoid misinterpretations and improve the comparability of research efforts.
A Call for Explicit Disclosure of Human Priors
To address these issues, we urge authors to disclose the human priors embedded in their data selection and learning objectives. This disclosure should outline which components of the pipeline rely on human assumptions, and how those assumptions affect the resulting representations. For example, a work that uses aggressive data augmentation to impose invariances should clearly state that choice and its rationale.
Standardized practices for such disclosure would offer multiple benefits:
- Improved Academic Communication: Clearer terminology and explicit documentation reduce misunderstandings and facilitate cross-paper comparison.
- Fairer Evaluation: Researchers can better assess the contributions of a method when the underlying priors are apparent, allowing a more accurate attribution of performance gains.
- Preserved Methodological Diversity: By valuing explicit disclosure, we also acknowledge that different priors can lead to diverse and legitimate approaches, rather than suppressing variety under a single label.
While we support pre-training as a cornerstone of modern computer vision, we firmly believe that a community-level push for conceptual clarity will strengthen the field. We encourage the introduction of a taxonomy that distinguishes between fully unsupervised, self-supervised, and human-conditioned methods—where the latter acknowledges the latent human guidance in data and objective design.
Conclusion
In summary, the absence of labels should not be mistaken for the absence of human supervision. We call for an inclusive yet precise discourse on the role of human priors in visual learning. By embracing transparency and clear terminology, we can ensure that future advances are built on sound and communicable foundations. The goal is not to diminish the value of unlabeled data; rather, it is to recognize and articulate the implicit choices that shape all learning methods, thereby enriching the field’s methodology and integrity.
via ArXiv CV