Article Title: Can Vision-Language Models Assess Proxemic Risk from Egocentric Robot Images?
Authors: Vladyslava Rudas, Dmytro Kuzmenko
Publication: arXiv:2608.12515 [cs.CV], submitted August 12, 2026. Accepted at the EMR 2026 workshop, ECCV 2026 (non-archival).
Abstract
Assessing proxemic danger from a robot's egocentric perspective is essential for safe embodied navigation in human environments, demanding both visual and contextual understanding. In this study, we evaluate three open-source vision-language models (VLMs)—InternVL, Qwen-VL, and SmolVLM—on the task of classifying egocentric robot images into four danger levels. We compare three prompting strategies and two rounds of QLoRA fine-tuning against a stratified random baseline. Without fine-tuning, all models perform near the baseline; fine-tuning yields only modest overall improvements. However, Qwen-VL with an advanced prompt achieves significantly higher recall for high-danger cases than the other models. An analysis of person localization reveals that correct danger classification does not correlate with better spatial grounding, suggesting that a model may produce a useful safety label without attending to the relevant scene region. These results indicate that current VLMs remain limited in fine-grained proxemic reasoning and spatial grounding, although targeted prompting and fine-tuning can improve high-danger detection in selected models.
1. Introduction
As robots increasingly operate in human-inhabited environments, their ability to perceive and judge the risk of physical proximity—termed proxemic risk—becomes paramount. This capability requires integrating visual cues with contextual reasoning about the scene. Recent advances in vision-language models (VLMs) present a promising avenue for such reasoning, yet their practical utility in egocentric robot perception remains largely unexplored.
In this paper, we ask: Can current VLMs reliably assess proxemic danger from a robot's first-person view? To answer this, we benchmark three open-source VLMs—InternVL, Qwen-VL, and SmolVLM—under various prompting and fine-tuning configurations. We also investigate whether these models achieve spatial grounding—i.e., whether they actually attend to the regions driving their safety classifications.
2. Methods
We use a dataset of egocentric robot images, each labeled with one of four danger levels (e.g., safe, low risk, moderate risk, high risk).
- Models: InternVL, Qwen-VL, SmolVLM—all open-source and accessible for research.
- Prompts: Three strategies, ranging from simple instruction to advanced multi-step reasoning prompts (e.g., chain-of-thought).
- Fine-tuning: Two rounds of QLoRA (quantized low-rank adaptation) on a subset of the dataset.
- Baseline: A stratified random classifier, which predicts danger levels based on their frequency in the training set.
We evaluate classification accuracy, per-class recall (with emphasis on high-danger), and spatial grounding accuracy (via person localization tasks).
3. Results
3.1 Overall Classification Performance
Without fine-tuning, none of the models significantly outperformed the stratified random baseline, indicating a lack of intrinsic proxemic reasoning. After fine-tuning, all models improved, but the gains were modest—typically a few percentage points in overall accuracy.
3.2 High-Danger Detection
A notable exception was Qwen-VL with the advanced prompt, which achieved substantially higher recall for high-danger images (e.g., recall > 0.8) compared to the other models. This suggests that combining advanced prompting with a capable base model can enhance safety-critical detection.
3.3 Spatial Grounding
We performed person localization using the models' attention maps. Interestingly, higher classification accuracy did not correspond to better localization. In many cases, models that correctly flagged high-danger scenes failed to attend to the actual person, indicating that their decisions may be based on scene-level heuristics rather than precise spatial reasoning.
4. Discussion
Our findings highlight two key limitations of current VLMs for proxemic risk assessment:
- Fine-grained reasoning: VLMs struggle with subtle distinctions between danger levels, as evidenced by near-baseline performance without fine-tuning.
- Spatial grounding: The lack of correlation between correct classification and attention to relevant regions suggests that VLMs may not truly "understand" the spatial layout, relying instead on superficial correlations.
However, the success of Qwen-VL with advanced prompting indicates that targeted interventions can yield substantial gains in high-risk detection—a crucial requirement for real-world safety applications.
5. Conclusion and Future Work
This study demonstrates that while current VLMs have limited capabilities for fine-grained proxemic reasoning from egocentric images, strategic prompting and fine-tuning can improve high-danger detection in select models. Future work should explore larger model variants, multimodal sensor data (e.g., depth), and more robust spatial grounding techniques to move toward reliable deployment in robotic systems.
Cite as: arXiv:2608.12515 [cs.CV]
DOI: https://doi.org/10.48550/arXiv.2608.12515
via ArXiv CV