MV2: A Multi-View Multi-Vehicle Driving Dataset for Novel View Synthesis
Authors: Sanjay Bhargav Dharavath, Hanvitha Saraswathi Mukkamala, Faizan Farooq Khan, Ioannis Kakogeorgiou, Aditya Arun, C V Jawahar, Zakaria Laskar
Subject: Computer Vision and Pattern Recognition (cs.CV)
Submitted: 12 August 2026 | Accepted: ECCV 2026 (18 pages, 7 figures) | arXiv: 2608.12442
Abstract
Differentiable rendering has significantly advanced novel view synthesis (NVS), yet its application to real-world driving scenarios remains challenging due to sparse capture viewpoints, dynamic objects, and a lack of multi-trajectory data. To address these limitations, we introduce the Multi-View Multi-Vehicle (MV2) dataset and benchmark for evaluating NVS models under substantial viewpoint changes in dynamic urban environments.
MV2 features synchronized captures from three platforms—a car, a scooter, and a drone—each following distinct but temporally aligned trajectories. By training NVS methods on one vehicle's camera stream and testing on another, our benchmark assesses performance under considerably larger viewpoint variations than existing single-trajectory datasets. All sequences are registered using Structure-from-Motion, with camera poses verified through manual pixel-level correspondence annotations. The dataset comprises 50 high-quality scenes with over 12,000 images.
Benchmarking recent NVS and camera pose estimation methods reveals that NVS performance degrades as viewpoint disparity increases, and that feed-forward pose estimators significantly lag behind optimization-based approaches. These findings position MV2 as a rigorous testbed for advancing NVS in driving contexts.
The dataset, benchmark protocol, and project resources are publicly available at https://mv2-dataset.github.io/.
1. Introduction
Novel view synthesis has seen rapid progress with the advent of differentiable rendering techniques, such as neural radiance fields (NeRFs) and 3D Gaussian splatting. However, applying these methods to autonomous driving introduces unique difficulties: vehicles operate in dynamic, uncontrolled environments, captured from limited camera viewpoints, and existing datasets typically provide only a single trajectory. This restricts the evaluation of NVS models to small viewpoint changes, failing to reflect real-world requirements where views may differ drastically—for instance, comparing a road-level camera with an aerial drone perspective.
To bridge this gap, we present MV2, a novel dataset designed specifically for multi-view, multi-vehicle driving scenes. Our contributions include:
- A new benchmark that evaluates NVS models under large viewpoint changes by cross-vehicle training and testing.
- A rich dataset containing 50 scenes with synchronized car, scooter, and drone footage, fully registered and pose-verified.
- Comprehensive baselines of state-of-the-art NVS and pose estimation methods, highlighting current limitations and future directions.
2. Dataset Design and Construction
MV2 was built to maximize viewpoint diversity while maintaining synchronization and accurate geometry. Each scene was captured simultaneously using three vehicles: a car with a roof-mounted camera, a scooter with a helmet-mounted camera, and a drone flying overhead. Trajectories were planned to cover overlapping areas from complementary angles, ensuring substantial pose differences between streams.
All images were processed using Structure-from-Motion (SfM) to estimate camera poses, which were then refined through manual pixel-level correspondence checks to ensure sub-pixel accuracy. This rigorous pipeline guarantees reliable ground truth for training and evaluation.
The final dataset includes:
- 50 scenes across urban and suburban environments.
- 12,000+ images total, with ~4,000 per vehicle type.
- Annotations for dynamic objects (e.g., vehicles, pedestrians) to facilitate future research on handling moving elements.
3. Benchmark Protocol
We propose a cross-view evaluation protocol: NVS models are trained on one vehicle's camera stream (source view) and evaluated on another vehicle's stream (target view). This setup forces models to generalize across large viewpoint changes, including altitude differences, occlusions, and lighting variations.
We benchmark several recent NVS methods (including NeRF-based and Gaussian splatting approaches) and camera pose estimators (both feed-forward and optimization-based). Metrics include PSNR, SSIM, and LPIPS for image quality, as well as pose accuracy metrics for localization.
4. Results and Findings
Our experiments reveal two key insights:
- Performance degradation with viewpoint disparity: All evaluated NVS methods show decreased quality as the viewpoint gap between training and testing increases. For instance, transferring from car to drone views results in significantly lower PSNR compared to car-to-car or scooter-to-car transitions.
- Feed-forward pose estimators lag behind: Optimization-based pose estimation methods consistently outperform feed-forward networks, especially under large baseline changes. This indicates a need for robust learning-based approaches that can handle extreme viewpoint shifts.
These findings underscore MV2's value as a challenging testbed, pushing the boundaries of current NVS and pose estimation research.
5. Conclusion and Future Work
MV2 addresses a critical gap in driving datasets by providing multi-trajectory, synchronized captures that enable rigorous NVS evaluation under realistic viewpoint changes. We hope this benchmark accelerates progress toward scalable, view-agnostic rendering systems for autonomous driving. Future extensions include adding more diverse weather conditions, nighttime scenes, and additional vehicle types.
Availability
The dataset, along with evaluation scripts and baseline models, is openly available at https://mv2-dataset.github.io/. We invite the community to use MV2 for benchmarking and further innovations.
Citation: Dharavath, S. B., Mukkamala, H. S., Khan, F. F., et al. (2026). MV2: Multi-View Multi-Vehicle Driving Dataset for Novel View Synthesis. arXiv:2608.12442.
via ArXiv CV