ObjectTwin: Measuring the Asset Correspondence Gap for Capture-Anchored Digital Twins
Author: Zhenghe Guo
Subject: Computer Vision and Pattern Recognition (cs.CV)
arXiv: 2610.35904 [cs.CV]
Submitted: 30 September 2026
DOI: https://doi.org/10.48550/arXiv.2610.35904
Abstract
Object-wise reconstruction increasingly inserts generated meshes into captured scenes as editable proxies for observed objects. This practice creates a requirement that isolated-asset scores and pose estimates do not capture: at the captured metric state, the proxy should reproduce calibrated observations of the same instance. We present ObjectTwin, a scan-grounded measurement protocol that freezes the instance, cameras, scan-projected support and recorded depth, metric anchor, and evaluation denominator, then replaces only the object representation. The resulting asset correspondence gap measures lost observation agreement relative to scan-derived visible geometry, without treating the scan as complete physical ground truth.
Across two indoor RGB-D capture domains and two generation backends, visible, anchored generated assets lose 0.302–0.497 MaskIoU under this replacement. Every tested scene-level estimate is positive, while same-input generator preference reverses across capture domains. Coverage and overlap-depth analyses locate the disagreement in both visible support and surface placement; the evaluated conservative pose refinement does not systematically recover it. These results establish capture correspondence as a separately measurable interface between generated-asset quality and stronger physical or task-level evaluation.
Key Contributions
- ObjectTwin protocol: A scan-grounded measurement framework that isolates the object representation while holding the capture context fixed.
- Asset correspondence gap: A new metric quantifying lost observation agreement between generated proxies and scan-derived visible geometry.
- Empirical findings: Generated assets lose 0.302–0.497 MaskIoU across two indoor RGB-D domains and two generation backends, with generator preference reversing between capture domains.
- Interface definition: Positions capture correspondence as a distinct, measurable layer between asset-level quality and physical or task-level evaluation.
Relevance in 2026
As generated 3D assets become standard editable proxies in scene reconstruction and digital-twin pipelines, measuring how faithfully those proxies reproduce captured observations—rather than only how plausible they look in isolation—has become a critical evaluation gap. ObjectTwin addresses this by formalizing capture correspondence as its own measurable interface.
via ArXiv CV