Object removal models have advanced faster than the metrics used to evaluate them. While modern diffusion-based erasers convincingly reconstruct shadows, reflections, and occluded structures, traditional metrics such as PSNR, SSIM, LPIPS, ReMOVE, and CFD often rank outputs incorrectly. The core issue is structural: erasure is inherently ill-posed and one-to-many, meaning no single ground truth exists for comparison. A research team from MiLM Plus, Xiaomi Inc., has introduced PROVE (Perceptual RemOVal cohErence), accepted at ACM MM 2026, to address this gap. PROVE pairs two perception-aligned metrics—RC-S (spatial coherence) and RC-T (temporal consistency)—with PROVE-Bench, a two-tier real-world video benchmark. Both metrics evaluate the edited region locally using sliding-window Maximum Mean Discrepancy (MMD) over DINOv2 features, and neither requires a reference video.
Deployability and Practical Use
Yes—as an evaluation harness, not a product feature. PROVE is available as an Apache 2.0 PyTorch repository with a single command-line interface (runprovemetrics.py). It requires Python 3.10+, PyTorch 2.6+, Transformers 4.51+, and DINOv2-giant weights. Masks are mandatory, with white pixels marking the removed object.
- Company scale: Any team with a single GPU and a mask pipeline can adopt PROVE. RC-S processes frames at 134.6 ms/frame on an RTX 4090, enabling practical nightly CI gates for startups. Enterprises with large editing catalogs benefit most, since no paired ground truth is required.
- Industries: Smartphone gallery and camera apps, short-video editing platforms, e-commerce catalog cleanup, advertising and stock media, film and VFX post-production, real estate imagery, and privacy redaction in mapping archives.
- Applications: Model bake-offs, CI gates on RC-S and RC-T scores, tuning inference steps or quantization without reference videos, filtering training data, and shaping reward models.
- Primary users: Computer vision researchers and applied AI/ML engineers on editing teams are the first adopters; MLOps engineers integrating scores into CI follow; product managers conducting vendor comparisons come third.
- Limitations: PROVE is not designed for real-time on-device scoring, and side effects such as large shadows or reflections extending beyond the cropped evaluation region may not be fully captured.
via MarkTechPost