Object removal is an ill‑posed, one‑to‑many task: many plausible restorations can fill the same hole, so there is no single ground truth to compare against. Traditional metrics such as PSNR, SSIM, LPIPS, ReMOVE and CFD assume point‑to‑point correspondence or rely on full‑frame statistics, which leads them to reward copy‑paste artifacts, ignore subtle shadows, and even improve when visual quality collapses. These shortcomings make it hard to trust benchmark results, tune models, or set reliable CI gates.
PROVE (Perceptual RemOVal cohErence) from MiLM Plus, Xiaomi Inc., closes this gap with two perception‑aligned scores—RC‑S for spatial coherence and RC‑T for temporal consistency—plus PROVE‑Bench, a two‑tier real‑world video benchmark. Both metrics operate locally on DINOv2 features using a sliding‑window Maximum Mean Discrepancy, need only a mask (white pixels mark the removed object), and require no reference video. RC‑S expands each mask box by one‑third, crops the region, extracts features, and averages MMD over windows that intersect the mask. RC‑T crops adjacent frames with the union of their masks and measures drift only in the intersection, ensuring temporal sensitivity where full‑frame metrics fail.
The evaluation harness ships as an Apache 2.0 PyTorch repository with a single CLI entry point (run_prove_metrics.py). It runs on Python 3.10+, PyTorch 2.6+, Transformers 4.51+, and DINOv2‑giant weights. On an RTX 4090, RC‑S processes ~134 ms per frame, making nightly CI checks feasible for startups and valuable for large editing catalogs in enterprises. Ideal users include smartphone gallery and camera apps, short‑video editors, e‑commerce catalog cleanup, advertising/stock media, film/VFX post‑production, real‑estate imagery, and privacy redaction in mapping archives. Primary adopters are CV researchers and applied AI/ML engineers, followed by MLOps wiring the score into CI and product managers comparing vendors.
PROVE does not fit real‑time on‑device scoring or cases where large shadows/reflections extend beyond the cropped evaluation region. By providing a reference‑free, perceptually grounded evaluation, PROVE enables reliable model bake‑offs, quantization tuning, training‑data filtering, and reward‑model shaping without the misleading bias of existing metrics.
#AI #Product #ComputerVision #MLOps #Evaluation #ObjectRemoval