Vishal Asnani

Vishal Asnani

Research Scientist at Adobe

Trustworthy AI · Watermarking · Media provenance

Research / 2023

PrObeD: Proactive Object Detection Wrapper

Vishal Asnani, Abhinav Kumar, Suya You, Xiaoming Liu

Published in Thirty-seventh Conference on Neural Information Processing Systems 2023, 2023

Read paper All publications
PrObeD generates an image-dependent template, applies it to the image, recovers the template, and fine-tunes an object detector.
An encoder-decoder wrapper learns task-relevant image changes alongside the object detector. Figure 2 from the NeurIPS 2023 proceedings paper. Select the figure to enlarge it.

Overview

Instead of only changing the detector, can we also learn a better input? PrObeD wraps object detectors with an image-dependent template generator, aiming to make useful scene information easier to exploit.

How it works

  • An encoder predicts a template for each image rather than sharing one fixed template across all scenes.
  • The template transforms the input; a decoder learns to recover it.
  • Joint training combines template objectives with the detector’s own task loss.

The framework is evaluated for both generic bounding-box detection and camouflaged-object segmentation.

A reported result

38.5 AP with PrObeD versus 37.3 AP without it for Faster R-CNN + FPN on the MS-COCO 2017 validation split, Table 2.

This AP averages over IoU thresholds from 0.50 to 0.95. It is not AP50 or classification accuracy. Camouflaged-object experiments use CAMO, COD10K, and NC4K with separate segmentation metrics.

Scope

The wrapper requires learned preprocessing and detector fine-tuning; it is not a claim that adding an arbitrary perturbation improves an unchanged detector.

Sources and resources

The result and protocol are detailed in Section 4 and Table 2.

Published paper Supplementary

Bibliographic reference

Asnani V., Kumar A., You S., Liu X., (2023). PrObeD: Proactive Object Detection Wrapper. In Thirty-seventh Conference on Neural Information Processing Systems.