Black-box Explanation of Object Detectors via Saliency Maps

News Release Summary

Researchers from Boston University and Adobe Research have developed a new technique called D-RISE that can produce visual explanations for why an object detection system makes the predictions it does — something that has proven surprisingly difficult to do well with existing tools. Object detectors are widely used in computer vision applications but are notoriously hard to interpret, and prior explanation methods were designed for simpler image classification tasks, making them poorly suited for detection systems that must simultaneously identify what an object is and where it sits in a scene. D-RISE works by generating thousands of randomly masked versions of an input image, feeding each through the detector, and then measuring how much each masked region influenced the detector's output using a custom similarity metric that accounts for both object location and category. The weighted combination of those masks produces a heatmap showing which parts of the image drove a particular detection decision. In testing on the MS-COCO benchmark dataset using two popular detector architectures — the one-stage YOLOv3 and the two-stage Faster R-CNN — D-RISE outperformed gradient-based baselines on standard evaluation metrics. The method also surfaced some interesting behaviors: detectors frequently rely on context outside an object's bounding box, such as using ski poles to help identify skis, and tend to focus on specific discriminative parts rather than whole objects. In a controlled experiment, the researchers deliberately introduced artificial biases into training data and confirmed that D-RISE could reliably flag those biases in the resulting model. Because the method treats the detector as a black box requiring no access to internal weights or gradients, it can in principle be applied to any detection system, including proprietary ones.