Leveraging Activations for Superpixel Explanations

• Main Authors: Boubekki, Ahcène, Fadel, Samuel G, Mair, Sebastian
• Format: Journal Article
• Language: English
• Published: 07-06-2024
• Subjects: Computer Science - Computer Vision and Pattern Recognition
• DOI: 10.48550/arxiv.2406.04933

Saliency methods have become standard in the explanation toolkit of deep
neural networks. Recent developments specific to image classifiers have
investigated region-based explanations with either new methods or by adapting
well-established ones using ad-hoc superpixel algorithms. In this paper, we aim
to avoid relying on these segmenters by extracting a segmentation from the
activations of a deep neural network image classifier without fine-tuning the
network. Our so-called Neuro-Activated Superpixels (NAS) can isolate the
regions of interest in the input relevant to the model's prediction, which
boosts high-threshold weakly supervised object localization performance. This
property enables the semi-supervised semantic evaluation of saliency methods.
The aggregation of NAS with existing saliency methods eases their
interpretation and reveals the inconsistencies of the widely used area under
the relevance curve metric.