[4C1-GS-7] Vision, speech media processing

[4C1-GS-7-04] Object-wise Interpretability in CNNs following Shapley value

〇Kosei Serizawa¹, Noboru Murata¹, Shotaro Akaho² (1. Waseda University, 2. National Institute of Advanced Industrial Science and Technology)

[[Online]]

Keywords:Computer Vision, Interpretability

The black box nature of neural networks is sometimes an obstacle to their utilization. In the field of computer vision, a variety of methods are proposed for visualizing the important parts of inputs and interpreting the model behavior, in order to give interpretability to convolutional neural networks (CNNs) like human recognition. In this study, we propose a method of calculating and visualizing object-wise importance. The input image is first divided into some objects and visualize them based on the method of Neuron Groups. After that, the attribution of each object is calculated by applying Expected Gradients, following the concept of Shapley value in cooperative game theory. By numerical experiments with real-world images, effectiveness of the proposed method is shown by visualizing the objects in the image detected by the CNN. It is also confirmed that the attributions of objects have appropriate additivity like Shapley value.