신경망 근사에 의한 다중 레이어의 클래스 활성화 맵을 이용한 블랙박스 모델의 시각적 설명 기법 = Visual Explanation of Black-box Models Using Layer-wise Class Activation Maps from Approximating Neural Networks

다국어 초록 (Multilingual Abstract)

In this paper, we propose a novel visualization technique to explain the predictions of deep neural networks. We use knowledge distillation (KD) to identify the interior of a black-box model for which we know only inputs and outputs. The information of the black box model will be transferred to a white box model that we aim to create through the KD. The white box model will learn the representation of the black-box model. Second, the white-box model generates attention maps for each of its layers using Grad-CAM. Then we combine the attention maps of different layers using the pixel-wise summation to generate a final saliency map that contains information from all layers of the model. The experiments show that the proposed technique found important layers and explained which part of the input is important. Saliency maps generated by the proposed technique performed better than those of Grad-CAM in deletion game.

참고문헌 (Reference)

1 M. T. Ribeiro, "Why Should I trust you? : Explaining the Predictions of any Classifier" 1135-1144, 2016

2 C. Wah, "The Caltech-ucsd Birds-200-2011 dataset" 2011

3 J. Hu, "Squeeze-and-excitation Networks" 7132-7141, 2018

4 L. Edwards, "Slave to the Algorithm: Why a Right to an Explanation is Probably not the Remedy you are Looking for" 16 (16): 2017

5 V. Petsiuk, "Rise : Randomized Input Sampling for Explanation of black-box models"

6 F. Wang, "Residual Attention Network for Image Classification" 3156-3164, 2017

7 X. Wang, "Non-local Neural Networks" 7794-7803, 2018

8 B. Zhou, "Learning Deep Features for Discriminative Localization" 2921-2929, 2016

9 C. Fong, "Interpretable Explanations of Black Boxes by Meaningful Perturbation" 3429-3437, 2017

10 J. Deng, "Imagenet : A Large-scale Hierarchical Image Database" 248-255, 2009

11 R. R. Selvaraju, "Grad-cam : Visual Explanations from Deep Networks via Gradient-based Localization" 618-626, 2017

12 S. Maji, "Fine-grained Visual Classification of Aircraft"

13 G. Hinton, "Distilling the Knowledge in a Neural Network"

14 Sample, Ian, "Computer says no: Why Mmaking AIs fair, Accountable and Transparent is Crucial"

15 S. H. Woo, "Cbam : Convolutional Block Attention Module" 3-19, 2018

16 A. Vaswani, "Attention is all you Need" 5998-6008, 2017

17 A. Holzinger, "A Glass-box Interactive Machine Learning Approach for Solving NP-hard Problems with the Human-in-the-loop"

18 J. Krause, "3d Object Representations for Fine-grained Categorization" 554-561, 2013