[2Q3-J-2] Machine learning: explainability, knowledge acquisition

[2Q3-J-2-02] Visualization of cluster structure transition in deep learning

〇Chihiro Watanabe¹ (1. NTT Communication Science Laboratories)

Keywords:deep learning, interpretability, hierarchical clustering

Deep neural networks have achieved high performance in various tasks, and to interpret their prediction mechanism is an important open problem. Recently, a series of methods have been proposed for decomposing a trained neural network into a simple interpretable module structure. In this paper, to acquire knowledge about the training process of a deep neural network, we proposed a method for visualizing the cluster structure transition during the training phase. Our proposed framework consists of two parts: we first decompose the network at each training step into modules based on hierarchical clustering, and then reveal the relationships between clusters at different training steps based on the ratio of their common units. The experimental results showed that our proposed method could provide us with knowledge about division and integration of neural network modules, and also information about the role of each module in terms of input-output mappings.