[4H3-GS-11d] AIと社会：テキスト解析

[4H3-GS-11d-02] A Method for Inspecting Neural NLP Models Based on Text That Activates Neurons

〇Daisuke Oba¹, Naoki Yoshinaga², Masashi Toyoda² (1. The University of Tokyo, 2. Institute of Industrial Science, the University of Tokyo)

Keywords:Analysis of Neural Networks, BERT, Internal Representations, Corpus Search, Text Mining

In this study, as an approach to provide a deeper insight into neural Natural Language Processing (NLP) models, we propose a method for analyzing the roles of individual neurons, which is the finest component of the models, by observing sentences which strongly activate individual neurons. Our method retrieves the sentences from the massive corpora, and abstracts the sentences for interpretation using data-mining techniques. In the experiments, we demonstrate that our method can give thoughtful insights into what linguistic aspects each neuron of a given model captures as well as how multiple neurons relate to each other.