[4C2-J-1] Fundamental AI, theory: brain-based design of intelligence

[4C2-J-1-02] Improving recognition performance of a deep neural network via integration with information representation in the brain

Improving recognition performance of a deep neural network via integration with information representation in the brain

〇Satoshi Nishida^1,2, Shinji Nishimoto^1,2 (1. National Institute of Information and Communications Technology, 2. Osaka University)

Keywords:Brain, Deep learning, Cognitive information, Pattern recognition, Vision

Deep learning has recently shown splendid performance in pattern-recognition tasks, such as object identification. However, even using the state-of-the-art deep neural network, it is still difficult to predict human subjective judgements, such as preferences or impressions, from sensory patterns. Here we investigate whether the performance of a deep neural network in such pattern recognition improves by integrating brain representations into deep-learning feature representations. The feature representations of visual inputs in a deep neural network are transformed into those in the brain via their association pre-learned from measured brain response. Then, the transformed representations are used to estimate human cognition induced by the visual inputs. We demonstrate that the estimation performance improves when the brain representations are integrated. Thus, brain data integration can provide an effective way to extend the general applicability of deep learning in the estimation of human subjective judgements.