[3Yin2] Interactive session 1

[3Yin2-06] Estimation of Brain Activity evoked Linguistic Stimuli utilizing the General-Purpose Language Model:BrainBERT

〇Ying LUO¹, Ichiro Kobayashi¹ (1.Ochanomizu University)

Keywords:BrainBERT, Encoding Model, brain activity data

Currently, many researchers have used language models to achieve excellent results in various fields, such as understanding the semantics of text and extracting multimedia information like videos. Furthermore, many investigations have also been conducted to capture the generative correspondence between text and the brain. In this paper, we constructed a model based on the correspondence between brain activity data and semantic representation by BERT, called BrainBERT. The BrainBERT was used to build an encoding model between text and brain activity states, and to estimate the brain activity. In brief, we have achieved the two primary achievements of this research. 1) We verified the superiority of the BrainBERT model for brain signal extraction compared to the other 20 popular language models. 2) Using visualization tools such as PyCortex, we visualized the correlation of brain activity data according to the regions of interest in the brain.