[2N5-OS-28a] OS-28

[2N5-OS-28a-01] A Study of Gender Bias Focusing on BERT Attention Mechanism

〇Mana Ueno¹, Ichiro Kobayashi¹ (1. Ochanomizu University)

Keywords:Gender Bias, BERT, Attention Mechanism

The corpora used to train large-scale language models contain various social biases, such as gender, nationality, and religion. Therefore, various methods have been proposed to investigate the biases contained within language models and to remove biases.
In this study, we focus on gender bias and aim to detect gender bias given by contextual information by investigating words within sentences that strongly focus attention on gender words, whereas previous studies have proposed to remove bias by performing direct manipulations on gender words.
We create a dataset consisting of pairs of sentences that differ only in male and female words, and use the BERT model to examine the differences in Attention values from other words in the sentence to the gender word. Thereby, we will analyze which words provide gender bias within a context.