[3Rin4] Interactive 1

[3Rin4-39] Further Pretraining BERT for Causality Existence Classification in Financial Domain

〇Yuta Niki¹, Hiroki Sakaji¹, Kiyoshi Izumi¹, Hiroyasu Matsushima¹ (1.School of Engineering, The University of Tokyo)

Keywords:Causal Extraction, Natural Language Processing, Language Resources

In this research, J-FinBERT, which is further pretrained from BERT model by using Japanese financial corpus, is proposed to classify the causality existence in the financial texts. We verify the effectiveness of further pretraining on prediction accuracy and robustness to fewer samples and noisy labels in our task. Through our experiments, we find that further pretraining in specific domain corpus is effective for improving prediction accuracy and robustness for small dataset in our task, but not for noisy labels.