[2E4-GS-6] Language media processing

[2E4-GS-6-04] Natural language processing with deep learning: Principles, limitations, and challenges

〇Kazutoshi Kan¹, Mitsuo Yoshida² (1. Bank of Japan, 2. University of Tsukuba)

Keywords:natural language processing, deep learning, security

Deep learning for natural language processing (NLP) outperforms traditional approaches in many tasks. High-performing deep learning models are realized by proficiently combining techniques in model architecture such as attention mechanisms. Open-access large scale pre-trained models and easier pipeline construction based on End-to-End learning have lowered barriers to develop such models. The practice of academia to share fundamental language resources such as morphological analysis tools and linguistic datasets as well as the relaxation of copyright on automatic collection of text data also encourage research and development of models for NLP. In real businesses, ethical considerations are required to ensure that models do not output harmful expressions. However, such consideration suitable for everyone is difficult to achieve because there are no universal norms in ethics. In addition, the performance of deep learning models has uncertainty in principle. Furthermore, the security risks specific to machine learning models should also be noted.