[3C4-GS-6] Language media processing

[3C4-GS-6-01] Towards Improved Numerical Reasoning Ability With Meta-Learning

〇Keito Kudo¹, Youichi Aoki¹, Ana Brassard^2,1, Tatsuki Kuribayashi^1,3, Masashi Yoshikawa^1,2, Kentaro Inui^1,2 (1. Tohoku University, 2. RIKEN, 3. Langsmith Inc.)

Keywords:Symbolic Reasoning, Numerical Reasoning, Meta-Learning, Deep Learning

Deep learning models such as Transformers have been successful in various natural language processing tasks, including inference tasks such as numerical reasoning. However, it is not evident whether current deep learning models capture compositionality and perform inference according to the structure of the problem. They may be learning shortcuts from superficial cues. This study evaluates the performance of existing deep learning models using a formal language that abstracts numerical reasoning. We also investigate whether meta-learning, which has recently been shown to be effective in making models understand compositionality, can be used to acquire numerical reasoning skills that are difficult to obtain via conventional supervised learning. Our experimental results show that (1) pre-training on natural language texts leads to improved performance on tasks on formal languages, and (2) problems that require multi-hop inference are difficult to solve with existing deep learning models.