[2A2] [General Session] 2. Machine Learning

[2A2-02] Deep few-shot learning with pseudo example optimization

〇Akisato Kimura^1,2, Zoubin Ghahramani², Koh Takeuchi¹, Tomoharu Iwata¹, Naonori Ueda¹ (1. Nippon Telegraph and Telephone Corporation, 2. University of Cambridge)

Keywords:Neural networks, Few-shot learning, Knowledge distillation

This paper proposes a novel method for training neural networks with a limited amount of training data. Our approach is based on knowledge distillation that transfers knowledge from a deep reference neural network to a shallow target one. The proposed method employs this idea to mimic predictions of non-neural networks reference models that are more robust against overfitting that the target neural network. Different from almost all the previous work for knowledge distillation that requires a large amount of labeled training data, the proposed method requires only a small amount of training data. Instead, we introduce pseudo training data that is optimized as a part of model parameters.