[3Win5] Poster session 3

[3Win5-01] Analysis of learning dynamics between clean and noisy labels in early stopping

〇Ryuken Uda¹, Yusuke Iida¹ (1.Univ. of Niigata)

Keywords:Deep Learning, Early Stopping, Label Noise Robustness, Loss Function, Flat Minimum

Deep learning models are known as robust to label noise in the early phase. Recent studies have shown that learning dynamics with clean label samples dominates in the early phase, which is insufficient to understand the mechanism of label noise robustness. In this study, we aimed to elucidate the mechanism by examining the differences in learning dynamics between clean and noisy labels. First, it is confirmed that the vectors of weight updates are parallel between clean and noisy labels. We visualized the shape of the loss function after learning clean and noisy datasets and compared the convergence locations. It can be seen that even when the label noise is as large as 70%, the model converges to the same local minimum or a plateau. This suggests that the flatness of the local minimum is important for the label noise robustness in the early phase.