[4Rin1] Interactive 2

[4Rin1-05] Evaluation of Concept Drift Detection Methods for Unlabeled Data in Operation

〇Tsutomu Ishida¹, Hiroaki Kingetsu¹, Yasuto Yokota¹, Yoshihiro Okawa¹, Kenichi Kobayashi¹, Katsuhito Nakazawa¹ (1.Fujitsu Laboratories Ltd.)

Keywords:Concept drift detection, Machine learning

Concept drifts in operational data streams cause accuracy degradation in predictive machine learning models used for AI systems. The concept drift refers to a change in the input data distribution of the class (concept) over time. To maintain high prediction accuracy in AI systems, it is essential to detect concept drifts and adapt the predictive model. This work focuses on the concept drift detection process. In addition, we emphasize the practicality of not requiring labeled data in operation. We survey existing unsupervised drift detection methods. The survey result shows that there are few unsupervised drift detection methods that can handle deep learning models for image classification tasks. We select a representative one and evaluate it using an image dataset. The experimental result shows that the existing method cannot achieve high drift detection accuracy for the artificial drifts that cause accuracy degradation. We confirmed that a new drift detection technique for image classification is required.