[2K5-ES-2] Machine learning: Multimedia

[2K5-ES-2-01] Application of deep learning to eye tracking video for estimating sales area where consumer looked

〇Ken Ishibashi¹, Zhen Li², Katsutoshi Yada³ (1. University of Hyogo, 2. Toyo University, 3. Kansai University)

Keywords:eye tracking, consumer behavior, data pre-processing, deep learning, general object recognition

The purpose of this study is to automatically estimate when and what sales area a consumer looked, from video recorded by using wearable eye tracking device. That is, this study attempts to collect shopping path data by using eye tracking device. In a video recorded by wearable eye tracking device, objects and scene looked by a consumer have been identified manually. However, this pre-processing work requires a huge amount of effort because data collected in a field experiment contains various scenes. Thus, separating from eye tracking, existing studies have collected consumers’ shopping path data by using other sensors such as RFID (radio frequency identifier). This study attempts to estimate sales area where a consumer existed from a scene looked by her/him. In this paper, we identify a sales area where consumer looked by applying publicly available model of general object recognition. This technique reduces the burden of data pre-processing which has been barrier for studies with eye tracking. Furthermore, this proposal is expected to facilitate collection of data available for data fusion between consumers’ shopping path and eye movement data. This paper verifies the estimation accuracy of proposed method with using eye tracking data identified sales areas by shipping path data.