[4Xin1] Poster session 2

[4Xin1-78] Edge Devices-Friendly Dynamic Sign Language Recognition System using Attention Module

〇Yuejie Meng¹, Masao Yanagisawa¹, Youhua Shi¹ (1.Waseda University)

Keywords:AI, Sign Language Recognition, Edge Device, Image Recognition, Transformer

In recent years, people’s life is becoming more and more convenient due to voice assistants like Siri, adopting artificial intelligence (AI) techniques. However, hearing-impaired people, especially those who cannot speak, are unable to have the benefits of this technology for physical reasons. Gesture recognition techniques using deep learning would be a hopeful alternative to help them. However, many previous studies used 3D-CNN or CNN+LSTM to recognize gestures from images or from videos, which requires large memory. In order to solve this problem, this paper proposes a gesture recognition model based on Transformer called DGT-STA. This model is able to achieve accuracy beyond that of 3D-CNN with a shallower neural network, and reduced memory usage to 50.91% compared to models using other Attention modules. In addition, a dataset of Japanese Sign Language is built to train and evaluate DGT-STA. Finally, this paper verified that it is feasible to deploy DGT-STA on IoT edge devices.