JSAI2018

Presentation information

Oral presentation

Organized Session » [Organized Session] OS-11

[4F1-OS-11c] [Organized Session] OS-11

Fri. Jun 8, 2018 12:00 PM - 1:20 PM Room F (4F Garreria)

1:00 PM - 1:20 PM

[4F1-OS-11c-04] Performance evaluation and internal state analysis of RCNN model in sign language classification

〇keisuke matsuda1, masahito yamamoto2, hiroyuki iizuka2 (1. Hokkaido University, 2. Graduate School of Information Science and Technology Hokkaido University)

Keywords:sign language, convolutional neural network, recurrent neural network

Sign language is a language used among hearing-impaired people. However, it is not common in our society and not many people can understand sign language. Developing a sign language translator is a big challenge for artificial intelligence. The purpose of this study is to classify sign language words in video as the first step toward sign language translation using deep learning. In sign language, not only hand shape and hand trajectory but also non-manual signals such as facial expression and nodding are important to understand meaning. A sign language translator needs to take into account the whole image of speakers. In this study, we applied the RCNN model which can use the information of the whole image and classified the sign language words.We also examined how input and model structure effect classification accuracy.