[3M1-CC] Synergy among Machine Learning, Computational Optimization, and Human Awareness / General Session – B

[3M1-CC-01] A Feasibility Study on Hand Gesture Recognition in Natural Conversation

Dian Christy Silpani¹, Keishi Suematsu¹, Kaori Yoshida ¹ (1. Kyushu Institute of Technology)

This paper proposes to approach natural hand gestures that occur in natural conversations. Hand gesture recognition in Human-Robot Interaction is considered a preliminary study for the next step, based on the vision system of a humanoid robot. We experimented as a feasibility study on hand gestures recognition in natural conversation. The conversations between an experimental assistant (the first author) and two different subjects (both female) were recorded in the experiment. Each conversation includes two people, one of whom is the subject of the hand gesture detection that appears during the conversation. We manually captured the hand gestures that appear naturally during the session and analyzed their duration and frequency of occurrence from each video. As a result of conversations with two subjects, it was clear that hand gestures significantly impact people’s conversations. We observed that hand gestures with the same word or similar meaning appeared several times with the same intention, such as pointing at something and emphasizing a sentence or word. The topic of natural hand-gesture recognition by a robot will be a challenge. Cultural diversity makes the scope of natural hand-gesture broad, but this can be considered a topic in developing the knowledge of human-robot interaction in the future.