Real-Time Facial Animation of a Reality Avatar Based on Japanese Vowels in a Speech Audio Stream

Ryoto Kato, Yusuke Kikuchi, Vibol Yem, Yasushi Ikei

doi:https://doi.org/10.36463/idw.2021.0526

[3Dp4/AISp4-2L] Real-Time Facial Animation of a Reality Avatar Based on Japanese Vowels in a Speech Audio Stream

*Ryoto Kato¹, Yusuke Kikuchi¹, Vibol Yem¹, Yasushi Ikei² (1.Tokyo Metropolitan University (Japan), 2.The University of Tokyo (Japan))

Realistic avatar, Speech facial expression, Real-time

https://doi.org/10.36463/idw.2021.0526

Automated, expressive facial animation is crucial for AI avatars in the VR space. We propose a real-time method for facial animation based on Japanese vowels. The user study revealed that our facial expression was more natural than the popular method (Oculus Lipsync) in mouth motion synchronous to speech audio stream.

Proceedings of the International Display Workshops Volume 28 (IDW '21)

[3Dp4/AISp4-2L] Real-Time Facial Animation of a Reality Avatar Based on Japanese Vowels in a Speech Audio Stream