[1O3-GS-7] Vision, speech media processing

[1O3-GS-7-01] Automatic Production of Audio Descriptions Using Image Recognition for Live Baseball Broadcasts

〇Yuki Shimano¹, Yuya Kuwano¹, Masaki Takahashi¹, Masaru Miyazaki¹, Masanori Sano¹, Atsushi Imai², Toru Takagi² (1. NHK Science & Technology Research Laboratories, 2. NHK Engineering System)

Keywords:Image Recognition, Automatic Production of Audio Descriptions, Media Accessibility

Audio descriptions enable a visually impaired audience to enjoy broadcast programs by providing supplementary information such as a person’s actions and facial expressions that are difficult for such audiences to understand from the main audio content. Although such descriptions would be ideal for the live sporting broadcasts, the production of audio descriptions for such events requires high production costs and expert commentary skills. We thus developed a system that creates audio descriptions of live baseball broadcasts and distributes them to users' smartphones in real time.These audio descriptions are created from the superimposed captions of baseball broadcasts automatically by using image recognition.The experimental results indicate that the proposed method recognizes information of superimposed captions and robustly produces audio descriptions in real time.