[2C6-GS-7] Language media processing:

[2C6-GS-7-02] Improving Saliency Map Prediction via Changing Backbone for Advertising Videos

taro watanabe¹, 〇kazuhiro onishi¹ (1. Hakuhodo Technologies Inc.)

Keywords:saliency map, action recognition model, advertising videos

For creation phase of the advertisement, feedback on which parts of the advertisement videos is mainly obtaining the viewers’ attention is important for leading to more efficient production of advertisement videos. We will improve the performance by replacing the backbone of the encoder part with a better-performing action recognition model to a UNet-like encoder-decoder structure. We selected six different action recognition models (S3D, Slow, X3D, Slowfast, MoViNet, and Uniformer) and evaluated their estimation accuracy using three different benchmarks. No correlation was found between the classification accuracy and saliency prediction accuracy for the action recognition models. We found improvement for small areas and low-contrast regions, but not much improvement when object motion prediction was still required.