[1Win4] Poster session 1

[1Win4-06] Multimodal Sentiment Analysis Based on Multimodal Label Distribution and Mixture of Experts

〇LEI YANG¹, SHINNOU Hiroyuki¹ (1.Ibaraki University)

Keywords:Multimodal Sentiment Analysis, Multimodal Label Distribution, Mixture of Experts, Collaborative Fusion

Multimodal sentiment analysis integrates text, audio, and visual data to capture emotional features comprehensively, enabling accurate polarity recognition in complex scenarios. However, existing methods often focus on the features of multimodal data while ignoring the influence of the overall situation on multimodal fusion and failing to address imbalanced modality contributions. To tackle these challenges, this paper proposes Multimodal Sentiment Analysis Based on Multimodal Label Distribution (MLD) and Mixture of Experts (MoE) as a novel approach for collaborative fusion. By leveraging side-information, MLD is dynamically generated to adjust cross-modal attention weights, guiding visual and audio modalities to better complement text information. Additionally, MLD is fused with multimodal features to assist the MoE module in activating the most relevant experts for modality-specific processing. Experimental results show that the proposed method significantly improves accuracy and robustness, providing an effective solution for multimodal sentiment analysis.