1:20 PM - 1:40 PM
[4F2-OS-11a-05] Use of Dialogue History for User's Interest Level Estimation Based on Multimodal Information
Keywords:multimodal dialogue, estimating interest, dialogue history, partially observable markov decision process
Estimation of a user's mental states including his/her interest level for the current topic is a key component to realize user-adaptive multimodal dialogue systems.
Multimodal sensing provides helpful information for that, but its results are often incorrect because the user's subtle behaviors need to be captured.
We incorporate a dialogue history when estimating the user's interest level to make the results more stable.
Specifically, we formulate the estimation by using the partially observable Markov decision process (POMDP).
The multimodal sensing results are used as likelihoods of input observation and the estimated interest levels are tracked as states.
Experimental results showed the estimation performance by considering the dialogue history.
We used two kinds of annotation results for each exchange: the interest levels and the next desirable system actions, which were given by multiple annotators.
We also confirmed a correlation between the estimated interest levels and the next desirable system actions.
Multimodal sensing provides helpful information for that, but its results are often incorrect because the user's subtle behaviors need to be captured.
We incorporate a dialogue history when estimating the user's interest level to make the results more stable.
Specifically, we formulate the estimation by using the partially observable Markov decision process (POMDP).
The multimodal sensing results are used as likelihoods of input observation and the estimated interest levels are tracked as states.
Experimental results showed the estimation performance by considering the dialogue history.
We used two kinds of annotation results for each exchange: the interest levels and the next desirable system actions, which were given by multiple annotators.
We also confirmed a correlation between the estimated interest levels and the next desirable system actions.