6:40 PM - 7:00 PM
[1E5-GS-6-06] Assessment of target speakerness by non-acquaintance through dialogue comparison
Keywords:Dialogue, target speakerness
Several dialogue system studies have attempted to replicate a specific speaker desired by users.
Typically, to assess the ``speakerness'' of a specific speaker subjectively, evaluators should be familiar with the target speaker.
However, when using a corpus collected from the web and crowdsourcing, it becomes challenging to find evaluators familiar with the target speaker.
To address this issue, we propose a novel method for assessing target speakerness through dialogue comparison that can be utilized by non-acquainted evaluators.
We evaluate the effectiveness of this method using both expert annotators and non-expert crowdworkers, discussing its validity as a subjective evaluation tool for speakerness.
Additionally, we train and examine the performance of a baseline model for assessment of target speakerness through dialogue comparison.
Typically, to assess the ``speakerness'' of a specific speaker subjectively, evaluators should be familiar with the target speaker.
However, when using a corpus collected from the web and crowdsourcing, it becomes challenging to find evaluators familiar with the target speaker.
To address this issue, we propose a novel method for assessing target speakerness through dialogue comparison that can be utilized by non-acquainted evaluators.
We evaluate the effectiveness of this method using both expert annotators and non-expert crowdworkers, discussing its validity as a subjective evaluation tool for speakerness.
Additionally, we train and examine the performance of a baseline model for assessment of target speakerness through dialogue comparison.
Authentication for paper PDF access
A password is required to view paper PDFs. If you are a registered participant, please log on the site from Participant Log In.
You could view the PDF with entering the PDF viewing password bellow.