[3Win5] Poster session 3

[3Win5-35] Analyzing How Question Expressions Change the Answer Tendencies of Large Language Models

〇Junya Takayama¹, Masaya Ohagi¹, Tomoya Mizumoto¹, Katsumasa Yoshikawa¹ (1.SB Intuitions Corp.)

Keywords:LLM, QA, Natural Language Processing

Large language models (LLMs) often generate inconsistent responses to questions that share the same intent but differ in linguistic expression. This phenomenon can lead to lower task completion rates and excessive agreement with users. In this study, we investigate how variations in the linguistic representation of questions influence response tendencies across multiple models, using a question-answering dataset where responses are limited to "yes" or "no." Specifically, we construct paraphrased versions of questions through various transformations, such as replacing words with synonymous or antonymous expressions and modifying modality markers. We then compare the output probabilities of "Yes" and "No" before and after paraphrasing. Our results show that, in many models, the addition of modality markers and substitution with antonymous expressions each tend to reduce response consistency. Furthermore, we demonstrate that this tendency is already present at the pre-training stage and that it can be mitigated through few-shot prompting.