[1I3-GS-4b] Webインテリジェンス：コミュニティ

[1I3-GS-4b-04] Mitigating Observation Biases in Crowdsourced Label Aggregation

〇Ryosuke Ueda¹, Koh Takeuchi¹, Hisashi Kashima¹ (1. Kyoto University)

Keywords:crowdsourcing, observation bias, answer aggregation

Crowdsourcing has been widely used to efficiently obtain labeled datasets for supervised learning from large amounts of human resources at low cost. However, to obtain high-quality results through crowdsourcing, the challenge is to deal with variations and biases caused by the fact that the work is performed by humans. We focus especially on the bias of workers in selecting tasks to work on. Workers may be biased in choosing tasks to work on based on their circumstances and preferences, and this may affect the quality of results. We propose to reduce this bias by using the observation bias reduction method, which is used in causal inference, to improve the accuracy of results by aggregating responses. Through experiments using artificial and semi-artificial data, we verify the existence of bias and confirm that the proposed method improves accuracy under certain conditions.