[1S1-IS-3] Agents

[1S1-IS-3-03] Evaluating Rank-Coherence among AI-enabled Ranking, Expert Rating and Crowd Voting for Selecting Winners in Incentivized Large-scale Idea Contest for Creative Works

〇Jawad Haqbeen¹, Sofia Sahab², Takayuki Ito² (1. Nagoya Institute of Technology, 2. Kyoto University)

Regular

Keywords:winner selection, idea contest, expert rating, crowd voting, system scoring

While expert rating and crowd voting is still a dominant approach for selecting winners in contests for creative works, however, due to participation scale in such large-scale idea contests, utilizing only the human-led mechanisms in the winning selection process are untenable. Thus, a few digital participatory platforms have recently used a “scoring system” to reward points for participants, that is, let the submitted opinions be evaluated by AI. Toward this end, drawing upon tournament theory, we investigate how a contest’s reliance on AI-enabled point-based system ranking can help experts and crowd for selecting winners, and also, we assess how each of these selection mechanisms (system ranking/scoring, expert rating, and crowd voting) affects contest winners’ selection. This enables us to collect large-scale participation, and to explore participation ranking in light of AI, expert rating, and crowd voting, and compare among each selection mechanism while looking to the final winners. The results showcase the contest’s reliance on system ranking is positively associated with expert and crowd voting winning mechanisms. Moreover, AI-enabled winning incentive mechanisms have proven essential for helping experts and crowds to select appropriate authors of ideas in large-scale contests, and is more appealing to large-scale idea contests for winner selection.