Presentation information

International Session

International Session » ES-3 Agents

[1S1-IS-3] Agents

Tue. Jun 14, 2022 10:00 AM - 11:40 AM Room S (Online S)

Chair: Takahiro Uchiya (Nagoya Institute of Technology)

10:40 AM - 11:00 AM

[1S1-IS-3-03] Evaluating Rank-Coherence among AI-enabled Ranking, Expert Rating and Crowd Voting for Selecting Winners in Incentivized Large-scale Idea Contest for Creative Works

〇Jawad Haqbeen1, Sofia Sahab2, Takayuki Ito2 (1. Nagoya Institute of Technology, 2. Kyoto University)


Keywords:winner selection, idea contest, expert rating, crowd voting, system scoring

While expert rating and crowd voting is still a dominant approach for selecting winners in contests for creative works, however, due to participation scale in such large-scale idea contests, utilizing only the human-led mechanisms in the winning selection process are untenable. Thus, a few digital participatory platforms have recently used a “scoring system” to reward points for participants, that is, let the submitted opinions be evaluated by AI. Toward this end, drawing upon tournament theory, we investigate how a contest’s reliance on AI-enabled point-based system ranking can help experts and crowd for selecting winners, and also, we assess how each of these selection mechanisms (system ranking/scoring, expert rating, and crowd voting) affects contest winners’ selection. This enables us to collect large-scale participation, and to explore participation ranking in light of AI, expert rating, and crowd voting, and compare among each selection mechanism while looking to the final winners. The results showcase the contest’s reliance on system ranking is positively associated with expert and crowd voting winning mechanisms. Moreover, AI-enabled winning incentive mechanisms have proven essential for helping experts and crowds to select appropriate authors of ideas in large-scale contests, and is more appealing to large-scale idea contests for winner selection.

Authentication for paper PDF access

A password is required to view paper PDFs. If you are a registered participant, please log on the site from Participant Log In.
You could view the PDF with entering the PDF viewing password bellow.