[4H2-E-5] Human interface, education aid: human evaluation

[4H2-E-5-03] Maximizing accuracy of group peer assessment using item response theory and integer programming

〇Masaki Uto¹, Duc-Thien Nguyen¹, Maomi Ueno¹ (1. University of Electro-Communications)

Keywords:Educational Technology, Educational Measurement, Test Theory, Item Response Theory

With the wide spread of large-scale e-learning environments, peer assessment has been widely used to measure learner ability. When the number of learners increases, peer assessment is often conducted by dividing learners into multiple groups. However, in such cases, the peer assessment accuracy depends on the method of forming groups. To resolve that difficulty, this study proposes a group formation method to maximize peer assessment accuracy using item response theory and integer programming. Experimental results, however, have demonstrated that the method does not present sufficiently higher accuracy than a random group formation method does. Therefore, this study further proposes an external rater assignment method that assigns a few outside-group raters to each learner after groups are formed using the proposed group formation method. Through results of simulation and actual data experiments, this study demonstrates that the method can substantially improve peer assessment accuracy.