[2K5-OS-1a] 医療におけるAIの社会実装に向けて(1/2)

[2K5-OS-1a-02] Waiting one second boosted data quality: A simple and general-purpose solution to allocate the trade-off between annotators' performance and workload

〇Rina Kagawa¹, Masaru Shirasuna², Atsushi Ikeda³, Masaru Sanuki¹, Hidehito Honda², Hirokazu Nozato⁴ (1. University of Tsukuba, 2. Otemon Gakuin University, 3. University of Tsukuba Hospital, 4. National Institute of Advanced Industrial Science and Technology)

[[Online]]

Keywords:resource rationality, annotation, data quality, mental workload

The development of statistical learning techniques generally requires large, accurately annotated data sets. However, for tasks where the definition of the correct label cannot be uniquely defined, especially when the task is highly specialized such as medical data, it is difficult to obtain large, accurately annotated data sets. We hypothesized that there exists an appropriate thinking time that balances the trade-off between accuracy and mental strain. We tested the effect of an intervention in which participants were prevented from answering for a certain period of time after the image was presented to them when deciding whether a medical image was abnormal or normal. In two behavioral experiments (physicians (N=634)), the expectation of a correct response increased when the image was made unanswerable for one second after presentation. This study showed that annotation quality can be improved in a simple and cost-effective way by utilizing human cognitive characteristics.