Presentation information

Organized Session

Organized Session » OS-1

[2K5-OS-1a] 医療におけるAIの社会実装に向けて(1/2)

Wed. Jun 15, 2022 3:20 PM - 5:00 PM Room K (Room K)

オーガナイザ:小寺 聡(東京大学)[現地]、木村 仁星(東京大学)、小林 和馬(国立がん研究センター)、杉原 賢一(エムスリー)

3:40 PM - 4:00 PM

[2K5-OS-1a-02] Waiting one second boosted data quality: A simple and general-purpose solution to allocate the trade-off between annotators' performance and workload

〇Rina Kagawa1, Masaru Shirasuna2, Atsushi Ikeda3, Masaru Sanuki1, Hidehito Honda2, Hirokazu Nozato4 (1. University of Tsukuba, 2. Otemon Gakuin University, 3. University of Tsukuba Hospital, 4. National Institute of Advanced Industrial Science and Technology)


Keywords:resource rationality, annotation, data quality, mental workload

The development of statistical learning techniques generally requires large, accurately annotated data sets. However, for tasks where the definition of the correct label cannot be uniquely defined, especially when the task is highly specialized such as medical data, it is difficult to obtain large, accurately annotated data sets. We hypothesized that there exists an appropriate thinking time that balances the trade-off between accuracy and mental strain. We tested the effect of an intervention in which participants were prevented from answering for a certain period of time after the image was presented to them when deciding whether a medical image was abnormal or normal. In two behavioral experiments (physicians (N=634)), the expectation of a correct response increased when the image was made unanswerable for one second after presentation. This study showed that annotation quality can be improved in a simple and cost-effective way by utilizing human cognitive characteristics.

Authentication for paper PDF access

A password is required to view paper PDFs. If you are a registered participant, please log on the site from Participant Log In.
You could view the PDF with entering the PDF viewing password bellow.