1:45 PM - 2:00 PM
[G04-01] Using LLM to analyze document text data for educational research
Keywords:Large Language Models (LLM), Qualitative approach , Qualitative Synthesis Method (KJ Method)
When measuring knowledge retention and participant satisfaction, not only numerical data such as Likert scales are collected in questionnaires, but also linguistic data such as free writing and notes from work observations. The method of analyzing such data in the form of natural language or expression of experience is called a qualitative research method. Analysis of linguistic data is also practiced in JpGU's educational sessions, and quantitative text analysis using KHCoder is frequently used. On the other hand, since natural language is used, many analytical methods are thought to be difficult to process mechanically, and there is room for development in qualitative analytical approaches due to the awareness of the hurdle of acquiring skills. In recent years, with the spread of generative LLM, there have been an increasing number of cases where qualitative analysis is practiced using AI, which is noteworthy in the sense of lowering the threshold for attempting analysis in the educational field. In this report, the reporter compared and examined some of the methods he has acquired and is currently promoting in other conventional analysis methods, using LLM. In the qualitative synthesis method (KJ method), the process of dividing text data is performed by humans, and the LLM is used to summarize the text, such as grouping and labeling texts with similar meanings. The difference between this and a practice led by a human is that it depends on human instructions, such as whether the text is checked properly, and on the efficiency of collecting text more efficiently. On the other hand, by checking and classifying a large amount of text at once, it is possible to repeat the process, and a collaborative system between LLMs and researchers can be established.