[1Win4-47] Construction and Evaluation of an AI Therapist Capable of In-Depth Exploration of Concerns Using a Large Language Model
Keywords:therapist, AI, Patient, Counseling
In recent years, the importance of psychological health issues has been increasing. The realization of AI therapists could help address problems such as the shortage of therapists and the cost of therapy.
Therefore, this study focuses on prompt design for large language models (LLMs) to develop an AI therapist. Specifically, two methods are explored: one that structures the order of conversations and another that incorporates feedback by summarizing the current conversational context and expected developments into the prompt for generating the next conversation turn. The impact of each method on the generation process is examined.
Furthermore, the AI therapist is evaluated through two types of experiments: one where human evaluators assess dialogues between the AI therapist and an AI client, and another where humans engage in conversations with the AI therapist and provide evaluations.
As a result, structuring the conversation order successfully facilitated deeper exploration of concerns, while the feedback-based approach improved scores across multiple evaluation criteria.
Therefore, this study focuses on prompt design for large language models (LLMs) to develop an AI therapist. Specifically, two methods are explored: one that structures the order of conversations and another that incorporates feedback by summarizing the current conversational context and expected developments into the prompt for generating the next conversation turn. The impact of each method on the generation process is examined.
Furthermore, the AI therapist is evaluated through two types of experiments: one where human evaluators assess dialogues between the AI therapist and an AI client, and another where humans engage in conversations with the AI therapist and provide evaluations.
As a result, structuring the conversation order successfully facilitated deeper exploration of concerns, while the feedback-based approach improved scores across multiple evaluation criteria.
Authentication for paper PDF access
A password is required to view paper PDFs. If you are a registered participant, please log on the site from Participant Log In.
You could view the PDF with entering the PDF viewing password bellow.