[1Win4] Poster session 1

[1Win4-46] Method for quantifying a reliability of LLM answers and OOD detection by internal calculation values

〇Shinji Nakagawa¹, Ryota Komatsu¹, Masashi Egi¹ (1.Hitachi, Ltd.)

Keywords:LLM, Reliability, Internal calculation value, Attention, Out-of-distribution

Ensuring high reliability of LLM is an important issue. In this paper, we propose a method to quantify a reliability of outputs based on attention vectors, which are an internal calculation value of LLM, and an out-of-domain (OOD) detection method using the quantified reliability. Values of attention vectors corresponding to a previously proven input and output of LLM are converted into features, and the values are determined as in-domain (ID) features. The greater the difference between the feature of unknown input and the ID features for input is, the lower the reliability of the unknown input is, and OOD detection is performed based on the reliability. Same processing is performed for unknown output. A final OOD judgment is made by both the OOD detection for input and output. Evaluations using actual business data was performed, and both OOD judgment rates and ID judgment rates was over 95%.