[S-CG50] Driving Solid Earth Science through Machine Learning

[SCG50-05] Recent Trials to Train Large Language Models for Scientific Data and Reasoning

★Invited Papers

*Kan Hatakeyama¹ (1.Tokyo Institute of Technology)

Keywords:Large language model, Machine learning, Chemistry

In recent years, the field of artificial intelligence has been undergoing a revolutionary paradigm shift with the utilization of large-scale language models and foundation models. Traditional AI was predominantly associated with specialized algorithms designed for specific tasks. For instance, deep learning-based face recognition algorithms and programs for games like Shogi and Go demonstrated performance surpassing human abilities, yet few considered these as manifestations of human-like intelligence. However, large-scale language models such as GPT-4, despite issues like hallucinations, are showing versatility and inferential capabilities that many associate with "artificial intelligence." For example, in January 2024, GPT-4 exceeded the average test-taker's scores in most subjects of a university entrance exam (https://note.com/lifeprompt/n/n87f4d5510100), and similar foundational models have achieved gold medal-level performances in Mathematics Olympiads (Loung et al., Nature 2024). Furthermore, the application of large-scale language models is rapidly progressing across various scientific fields.
This presentation will focus particularly on the field of chemistry, introducing both domestic and international trends regarding the roles large-scale language models have begun to play. Additionally, it will report on the latest advancements in large-scale language model-related research conducted since 2023. This includes tasks such as training AI on scientific papers (arXiv 2023, https://arxiv.org/abs/2312.03360), studies allowing AI to analyze mechanisms in experimental data (https://github.com/KanHatakeyama/LLMChem), and efforts to build language models from scratch.