9:20 AM - 9:40 AM
[3G1-OS-24a-02] Cooking Recognition Planning Action Robot System Considering the Change of Food Condition from the Description of Cooking Recipe Using Large-scale Foundation Models
Keywords:Robotics, Daily Life, Cooking Support, Foundation Model
In the cooking task, the goal is to make the dish complete by changing the state of the ingredients by following the recipe description. Therefore, it is desirable for robots to be able to plan cooking tasks based on recipes written in natural language and to be able to recognize changes in the state of ingredients. In this study, we propose a robot system that uses human knowledge and common sense contained in the vast amount of language-related data on the Internet by applying large-scale foundation models that have been actively developed in recent years to robots, and executes cooking based on recipe descriptions while considering changes in the state of ingredients. and executes cooking based on recipe descriptions while considering changes in the state of food ingredients. We have confirmed the effectiveness of the proposed system through experiments by converting recipes into cooking function expressions using a language model and by recognizing changes in the state of food ingredients based on language descriptions by time-series use of a vision-language model.
Authentication for paper PDF access
A password is required to view paper PDFs. If you are a registered participant, please log on the site from Participant Log In.
You could view the PDF with entering the PDF viewing password bellow.