[2C1-GS-7] Language media processing:

[2C1-GS-7-05] Towards General Text-to-Design System with LLM-based Agents

〇Hisaki Seki¹, Kotaro Kikuchi², Naoto Inoue², Mayu Otani², Kota Yamaguchi², Edgar Simo-Serra¹ (1. Waseda University, 2. CyberAgent)

Keywords:LLM-based Agents, Text-to-Design, User Interface, Graphic Design

Design tasks, including user interface design, can be complex and time-consuming for non-designers. There are several methods for generating designs, including creating pixel images or training models to output designs in specific formats from natural language queries. However, there is a need for a more general approach to performing design tasks that does not rely on a specific format and produces editable results. The proposed method, DesignPlanner, is a system that can execute design tasks using a large language model. Utilizing the existing methodology as the core, the system consists of two components: Planner and Executor. Planner breaks down the query into subqueries, and Executor executes the subqueries using registered functions. Our system performs simple operations related to Web UI design, such as using existing components and editing them. Our prototype evaluation demonstrates that our approach is useful for simple tasks, but can fail more complex tasks due to incorrect function calls.