[4-1600-D] Other Categories (1)

[4-1600-D-02] Research on an Intelligent Robot Eye-hand System for Harvesting Pumpkin in the Outdoor Condition

*Liangliang Yang¹, Qian Wang², Yohei Hosino¹, Hiroki Ishikuro¹, Ying Cao¹ (1. Kitami Institute of Technology(Japan), 2. Ning Xia University(China))

Keywords:Robot, Machine vision, Harvester, Pumpkin

Hokkaido is the largest pumpkin planting region in Japan. However, the planting area is shrinking these years for the shortage of labor force for harvesting the pumpkin fruits. An autonomous robot harvesting system is required by farmers. In addition, the farmers asked us that the surface of the fruits cannot be scrubbed, because of the restrict market requirement of vegetables. Moreover, the storage span will be shorter if the surface has broken points. Therefore, a robot eye-hand system is going to be developed for harvesting the pumpkin fruit. There are three modules of the harvester robot, which are fruits detection module, position coordinates transform and communication module, and robot arm module. In the first module, a color USB camera (IDS, xs, Germany) was utilized to grab images of the field on the speed under 30 frame per second (FPS). The grabbed images were processing in real-time by using faster regional convolutional neural-network (faster R-CNN) method. The pumpkin fruits can be detected correctly around 90% using the method. The detected results were transferred to the second module. In the second module, a PC was connected to the robot arm via an Ethernet cable. The PC was configured as a TCP/IP server and the robot arm was configured as a TCP/IP client. The server and client were communication each other by a speed of 125 Hz, so that the robot arm can be controlled in real-time. The server transformed the position data of the pumpkin fruits from the camera coordinates to the robot coordinates. In the third module, a commercial robot arm (universal robot, UR5, Denmark) was utilized to catch the target fruits.