11:45 AM - 12:00 PM
[MTT46-05] Point-Agri: Building digital twin greenhouse and Growth diagnosis using point cloud analysis
Keywords:Agriculture, DigitalTwin, PointCloud
In recent years, smart agriculture has been promoted due to the aging and declining population of farmers caused by the declining birthrate. In order to stabilize the yield and increase sales, it is necessary to improve the efficiency and accuracy of growth diagnosis.
On the other hand, the concept of digital-twin, which refers to a digital copy of the real space, has been attracting attention in recent years due to the advancement of digitalization and networking. Digital-twin is a method of preparing a real space and a virtual space in which a digital copy of the real space is placed and providing feedback to both through simulations. In this study, we propose a method for constructing a digital twin of a greenhouse and diagnosing its growth by utilizing this digital twin technology.
In the field of phenotyping, many studies have been conducted to obtain and analyze 3D point cloud information of crops. In these studies, a method called photogrammetry, which constructs a 3D point cloud from RGB images, has been used to extract the parameters necessary for growth diagnosis with high accuracy. Although the photogrammetric point cloud construction method is suitable for outdoor environments and can construct highly accurate point clouds, it is difficult to use in greenhouse cultivation because it requires a long time for the point cloud construction process and is prone to many misjudgments in environments where the same object, such as a leaf, is continuously used, such as in an actual greenhouse. In this study, we used a point cloud construction method using a depth sensor, which can measure the distance between the object and the sensor, to create a 3D point cloud of the entire greenhouse. To solve the problem that RGB images obtained by depth sensors are often misidentified, we used a method of extracting point clouds by hue and matching the shapes of the point clouds to reduce misidentification and align the RGBD images of the entire greenhouse. The 3D point clouds were analyzed to extract two parameters, leaf area and the number of flowering plants, which are necessary for growth diagnosis. The leaf area was first classified into individual leaves using the x-mean method, and then the normal direction was determined for each leaf to obtain an index similar to the leaf area. The normal direction was determined as the direction that maximized the area by projection. The number of flowering plants was determined by extracting the hue information and identifying the location of the flower from the point cloud density, which was counted every two meters and averaged to obtain an index similar to the number of flowering plants. We conducted an experiment on a farm in Kochi Prefecture using the implemented system.
The average length of the 24-meter rows and the average length of the digital twin rows were in good agreement with each other.
The area of individual leaves obtained from the constructed digital twin and the actual measurements were also in good agreement.
The number of flowering plants was also in good agreement with the number of flowers obtained from the digital twin.
From these results, we conclude that the leaf area and the number of ignitions obtained from the digital twin in this study are accurate enough for growth diagnosis from both spatial and temporal perspectives.
On the other hand, the concept of digital-twin, which refers to a digital copy of the real space, has been attracting attention in recent years due to the advancement of digitalization and networking. Digital-twin is a method of preparing a real space and a virtual space in which a digital copy of the real space is placed and providing feedback to both through simulations. In this study, we propose a method for constructing a digital twin of a greenhouse and diagnosing its growth by utilizing this digital twin technology.
In the field of phenotyping, many studies have been conducted to obtain and analyze 3D point cloud information of crops. In these studies, a method called photogrammetry, which constructs a 3D point cloud from RGB images, has been used to extract the parameters necessary for growth diagnosis with high accuracy. Although the photogrammetric point cloud construction method is suitable for outdoor environments and can construct highly accurate point clouds, it is difficult to use in greenhouse cultivation because it requires a long time for the point cloud construction process and is prone to many misjudgments in environments where the same object, such as a leaf, is continuously used, such as in an actual greenhouse. In this study, we used a point cloud construction method using a depth sensor, which can measure the distance between the object and the sensor, to create a 3D point cloud of the entire greenhouse. To solve the problem that RGB images obtained by depth sensors are often misidentified, we used a method of extracting point clouds by hue and matching the shapes of the point clouds to reduce misidentification and align the RGBD images of the entire greenhouse. The 3D point clouds were analyzed to extract two parameters, leaf area and the number of flowering plants, which are necessary for growth diagnosis. The leaf area was first classified into individual leaves using the x-mean method, and then the normal direction was determined for each leaf to obtain an index similar to the leaf area. The normal direction was determined as the direction that maximized the area by projection. The number of flowering plants was determined by extracting the hue information and identifying the location of the flower from the point cloud density, which was counted every two meters and averaged to obtain an index similar to the number of flowering plants. We conducted an experiment on a farm in Kochi Prefecture using the implemented system.
The average length of the 24-meter rows and the average length of the digital twin rows were in good agreement with each other.
The area of individual leaves obtained from the constructed digital twin and the actual measurements were also in good agreement.
The number of flowering plants was also in good agreement with the number of flowers obtained from the digital twin.
From these results, we conclude that the leaf area and the number of ignitions obtained from the digital twin in this study are accurate enough for growth diagnosis from both spatial and temporal perspectives.