Abstract:In order to provide reliable positioning technical guidance for the implementation of pollination by the facility tomato pollination robot, a method for positioning tomato pollination flowers was proposed based on 3D vision. Firstly, the RGB-D structured light camera was used to quickly obtain the color map and depth map information of the tomato plants in the glass greenhouse, through the fast small target detection YOLO v4 (You only look once) neural network to detect the tomato bouquet on the plant, and extract the two-dimensional area of the pollination bouquet. Then an active alignment method was used in conjunction with PCL to roughly align the RGB map and the depth map. In the RGB image, the bouquet area was filled with yellow flowers. The color system of the pixels in the prediction frame was judged, the non-flower pixels was removed, and the precise alignment of the point cloud of the bouquet area was performed. In order to obtain the high-precision point cloud information of the spatial tomato bouquet, a single-view linear traversal method of the color system within the region was used to perform fine registration on the extracted bouquet region, in the three-dimensional point cloud collection, the double-plane centroid algorithm was used to obtain the spatial point cloud coordinates of the target bouquet (x, y, z). Finally, after the group filtering method denoised the point cloud information, the two-way average method was combined to calculate the pollination centroid coordinates of the bouquet 3D box. The positioning test results showed that the method can successfully identify and locate the bouquet in the greenhouse environment, the average detection accuracy of the neural network was 97.67%, the extraction time of a single image bouquet was 14.95ms, the time and energy consumption of the algorithm to obtain the pollination centroid coordinates was about 300ms. In the actual verification process, in the absence of strong light, the algorithm can basically realize the robot's positioning problem of tomato flowers in the greenhouse, and provide a method for the tomato pollination robot to locate and solve the pollination point.