Abstract:Aiming at the challenges of mechanized citrus fruit harvesting in natural environments, such as complex environments and diverse fruit states, a multichannel information fusion network (YOLO -v5-citrus) was developed, to solve the problems of low accuracy of citrus fruit recognition, fuzzy fruit classification and low accuracy of localization. Different citrus targets were categorized into “pickable” and “hard-to-pick” by different occlusion conditions, and this classification strategy guided the robot to pick them sequentially in a real orchard, which improved the picking rate and reduced the damage rate of the robot body and end-effector. In YOLO v5-citrus, a multi-channel information fusion module was inserted into the neck network to process the depth feature information of citrus to improve the recognition accuracy of the citrus picking state. At the same time, the splicing method of the neck network was modified to recognize the size of the target citrus. The clustering algorithm module was embedded in the recognition part after training to make the final distinction between the citrus targets blurred by the recognition in the training part. Pixel alignment of a depth image and a color image was performed after recognition and 3D coordinates of citrus targets were obtained by coordinate system transformation. In the dataset processed using multiple enhancement techniques, YOLO v5-citrus improved mAP and precsion by 2.8 percentage points and 3.7 percentage points, respectively, compared with the original YOLO v5, respectively. It maintained higher detection accuracy and faster detection speed than other mainstream network architectures such as YOLO v7 and YOLO v8. Through the detection and localization test in the real orchard, the localization error of the 3D coordinate recognition localization system for the citrus target was obtained as (1.97mm,0.36mm,9.63mm), which satisfied the grasping condition of the endeffector. The experimental results showed that the model had strong robustness, meeting the requirements of citrus state recognition in complex environments, and can provide technical support for mechanical harvesting equipment in large-field citrus orchards.
