Abstract:Aiming at the problem of lack of presence when the orchard robot used a monocular camera for remote operation, only using two-dimensional video to obtain environmental information lacked presence, a set of orchard environment information visualization system based on the enhanced sense of presence was designed. The system consisted of computing server, cloud server, network camera, LiDAR, embedded development platform, etc. The computing server adopted the T7920 workstation, and deployed the Tensorflow computing framework and the Open3D algorithm library of point cloud on it. After receiving the environmental image and point cloud data forwarded by the cloud server, the computing server enhanced the navigation information of the image, and surface reconstructed the point cloud. The embedded development platform could collect raw data from webcam and LiDAR, and uploaded them to cloud servers. A ZeroMQ-based message transfer program and HTML5 background service were deployed on the cloud server, providing cross-Internet message communication services and mobile teleoperation environment information visualization services. The test results showed that the average extraction time of the extraction model for navigation information deployed on the computing server was 86ms, and the average precision of the navigation line extraction was 16°, which were better than the results of the comparison model. The algorithm of point cloud reconstruction can effectively establish scene contours with an average accuracy of 4.9cm and an average reconstruction time of 24ms. The delay of compressed image transmission and enhancement processing did not exceed 230ms, and the transmission delay of point cloud did not exceed 400ms. The parameters could meet the basic requirements of the remote operation robot for orchard operation. The system significantly enhanced telepresence compared with that only with monocular cameras, which provided an effective reference for the remote operation of the orchard robot.
