Towards High-Quality 3D Scans of Orchards for Automated Pruning
Abstract
Due to increasing labour costs and shortages, automatisation of tree pruning is desirable. This requires accurate 3D models of orchards, which robots can use for navigation and for making decisions about which branches to prune. We propose a pipeline to create high-quality coloured 3D point clouds of orchards during the pruning season. We demonstrate that our method, which combines LiDAR and RGB sensing, is able to reconstruct orchards with finer details and fewer artefacts compared to classical photogrammetry methods relying solely on RGB images.
Full text
Towards High-Quality 3D Scans of Orchards for Automated Pruning Laurens Diels1, Michiel Vlaminck1, Bas Boom2, Wilfried Philips1, Hiep Luong1 1Ghent University (TELIN-IPI-URC) - imec 2imec - OnePlanet Introduction Capture setup Pose estimation Conclusion •Pruning orchard trees requires expert knowledge •But: increasing shortage of expert pruners → Need to automate •Requires accurate 3D models •These models can also be used for •training new pruners in virtual reality •monitoring orchards over time, increasing understanding of optimal pruning strategies Processing pipeline 3D reconstruction Results Sensing modalities •RGB cameras (Red-Green-Blue) + Fine details, colours - Scenes are very cluttered → Image-based pose estimation fails → 3D reconstruction fails •LiDAR (Light Detection And Ranging) + Immediate 3D information - Coarse, misses smaller branches - Sensor noise →Use sensor fusion to get the best of both worlds! LiDAR RGB image Image clutter: which pixels belong to the foreground tree? Smaller branches are not sampled by the LiDAR, or disappear in noise. NVIDIA Jetson Orin onboard computer Ouster OS-128 LiDAR Lucid Triton 24.5 MP high-resolution cameras In the current version of the pipeline, we only use this one. We first apply our LiDAR-SLAM (Simultaneous Localisation and Mapping) algorithm, then refine using RGB SfM (Structure from Motion). Top-down view of SfM point cloud: RGB SfM only Ours: LiDAR SLAM + RGB SfM Orchard row Same row Successfully aligned cameras Failed to align cameras Tie points from images taken from opposite sides of the orchard start to diverge Camera exposure time such that sky is overexposed → easily masked out •3D Gaussian splatting starting from (lower-quality) LiDAR point cloud •Modified to keep masked-out regions empty •Sample trained Gaussians → high-quality dense coloured point cloud The smaller branches are captured in our Gaussian splatting model. 3DGS render RGB mult-view stereo (starting from LiDAR-enhanced poses) Ours Indoor apple trees Outdoor redcurrant bushes Zoom to small branches Artefacts at edges Gaps Missing in reconstruction No artefacts! Fewer gaps More complete reconstruction Small branches are captured •Developed pipeline for high-quality 3D reconstruction of trees in an orchard, during the pruning season •Outperforms classical photogrammetry and LiDAR SLAM •Sensor fusion between LiDAR and RGB imagery improves quality and robustness Our method is designed and tested to reconstruct mostly leafless trees during the pruning season, but can likely be applied to other use-cases as well. Contact: [email protected]