A Generic ROS-Based Control Architecture for Pest Inspection and Treatment in Greenhouses Using a Mobile Manipulator

Received June 9, 2021, accepted June 26, 2021, date of publication July 1, 2021, date of current version July 12, 2021. Digital Object Identifier 10.1109/ACCESS.2021.3093978 A Generic ROS-Based Control Architecture for Pest Inspection and Treatment in Greenhouses Using a Mobile Manipulator JON MARTIN 1, ANDER ANSUATEGI1, IÑAKI MAURTUA1, AITOR GUTIERREZ1, DAVID OBREGÓN2, OSKAR CASQUERO 3, AND MARGA MARCOS 3, (Senior Member, IEEE) 1Autonomous and Intelligent Systems Unit, Fundación Tekniker, 20600 Eibar, Spain 2Centro Tecnológico CTC, 39011 Santander, Spain 3Systems Engineering and Automatic Control Department, Faculty of Engineering, University of the Basque Country (UPV/EHU), 48940 Bilbao, Spain Corresponding author: Jon Martin ([email protected]) This work was supported in part by the GreenPatrol European Project through the European GNSS Agency by the European Union’s (EU) Horizon 2020 Research and Innovation Program under Grant 776324 [11]. ABSTRACT To meet the demands of a rising population greenhouses must face the challenge of producing more in a more efficient and sustainable way. Innovative mobile robotic solutions with flexible navigation and manipulation strategies can help monitor the field in real-time. Guided by Integrated Pest Management strategies, robots can perform early pest detection and selective treatment tasks autonomously. However, combining the different robotic skills is an error prone work that requires experience in many robotic fields, usually deriving on ad-hoc solutions that are not reusable in other contexts. This work presents Robotframework, a generic ROS-based architecture which can easily integrate different navigation, manipulation, perception, and high-decision modules leading to a faster and simplified development of new robotic applications. The architecture includes generic real-time data collection tools, diagnosis and error handling modules, and user-friendly interfaces. To demonstrate the benefits of combining and easily integrating different robotic skills using the architecture, two flexible manipulation strategies have been developed to enhance the pest detection in its early state and to perform targeted spraying in simulated and field commercial greenhouses. Besides, an additional use-case has been included to demonstrate the applicability of the architecture in other industrial contexts. INDEX TERMS Precision agriculture, robotic control architecture, mobile manipulator, pest detection and treatment, greenhouse. I. INTRODUCTION The European agriculture land surface is decreasing due to deforestation and urbanization while population continues increasing. In order to achieve a more sustainable business model, protect the crops from adverse weather conditions and control the temperature and water of the plant, greenhouse production is growing a 22% accumulated increase in area since 2011 [1]. However, the presence of warm, humidity conditions and abundant food under protected structures provide favorable habitats for pest development, this being the The associate editor coordinating the review of this manuscript and approving it for publication was Vincenzo Conti . main threat to production and productivity of greenhouse crops worldwide [2]. Digital farming [3]–[5] can help through sensors, robotics and data analysis to automatically maintain and monitor greenhouses, making cropping system smart and, thus, enhancing the agricultural productivity. Traditional pest detection methods in tomato crops rely on farmers observing skills, which are very time-consuming and inefficient in large crops. Nowadays, robotic solutions combined with computer vision can be used to automate this repetitive inspection task, increasing the reliability, maximizing the health of crops and optimizing the use of pesticides to as little as 5%-10% [6]. For that purpose, robots need to implement plenty of different tasks such as localize VOLUME 9, 2021 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ 94981 J. Martin et al.: Generic ROS-Based Control Architecture for Pest Inspection and Treatment themselves [7] and navigate inside greenhouses [8], [9]; acquire quality pictures to identify pests and their locations [10]; or process the obtained results to generate efficient high-level instructions to command the robot according to an Integrated Pest Management (IPM) system [11]. However, most research works focus on individual problems neglecting its integration within a single complete solution. The combination of different robotic skills can be difficult and usually derive to ad-hoc solutions, but this is necessary to perform early pest detection. The insect in their early eggs state can measure as less as 0.3 mm and, to detect them, advance perception and dexterity skills need to be merged to automatically obtain close and good quality pictures of the pests from different sides of the leaves. This work presents Robotframework, a novel robotic architecture that integrates navigation, manipulation, and perception skills while following high level instructions from an IPM decision support system for early pest detection and treatment in greenhouses. The architecture includes additional features that makes it easily applicable for similar precision agriculture applications where robot navigation, manipulation and perception skills are required. This generic architecture can remarkably reduce the development time required to perform Robot Operating System (ROS) based field robotic experiments due to efficient reuse of common modules across projects and robot platforms. To demonstrate the easy integration and the benefits of combining different robotic skills within the architecture, flexible manipulation strategies to enhance pest detection and targeted spraying have been developed. Finally to evaluate the architecture, several tests in simulated and field commercial greenhouses have been performed in the context of the European GreenPatrol project [13]. This paper is structured as follows: The related work is analyzed in Section 2. Section 3 introduces the challenges for performing autonomous pest detection and treatment and the main robotic system requirements. Section 4 describes the developed Robotframework architecture while Section 5 focuses on the manipulation strategies for enhancing pest detection and treatment. Section 6 introduces the simulated and field tests carried out to evaluate the system in greenhouse and industrial scenarios. Finally, the conclusions obtained from the assessment are discussed and the future work is presented. II. RELATED WORK Research on precision agriculture robotics has recently focused on two areas: (i) weed inspection and targeted spraying and (ii) fruit and vegetables harvesting robots [5]. The first area is mostly represented by outdoor robots for weed control such as the Graph Weeds Net [14], the RHEA project centered on both agriculture and forestry [15], BoniRob project dedicated to multipurpose farming [16], or CROPS project focused on precision spraying in vineyards [17]. The navigation of these outdoor robots is largely based on the use of satellite localization systems and their signal is much weaker and unprecise in indoor environments, making them less suitable for greenhouses [18]. Moreover, greenhouses are specially challenging for simultaneous localization and mapping solutions, as they are partially structured environments with constantly growing plants [19]. That is why most of the robots found in greenhouses use rails to navigate on it [20]. Some examples are the tomato harvesting robot [21], the pepper harvesting robot [22] or the cherry tomato harvesting robot [23]. Other examples are AURORA, a spraying robot that implements a wall following algorithm for navigation [24] or a greenhouse spraying robot that follows lines and QR codes for navigating [25]. The use of fixed paths such as rails for navigation in greenhouses has resulted in decoupling navigation from the manipulation and inspection tasks. This is the case of the CROPS robot framework [26], where the control architecture covers only the fruit localization and arm control functionalities. As demonstrated by the open source control architecture FroboMind [12], a common reusable architecture that combines different robotic skills and tailored to precision agriculture robots can significantly decrease development time and resources due to efficient reuse of existing work across projects. However, despite using ROS as communication middleware, BoniRob is outdated as it does not integrate the state-of-the-art accepted navigation_stack [27] for navigation or MoveIt! [28] for manipulation. The first package takes information from odometry, sensor streams, and a goal pose and outputs safe velocity commands that are sent to the mobile base. The second one is the most widely used software for manipulation and provides the latest advances in motion planning, manipulation, or 3D perception, among others. In order to combine both algorithms for mobile manipulators, some authors have tried to simultaneously plan and execute mobile manipulation goals [29], [30] but these are computationally complex methods tested in simulation or laboratory conditions and currently unfeasible for greenhouse-like unstructured environments. Developing an effective IPM requires frequent and precise observations of plants. To build an early pest detection it may not be enough to focus on plants or leaves that are already infected with insects at adult stages as in [32]–[34], but it is necessary to detect the cause of the infection. In order to enhance the early pest detection, it is necessary to go a step further and detect the insects also in their egg and larva stages [10], [35]. Moreover, most pest detection works focus on the detection and classification of pests on already acquired pictures dataset neglecting the difficulties of automatically obtaining them with enough quality and closeness. In this sense, this work presents the manipulation strategies developed to get closer pictures to the surfaces of the leaves from above and from below, so as to inspect the surfaces of the leaves from both sides. Finally, there are several European projects as DROPSA [36], ISEFOR [37], PALMPROTECT [38] or EMPHASIS [39] focused on the development of new fighting strategies against some specific pests, but the bridge between new pest 94982 VOLUME 9, 2021 J. Martin et al.: Generic ROS-Based Control Architecture for Pest Inspection and Treatment FIGURE 1. Tomato crop greenhouse evolution at the beginning (up) and at the end (down) of the season. detection strategies and automated and robust management is barely addressed. This work presents, similarly to [31], a decoupled mobile manipulation control for greenhouse related tasks using the ROS de-facto algorithms [27] and [28]. The navigation is based on latest robotic solutions which have proven to successfully use Galileo Satellites combined with IMU, odometry and range laser sensors for localization [8]. The control architecture follows the hybrid paradigm presented in [40], where rational and efficient deliberative decisions represented by an IPM strategy are combined with reactive behaviors represented by the different navigation, manipulation and vision modules. III. PROBLEM DESCRIPTION AND ARCHITECTURE REQUIREMENTS There are several challenges for developing a robotic system able to perform autonomous and continuous monitoring in greenhouses for the detection, identification, and control of pests. As shown in Figure 1, the plants grow remarkably during the growing season affecting: (1) the localization and navigation systems because of a constant change of the environment and the narrowing of the corridors; (2) the manipulation strategy, as the arm needs to approach the leaves to obtain good quality pictures while avoiding damaging the crops; and (3) the vision modules dealing with changes in illumination and focus distance. In addition, the system must be able to execute high-level instructions proposed by the IPM strategy, providing diagnosis and logging capabilities and offering an easy-to-use user interface. The main robotic system requirements presented in Table 1 have been identified by observing a single robot needs for the GreenPatrol application. It is however noticeable that most requirements remarked in bold are desirable for almost TABLE 1. Main mobile manipulator system requirements. any other mobile manipulator system. Robotframework takes all these requirements into account and presents an architecture that is not only valid for the current application but also for other agricultural or even industrial applications. IV. SYSTEM ARCHITECTURE This Section contains a description of the general control architecture presented in Figure 2. The four-layered architecture seeks the easy integration of the different robot functionalities ensuring the system requirements presented in Table 1. It follows a distributed computing design allowing several tasks to run in different computers while still appearing to its users as a single coherent system and allowing an easy extensibility. ROS [41] is proposed as core communication middleware among the different modules. In recent years, ROS has become the de facto standard framework for the development of software in robotics. ROS is a flexible open-source framework for writing robot software that provides, collection of communication mechanisms, tools, libraries, and rules that aim to simplify the task of creating robot software for a wide variety of robotic platforms. In the architecture, there are several modules that are common to any application. These are represented in turquoise color and include the robot user interface, as well as application layer, the error managing and logging modules and some common parts of the abilities layer. Some other modules composed by standard ROS modules or packages developed and tested by the GreenPatrol project are available in the architecture, but it is up to the user to use them or implement new modules using the available ones as templates. The drivers layer, for instance, depends on the robot used. Also, the high-level decision modules, here represented as an IPM system, depends on the application. A. DECISION LAYER The decision layer contains the high-level decision modules that generate new plans for the application. In this case, VOLUME 9, 2021 94983 J. Martin et al.: Generic ROS-Based Control Architecture for Pest Inspection and Treatment FIGURE 2. Four-layer Robotframework control architecture. Modules represented with turquoise color are common to any application. Grey modules are standard ROS modules while the rest have been developed and tested during the GreenPatrol project and could be used as templates. FIGURE 3. Greenhouse representation with a robot approaching the red circles representing the targets with navigation and pest inspection tasks (left). Representation of an IPM generated T1-T4 plan (right). an IPM strategy generates pest scouting and treatment plans based on domain expert knowledge, crops distribution in the greenhouse and information obtained from previous plan executions as seen in the top layer of Figure 2. The plans are composed by targets that contain a navigation goal to move the robot to a desired position and a task to be performed there. An example plan for the current application can be seen in Figure 3 where the robot must navigate to four different greenhouse zones and perform there an inspection task. The Robot GUI module in Figure 2 is common for any application and provides an easy-to-use user interface to load, FIGURE 4. Robot GUI interface used to load and start a json plan (left) and the application workflow messages once the plan has been initialized (right). execute or cancel plans fulfilling system requirement SR7. The plan is then sent to the application layer to be interpreted and executed by the robot while the GUI displays the current state of the system including status, alerts, or the batteries level as shown in Figure 4. The plans are implemented using a JavaScript Object Notation (JSON) format, which is a very common open standard 94984 VOLUME 9, 2021 J. Martin et al.: Generic ROS-Based Control Architecture for Pest Inspection and Treatment FIGURE 5. State machine representing the operation mode targets plan and its navigation, tasks, and error-handling behavior states. and language-independent, that uses human-readable text to store and transmit data objects consisting of attribute–value pairs and array data. The JSON keywords related to navigation are: •navigate: Indicates whether the target contains a navigation step. The following navigation keys are only considered if navigate is true. •navigationType: Three types of robot navigation are available. (1) Natural navigation making use of the well-known navigation_stack from ROS; (2) Relative navigation to perform a continuous motion to reach a position relative to the robot’s current position; and (3) Precise navigation to generate a continuous motion to position the robot accurately with respect to an artificial mark. Only the first type is used in the GreenPatrol context. •navigationTrials: Number of trials for navigation in case of failure. •targetPose: Navigation destination (x, y, theta) in the given frame_id sent to the navigation node. The JSON keywords related to task definition: •tasks: An array of tasks to be executed. – name: Name of the task. – type: Type of the task plugin that will be loaded and executed. In this case it may be an inspection or a spraying task. – params: Necessary parameters to perform the task. This enables a high configurability of the IPM strategy to define the zones to be inspected or the amount of pesticide to use depending on the infection level of the plant. This method permits orders parametrization that covers a wide range of mobile robotics applications. The plans can be generated manually or automatically by different high-level decision support systems, addressing system requirement SR4. Section 5 presents two simple plan examples for pest inspection and treatment operations and subsection 6-C presents an additional plan for an aileron inspection in an industrial use case. B. APPLICATION LAYER The application layer includes the robot manager module, which is responsible for controlling the overall robotic system, maintaining its status continuously. This layer also includes the operation mode, which interprets the high-level plans and implements a specific robotic application. The operation mode is designed to be as general as possible, easily configurable for a variety of robotic processes and, thus, avoiding an ad-hoc implementation only useful for specific workspace configurations. A plan can be specified by a set of targets composed by a navigation destination for the mobile platform and a set of tasks to be performed at each destination. Figure 5 shows the state machine implemented for the operation mode. It starts in a Waiting state until a new-plan event indicates the beginning of the operation. The system switches then to Checking next target state, analyzing the next target in the sequence. The Navigating state is responsible for coordinating the autonomous movements of the mobile platform. The architecture permits an easy integration of different navigation modules and provides the management of their results. If the navigation is not able to reach the target pose due to obstacles on the way or localization problems, this will be notified in the result and, if required, a Navigation Recovery Behavior is triggered. Due to the criticality of the satellite-based localization in greenhouses to reach a position accurately, it has been VOLUME 9, 2021 94985 J. Martin et al.: Generic ROS-Based Control Architecture for Pest Inspection and Treatment necessary to include an additional feature in the navigation stack to provide information about the localization quality. In case of a remarkable localization quality loss, a specific error recovery behavior can be triggered. This behavior consists in sending the robot to a well-known greenhouse position where the localization signal is known to be strong and retrying from there the previous navigation goal. There is a second recovery behavior triggered when, despite having a proper localization signal, the robot does not reach the destination with enough accuracy. This can happen because a slightly better localization is needed or because there are obstacles on the way that the navigation module cannot overcome. In both cases, the recovery behavior consists off waiting for a predefined time still, while playing an advertisement sound. Waiting may help improving the localization while the sound notifies the operators in the vicinity about the current robot state and, if needed, about the need of removing the obstacle on the way. The number of trials to perform the navigation are configurable. If there are no more trials left, meaning the robot failed to reach the destination, the failure is notified in the running navigation failure behavior state, and the tasks to be performed at this point are skipped, addressing the following target. The navigation state has been developed in a generic way to support different global, relative, and precise navigation modules as it will be explained later. Once the navigation finishes correctly, the configured set of tasks are executed in the Doing Tasks state. A Task is the implementation of a robots’ specific set of actions. The proposed architecture is designed to implement new robotic tasks by using the ROS pluginlib mechanism. The tasks are developed as plugins which are parametrized, dynamically loadable and executed from a runtime library. The plan generated by the high-level decision module must contain enough information for the state machine to understand where to go and which actions to take at each place. There is therefore no need to touch or recompile the core of the framework. This is useful for extending/modifying the application and provide a great extendibility to the system. Section V present two task implementations in the context of precision agriculture for pest detection and treatment while Section VI-C illustrates an additional task example for an aileron inspection in an industrial context. The benefits and reusability of the architecture is finally described in the discussions section. Moreover, the here presented task plugins can be used as templates and be adapted for future tasks. C. ABILITIES LAYER This layer is composed by the ROS nodes involved in the basic control functionalities of a robot. These nodes manage the sensor and actuator components, and provide robot capabilities such as autonomous navigation, manipulation, and inspection. At this level, ROS provides a wide range of stateof-the-art robotic algorithms: GMaping [42] for generating maps using the on board 2D laser scanners. The maps can be manually modified to include, for instance, forbidden areas FIGURE 6. GreenPatrol robot description in ROS-visualization RViz. It presents the main platform components (mobile platform, arm, sensors...) and the transformation links between them. for the robot; the Navigation Stack used in the Navigation State for planning global and local paths. It uses combined 2D laser scanners and satellite based localization to generate the velocity commands for the mobile base while avoiding the obstacles on the unstructured greenhouse environment; the Unified Robot Description URDF for generating a combined robot description as presented in Figure 6; MoveIt! is used for generating and executing collision free manipulation trajectories in the Doing Tasks State as it will be later presented in Section V. MoveIt! tools can be also used, to integrate 3D point cloud based obstacles or useful simulation tools among other utilities. On top of them, several additional nodes have been developed. To ensure the system requirement SR1 and freely navigate within the greenhouse, the localization module benefits from the multiple signal frequencies and the higher accuracy provided by the European Global Navigation Satellite System (EGNSS) of the Galileo constellation as explained in [8]. The system requirement SR3 is achieved using a deep learning model for detecting the most harmful pests in greenhouse tomato crops: Bemisia Tabaci, Tuta Absoluta and Whitefly [10]. In addition, a leaf detection deep learning model has been implemented to safely and accurately detect and approach individual leaves using a 3D camera. Then, closer and high resolution pictures of the pests are taken as presented in Section 5, answering to system requirement SR2. The architecture includes additional modules for relative and precise navigation which are valuable for a wide range of mobile robotic applications as shown in subsection 6-C. D. DRIVERS LAYER The drivers layer includes the modules that allow interacting with the robot platform sensors and actuators. An overview of the specific robotic system used for validation purposes is shown in Figure 7. 94986 VOLUME 9, 2021 J. Martin et al.: Generic ROS-Based Control Architecture for Pest Inspection and Treatment FIGURE 7. GreenPatrol robotic platform entering the greenhouse. FIGURE 8. GreenPatrol pest inspection and treatment tools mounted at the robot arms end-effector. The mobile platform consists on the Segway RMP 440 Omni Flex [43] with mecanum wheels to improve mobility in greenhouse narrow corridors. The platform is equipped with an on-board PC, a Velodyne 3D laser scanner [44] for obstacle detection and two OS32C safety laser scanners [45] for obstacle detection, mapping and navigation. The absolute localization unit consist of a multi-constellation, GNSS receiver, IMU and odometry. A KUKA LBR iiwa manipulator [46] has been mounted on the middle of the platform to allow inspecting the leaves on the right and left sides. The vision system consists of a 3D RealSense camera [47] to find leaves positions and an IDS RGB autofocus camera [48] to acquire good quality pictures of the pests as seen in Figure 8. The spraying equipment consists of a plastic tank on the back-right corner of the platform, an electric compressor with a pipe and a spraying nozzle at the arm’s end-effector shown in Figure 8. A benefit of using ROS is the availability of a wide variety of robotic components drivers such as mobile robots, manipulators, cameras and, lasers. This makes the architecture hardware agnostic enabling the possibility to replace them without affecting the rest of the architecture. E. MONITORING The three modules shown on the left side of Figure 2 are available with the architecture to monitor the functional state of the system. The Diagnostics module has been designed for collecting and preprocessing specific data from drivers and abilities layers which are then passed to the Diagnostics Manager for automatic decision making and incidents notification fulfilling system requirement SR5. These two modules must be adapted to the application on demand. Moreover, the generated DEBUG, INFO, WARNING and ERROR messages are recorded by the Logging module using a RabbitMQ [49] queue that implements the Advanced Message Queuing Protocol (AMQP). The logs are used to record historical track of the process, ensuring SR6, and can either be stored locally in the robot or in the cloud using a nonrelational Elasticsearch database [50]. This data has been later used to obtain tests results and statistics. The following Section shows how to include new ad-hoc modules and tasks within the architecture. In particular, the integration of manipulation strategies for enhancing pest detection and treatment operations are presented. V. MANIPULATION STRATEGIES FOR PEST DETECTION AND TREATMENT The high-level decision support system (in this case the IPM strategy) defines the manipulation, inspection, and treatment tasks to be performed. First, the mobile platform needs to navigate to the target plants as seen in Section 4-A. Once in front of the plant, the robotic arm mounted on the middle-top of the mobile platform performs the corresponding pest inspection or treatment task on right and left sides of the platform. This Section presents the strategies taken, the execution workflow and examples of simple plans for each manipulation task. The tasks have been developed as plugins and represent standalone integration cases withing the architecture presented here. A. PEST INSPECTION TASK The plant zones to be inspected and the number of pictures that need to be taken at each zone are represented as the Pest Monitoring Index (PMI) in Table 2 and Figure 9. Lower and darker zones tend to provide more suitable habitats for the pests, resulting on a higher number of pictures required. As an example, in the high-up zone the robot must inspect leaves above 1m (PMI6) and requires two pictures to be taken, while in the middle-bottom zone the robot must inspect leaves from bellow in between 0.5 m and 1 m (PMI2) and requires 4 pictures to be taken. A simple inspection plan is shown in Figure 11. VOLUME 9, 2021 94987 J. Martin et al.: Generic ROS-Based Control Architecture for Pest Inspection and Treatment TABLE 2. Pest monitoring index for defining the number of pictures to take at each plant zone. FIGURE 9. The GreenPatrol robot appears facing the plant at the high-up zone inspection position in Gazebo simulation. The manipulation strategy for pest detection consists of the workflow defined in Figure 10 (up). First, the arm is moved to the next inspection zone. Second, the leaf detector model and the RGBD image are used to find leaves poses. If no leaf is found, the arm is moved to the following inspection zone. Third, the arm approaches the leaves found in the previous step and takes closer pictures of them using the RGB autofocus camera. An algorithm determines the quality of the picture. If the quality is not good enough, the arm makes a predefined small movement, and takes a new picture from there. This process is repeated until all required plant zones have been inspected. The pictures taken in this process are saved locally on the robot. After completing the plan, the pictures are sent to the cloud, where a Deep Learning (DL) model has been deployed to identify infection areas in the greenhouse offline. The IPM strategy module uses the DL module results along FIGURE 11. Example of a simple GreenPatrol inspection plan. FIGURE 12. Example of a simple GreenPatrol spraying plan. with additional information such as the current harvest season conditions, the working area size, the size of the plant or legal aspects on pesticides on the working country. As a result, new inspection (Figure 11) and treatment (Figure 12) plans are generated. FIGURE 10. Manipulation strategies workflows for pest detection task (up) and pest treatment task (down). 94988 VOLUME 9, 2021 J. Martin et al.: Generic ROS-Based Control Architecture for Pest Inspection and Treatment B. PEST TREATMENT TASK The pest treatment process can be defined as the precise spraying of pesticide on different plant zones (high, middle and low), being the pesticide spraying dose at each plant determined by IPM strategy as shown in the parameters field in Figure 12. The manipulation strategy is represented by the workflow defined in Figure 10 (down). First, the arm is moved to the next spraying zone. Second, the sprayer is activated and in order to cover the whole plant zone, the manipulator performs small, controlled movements until the complete dose has been sprayed. This process is repeated until all required plant zones have been sprayed. VI. SYSTEM VALIDATION This section presents the validation tests performed within the simulated and real greenhouses of 52×30m and 31 corridors FIGURE 13. Details of the simulated environment in Gazebo simulator with the robot performing navigation and inspection tasks. between plants shown in Figure 13 and Figure 1 respectively. The aim of these tests has been the assessment of the following features: first, the correct integration of Robotframework with the different robotic modules; second, successful execution of pest inspection and treatment plans; third, the logging capabilities of the system to generate and use the collected data; finally, the system requirements proposed in Table 1. The results and the most remarkable conclusions are detailed at the end of each test. Furthermore, the use of Robotframework in an industrial application is presented to demonstrate its adaptability and generalization. A. GREENHOUSE SIMULATION TESTS Gazebo simulator [51] has been used to simulate the crops, robot sensory information (laser, images...), physics involved (collisions, inertia...) and localization data (global coordinates, errors...). The leaves, despite realistic, do not perfectly represent the real world and do not contain insects on them. Thus, a simulated vision module for leaf detection provides their position. Also, the images used for validating the pest detection and identification modules are semi-randomly acquired from our custom dataset of labelled images (a set not used for training the model) with infected and healthy images. The same software as in the real scenario has been used. The simulation test presented in Figure 14 consists of the following steps: First, an IPM algorithm generates a new semi random scouting plan based on the greenhouse dimensions, which in this case corresponds to 31 rows (R1-R31) in the horizontal axis and 6 vertical zones that, in turn, consist of FIGURE 14. Representation of the greenhouse inspected/sprayed zones during the different simulation test-steps: Initial semirandom scouting plan (up). Scouting results representing the infected and healthy zones (middle). Spraying plan generated for infected zones (down). VOLUME 9, 2021 94989