Application of A Computer Vision Method for Soiling Recognition in Photovoltaic Modules for Autonomous Cleaning RobotsF

Signal & Image Processing: An International Journal (SIPIJ) Vol.10, No.3, June 2019 DOI: 10.5121/sipij.2019.10305 43 APPLICATION OF A COMPUTER VISION METHOD FOR SOILING RECOGNITION IN PHOTOVOLTAIC MODULES FOR AUTONOMOUS CLEANING ROBOTS Tatiani Pivem1, Felipe de Oliveira de Araujo2, Laura de Oliveira de Araujo2, Gustavo Spontoni de Oliveira 2 1Federal University of Mato Grosso do Sul - UFMS, Campo Grande, Brazil 2Nexsolar Energy Solutions, Campo Grande, Brazil ABSTRACT It is well known that this soiling can reduce the generation efficiency in PV system. In some case according to the literature of loss of energy production in photovoltaic systems can reach up to 50%. In the industry there are various types of cleaning robots, they can substitute the human action, reducing cleaning cost, be used in places where access is difficult, and increasing significantly the gain of the systems. In this paper we present an application of computer vision method for soiling recognition in photovoltaic modules for autonomous cleaning robots. Our method extends classic CV algorithm such Region Growing and the Hough. Additionally, we adopt a pre-processing technique based on Top Hat and Edge detection filters. We have performed a set of experiments to test and validate this method. The article concludes that the developed method can bring more intelligence to photovoltaic cleaning robots. KEYWORDS Solar Panel, Soiling Identification, Cartesian Robots, Autonomous Robots, Computer Vision 1. INTRODUCTION Photovoltaic (PV) solar energy is one of the best options to meet future energy demand, and a more sustainable future, since it is superior in terms of availability, cost effectiveness, accessibility, capacity, and efficiency compared to other renewable energy sources [1]. The market of PV solar energy is in constant growing [2], however after the installation of the generator PV system, due to climate events and region [3], it is common the presence of dust and soiling covering the panels. It is well known that this soiling can reduce the generation efficiency in PV system. In [4] show that average deduction in power output is different varying from region to region, for example from an increase scale, power output reduces in USA 1– 4.7%, Egypt 33– 65%, Saudi 40% and in Kuwait round 65%. This information show that the efficiency can be impacted in various levels, and in some cases, if there is a large soiling deposit, the lost can achieve more than 50% of the production when the PV solar panel is not clean. In the industry there are various types of cleaning robots, they can substitute the human action, achieving difficult regions and increasing significantly the gain of the systems. These robots are positioned in the regions for the clean, or send to the direct position, but they don’t have a central decision with embedded logic. Signal & Image Processing: An International Journal (SIPIJ) Vol.10, No.3, June 2019 44 Robotic platforms can be combined with embedded algorithms, using Internet of Things (IoT) it and artificial intelligence algorithms, like computer vision techniques or convolutional neural networks. That are a various mode of operation like visual inspection [5,6], infrared thermography [7, 8, 9] for hot spots detection and operation of the PV plants and Electroluminescence (EL), method capable of detected beyond soiling and achieve damage detection, but requires detailed analysis of each solar panel cell. In the work [10] provides a probabilistic methodology for identifying various types of damage for solar cells. The author utilizes Support Vector Machines (SVMs) techniques and Convolutional Neural Network (CNN) for the data analysis, and conclude that SVM methodology requires less processing effort than CNN, but CNN achieves better results. For [10] it is feasible embedded the software in tablet computer or drones, however there is still challenges outside of a manufacturing setting. In the market is showed a large variety of this robots, classified in operational and maintenance [11,12], climbing robots, and aerial monitoring. For the monitoring it is more common the use of satellite images, images acquired by drone, or even a local fixed monitoring. It is also common to see operational cleaning robots with a rigid sustentation structure and a rotating brush. Operational and maintenance using ground robots is another inspection alternative for monitoring installations PV generator. Nowadays, significant effort has already been made in several fields, they have been made for support to human process, being at levels of autonomous [13]. Two categories can be highlighted, the defect detection and the cleaning process [14], instead of spend time from collaborators of a plant to clean the installation, it is even more practice to use the robot mechanism. The defects from a panel can vary, being superficial or deeper inside the panel. These defects associated with shadowing and soiling can be a focal source of problem [15]. Early warning of these failure conditions enables to find appropriate solutions to improve efficiency, while enhancing the PV system economics and reducing the human labour cost. Technologies for plant monitoring can be classified in two groups: invasive and non-invasive. Invasive technology uses the module information, electrical like current and voltage or even the temperature. This information is used to find a fault condition. This way there are methods based in the AC condition [16], Predictive control [17], Bayesian Networks [18], and comparation methods [19, 20].Cartesians robots make linear motion in direction of three axes X, Y and Z [21]. The developed robot is a simple cartesian. Its main goal is to move in the horizontal and then vertical path. The start is the horizontal movement in the string of panels. After achieving the right PV panel previous selected, it is started the cleaning system. Cleaning system is responsible for making the PV panels clean, it is performed for a region that has a brush, fixed in the rigid structure, but with vertical moving freedom. While the vertical is executed, there is a water flow coming to the panel via a hydraulic system. Then, the brush starts to rotate in order to remove the soiling from the panel. The implemented robot is showed in the Figure 1. Signal & Image Processing: An International Journal (SIPIJ) Vol.10, No.3, June 2019 45 Figure 1: Cartesian robot developed The robot also has a control system, monitoring the climate variables like irradiation, temperature and humidity. It is proposed to join an algorithm for image soiling recognition, the system is expected to be more efficient and autonomous. The development of cleaning techniques is made more essential every day. Computer vision is in the focus of the state-of-the-art techniques that make analysing the existence of dirtiness and decides the right moment to clean. These techniques are used with the goal to determine the particle size as presented in [22], and even the clearness type in various ways like the study of electrical characteristics as presented in [23], or by analysing the existence of dirtiness and the right moment to clean as showed by [6]. Another example is about the dirtiness classification as showed by [24], and with histogram analysis as presented in [25]. The author in [6] work promotes the dirtiness recognition making use of Mahalanobis calculus and sets a threshold value for accepted loses in the system in order to generate a date clean indicator with the correct date to make the cleaning operation. The system in question has a little brush and vacuum to clean the PV solar panels. The author in [24], classifies various dirtiness types based on each pattern achieved in its covered area surface as snow, bird drop, crack, etc. The author in [25] verifies the covered area over a surface using one dirtiness type and analysing the area percentage covered. From this step the histograms are taken for analysis in the image pixels variation, they present that a heavy dusty deposition can generate a thin peak in high amplitudes for the histogram. This work proposes a new algorithm for the cleaning panels that can be used embedded in an autonomous robot for cleaning installation of solar panels. The article is divided as follows: in section 1 is presented in the introduction, in section 2 is presented concepts and technique of image processing used, in section 3 is presented the strategy of implementation of the algorithm developed to identify the types of dirt present in PV panels, section 4 shows the experiments and results and in section 5 a conclusion of the work is presented. Signal & Image Processing: An International Journal (SIPIJ) Vol.10, No.3, June 2019 46 2. IMAGE PROCESSING CONCEPTS 2.1. Segmentation The segmentation principle was first introduced in the early 20th century by psychologists Khler, Wertheimer and Koffka, which showed that the human vision performs groupings based on the similarity and continuity of the captured images. These groupings were the motivation of segmentation studies that extended the concept computationally. Search algorithms used to find edges, textures, patterns or differences, for example, in the cases of cancer recognition in the medical area. In this work we use the Gonzales taxonomy [26] which classifies the methods in three major areas: Thresholding, edge detection and Region-based detection. It can be said mathematically that the constitution methods segmentation follows: Being divisions performed in the image and R the complete image given by region summation. The same inference can be made by edges which give us the main edge as the result obtained by summation of minor edges 2.2. Thresholding This process consists in the object segmentation based on intensity levels, taking all pixels belonging to a level as a distinct object. In other words, the segmented region is classified if it belongs to a certain region level, it can be said mathematically as follows: If Where the selected is object and are the thresholding values. The most intuitive method is based on manual tries and observations of results and can also be reached by making use of histograms. The histogram shows the number of pixels present in the image for each intensity light level, so that for a distinct image with homogeneous and distinct background it is presented a bimodal histogram with a valley. The regions can be separated in background and front, which have distinct intensity levels separated by the valley. In this way it is possible to perform the selections based on the histogram valleys, first coming from the valley analysis and following by spacing the valley with Otsu’s method, which insure the maximum variance. 2.3. Edge Segmentation The edge segmentation methods occur where there is an abrupt grey level variation. Once the process is done, it is necessary to fill the area between these points. The process that performs this task is called linking algorithm and uses more complex math to achieve its goal. When the intensity varies abruptly, the algorithm uses differential operators to make the detection, given that in one discontinuity the differential value skyrockets. The gradient method can be showed mathematically as: Signal & Image Processing: An International Journal (SIPIJ) Vol.10, No.3, June 2019 47 The module and direction by: Linking Algorithms: The linking algorithms can produce vectorial contours by highlight the edges, that is, first an algorithm for abrupt detection is used and each abrupt detection is made and highlighted. Knowing that images can have noise, it is not common to achieve perfect edge detection, but reasonable results can be obtained. Then the linking methods are applied in order to fill the discontinuities in those edges. The most known methods are the Hough transform and the path finding graph. Local Processing: One of the simplest approaches for linking edges points is to analyse the characteristics of pixels in a small neighbourhood and select all points that are similar according to a set of predefined criteria. In a gray scale, we can say that objects can only belong to a same type if they are presented in the same range of gray level criteria. The edge is found by the difference in the values of pixels (threshold), if the mean value from the levels in this edge doesn’t differ too much amongst themselves, the code finds the edge. It can be said mathematically as follows: The difference between a point and its neighbor needs to be smaller than a predefined value. This calculation is made in a small neighborhood with connectivity 3x3, 5x5 or other by the analyst choice. A good point to add is the possibility to reduce computational cost by changing the coordinate system with acquisition of a gradient angle to a vectorized system. The Hough method is used in order to achieve specific contours inside the image, as a straight, circle or ellipse, and make the coordinate system polynomial, in order to achieve the form that better adapts to the region. The pathfinding graph method is in general more complicated, with more processing time, but with better results in a noisy ambient. The method consists in the choice of graphs with the smaller computational cost, by testing all the viable paths as every graph is a combination of vertices and arcs. Signal & Image Processing: An International Journal (SIPIJ) Vol.10, No.3, June 2019 48 2.4. Region Segmentation While thresholding and edge detection find differences between pixels, the region segmentation is more focused in the similarities in the regions. The Region Growing technique is the most well-known approach and is the major tool in many publications in the state of the art [26]. Its proposal is to joint pixels with the same characteristics, as intensity gray level, texture, color, etc. It can be consisted as the mean calculation of every pixel in the image considering each pixel in the same neighborhood and going forward with the actualization of this calculus. The main goal is to verify which pixels have the same mean expected, that is, the difference between the pixel intensity and the mean neighborhood must be less than a critical value, called threshold. This is made by grouping the same tons of color, but the process can be made with other specifications. The output mask is made by marking the points which are in the same region, which is then applied in the selected image. The mask composes the Region of Interest (ROI), the main goal from region segmentation algorithm 3. IMPLEMENTATION 3.1. Uniform Dirtiness The objective of this method is to expand small regions of pixels into bigger ones. The focus was in the region covered by one type of dirtiness. This approach tries to achieve a worst-case scenario in uniform deposition occurring by time in PV panels. First there are a set of seeds that search for a common characteristic, in this case scale gray level. The pixels which have the same tons are grouped together making the ROI. We created an image database (Figure 2) varying the wet sand applicated over a photovoltaic solar panel in the horizontal position on the ground so the sand wouldn’t slide. We repeated this process until obtaining a scale of dirtiness, as if it was achieved through a long period of time with real dirtiness exposure. Considering the color of the ground having gray level intensity similar to the dirtiness over the panel (both brown), the method considered just the intensity from dirtiness (lighter brown). Seeds were thrown and grew over the dirtiness making a direct identifying. It was made a pre-processing using the median filter of size 3x3, which doesn’t influence the edge detection, but uniformizes the inside of the regions increasing the precision of segmentation. The first analysis is performed with the application of region growing algorithm, where is obtained a “mask”, the region of interest to be used in the original image. There was a need to create a double matrix for the image and the method was applied by channel, that is, Red, Green and Blue (RGB). In this way, every layer from the image could be concatenated, generating three histograms to be analyzed at chapter Tests. The histogram is taken by the grown dirtiness, so with more dirtiness the mask gets amplified, letting more dirtiness in the mask and thus, increases the histograms. It can be verified that with more dirtiness, the amplitude of the channels on the histograms will be higher. Signal & Image Processing: An International Journal (SIPIJ) Vol.10, No.3, June 2019 49 Mathematically we developed this approach in order to calculate the extension of dirtiness: Figure 2: Image database example 3.2. Non-Uniform Considering the non-uniform dirtiness distribution over the photovoltaic solar panel, we applied more wet sand, threw more water and some stones, representing the worst non-uniform scenario. Once again, we use sand as the dirtiness type and we analyze the recognition of the covered area. For this method the region growing technique had difficulties, because the growing stops when another region is achieved. Just throwing a seed in the center does not guarantee success as in the first method. Beside this, the ground is like the dirtiness, taking this information we used the data corresponding to the layers RGB. As expected, the brown color is composed by a major parcel of the red color, showing more contrast with the blue from the panel as presented by Figure 3. Figure 3: RGB separation layers, non-uniform dirtiness For this new method, the red layer was used, like before. In the image we initialize some seeds, these seeds grew up and returned the regions. We sorted the regions and took the predominant two areas. These two regions are analyzed in the section 4.2. Signal & Image Processing: An International Journal (SIPIJ) Vol.10, No.3, June 2019 50 3.3. Punctual Dirtiness In some residential areas especially, there are quite a few quantities of heavy particles in the air. It is common for solar panel to be afflicted by bird dropping, or the amount caused by time stacked with hard attraction in the particles making some types of circular draws as showed by Figure 4, a real photovoltaic solar panel dirty by the time. For localization of these punctual dirtiness we propose the Hough algorithm, which performs a calculation in neighborhood pixels looking for a ratio that constitutes a fill circle. If this ratio exists the method highlights the region, finding the circular area. If the ratio is not found, the next pixel is analyzed. Figure 4: PV Panel dirty by the time 4. EXPERIMENTS AND RESULTS 4.1. Applications for photovoltaic solar panels with uniform dirtiness In this section we discuss the results of the application showed before about uniform dirtiness. We did those tests by applying the region growing algorithm in three scaled images, as presented by Figure 5. The first panel 5(a) is the higher level of dirtiness of this experiment, the second panel 5(b) is a medium level of dirtiness and the third panel 5(c) is the smaller level of dirtiness. In Figure 6 we show the masks already applied in real dirtiness, obtained in sequence 6(a),6(b),6(c) for the Figure 5. We also have, in Figure 7, Figure 8 and Figure 9 the histograms and means analysis. We can see that there is a strong correlation between dirtiness and histograms amplitudes and means. This way, we can demonstrate the identification of uniform dirtiness in PV panels. Signal & Image Processing: An International Journal (SIPIJ) Vol.10, No.3, June 2019 51 Figure 5: Levels of dirtiness Figure 6: Masks obtained Figure 7: First panel - high level of dirtiness Figure 8: Second panel - medium level of dirtiness Signal & Image Processing: An International Journal (SIPIJ) Vol.10, No.3, June 2019 58 [13] SMP Robotics. S5 PTZ Security Robot—Rapid Deployment Surveillance System. 2016. Available online:https://smprobotics.com/security_robot/security-patrolrobot/rapid_deployment_surveillance_system/ (accessed on 20 May 2019). [14] Maurtua, I., Susperregi, L., Fernández, A., Tubío, C., Perez, C., Rodríguez, J., Ghrissi, M. (2014). MAINBOT – Mobile Robots for Inspection and Maintenance in Extensive Industrial Plants. Energy Procedia, 49, 1810–1819. doi:10.1016/j.egypro.2014.03.192 [15] Felsch, T., Strauss, G., Perez, C., Rego, J., Maurtua, I., Susperregi, L., & Rodríguez, J. (2015). Robotized Inspection of Vertical Structures of a Solar Power Plant Using NDT Techniques. Robotics, 4(2), 103–119. doi:10.3390/robotics4020103 [16] Kim, K.A.; Seo, G.S.; Cho, B.H.; Krein, P.T. Photovoltaic Hot-Spot Detection for Solar Panel Substrings Using AC Parameter Characterization. IEEE Trans. Power Electron. 2016, 31, 1121–1130. DOI 10.1109/TPEL.2015.2417548 [17] Samani L, Mirzaei R, Model Predictive Control Method to Achieve Maximum Power Point Tracking Without Additional Sensors in Stand-Alone Renewable Energy Systems, Optik (2019), https://doi.org/10.1016/j.ijleo.2019.04.067 [18] Daniel Riley and Jay Johnson, “Photovoltaic Prognostics and Heath Management using Learning Algorithms”, Photovoltaic Specialists Conference (PVSC), 2012 38th IEEE, DOI: 10.1109/PVSC.2012.6317887 [19] Zapata, J.W.; Perez, M.A.; Kouro, S.; Lensu, A.; Suuronen, A. Design of a Cleaning Program for a PV Plant Based on Analysis of Energy Losses. IEEE J. Photovolt. 2015, 5, 1748–1756. [20] Mohammad Hammouda,, Bassel Shokra, Ali Assia, Jaafar Hallala, Paul Khouryb . “Effect of dust cleaning on the enhancement of the power generation of a coastal PV-power plant at Zahrani Lebanon”, Solar Energy 184 (2019) 195–201. DOI: https://doi.org/10.1016/j.solener.2019.04.005 [21] Sonick Suri, Anjali Jain, Neelam Verma, Nopporn Prasertpoj. “SCARA Industrial Automation Robot”, 2018 International Conference on Power Energy, Environment and Intelligent Control (PEEIC) G. L. Bajaj Inst. of Technology and Management Greater Noida, U. P., India, Apr 13-14, 2018 [22] Biryukov, S.,Faiman, D., Goldfeld, A.: 'An optical system for the quantitative study of particulate contamination on solar collector surfaces' Solar Energy Vol. 66, No. 5, pp. 371–378, 1999 [23] Atten P., Pang H.L., Reboud J.L., D.: ' Study of dust removal by standing wave electric curtain for application to solar cells on mars '. IEEE Transactions on Industry Applications Vol.45, France, Jan 2009, pp. 75–86 [24] Mehta, Sachin & P. Azad, Amar & Chemmengath, Saneem & Raykar, Vikas & Kalyanraman, Shivkumar.: 'DeepSolarEye: Power Loss Prediction and Weakly Supervised Soiling Localization via Fully Convolutional Networks for Solar Panels '. IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, 2018, pp. 333-342. [25] WK Yap, R Galet, KC Yeo.: 'Quantitative analysis of dust and soiling on solar pv panels in the tropics utilizing image-processing methods'. Asia-Pacific Solar Research Conference, 2015 [26] Gonzales, R., Woods, R.: Digital Image Processing'. Vol.2 [27] Philipe A. Dias., Henry Medeiros.: 'Semantic Segmentation Refinement by Monte Carlo Region Growing of High Confidence Detections'. Cornell University Library, 2018, available at: https://arxiv.org/abs/1802.07789 Signal & Image Processing: An International Journal (SIPIJ) Vol.10, No.3, June 2019 59 AUTHORS Tatiani Pivem – was born in Assis-SP, on September 27, 1991. She has a degree in Electrical Engineering – USP 2015, Tatiani has experience in Research and Development, was CPqD intern in 2014, now she is at the master in Energy Efficiency and Sustainability at UFMS and works with Software Coordinator research and development at Nexsolar Felipe de Oliveira de Araujo – was born in Campo Grande, MS, Brazil on July 11, 1991.He has a degree in Electrical Engineering – UFMS. He has master in Energetic Efficiency and Sustainability. Worked in Spain and now he is CEO at Nexsolar. Your main goal is to make energy accessible for everyone. Felipe works as Director of R&D programs. Laura de Oliveira de Araujo – was born in Campo Grande, MS, Brazil on July 11, 1991. Ambiental Engineer – UFMS 2015. Laura has specialization at Work Security UFMS 2018. Laura works in solar energy area, she is CFO at Nexsolar, your main goal is to make energy accessible for everyone. Laura has specialization at Work Security UFMS 2018. Laura works in solar energy area, she is CFO at Nexsolar, your main goal is to make energy accessible for everyone. Gustavo Spontoni de Oliveira – was born in Campo Grande, MS, Brazil on March 03, 1995. He is currently a designer and researcher at Nexsolar. Civil Engineer, graduated from Anhanguera University - UNIDERP. Experience in structural masonry, concrete block manufacturing, photovoltaic systems and energy efficiency projects.