Transparent Layer Compensation at the Active Triangulation Systems

IFAC PapersOnLine 52-27 (2019) 508–513 ScienceDirect Available online at www.sciencedirect.com 2405-8963 © 2019, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Peer review under responsibility of International Federation of Automatic Control. 10.1016/j.ifacol.2019.12.714 © 2019, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. 10.1016/j.ifacol.2019.12.714 2405-8963 Transparent Layer Compensation at the Active Triangulation Systems I. Janakova*, S. Sediva**, P. Benes*** Department of Control and Instrumentation, Faculty of Electrical Engineering and Communication, Brno University of Technology, Brno, Czech Republic *(+420 541 146 390, e-mail: [email protected]) **(+420 541 146 418, e-mail: [email protected]) ***(+420 541 141 152, e-mail: [email protected]) Abstract: The paper deals with an active triangulation technique of non-contact optical measurement of 3D properties. The basic features of the method - principle, advantages and disadvantages - are described first. But the main part of the paper is concerned with the impossibility of using the basic principle of the method for measured objects, which are covered by a layer of transparent material of unknown thickness. Most often it is a layer of oil for the preservation of metal products. This layer distorts the measured value due to the light refraction. In this work, a method of eliminating this distortion based on the use of two sensors is proposed. Experimental verification of this proposal is also presented. Keywords: 3D vision, optical distance measurement, active triangulation, image processing, transparent layer, oil compensation. 1. INTRODUCTION Optical non-contact distant measuring techniques have found widespread use in many areas, for example in traffic, architecture, robot navigation, object recognition, or in industry in the production inspection (Kalová and Lisztwan, 2006). Their expansion was supported not only by the great development of technics (electronics, optics), but also by the progress made in techniques of data and image processing. The advantages of optical methods are that they can be very accurate and fast and against other principles that are noncontact and non-destructive. It is also easier to reconfigure the system to another dimension or type of products. In standard projective imaging, one dimension (usually the depth information) from 3D scenes is lost. Without the use of a special technique, it is not possible to determine this information retroactively. Fortunately, depending on the specific requirements, different technical approaches can be used for optical distance measurement (Berkovic and Shafir, 2012). The most used approaches are based on one of three basic principles: triangulation, time-of-flight (ToF) or optical interferometry (Schwarte et al., 1999). Among the most important parameters for selecting the most appropriate method include the required range of measured distances (dimensions of measured object and the working distance), resolution, measuring time, topology (concavities, protrusions, edges) and surface properties (roughness, specular reflections) of the measured object, properties of surrounding lighting, size of measuring system, possibilities for calibration, and also costs. The most widely used technique for optical form measurements is triangulation. Triangles are the basis of many measurement techniques. The following variants, which look greatly different, but use same principle can be distinguished: active triangulation techniques with different illumination, passive triangulation techniques on the basis of digital photogrammetry and stereoscopy, theodolite measuring systems, shape from shading techniques and focus techniques. The active triangulation technique is very often used in practice due to its robustness and efficiency, but it also has several limitations and disadvantages. In particular, this article focuses on the limitations of using this method for objects that are covered with a transparent layer, such as preserving oil. 2. ACTIVE TRIANGULATION 2.1 Principle The active triangulation technique is based on a photogrammetric reconstruction of the measured object by illumination its surface and contemporaneous scanning by a sensor (mostly PSD module or CCD or CMOS camera). The principle of this technique is shown in Fig. 1. The light source, the detector and the illuminated part of the measured object form a triangle. The join b between the light source and the detector is called a triangulation optical basis. The light source ray angle α is fixed whereas the angle on the detector side β changes depending on the variable distance z and it is defined by an illuminated point on the camera chip, respectively by a bright point on a produced image. If the investigated object is situated farther away from the camera and light source, the angle β is bigger so the reflected light ray is projected to the pixel more up (according to the Transparent Layer Compensation at the Active Triangulation Systems I. Janakova*, S. Sediva**, P. Benes*** Department of Control and Instrumentation, Faculty of Electrical Engineering and Communication, Brno University of Technology, Brno, Czech Republic *(+420 541 146 390, e-mail: [email protected]) **(+420 541 146 418, e-mail: [email protected]) ***(+420 541 141 152, e-mail: [email protected]) Abstract: The paper deals with an active triangulation technique of non-contact optical measurement of 3D properties. The basic features of the method - principle, advantages and disadvantages - are described first. But the main part of the paper is concerned with the impossibility of using the basic principle of the method for measured objects, which are covered by a layer of transparent material of unknown thickness. Most often it is a layer of oil for the preservation of metal products. This layer distorts the measured value due to the light refraction. In this work, a method of eliminating this distortion based on the use of two sensors is proposed. Experimental verification of this proposal is also presented. Keywords: 3D vision, optical distance measurement, active triangulation, image processing, transparent layer, oil compensation. 1. INTRODUCTION Optical non-contact distant measuring techniques have found widespread use in many areas, for example in traffic, architecture, robot navigation, object recognition, or in industry in the production inspection (Kalová and Lisztwan, 2006). Their expansion was supported not only by the great development of technics (electronics, optics), but also by the progress made in techniques of data and image processing. The advantages of optical methods are that they can be very accurate and fast and against other principles that are noncontact and non-destructive. It is also easier to reconfigure the system to another dimension or type of products. In standard projective imaging, one dimension (usually the depth information) from 3D scenes is lost. Without the use of a special technique, it is not possible to determine this information retroactively. Fortunately, depending on the specific requirements, different technical approaches can be used for optical distance measurement (Berkovic and Shafir, 2012). The most used approaches are based on one of three basic principles: triangulation, time-of-flight (ToF) or optical interferometry (Schwarte et al., 1999). Among the most important parameters for selecting the most appropriate method include the required range of measured distances (dimensions of measured object and the working distance), resolution, measuring time, topology (concavities, protrusions, edges) and surface properties (roughness, specular reflections) of the measured object, properties of surrounding lighting, size of measuring system, possibilities for calibration, and also costs. The most widely used technique for optical form measurements is triangulation. Triangles are the basis of many measurement techniques. The following variants, which look greatly different, but use same principle can be distinguished: active triangulation techniques with different illumination, passive triangulation techniques on the basis of digital photogrammetry and stereoscopy, theodolite measuring systems, shape from shading techniques and focus techniques. The active triangulation technique is very often used in practice due to its robustness and efficiency, but it also has several limitations and disadvantages. In particular, this article focuses on the limitations of using this method for objects that are covered with a transparent layer, such as preserving oil. 2. ACTIVE TRIANGULATION 2.1 Principle The active triangulation technique is based on a photogrammetric reconstruction of the measured object by illumination its surface and contemporaneous scanning by a sensor (mostly PSD module or CCD or CMOS camera). The principle of this technique is shown in Fig. 1. The light source, the detector and the illuminated part of the measured object form a triangle. The join b between the light source and the detector is called a triangulation optical basis. The light source ray angle α is fixed whereas the angle on the detector side β changes depending on the variable distance z and it is defined by an illuminated point on the camera chip, respectively by a bright point on a produced image. If the investigated object is situated farther away from the camera and light source, the angle β is bigger so the reflected light ray is projected to the pixel more up (according to the Transparent Layer Compensation at the Active Triangulation Systems I. Janakova*, S. Sediva**, P. Benes*** Department of Control and Instrumentation, Faculty of Electrical Engineering and Communication, Brno University of Technology, Brno, Czech Republic *(+420 541 146 390, e-mail: [email protected]) **(+420 541 146 418, e-mail: [email protected]) ***(+420 541 141 152, e-mail: [email protected]) Abstract: The paper deals with an active triangulation technique of non-contact optical measurement of 3D properties. The basic features of the method - principle, advantages and disadvantages - are described first. But the main part of the paper is concerned with the impossibility of using the basic principle of the method for measured objects, which are covered by a layer of transparent material of unknown thickness. Most often it is a layer of oil for the preservation of metal products. This layer distorts the measured value due to the light refraction. In this work, a method of eliminating this distortion based on the use of two sensors is proposed. Experimental verification of this proposal is also presented. Keywords: 3D vision, optical distance measurement, active triangulation, image processing, transparent layer, oil compensation. 1. INTRODUCTION Optical non-contact distant measuring techniques have found widespread use in many areas, for example in traffic, architecture, robot navigation, object recognition, or in industry in the production inspection (Kalová and Lisztwan, 2006). Their expansion was supported not only by the great development of technics (electronics, optics), but also by the progress made in techniques of data and image processing. The advantages of optical methods are that they can be very accurate and fast and against other principles that are noncontact and non-destructive. It is also easier to reconfigure the system to another dimension or type of products. In standard projective imaging, one dimension (usually the depth information) from 3D scenes is lost. Without the use of a special technique, it is not possible to determine this information retroactively. Fortunately, depending on the specific requirements, different technical approaches can be used for optical distance measurement (Berkovic and Shafir, 2012). The most used approaches are based on one of three basic principles: triangulation, time-of-flight (ToF) or optical interferometry (Schwarte et al., 1999). Among the most important parameters for selecting the most appropriate method include the required range of measured distances (dimensions of measured object and the working distance), resolution, measuring time, topology (concavities, protrusions, edges) and surface properties (roughness, specular reflections) of the measured object, properties of surrounding lighting, size of measuring system, possibilities for calibration, and also costs. The most widely used technique for optical form measurements is triangulation. Triangles are the basis of many measurement techniques. The following variants, which look greatly different, but use same principle can be distinguished: active triangulation techniques with different illumination, passive triangulation techniques on the basis of digital photogrammetry and stereoscopy, theodolite measuring systems, shape from shading techniques and focus techniques. The active triangulation technique is very often used in practice due to its robustness and efficiency, but it also has several limitations and disadvantages. In particular, this article focuses on the limitations of using this method for objects that are covered with a transparent layer, such as preserving oil. 2. ACTIVE TRIANGULATION 2.1 Principle The active triangulation technique is based on a photogrammetric reconstruction of the measured object by illumination its surface and contemporaneous scanning by a sensor (mostly PSD module or CCD or CMOS camera). The principle of this technique is shown in Fig. 1. The light source, the detector and the illuminated part of the measured object form a triangle. The join b between the light source and the detector is called a triangulation optical basis. The light source ray angle α is fixed whereas the angle on the detector side β changes depending on the variable distance z and it is defined by an illuminated point on the camera chip, respectively by a bright point on a produced image. If the investigated object is situated farther away from the camera and light source, the angle β is bigger so the reflected light ray is projected to the pixel more up (according to the Transparent Layer Compensation at the Active Triangulation Systems I. Janakova*, S. Sediva**, P. Benes*** Department of Control and Instrumentation, Faculty of Electrical Engineering and Communication, Brno University of Technology, Brno, Czech Republic *(+420 541 146 390, e-mail: [email protected]) **(+420 541 146 418, e-mail: [email protected]) ***(+420 541 141 152, e-mail: [email protected]) Abstract: The paper deals with an active triangulation technique of non-contact optical measurement of 3D properties. The basic features of the method - principle, advantages and disadvantages - are described first. But the main part of the paper is concerned with the impossibility of using the basic principle of the method for measured objects, which are covered by a layer of transparent material of unknown thickness. Most often it is a layer of oil for the preservation of metal products. This layer distorts the measured value due to the light refraction. In this work, a method of eliminating this distortion based on the use of two sensors is proposed. Experimental verification of this proposal is also presented. Keywords: 3D vision, optical distance measurement, active triangulation, image processing, transparent layer, oil compensation. 1. INTRODUCTION Optical non-contact distant measuring techniques have found widespread use in many areas, for example in traffic, architecture, robot navigation, object recognition, or in industry in the production inspection (Kalová and Lisztwan, 2006). Their expansion was supported not only by the great development of technics (electronics, optics), but also by the progress made in techniques of data and image processing. The advantages of optical methods are that they can be very accurate and fast and against other principles that are noncontact and non-destructive. It is also easier to reconfigure the system to another dimension or type of products. In standard projective imaging, one dimension (usually the depth information) from 3D scenes is lost. Without the use of a special technique, it is not possible to determine this information retroactively. Fortunately, depending on the specific requirements, different technical approaches can be used for optical distance measurement (Berkovic and Shafir, 2012). The most used approaches are based on one of three basic principles: triangulation, time-of-flight (ToF) or optical interferometry (Schwarte et al., 1999). Among the most important parameters for selecting the most appropriate method include the required range of measured distances (dimensions of measured object and the working distance), resolution, measuring time, topology (concavities, protrusions, edges) and surface properties (roughness, specular reflections) of the measured object, properties of surrounding lighting, size of measuring system, possibilities for calibration, and also costs. The most widely used technique for optical form measurements is triangulation. Triangles are the basis of many measurement techniques. The following variants, which look greatly different, but use same principle can be distinguished: active triangulation techniques with different illumination, passive triangulation techniques on the basis of digital photogrammetry and stereoscopy, theodolite measuring systems, shape from shading techniques and focus techniques. The active triangulation technique is very often used in practice due to its robustness and efficiency, but it also has several limitations and disadvantages. In particular, this article focuses on the limitations of using this method for objects that are covered with a transparent layer, such as preserving oil. 2. ACTIVE TRIANGULATION 2.1 Principle The active triangulation technique is based on a photogrammetric reconstruction of the measured object by illumination its surface and contemporaneous scanning by a sensor (mostly PSD module or CCD or CMOS camera). The principle of this technique is shown in Fig. 1. The light source, the detector and the illuminated part of the measured object form a triangle. The join b between the light source and the detector is called a triangulation optical basis. The light source ray angle α is fixed whereas the angle on the detector side β changes depending on the variable distance z and it is defined by an illuminated point on the camera chip, respectively by a bright point on a produced image. If the investigated object is situated farther away from the camera and light source, the angle β is bigger so the reflected light ray is projected to the pixel more up (according to the Transparent Layer Compensation at the Active Triangulation Systems I. Janakova*, S. Sediva**, P. Benes*** Department of Control and Instrumentation, Faculty of Electrical Engineering and Communication, Brno University of Technology, Brno, Czech Republic *(+420 541 146 390, e-mail: [email protected]) **(+420 541 146 418, e-mail: [email protected]) ***(+420 541 141 152, e-mail: [email protected]) Abstract: The paper deals with an active triangulation technique of non-contact optical measurement of 3D properties. The basic features of the method - principle, advantages and disadvantages - are described first. But the main part of the paper is concerned with the impossibility of using the basic principle of the method for measured objects, which are covered by a layer of transparent material of unknown thickness. Most often it is a layer of oil for the preservation of metal products. This layer distorts the measured value due to the light refraction. In this work, a method of eliminating this distortion based on the use of two sensors is proposed. Experimental verification of this proposal is also presented. Keywords: 3D vision, optical distance measurement, active triangulation, image processing, transparent layer, oil compensation. 1. INTRODUCTION Optical non-contact distant measuring techniques have found widespread use in many areas, for example in traffic, architecture, robot navigation, object recognition, or in industry in the production inspection (Kalová and Lisztwan, 2006). Their expansion was supported not only by the great development of technics (electronics, optics), but also by the progress made in techniques of data and image processing. The advantages of optical methods are that they can be very accurate and fast and against other principles that are noncontact and non-destructive. It is also easier to reconfigure the system to another dimension or type of products. In standard projective imaging, one dimension (usually the depth information) from 3D scenes is lost. Without the use of a special technique, it is not possible to determine this information retroactively. Fortunately, depending on the specific requirements, different technical approaches can be used for optical distance measurement (Berkovic and Shafir, 2012). The most used approaches are based on one of three basic principles: triangulation, time-of-flight (ToF) or optical interferometry (Schwarte et al., 1999). Among the most important parameters for selecting the most appropriate method include the required range of measured distances (dimensions of measured object and the working distance), resolution, measuring time, topology (concavities, protrusions, edges) and surface properties (roughness, specular reflections) of the measured object, properties of surrounding lighting, size of measuring system, possibilities for calibration, and also costs. The most widely used technique for optical form measurements is triangulation. Triangles are the basis of many measurement techniques. The following variants, which look greatly different, but use same principle can be distinguished: active triangulation techniques with different illumination, passive triangulation techniques on the basis of digital photogrammetry and stereoscopy, theodolite measuring systems, shape from shading techniques and focus techniques. The active triangulation technique is very often used in practice due to its robustness and efficiency, but it also has several limitations and disadvantages. In particular, this article focuses on the limitations of using this method for objects that are covered with a transparent layer, such as preserving oil. 2. ACTIVE TRIANGULATION 2.1 Principle The active triangulation technique is based on a photogrammetric reconstruction of the measured object by illumination its surface and contemporaneous scanning by a sensor (mostly PSD module or CCD or CMOS camera). The principle of this technique is shown in Fig. 1. The light source, the detector and the illuminated part of the measured object form a triangle. The join b between the light source and the detector is called a triangulation optical basis. The light source ray angle α is fixed whereas the angle on the detector side β changes depending on the variable distance z and it is defined by an illuminated point on the camera chip, respectively by a bright point on a produced image. If the investigated object is situated farther away from the camera and light source, the angle β is bigger so the reflected light ray is projected to the pixel more up (according to the Transparent Layer Compensation at the Active Triangulation Systems I. Janakova*, S. Sediva**, P. Benes*** Department of Control and Instrumentation, Faculty of Electrical Engineering and Communication, Brno University of Technology, Brno, Czech Republic *(+420 541 146 390, e-mail: [email protected]) **(+420 541 146 418, e-mail: [email protected]) ***(+420 541 141 152, e-mail: [email protected]) Abstract: The paper deals with an active triangulation technique of non-contact optical measurement of 3D properties. The basic features of the method - principle, advantages and disadvantages - are described first. But the main part of the paper is concerned with the impossibility of using the basic principle of the method for measured objects, which are covered by a layer of transparent material of unknown thickness. Most often it is a layer of oil for the preservation of metal products. This layer distorts the measured value due to the light refraction. In this work, a method of eliminating this distortion based on the use of two sensors is proposed. Experimental verification of this proposal is also presented. Keywords: 3D vision, optical distance measurement, active triangulation, image processing, transparent layer, oil compensation. 1. INTRODUCTION Optical non-contact distant measuring techniques have found widespread use in many areas, for example in traffic, architecture, robot navigation, object recognition, or in industry in the production inspection (Kalová and Lisztwan, 2006). Their expansion was supported not only by the great development of technics (electronics, optics), but also by the progress made in techniques of data and image processing. The advantages of optical methods are that they can be very accurate and fast and against other principles that are noncontact and non-destructive. It is also easier to reconfigure the system to another dimension or type of products. In standard projective imaging, one dimension (usually the depth information) from 3D scenes is lost. Without the use of a special technique, it is not possible to determine this information retroactively. Fortunately, depending on the specific requirements, different technical approaches can be used for optical distance measurement (Berkovic and Shafir, 2012). The most used approaches are based on one of three basic principles: triangulation, time-of-flight (ToF) or optical interferometry (Schwarte et al., 1999). Among the most important parameters for selecting the most appropriate method include the required range of measured distances (dimensions of measured object and the working distance), resolution, measuring time, topology (concavities, protrusions, edges) and surface properties (roughness, specular reflections) of the measured object, properties of surrounding lighting, size of measuring system, possibilities for calibration, and also costs. The most widely used technique for optical form measurements is triangulation. Triangles are the basis of many measurement techniques. The following variants, which look greatly different, but use same principle can be distinguished: active triangulation techniques with different illumination, passive triangulation techniques on the basis of digital photogrammetry and stereoscopy, theodolite measuring systems, shape from shading techniques and focus techniques. The active triangulation technique is very often used in practice due to its robustness and efficiency, but it also has several limitations and disadvantages. In particular, this article focuses on the limitations of using this method for objects that are covered with a transparent layer, such as preserving oil. 2. ACTIVE TRIANGULATION 2.1 Principle The active triangulation technique is based on a photogrammetric reconstruction of the measured object by illumination its surface and contemporaneous scanning by a sensor (mostly PSD module or CCD or CMOS camera). The principle of this technique is shown in Fig. 1. The light source, the detector and the illuminated part of the measured object form a triangle. The join b between the light source and the detector is called a triangulation optical basis. The light source ray angle α is fixed whereas the angle on the detector side β changes depending on the variable distance z and it is defined by an illuminated point on the camera chip, respectively by a bright point on a produced image. If the investigated object is situated farther away from the camera and light source, the angle β is bigger so the reflected light ray is projected to the pixel more up (according to the I. Janakova et al. / IFAC PapersOnLine 52-27 (2019) 508–513 509 simplified Fig. 1). In this way, the distance of the object can be obtained just from the position of the light point in the image. However, with the α angle other than 90 °, it should be borne in mind that when the object's distance changes, the x position on the surface being measured also changes. The relationship between the coordinate in the image and the distance can be determined by calculation (based on the knowledge of the geometry and parameters of the camera and lens) or due to calibration. Fig. 1. Principle of the triangulation method (1D variant) The measurement uncertainty in z is inversely proportional to both the camera baseline and the focal length of the lens, but directly proportional to the square of the distance (Kalová and Lisztwan, 2005). Unfortunately, f and b cannot be made as large as desired. The baseline is limited mainly by the mechanical construction (stability of the whole system decreases as b increases), and by shadow effects (selfocclusion problems increase with b) (Beraldin et al., 2003). The size of the focal length is limited by the limited dimensions of the offered detectors. Moreover, with increasing f the detector's field of view decreases and thus the range in the Z-axis also decreases. 2.2 Variants Depending on the light source used to mark the object, three variants can be distinguished: a light point, a light stripe, and a structured light. The dimension of the used light determines the dimension and the number of obtained points. When using the light point, only one coordinate (typically distance, depth, or z coordinate) of one point is determined at a time, therefore this variant is called 1D triangulation. The advantage of this option is that a line camera with a usually higher horizontal resolution and high line scan frequency can be used. When the light stripe (2D triangulation) is used, two coordinates are determined at once and the number of measured points depends on the camera's pixel resolution (a profile is measured). The advantage of using structured light - 3D triangulation (Geng, 2011) is that the complete 3D coordinates of the whole object can be determined in block (the number of points depends on the used pattern and the camera resolution; the dimensions of the measured part of the object are defined by the selected chip and lens and overall geometry). If it is required to specify more coordinates (for 1D and 2D variants) or more points, it is necessary to ensure scanning (movement of the measuring system or object in one or two axes). Conventional light sources can be used for optical triangulation, but laser sources have unique advantages for 3D imaging (Amann et al., 2001). Above all, it is high brightness, coherence and monochromatism, and low divergence. 2.3 Advantages and disadvantages Active triangulation systems are relatively simple, robust and efficient. The advantage is that they can operate over a wide range of distances. Laser-based optical displacement sensors using a very small spot which enables measurements on the very small parts. The 3D version can measure an entire object in one instant from one frame. The 2D variant is often used on production lines, where scanning and measurement of the whole object are ensured thanks to the translational movement below the measuring unit. The variant with point laser and line camera is very fast, with a sampling rate of 100 kHz commonly achieved. High measuring rates are required for the fast moving targets or measurements on difficult surfaces. Commercial laser scanners are very robust with compact dimensions. Despite their very small dimensions, some series have a fully integrated controller. As a result, simple, rapid installation and wiring are possible. The sensors can be integrated easily even into restricted installation space. The comprehensive solution enables easy measurement control, synchronization with other elements as well as between several triangulation sensors (e.g. differential thickness measurement) and usually a clear control and evaluation application. The sensors can be also equipped with different outputs to fulfil industrial user requirements. The pitfalls of active triangulation techniques are generally to evaluate parts of the object that are not directly visible by the camera or illuminated by the light source. Surface topology and geometry of real physical objects can be quite complex, with multiple holes, concavities, protrusions and edges. Problems also bring surfaces perpendicular to the image plane and surfaces parallel to the light source. The measurement quality is also influenced by the surface properties of the measured object. Essential are especially surface roughness, reflectance, and colour. However, today's systems are capable of measuring even complicated surfaces, such as shiny metallic materials or contrariwise rubber. The exposure time or the amount of light produced by the laser is optimally matched to the reflection characteristics of the target surface. New possibilities also include the use of a blue laser (next to more common red). Due to its shorter wavelength, it does not penetrate the target surface, projecting a small light spot on the surface and therefore providing stable and precise results. This technology is preferably used with organic and (semi-)transparent objects, as well as for red-hot glowing metals. Another problem, that was encountered when testing the laser system to check the dimensions of products, can be caused by the thin layer of some transparent material on the products, where the measured values are distorted due to the refraction of the light in that layer. 510 I. Janakova et al. / IFAC PapersOnLine 52-27 (2019) 508–513 3. PROBLEM DEFINITION The task of the proposed system was to measure the parameters of the particular metal parts of the axial bearings and, based on the measurements, to check whether some parts weren't confused. Individual types of bearings differ in the combination of their dimensions (height, inner and outer diameter, raceway parameters) even in the order of only a few micrometers, so it was necessary to ensure very accurate measurements. Unfortunately, the control measuring system had to be placed until just before assembling all parts, where the parts are oil-preserved. Due to the method of preservation by dipping and free dripping of the components in an undefined position, the layer thickness is not uniform and cannot be predetermined. Several approaches were considered for measurement. One option was to use the 1D or 2D laser triangulation system. However, the problem is that the partially transparent layer of preservative oil causes not only disturbing light reflections from its surface and also reduces the intensity of received light, but above all, it causes a change in the direction of the light beam at the interface between two media with different refractive indices. If the laser beam passes from ambient air to an optically denser oil medium (the refractive index of the oil is about 1.5), the angle of refraction is less than the angle of incidence - the ray in the higher-index medium is closer to the normal. Respectively, when recording the light reflected from the metal surface, the opposite phenomenon is observed. For the sake of clarity, the problem is shown in Fig. 2 in a simpler and more favourable situation, where the laser beam is perpendicular to the surface (therefore there is no double refraction) and parallel projection is used. Fig. 2. The distance measuring problem of components with the oil layer Point A in Fig. 2 indicates the location where the light beam would be projected if there were no oil layers. Conversely, point B represents the illuminated spot on the chip after refraction of the light in the upper transparent layer. Due to the fact that the projection is shifted on the chip, the wrong distance l B is determined against the real distance l of the metal surface of the bearing from the laser. The magnitude of the error is directly proportional to the angle between the camera's optical axis and the normal to the measured surface α, as well as to the refractive index of the layer n 2 and to its thickness t. Trigonometric functions can be used to calculate the error (for simplicity, a telecentric lens with parallel beams is considered, as in Fig. 2): α β tg t gtte ⋅ −= , (1) where angle β is given by Snell’s law: βα sinsin 21 ⋅=⋅ nn , (2) For example, for the angle α = 50° and the oil layer ( n 2 = 1.5) with the thickness of only 0.1 mm, the error is greater than 0.05 mm, so for the correct distinction of bearing types is this error very significant. The thickness of the oil layer may vary in different parts of the bearing ring due to the surface tension and running down oil into the bearing raceway (see Fig. 2), and thus depends also on the size and shape of the ring. In addition, the oil thickness can be changed by further manipulation, for example by gripping the ring in the hand. However, due to the unknown actual oil thickness, it is not possible to determine the amount of distortion, respectively enquired bearing height, from only one measurement. 4. SUGGESTED SOLUTION The proposed solution to the problem described in the previous chapter is to use two sensors (two cameras) each with the different angle to the laser beam. The principle is shown in Fig. 3. Fig. 3. Principle of the proposed solution using two sensors Each separate measuring unit, composed of the camera and the laser (where the laser can be common to both units) measures, due to different viewing angle α i , the diversely distorted coordinates z ei ( i is the index of measurement) iiii iii i e ezztnnfz +== ), ,,,( 21 α . (3) I. Janakova et al. / IFAC PapersOnLine 52-27 (2019) 508–513 511 The medium n 1 , where the whole measuring unit is located, is surrounding air so n 11 = n 12 = n 1 = 1 . The medium n 2 represents the transparent layer above the measured object (in our case oil, where n 2 is approximately 1.5) and it is uniform for both measurements ( n 21 = n 22 = n 2 ). In the near neighbourhood of the measurement, unknown thicknesses of the layer can also be considered identical ( t 1 = t 2 = t ). And of course, equal is the second unknown in our task - the height of the object above the pad z 1 = z 2 = z . If the angles α i and index n 2 are known, two equations of two unknowns can be constructed based on two measurements and z and t can be determined simultaneously. The advantage is that the same system can be used for another type of task with two unknowns. For example, the layer refractive index of the unknown thickness can be determined or vice versa. 5. EXPERIMENTAL VERIFICATION A laboratory triangulation workplace with two cameras was assembled to verify the proposed solution. The design was tested on the test task of measuring the parameters of axial bearings. For the measurement, the two-dimensional variant with the line laser was chosen to measure the entire bearing profile at once. The arrangement of the individual components of the assembled experimental workplace is shown in the photograph (Fig. 4) and the sketch (Fig.5). The bearing was placed on a flat backplate with the raceway pointing upwards and was pushed to two chocks (diameter 6 mm and distance 25 mm). This ensures precise positioning, centering of the ring and, at the same time, the outer diameter of the ring can be determined from the size of insertion of the ring between the stops (profile position within the measured field). The line laser was mounted above the pad to shine perpendicular to the pad, and the laser stripe illuminated the space exactly between the stoppers. The optical plane of the laser then passes through the symmetry axis of the ring. The cameras were mounted separately on the stands and were oriented so that the laser track reflected from the pad was at the bottom of the captured images and was parallel to the Y-axis of the images and, of course, to watch the area at the ring profile. Fig. 4. Photograph of the assembled experimental workplace Fig. 5. Schematic sketch of the arrangement of the individual components of the designed measuring system Two identical CCD cameras were used (Imaging Source DFK 41BU02 with 1/2" chip size and 1280x960 pixel resolution). A narrowband filter was inserted in front of each chip to filter out light components other than the wavelength of the laser used (650 nm). Both cameras were supplemented with the same lenses (VICO imaging MFA1-230-5M35 with 35 mm focal length and 1:1.4 aperture). A distance ring has always been inserted between the camera and the lens to reduce the working distance. The camera distance was adjusted to obtain the necessary X-axis range (approximately 26 mm). The use of a classic fixed focus lens (endocentric) is not the most suitable for measurement tasks like this. A telecentric lens and/or a lens using the Scheimpflug principle for oblique plane sensing would be preferable. However, such two identical lenses were not available for testing. It is also possible to select more suitable cameras with larger chip and resolution for real application. The angle between the optical axis of the first camera and the laser plane was approximately 30° (α 1 = 30°) and for the second camera α 2 = 55°. These angles were chosen as a compromise between the desired range and resolution in Z-axis, the quality of the reflected light strip in the captured images, and a sufficient difference between the two angles for successful compensation of the oil layer. The measuring system of the assembled workplace had a pixel resolution in the X-axis of approximately 20 µm and in the Z-axis of the first camera 40 µm and of the second 24 µm. The conversion relations between the image coordinates and the real coordinate system introduced for the measured object can be determined by calculation, but it is necessary to know precisely all the required input parameters (e.g. parameters of cameras and lenses, orientation of cameras, etc.) and in case of some changes during use of measuring system it is necessary to measure all variables again and recalculate the relationships. Or it is possible to determine the interrelationships for individual triangulation systems (camera, laser, and measured object) by calibration. If a suitable automatic procedure and solution of calibration are designed, this approach is more appropriate because it allows a continuous recalibration of the entire measuring system at every change or even preventively (a slight change in the relationship can also occur due to environmental influences, e.g. temperature). Calibration of experimental systems was 512 I. Janakova et al. / IFAC PapersOnLine 52-27 (2019) 508–513 performed on several rings whose real spatial coordinates were subtracted from previously measured profiles by ScanControl 2900-25/BL scanner. More appropriate, even with respect to the planned recalibration, would be to use a specifically designed for this purpose (suitable dimensions and distribution of calibration points) and precisely manufactured calibration object that would best cover the entire required space with the calibration points. For the set of points with known spatial coordinates (x, z) are assigned corresponding coordinates (u, v) in the images. Two (one for each camera) transformation matrices (size 3x3) determined by the least squares method are the result of the calibration. Similarly, the conversion relationship between the oil thickness and the error produced can be determined by calculation or calibration. For the same reason, however, these relationships were again determined by calibration. Also because the refractive index of the particular used oil was not exactly known. For this purpose, the oil-free rings were first measured and then the oil of unknown thickness was applied to them and the differences in values were determined by re-measurement. Based on several measurements, the constant K representing the average error ratio of the second and the first camera K = e 2 / e 1 was determined (at the cost of some inaccuracy in endocentric lenses). The advantage of this approach to calibration is that there is no need to know the current oil thickness. For the purpose of measuring bearing profiles, undistorted as well as distorted by oil layer, a program for sub-pixel laser line detection and data filtering had to be created. An example of laser track evaluation is shown in Fig. 6. As can be seen from the presented figure, the quality of laser trace tracing is highly dependent on the surface of the object being measured. Problems are caused by uneven surfaces, glare, less reflective and inclined surfaced in the raceway, and shadows between the bearing and base (in the case of telecentric lens use, blind spots and shadows should not be created). After applying the oil, the laser track, especially in the raceway, where the largest amount of oil tends to be, is less noticeable and more blurred. The quality of laser tracking can be improved by taking images with several different exposure times and selecting the most appropriate image for each bearing region. Fig. 6. The result of the laser track detection program After laser track detection, the pixel positions of the line are transformed at spatial coordinates based on calibration data for each camera. The following figure shows two measured profiles of the same ring without an oil layer from the first and the second camera. Ideally (especially perfect line detection and calibration), both profiles should be the same after transformation. However, due to e.g. reflections, limited and unequal camera resolution, etc., the real obtained profiles are slightly different. The differences are shown by the green dots in Fig. 7. In this particular case, the average profiles difference is 6.5 µm, the standard deviation is 12.7 µm. Fig. 7. Profile measured by first camera (blue) and second camera (red) and difference of values (green) After applying the oil, as expected, the measured data is deformed by the oil layer (light refraction). The measured profiles of the same oil-free (blue) and oiled (red) bearing are shown in the following figure. As you can see from the difference of values (green), the error at the deepest point of the raceway is over 200 µm for the first camera and nearly 300 µm for the second (due to the larger camera-laser angle). This corresponds to the oil thickness of approximately 0.55 mm. However, it should be noted that this oil thickness is unrealistically high and serves only for presentation. The usual oil thickness after preservation and dripping in operation is up to about 100 µm. Fig. 8. Measured profile of the same bearing without oil (blue), after oil application (red) and difference (green) When two oil-coated ring profiles are measured with cameras at different angles, the real bearing profile can be calculated based also on the deformed data thanks to the conversion relationships specified by calibration (for each X-coordinate, the Z-coordinate can be determined what would be without oil). The calculation can be done using the formula: 1 21 − − ⋅ = K zKz z ee , (4) I. Janakova et al. / IFAC PapersOnLine 52-27 (2019) 508–513 513 The result if graphed in Fig. 9, wherein the calculated oil layer compensation profile is shown in black. For the sake of clarity, the graph is supplemented by profiles measured by both cameras without the oil layer (blue and red waveforms as in Fig. 7) and by the difference between the compensated profile and the oil-free profile from the first camera (green waveform). Although the dispersion of the difference between the profile of the first oil-free camera and the compensated profile is larger in this case, as shown by the larger standard deviation value of 52.8 µm, however, the average difference is only -2.7 µm. Fig. 9. Compensated profile (black) and difference (green) from the profile without oil from first camera (blue) The oil layer, if not compensated, has a fundamental influence on the determination of the basic bearing parameters - ring height and raceway radius, respectively raceway depth. Separate algorithms were programmed to determine these parameters. For example, the ring height is determined by approximation of data at the ring's top edges and at the baseplate by the straight lines. The data in the raceway, on the other hand, is interlaced with a circle. If the bearing dimensions were measured with only one triangulation system, without the possibility of eliminating the effect of the layer, the measured bearing height would be higher than the reality and the raceway radius would also be larger. Fig. 10 shows a detail of profile with oil without (red) and after (black) compensation. The calculated raceway radii determined by fitting the raceway points with the circle are indicated by the same colour. The blue part of the circle indicates the radius specified for the oil-free bearing. It can be seen that while the compensated profile has a greater dispersion of values, the values better follow the real shape Fig. 10. Raceway radius determination - profile without (red) and after (black) compensation of the raceway. However, the short section of the circle is not sufficient to accurately determine the diameter of the raceway but will be used to determine its depth and other bearing parameters. 6. CONCLUSIONS The article deals with the fundamental limitation of the application of the basic simple principle of the active triangulation method to objects that are covered by some transparent layer (e.g. preserving oil, lubricant, water). For this reason, the innovative system to measure 3D parameters of objects through this layer has been designed. The basic idea is to use two cooperating sensors each with the different angle to the laser. The advantage is that the same system can be used for another type of task with two unknowns (e.g. the layer thickness of unknown refractive index or vice versa). Although the most appropriate hardware was not available for the experiments, and even the software could not yet be resolved at the level of the development of a commercial finished device, nevertheless the performed tests demonstrate that the proposed principle is correct and that this procedure is applicable to eliminate the transparent layer. ACKNOWLEDGEMENT The completion of this paper was made possible by the grant TH02010830 – “Automation, measurement and monitoring of assembly processes” which is funded by Technology Agency of the Czech Republic - programme EPSILON and by the grant No. FEKT-S-17-4234 - „Industry 4.0 in automation and cybernetics” financially supported by the Internal science fund of Brno University of Technology. REFERENCES Amann, M.-Ch., Bosch, T., Lescure, M., Myllylä, R. and Rioux, M. (2001). Laser Ranging: A Critical Review of Unusual Techniques for Distance Measurement. Optical Engineering, Vol. 40(1), pp. 10-19. Beraldin, J.-A., Blais, F., Cournoyer, L., Godin, G., Rioux, M. and Taylor, J. (2003). Active 3D Sensing. In: The eWay into the Four Dimensions of Cultural Heritage Congress. Vienna, Austria. Berkovic, G. and Shafir, E. (2012). Optical methods for distance and displacement measurements. Advances in Optics and Photonics, Vol. 4, Issue 4, pp. 441-471. Geng, J. (2011). Structured-light 3D surface imaging: a tutorial. Advances in Optics and Photonics, Vol. 3, Issue 2, pp. 128-160. Kalová, I. and Lisztwan, M. (2005). Active Triangulation Technique. In: 5th International Conference of PhD Students, pp. 99-104. University of Miskolc, Miskolc. Kalová, I. and Lisztwan, M. (2006). Industrial Applications of Triangulation Technique. IFAC Proceedings Volumes, Vol. 39(21), pp. 258-263. Schwarte, R., Heinol, H., Buxbaum, B., Ringbeck, T., Xu, Z. and Hartmann, K. (1999). Principles of ThreeDimensional Imaging Techniques. In: Handbook of Computer Vision and Applications, Vol. 1, pp. 463-484. Academic Press, San Diego.