3D modelling with structured light GAMMA calibration

DIGITAL IMAGE PROCESSING AND COMPUTER GRAPHICS VOLUME: 12 |NUMBER: 4 |2014 |SPECIAL ISSUE 3D Modelling with Structured Light Gamma Calibration Eser SERT1, Ibrahim Taner OKUMUS1, Deniz TASKIN2 1Computer Engineering Department, Engineering and Architecture Faculty, Kahramanmaras Sutcu Imam University, Avsar Kampusu 46100 Kahramanmaras, Turkey 2Computer Engineering Department, Faculty of Engineering, Trakya University, 22030 Balkan Yerlekesi, Edirne, Turkey [email protected], iokum[email protected], [email protected] Abstract. Structured light method is one of the noncontact measurement methods used for high resolution and high sensitive 3D modeling. In this method, a projector, camera and computer are used. Projector projects patterns that are generated with specific coding strategies onto the object that will be 3D modeled. Camera receives these patterns. By processing the images received by the camera, object is 3D modeled. Light intensity that is emitted from the projector generally not a linear function of the signal input. This causes brightness problems in the patterns projected. Thus, images received from the camera needs to the gamma corrected. In this study, gamma calibration method is proposed to overcome this problem. Test results show that proposed calibration system improves the accuracy and quality of the 3D modeling. Keywords 3D modeling, gamma correction, structured light. 1. Introduction With 3D modelling systems, complex surfaced objects can be realistically modelled and be shown on electronic systems with a high sense of reality [1]. Because of these features the need for 3D modelling systems is increasing. One of the 3D modeling systems used in this field is structured light. Using this system, it is possible to produce 3D models of objects with high quality and high sensitivity [2]. A structured light method uses a projector and a camera as seen in Fig. 1. Projector projects specially formed structured light patterns onto the object. Camera transfers the pictures of the scene to the computer. Fig. 1: Working principle of structured light modelling system. These scene pictures are processed by the computer and 3D model is obtained [3]. Projected structured light pattern is distorted depending on the amount of depth of the object. Distortion of the structured light increases with the increase of the depth of the surface [4]. 3D model is obtained by using the distortions in the structured light. In order for to work properly of 3D modeling system, it needs to be calibrated [5], [6]. Calibration steps that are needed to be applied are gamma calibration, camera calibration, projector calibration, etc. [7]. Among these, gamma calibration is very important. In this study, a structured light 3D modeling system is designed, and gamma calibration method is tested on this system. In the designed system, three phase shifting algorithm is used. This algorithm is chosen because it can provide high quality and high sensitive 3D modeling compared to other algorithms. c 2014 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 347 DIGITAL IMAGE PROCESSING AND COMPUTER GRAPHICS VOLUME: 12 |NUMBER: 4 |2014 |SPECIAL ISSUE In the literature, there are various work on gamma calibration. Song Zhang et al. [8] suggested an error compensation method for a 3D measurement system working on phase shifting method. Study suggests that the main source of error is the nonlinearity of the gamma of the image produced by a projector. This error causes problems in sinusoidal signal and 3D model. With the proposed method problems in sinusoidal signals are corrected, problems in fringe images are reduced, studies performed for another source of errors caused by projectors gamma. Li et al. [9] used gamma correction to reduce phase errors. Study conducted investigation on gammadistorted fringe images. Also study investigated phase errors, gamma distortion and defocusing-phase error and defocusing-gamma. Wang et al. [1] proposed a 4 step application that can be used in fringe projection profilometry. At the first step, gamma correction of digital protection is applied. Second step included setting up system components. In the third step, a phase unwrapping is applied. At the last step, the system is calibrated by least-square inverse approach. In this study by reducing the unnecessary process steps in the calibration, gamma calibration process is accelerated. Projection sourced gamma problems are corrected by calibration process, and an ideal 3D modelling is achieved. Rest of the paper is organized as follows. Section 2 describes the working principle of 3D modeling system. Section 3 discusses Gama Calibration. Section 4 contains experimental results and section 5 is the concluding remarks. 2. Working Principle of 3D Modelling System Flow diagram of a three-step phase-shifting algorithm is provided in Fig. 2. In the beginning, a calibration is performed for the system to work properly. Gamma calibration is one the most important one among these. After calibration, projector projects first, second and third phase patterns onto the object in the given order. Step II in Fig. 2 contains three phase patterns. Brightness signal formulations of these phase patterns are given in Eq. (1), Eq. (2), Eq. (3) respectively [10], [11], [12]: I1(x, y) = I0(x, y) + I00 (x, y) cos[Φ(x, y)−2π/3],(1) I2(x, y) = I0(x, y) + I00 (x, y) cos[Φ(x, y)],(2) I3(x, y) = I0(x, y) + I00 (x, y) cos[Φ(x, y)+2π/3],(3) where I0(x, y)is the average intensity, I00 (x, y)is intensity modulation. Iis the intensity value that corresponds to x, y coordinates. Figure 3 shows the fringe pictures that correspond to sinusoidal signals. Three different fringe patterns are projected onto the person to be modeled and these Pictures are taken. Step III in Fig. 2 shows the pictures obtained after this process. Fig. 2: Flow diagram of three-step phase shifting 3D modeling system. Phase matrix is obtained with Eq. (4) by using I1, I2and I3intensity values of the phase pictures taken by camera [13], [14], [15]. Phase: Φ(x, y) = tan−1 √3(I1−I3) 2I2−I2−I3!.(4) Wrapped phase matrix is obtained by applying phase wrapping to the phase matrix. The wrapped phase matrix is used to obtain depth matrix. Wrapped phase matrix Picture is shown in step IV of Fig. 2. c 2014 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 348 DIGITAL IMAGE PROCESSING AND COMPUTER GRAPHICS VOLUME: 12 |NUMBER: 4 |2014 |SPECIAL ISSUE Fig. 3: Fringe patterns that correspond to sinusoidal signals. Unwrap process is applied in step V. This process produces the depth map that contains the 3D modeling information. Figure of the depth map is shown in step V of Fig 2. At the last step, 3D model is produced using all the depth maps that are produced in step V. 3. Gamma Calibration Intensity of light produced by a physical device is not a linear function of the signal fed into the input of the device [16], [17]. Examples of this phenomenon can be distortion of the brightness level of the light produced by a projector. Gamma correction can be formulated as in Eq. (5), [1]. In the equation, Iin is the signal at the input of the projector, Iout is the signal at the output and γis the gamma level of projector: IOUT =Iγ IN .(5) Images projected by projectors have nonlinear functions. γparameter, which is shown in Eq. (5), of the projectors has values in the range of 2.2 to 4.6 [1]. This causes problems in projecting ideal fringe patterns. Thus, gamma calibration becomes a necessity. Gamma correction is performed through Eq. (6): ImageOut = 255 ·(ImageIn/255)(1/γcp),(6) where ImageIn is the input image, ImageOut is gamma corrected image and γcp is gamma correction parameter. Ideal phase patterns of a system that models using structured light are shown in Fig. 4. For the analysis, top 5 horizontal strips are turned vertical and intensity produces a sinusoidal signal. Brightness analysis on each strip of this phase pattern results in a sinusoidal signal. This is shown in Fig. 4. This sinusoidal oscillates within the ideal boundaries. Imax is the maximum Intensity value that Fringe Pattern gets and Imin is the minimum intensity value. Figure 5(a) shows the brightness curve when the contrast and brightness level of projector is low. Figure 5(b) shows the case where these levels are high. As it can be seen from both figures, gamma curve does not oscillate within ideal boundaries. Gamma calibration can overcome possible 3D modelling problems by taking problematic gamma curves into the ideal boundaries [1], [18]. Figure 6 shows the flow diagram of the designed Gamma calibration method. In the beginning, fringe pattern is projected onto the scene. In the second step, by analyzing the projected ideal brightness curve and obtained brightness curve is compared. In the fourth step, the brightness curve is taken to the ideal limit with appropriate gamma correction parameter. At the last step, this adjusted fringe pattern is projected. 4. Experimental Results We performed several tests to see the effects of Gamma Calibration on the system. In these tests, by choosing different values of gamma correction parameter shown in step 4 of Fig. 6, resulting effects on 3D modeling performance are tested. At first 3D modeling is performed with low gamma correction parameter (0.4). Resulting phase pictures and modeling results are shown in Fig. 7. As it can be seen from Fig. 7(c), number of points that generates the 3D model is reduced and ruptures occurred in 3D model. Fig. 4: Brightness analysis on structured light pattern. c 2014 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 349 DIGITAL IMAGE PROCESSING AND COMPUTER GRAPHICS VOLUME: 12 |NUMBER: 4 |2014 |SPECIAL ISSUE In the second step, 3D modeling is performed with high gamma correction parameter (1.2). Resulting phase and modeling pictures are shown in Fig. 8. As it can be seen from Fig. 8(c) there is a rupture around the neck of the model and this region is also misaligned. (a) Fringe pattern obtained from low-level contrast and brightness projector. (b) Fringe pattern obtained from high-level contrast and brightness projector. Fig. 5: Problematic Gamma curves. In the third step, 3D modeling is performed with ideal gamma correction parameter (0.8). Figure 9 shows the phase pictures and 3D modeling results for the ideal gamma correction value. As it can be seen from Fig. 9(c) an ideal 3D model is a result in the end. Fig. 6: Gamma calibration flow diagram. c 2014 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 350 DIGITAL IMAGE PROCESSING AND COMPUTER GRAPHICS VOLUME: 12 |NUMBER: 4 |2014 |SPECIAL ISSUE Fig. 7: 3D modeling results at low gamma correction parameter (0.4). 5. Conclusion In this paper, gamma calibration method is proposed for three-step phase-shifting method based structured light system. In the 3D modeling system, fringe patterns projected onto the subject can have faulty brightness levels. In order for the system to perform correct 3D modeling these fringe patterns need to go through gamma calibration process. Test results Show that gamma calibration enhances the correctness and quality of the 3D modeling. Fig. 8: 3D modeling results with high gamma correction parameter (1.2). References [1] WANG, Z., D. A. NGUYEN and J. C. BARNES. Some practical considerations in fringe projection profilometry. Optics and Lasers in Engineering. 2010, vol. 48, iss. 2, pp. 218–225. ISSN 0143-8166. DOI: 10.1016/j.optlaseng.2009.06.005. [2] SU, X. and Q. ZHANG. Dynamic 3-D shape measurement method: A review. Optics and Lasers in Engineering. 2010, vol. 48, iss. 2, pp. 191–204. ISSN 0143-8166. DOI: 10.1016/j.optlaseng.2009.03.012. [3] GENG, J. Structured-Light 3D surface imaging: A Tutorial. Advances in Optics and Photonics. 2011, vol. 3, iss. 2, pp. 128–160. ISSN 1943-8206. DOI: 10.1364/AOP.3.000128. [4] SERT, E., D. TASKIN, O. OZCAN, C. TASKIN and K. BAYSAL. 3D Modellemede Kullanilan Ardisik Yansitmali Yapilandirilmis Isik Yontemleri. c 2014 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 351 DIGITAL IMAGE PROCESSING AND COMPUTER GRAPHICS VOLUME: 12 |NUMBER: 4 |2014 |SPECIAL ISSUE Fig. 9: 3D modeling results with ideal gamma correction parameter (0.8). Trakya University Journal of Engineering Sciences. 2012, vol. 13, iss. 2, pp. 121–141. ISSN 13057766. [5] ZHANG, Z., C. TOWERS and D. P. TOWERS. Compensating Lateral Chromatic Aberration of A Colour Fringe Projection System For Shape Metrology. Optics and Lasers in Engineering. 2010, vol. 48, iss. 2, pp. 159–165. ISSN 01438166. DOI: 10.1016/j.optlaseng.2009.04.010. [6] GORTHI, S. S. and P. RASTOGI. Fringe projection techniques: Whither we are? Optics and Lasers in Engineering. 2010, vol. 48, iss. 2, pp. 133–140. ISSN 0143-8166. DOI: 10.1016/j.optlaseng.2009.09.001. [7] MORENO, D. and G. TAUBIN. Simple, Accurate, and Robust Projector-Camera Calibration. In: 3D Imaging, Modeling, Processing, Visualization and Transmission. Washington: IEEE, 2012, pp. 464–471. ISBN 978-1-4673-4470-8 DOI: 10.1109/3DIMPVT.2012.77. [8] ZHANG, S. and P. S. HUANG. Phase Error Compensation For A 3-D Shape Measurement System Based On The Phase-Shifting Method. Optical Engineering. 2007, vol. 46, iss. 6, pp. 1–9. ISSN 1560-2303. DOI: 10.1117/1.2746814. [9] LI, Z. and Y. LI. Gamma-Distorted Fringe Image Modeling and Accurate Gamma Correction for Fast Phase Measuring Profilometry. Optics Letters. 2011, vol. 36, iss. 2, pp. 154–156. ISSN 15394794. DOI: 10.1364/OL.36.000154. [10] HUANG, P. S. and S. ZHANG. Fast three-step phase-shifting algorithm. Applied Optics. 2006, vol. 45, iss. 21, pp. 5086–5091. ISSN 2155-3165. DOI: 10.1364/AO.45.005086. [11] ZHANG, S. and S.-T. YAU. High-Speed ThreeDimensional Shape Measurement System Using A Modified Two-Plus-One Phase-Shifting Algorithm. Optical Engineering. 2007, vol. 46, iss. 11, pp. 113603–113611. ISSN 1560-2303. DOI: 10.1117/1.2802546. [12] ZHANG, Z., G. NEJAT, H. GUO and P. HUANG. A Novel 3D Sensory System For Robot-Assisted Mapping of Cluttered Urban Search and Rescue Environments. Inteligent Service Robotics. 2011, vol. 4, iss. 2, pp. 119–134. ISSN 1861-2784. DOI: 10.1007/s11370-010-0082-3. [13] KARPINSKY, N. and S. ZHANG. HighResolution, Real-Time 3D Imaging With Fringe Analysis. Journal of Real-Time Image Processing. 2012, vol. 7, iss. 1, pp. 55–66. ISSN 1861-8219. DOI: 10.1007/s11554-010-0167-4. [14] ZHANG, S. Recent Progresses on Real-Time 3D Shape Measurement Using Digital Fringe Projection Techniques. Optics and Lasers in Engineering. 2010, vol. 48, iss. 2, pp. 149–158. ISSN 01438166. DOI: 10.1016/j.optlaseng.2009.03.008. [15] ZHANG, S. and P. S. HUANG. Novel Method for Structured Light System Calibration. Optical Engineering. 2006, vol. 45, iss. 8, pp. 1–8. ISSN 15602303. DOI: 10.1117/1.2336196. [16] POYNTON CH. Frequently Asked Questions about Gamma. 2014. Available at: http://www. poynton.com/PDFs/GammaFAQ.pdf. [17] WADDINGTON, C. and J. KOFMAN. Analysis Of Measurement Sensitivity To Illuminance And Fringe-Pattern Gray Levels For FringePattern Projection Adaptive To Ambient Lighting. Optics and Lasers in Engineering. 2010, vol. 48, iss. 2, pp. 251–256. ISSN 0143-8166. DOI: 10.1016/j.optlaseng.2009.07.001. c 2014 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 352 DIGITAL IMAGE PROCESSING AND COMPUTER GRAPHICS VOLUME: 12 |NUMBER: 4 |2014 |SPECIAL ISSUE [18] MARTINEZA, A., J. A. RAYASA, H. J. PUGAB and K. GENOVESE. Iterative estimation of the topography measurement by fringe-projection method with divergent illumination by considering the pitch variation along the x and z directions. Optics and Lasers in Engineering. 2010, vol. 48, iss. 9, pp. 877–881. ISSN 0143-8166. DOI: 10.1016/j.optlaseng.2010.04.002. About Authors Eser SERT received the M.Sc. degree in 2010, and the Ph.D. degree in 2013, all in Computer Engineering at Trakya Univesity, Turkey. Currently he is Assistant Professor at Computer Engineering of Kahramanmaras Sutcu Imam University. His research interests include 3D modeling system, image processing, computer vision, field programmable gate arrays (FPGA)and artificial intelligence. Ibrahim Taner OKUMUS was born in Kahramanmaras, Turkey. He received his M.Sc. and Ph.D. degrees from Sysracuse University, USA. He is currently working as an Associate Professor at Computer Engineering Department of Kahramanmaras Sutcu Imam University, Turkey. His research interests include wireless networks, software defined networks, wireless sensor networks and computer networks in general. Deniz TASKIN received the M.Sc. degree in 2004, and the Ph.D. degree in 2007, all in Computer Engineering at Trakya Univesity, Turkey. Currently he is Assistant Professor at Computer Engineering of Trakya University. c 2014 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 353