Augmented reality in error troubleshooting inside flexible manufacturing cells

Recent Innovations in Mechatronics (RIiM) Vol. 2. (2015). No. 1-2. DOI: 10.17667/riim.2015.1-2/3. 1 Augmented reality in error troubleshooting inside flexible manufacturing cells Lehel Csokmai PhD University of Oradea, Faculty of IMT, Presearch Center Productica, Oradea, Romania [email protected] Radu Tarca PhD University of Oradea, Faculty of IMT, Department of Mechatronics, Oradea, Romania [email protected] Moldovan Ovidiu PhD University of Oradea, Faculty of IMT, Department of Mechatronics, Oradea, Romania [email protected] Abstract—A new type of software framework for error troubleshooting in flexible manufacturing systems using a frame marker device. Our framework system meant to resolve the failures originated from the human-machine interface and to generate self-training from previous experience. Keywords—flexible manufacturing cell;error troubleshooting; augmented reality; software; framework. I. INTRODUCTION The software framework and the frame marker device has been connected to the TMA 550 AL flexible manufacturing cell. The cell is composed of the TMA 550 AL manufacturing centre, two ABB robots and one T shaped conveyor system which connects the two ABB robots. The position of the cell components is presented on the (Fig. 1). Fig 1. The TMA 55 Al flexible manufacturing cell The frame marker device is connected to the flexible manufacturing cell through the cell control software which is running on the central command PC. The command software is made in-house with Embarcadero's RAD Studio and using as database MySQL. The cell command software is based on modules. In the case of the manufacturing cell control application each expansion module is a TCP/IP server or client. These modules may operate without the help of other modules (standalone), or are able to connect to other internal modules (e.g., database). The following extension modules has been made for the flexible manufacturing cell: Conveyor, ABB, Fanuc, RFID, AR. (Fig. 2) Fig 2. The TMA 55 Al flexible manufacturing cell control software Recent Innovations in Mechatronics (RIiM) Vol. 2. (2015). No. 1-2. DOI: 10.17667/riim.2015.1-2/3. 2 II. AUGMENTED REALITY IN ERROR TROUBLESHOOTING Virtual reality (VR) and augmented reality (AR) are tools and techniques for virtual environments that has key features in common with physical environment. View and interact with 3D objects is closer to reality than abstract mathematics and 2D approaches. Augmented reality is an advanced form of VR, which integrates images of virtual objects in the real world. With increasing amount of digital data that can be stored and accessed, there is a need to harness this data and turn into an engine capable of developing a new perception of the real world. Augmented reality depends on the computing power of the devices, which must be able to retain information about the real world, such as video data, position, orientation etc. and also must display the combined information of the captured screen and virtual information in a way that is meaningful for the operator. The augmented reality module of the control application is composed from 3 parts:  a web module (AR-WEB) responsible for displaying the documentation for error codes and oher information,  the control application module (AR-PC) responsible for gathering error messages and other information related to errors,  and the software running on the mobile device (ARMOBILE) responsible for the recognition of frame marker tags and displaying the documentation from the web module. III. AR-WEB MODULE Fig 3. AR-WEB module of the flexible cell control application The AR-WEB (Fig. 3) module is done using the PHP programming language with MySQL database support. The module is intended to serve the queries sent from mobile devices. At every 4-5 seconds the mobile devices making queries towards the web module which checks the database for errors and other informations happening in the flexible manufacturing cell. This module runs on an Apche web server which can run on the same computer with the control application. In addition to serving queries for error messages, the module displays documentation that may help solve the problems of the flexible manufacturing cell. In Fig. 4 is shown a page from the ABB documentation to resolve an error of the ABB IRB1600 robot where the robot memory battery is discharged and must be replaced and the robot must be recalibrated. Fig 4. ABB documentation displayed on mobile device For a faster resolutions of errors, a form completed by the operator which resolved the error can be added on the documentation page. On any future error this form will be displayed along with the documentation. IV. AR-PC MODULE The AR-PC module is designed to retrieve error messages and other informations from the equipments of the flexible manufacturing cell connected to the control application then insert into the database. These errors and informations will be returned on the next query made from the mobile device. V. AR-MOBILE MODULE The AR-MOBILE (Fig. 5) module was created using the Unity development environment. This development environment is often used for creating games and was selected Recent Innovations in Mechatronics (RIiM) Vol. 2. (2015). No. 1-2. DOI: 10.17667/riim.2015.1-2/3. 3 because has the Vuforia augmented reality libraries integrated. Another advantage is the ease of distributing the application on different mobile platforms. With minor modifications the application may be distributed on the following platforms: Android, iOS, Windows or Linux. Fig 5. Unity development environment After the AR-MOBILE application is connected to the ARPC module and the error codes are retrieved, the virtual objects representing the errors are displayed when the mobile device is directed towards the tag generator device (Fig. 6). Selecting the virtual object on the AR-MOBILE application will open the documentation page shown in Fig. 4. Fig 6. Tag generator for AR-MOBILE module VI. CONCLUSION The proposed use of augmented reality in the error troubleshooting inside the flexible manufacturing cells can improve the control activities using an integrated error handling system. The error database can be dynamically extended with new items corresponding to unpredictable errors developed during the manufacturing processes. Acknowledgment This work was partially supported by the strategic grant POSDRU/107/1.5/S/80272, inside POSDRU Romania 20072013 co-financed by the European Social Fund – Investing in People. References [1] Lehel Csokmai, Ovidiu Moldovan, Constantin Bungău, “Architecture of a flexible manufacturing cell control application”, International Scientific Conference on Advances in Mechanical Engineering 10-11 October 2013, Debrecen, Hungary [2] L.Csokmai, O. Moldovan, I. Tarca, R. Tarca, “A Comprehensive Approach of Advanced Error Troubleshooting in Intelligent Manufacturing Systems”, Applied Mechanics and Materials, Volume 404, 2013, pp. 631-634. [3] L.S. Csokmai R.C. Țarcă C. Bungău G. Husi, “A Comprehensive Approach to Offline Advanced Error Troubleshooting in Intelligent Manufacturing Systems”, ICCCC 2014, Oradea, Romania. [4] Lehel Csokmai, Ovidiu Moldovan, Ioan Tarca and Radu Tarca, “Software Framework for Advanced Error Troubleshooting In Flexible Manufacturing System”, ICADME 2013. [5] S. Lee, D.M. Tilbury, “A modular control design method for a flexible manufacturing cell including error handling”, Int J Flex Manuf Syst (2007) 19, pp.308–330. [6] C. Syan, Y. Mostefai, “Status monitoring and error recovery in flexible manufacturing systems, Integrated Manufacturing Systems”, 1995, Vol. 6 Iss: 4 pp. 43-48. [7] H. Kaindl, M. Vallée, and E. Arnautovic, “Self-Representation for SelfConfiguration and Monitoring in Agent-Based Flexible Automation Systems, IEEE Transactions on Systems”, Man, and Cybernetics: Systems, Vol. 43, no.1, 2013, pp.164-175. [8] Borchelt, R. D., Thorson, J. “Toward reusable hierarchical cell control software”, International Journal of Production Research, 1997, 35:2, 577-594. [9] Kao, J.F., “Optimal recovery strategies for manufacturing systems”, European Journal of Operational Research 80 (2), 1995, 252263. [10] Toguyeni, A.K.A., Craye, E., Gentina, J.C., “A framework to design a distributed diagnosis in FMS”, IEEE International Conference on Systems Man and Cybernetics, 1996, Vol. 4, 2774-2779