Difficulties in the Incorporation of Augmented Reality in University Education: Visions from the Experts

ORIGINAL JOURNAL OF NEW APPROACHES IN EDUCATIONAL RESEARCH 2019, VOL. 8, NO. 2, 126-141, e-ISSN: de https://doi.org/10.7821/naer.2019.7.409 Received13-02-2019 Revised28-02-2019 Accepted05-04-2019 Published15/07/2019 CorrespondingAuthor JulioBarroso-Osuna, [email protected] FacultaddeCienciasdela Educación,Departamentode DidácticayOrganización Educativa,C/Pirotecnias/n, 41013,Seville,Spain DOIhttps://doi.org/10.7821/ naer.2019.7.409 Pages: 126-141 Funding:MinistryofEconomy andCompetitivenessofthe GovernmentofSpain (EDU2014-57446-P) Distributedunder CreativeCommonsCCBY4.0 Copyright: ©NAERJournalof NewApproachesinEducational Research2019 Difficulties in the Incorporation of Augmented Reality in University Education: Visions from the Experts JulioBarroso-Osuna1 ,JuanJesúsGutiérrez-Castillo1 ,MªdelCarmenLlorente-Cejudo1 and RubiceliaValenciaOrtiz2 1Departamento de Didáctica y Organización Educativa, University of Seville, Spain 2Latin American Institute for Educational Communication (ILCE), Mexico ABSTRACT “Augmented reality” (AR) is presented as technology that harmonizes, in real time and with user collaboration, digital information with physical information through different technological supports. Such are AR possibilities, that when it is incorporated into teaching, students show high satisfaction rates and positive attitudes for its use, but at the same time, a series of limitations and obstacles can be noticed. As an attempt to investigate the technical, curricular and organizational difficulties of AR incorporation in training contexts, an instrument “AR obstacle questionnaire” (CORA) was designed, which in turn was administered to a total of 264 subject experts for validation through the Delphi method, subsequently applying the “K coefficient” (K = 1 2Kc (knowledge coefficient) + Ka (argumentation coefficient)), selecting those experts who obtained a ≥0,8 score in the coefficient. Among the results obtained in the study we manage to point out the main obstacles involving AR implementation into university education: the lack of teacher training and improvement, the few educational experiences found, lack of conceptual foundation, limited educational research and lack of institutional support. Keywords AUGMENTED REALITY, HIGHER EDUCATION, EMERGING TECHNOLOGIES, INFORMATION AND COMMUNICATION TECHNOLOGIES, DIGITAL COMPETENCES 1AUGMENTED REALITY APPLIED TO UNIVERSITY EDUCATION The insertion that information and communication technologies (ICTs) have recently had intoeducationalactionsandinstitutions, isof amagnitudethat had notpreviouslyoccurred, this being a direct consequence of digitalization, transmedia, the exponential growth of technologies, the penetration of web 2.0 and the significance that ICTs hold in knowledge society. Technologies such as “learning analytics”, “gamification”, “serious video games”, or “robotics”, are rapidly being incorporated into teaching as the “Horizon Reports” prepared by “The New Media Consortium”1point out, or the EduTrends reports produced by 1https : //www.nmc.org/nmc-horizon/ How to cite this article (APA): Barroso-Osuna, J., Gutiérrez-Castillo, J. J., Llorente-Cejudo, M. d. C., & Valencia Ortiz, R. (2019).DifficultiesintheIncorporationofAugmentedRealityinUniversityEducation: VisionsfromtheExperts.JournalofNew ApproachesinEducationalResearch, 8(2),126-141.doi: 10.7821/naer.2019.7.409 126 Julio,Barroso-Osuna;etal. DifficultiesintheIncorporationofAugmented RealityinUniversity Education: VisionsfromtheExperts the “Observatory of Educational Innovation” of Tecnológico de Monterrey2. Augmented Reality applied to university education. One such technology is “augmented reality” (AR), which is a type of technology that harmonizes in real time, and with the collaboration of the user, digital information with physical information through different technological supports, being those which are the most common mobile devices such as smartphones and tablets. (Cabero-Almenara, Fernández, & Marín, 2017;Cabero-Almenara & García, 2016). It is precisely the type of device typically used, which facilitates its incorporation into university education since they are owned by the vast majority of students, and in addition, they tend to hold significant attitudes towards them (Lagunes-Domínguez, Torres-Gastelú, Angulo-Armenta, & Martínez- Olea, 2017;Seifert, Hervás, & Toledo, 2019). In comparison to other technologies, it holds a series of specific characteristics: • Being of mixed reality, enabling the enrichment or complementing the information of reality with digital information. • Integration occurs in real time and in a cohesive manner. • Facilitates the unification of different resources: text, websites, video clip, audio, and 3D. • It is interactive. • In the creation of content, intervention from a person is necessary. In regards to its impact on education, and although the research carried out is limited, an aspect that is possibly one of the biggest issues relates to its incorporation into teaching, studies like the one conducted by Joo, García-Bermejo, and Martinez-Abad (2016) give us clues ensuring that the AR objects that we design can be used with some guarantee of success. With this line of thought we can say that objects, in terms of content, have to be short and direct, lasting no longer than 5 minutes, they must consider the different capacities of the students, the activities must be presented with an intuitive interface that allows the rapid correction of errors and with a constructivist character, in which the participation and the exchange between the students is encouraged, presentation of objects with multimedia character, with clear practical orientation and finally, but not any less importantly, it is advised that the AR objects adapt to the characteristics of the devices in which they will be viewed. And also through its use, the information of physical reality where it is integrated is enriched or altered. The possibilities of incorporation into training are diverse: a) nonsignificant information for understanding of a phenomenon; b) enriches the information of the reality to facilitate its understanding; c) provides different points of view of an object, choosing the person the moment and perspective of observation; d) enhances ubiquitous learning; e) creates safe laboratories and/or simulators for students; f) enriches printed documents with additional information on different media; g) allows the student to visualize a phenomenon from multiple perspectives thus enhancing spatial intelligence; h) students 2https://observatorio.itesm.mx/redutrends/ JournalofNewApproachesin EducationalResearch,8(2) |2019 |https://doi.org/10.7821/naer.2019.7.409 127 Julio,Barroso-Osuna;etal. DifficultiesintheIncorporationofAugmented RealityinUniversity Education: VisionsfromtheExperts can be actors producing these objects, i) allows the exposure of temporal phenomena and especially heterogeneous, and j) contextualizes information (Fonseca, Redondo, & Valls, 2016;Han, Jo, Hyun, & So, 2015;Jamali, Fairuz, undefined K., & Oskam, 2015;Maquilón, Mirete, & Avilés, 2017;Santos et al., 2016). Possibilities that when applied to teaching, show students to have high levels of satisfaction and positive attitudes for their use (Akçayır & Akçayır, 2017;Y. Hsu, Lin, & Yang, 2017;Joo, Martínez, & García-Bermejo, 2017), are motivated towards these educational actions (Cheng, 2017;Garay, Tejada, & Maiz, 2017) and improves academic performance (Lu & Liu, 2015;Tekedere & Göker, 2016;Yilmaz & Goktas, 2017). It should be indicated that it has been incorporated into different university disciplines: engineering and architecture (Odeh, Shanab, & Anabtawi, 2015), mathematicsgeometry (Coimbra, Cardoso, & Mateus, 2015), languages (T. C. Hsu, 2017), biology (Fracchia, Alonso, & Martins, 2015), geography (Tsai, Liu, & Yau, 2013), medicine (Barroso & Cabero-Almenara, 2016), education sciences (Garay et al., 2017;Martínez & Fernández, 2018), or engineering (Ibáñez & Delgado-Kloos, 2018). 2RESEARCH 2.1 Research Objectives This research was conducted as a part of a R & D + I project called “Augmented reality to increase training. Design, Production and Evaluation of augmented reality programs for university education”, funded by the Ministry of Economy and Competitiveness of the Government of Spain, the objective being sought after in the present work is: “To investigate the technical, curricular and organizational challenges that AR application may be faced with in university education”. To this end, different actions were developed, ranging from the review of literature, the construction of a specific instrument on limitations that were exposed in the scientific literature that could hinder the incorporation of AR into teaching, the selection of a series of experts, and the assessment by the experts on the significance of the limitations pointed out in the scientific literature. 2.2 Research Strategy One of the strategies used in educational research is the so-called “expert judgment”, with a “Delphi” modality application, which is one of the most used methods to configure the content validity of a scale or a questionnaire (López-Gómez, 2018;Pérez-Pérez, Gómez, & Sebastián, 2018). As Gil-Gómez and Pascual-Ezama (2012) point out: “the Delphi methodology is a technique framed within the methods of experts that is used to obtain the most consensual opinion possible from a group of people, considered experts, in relation to a certain research objective”. (p. 1011) Methodology that is commonly used for different aspects: analyzing the validity of the items in a questionnaire, studying the accuracy of meaning of terms, evaluating educational material, or identifying solutions to a problem (Barroso, Cabero, & Llorente, 2015;Escobar JournalofNewApproachesin EducationalResearch,8(2) |2019 |https://doi.org/10.7821/naer.2019.7.409 128 Julio,Barroso-Osuna;etal. DifficultiesintheIncorporationofAugmented RealityinUniversity Education: VisionsfromtheExperts & Cuervo, 2008;Gil-Gómez & Pascual-Ezama, 2012;Hernández, Fernández, & Baptista, 2010). Its application is carried out from different perspectives: individual aggregation of the experts, Delphi method, nominal group technique, and consensus method (Cabero & Barroso, 2013). Here we are using a combination of the first two, in what is called a “partially modified Delphi” in which the information is obtained individually without the experts being in contact. One of the problems for its use is how to select the experts that will participate in the investigation. And for this, different procedures are used, ranging from the analysis of their curriculum, to the use of procedures such as the “coefficient of expert competence” (Coefficient K) (Blasco, Padrón, & Mengual, 2010;García & Fernández, 2008); procedure that has been used recently by different authors in different investigations: Cabero and Barroso (2013), Llorente and Journal of New Approaches in Educational Research (2013), Zartha, Montes, Toro, and Villada (2014), and Mengual-Andrés, Roig-Vila, and Blasco (2016). In this study both procedures were used; firstly, the “AR obstacle questionnaire was sent out” (CORA), which we will present later, to different experts selected using the following criteria: having experience in the field of educational technology and the use of ICT in the field of education, having experience in the field of AR, belonging to the research team of the RAFODIUN project, and being from different Spanish and Latin American universities. At the same time, a link was placed in the “RAFODIUN Virtual Community” developed in Google+ for the project so that it could be completed voluntarily by people who wanted it. As regards to the number of experts needed to carry out the Delphi study, there is no definitive rule, and as López-Gómez (2018, p.24) points out: “[...] estimating the optimal number of the panel requires of a contingent approach, that takes into account the nature of the research and the objectives to be achieved, the geographic scope and the resources available to the researcher”. Our research, based on these arguments, makes an extensive selection of them, emphasized by the coefficient of Expert competence in their final selection. The number of “experts” who initially completed the CORA was 264, and with these we apply the “K coefficient” (K = 1 2Kc (knowledge coefficient) + Ka (argumentation coefficient)), which assumes that the expert assesses on a scale from 0 to 10 the degree of knowledge they have on the subject (Kc) and argue according to a table of value where you have acquired the training, and which is scored according to the values shown in Table 1. Table 1 Questions and scores for obtaining the Ka of the “Expert Competency Coefficient”. High Medium Low Theoretical analysis carried out by you 0,3 0,2 0,1 Your experience gained from your practical activity 0,5 0,4 0,2 Study the work on the subject, by Spanish authors 0,05 0,05 0,05 Study the work on the subject, by foreign authors 0,05 0,05 0,05 Your own knowledge about the status of the problem abroad 0,05 0,05 0,05 Your intuition about the subject 0,05 0,05 0,05 JournalofNewApproachesin EducationalResearch,8(2) |2019 |https://doi.org/10.7821/naer.2019.7.409 129 Julio,Barroso-Osuna;etal. DifficultiesintheIncorporationofAugmented RealityinUniversity Education: VisionsfromtheExperts In general terms, it is suggested (Cabero & Barroso, 2013;Llorente & Journal of New Approaches in Educational Research, 2013;Mengual-Andrés et al., 2016), that those experts who obtain a score ≥a 0.8 in the coefficient be the ones selected. In our case, this value was reached by 208 subjects, who became those that were considered for the study, having, therefore, rejected 56 of those who completed the questionnaire. In regards to the characteristics of these 208 experts, it is indicated that 80.77% (f = 168) held a doctorate degree, 11.54% (f = 24) a masters degree, 3.88% (f = 8) a bachelors degree, and 8 (f = 3.88%) had other studies. Of them the great majority (f = 188, 90.39%) worked in a university center, while the remainder worked in non-university centers (f = 8, 3.88%), a company related to training (f = 8,3, 88%), a company related to technological production (f = 2, 0.96%), and 2 indicated ‘other’ (0.966%). Its basic activity was teaching (f = 148, 71.15%), although another large percentage worked both in teaching and in the technical part (f = 42, 20.19%), the rest worked in technical aspects (f = 8, 3.88%) and in management (f = 8, 3.88%). It was indicated that 97.11% (f = 2012) had taught subjects related to Educational Technology or ICT applied to training and that they had participated in some research on ICT, virtual training or AR. At the same time, 93.27% (f = 194) had published on these aspects. 2.3 Information collection instrument The instrument for collecting information on the obstacles to the incorporation of AR in training was a questionnaire with a Likert type format, with 7 response options (from 0 = nothing significant to 6 = very significant). For its construction we follow three stages: a) telephone survey to a series of “experts” in Educational Technology and AR on the most significant obstacles they would face in the implementation of the AR in university education, b) review of scientific articles on AR published in database journals “Web of Science”, “Scopus”, and in the “Seal of Quality of Spanish Scientific Journals”, and c) obtaining the reliability index. The first phase consisted of the telephone call to different teachers of Educational Technology of different Spanish Universities, to which the following question was asked: what do you think are the most significant obstacles that can hinder the incorporation of AR in university teaching? In total twenty-five calls were made. Simultaneously, a review of the AR literature was carried out to identify problematic situations that, according to different authors, presented the main drawbacks and difficulties that can be encountered in the incorporation of AR to teaching. The results obtained are shown in Table 2. The gathering of the data obtained by both procedures allowed for the identification of a series of dimensions: • Educational futility (EF) • Difficulty using technology (DUT) • Missing student training (MST) • Student attitude/belief for incorporation (SAI) JournalofNewApproachesin EducationalResearch,8(2) |2019 |https://doi.org/10.7821/naer.2019.7.409 130 Julio,Barroso-Osuna;etal. DifficultiesintheIncorporationofAugmented RealityinUniversity Education: VisionsfromtheExperts Table 2 Dimensions identified in different works Dimensions Authors Student training Durall et al. (2012) and Liu and Tsai (2013) Few educational experiences, Johnson et al. (2012), Cuendet et al. (2013), Wojciechowski and Cellary (2013), Bower et al. (2014), Radu (2014), Cabero-Almenara and García (2016) Few investigations Serio et al. (2013), Lin et al. (2013), Chen and Tsai (2012), Wu et al. (2013), X. Wang et al. (2013), Bacca et al. (2014), and Saidin et al. (2015) Difficulty creating interactive content and few educational materials Carracedo and Méndez (2012), Cabero-Almenara and García (2016) and Akçayır and Akçayır (2017) The technology is not yet very precise Telefónica (2011), O’Shea et al. (2012) and Saidin et al. (2015) Teacher training Durall et al. (2012), Bower et al. (2014), Billinghurst and Duenser (2012) and Cabero-Almenara and García (2016) It is limited to advanced devices Telefónica (2011) and Y. Wang (2017) Search for active methodologies Durall et al. (2012) Diversity of technologies Fombona et al. (2012) Teaching attitude Wu et al. (2013) and Alkhattabi (2017) Unflexible curriculum and instructional design Durall et al. (2012), Cuendet et al. (2013) and Wu et al. (2013) Cognitive overload of students Dunleavy et al. (2010), Wu et al. (2013), Akçayır and Akçayır (2017) and Y. Wang (2017) Lack of conceptual references Rasimah et al. (2011) and Bower et al. (2014) It is not easy to manage J. L. Hsu and Huang (2011), Saidin et al. (2015) and Y. Wang (2017) Economic factors Cabero-Almenara and García (2016) and Y. Wang (2017) • Teacher attitude/belief for incorporation (TAI) • Lack of educational experience (LEE) • Lack of conceptual foundation (LCF) • Lack of teacher training (LTC) • Technological/economic • Lack of educational research (LER) • Institutional difficulty / lack of institutional support (ID) • Characteristics educational curriculum (CEC) Around these we built the CORA, which brought about a first version that was reviewed by members of the research team RAFODIUN, leaving the final version consisting of four major blocks: characterization of the expert, questions for the analysis of expert competence, assessment of the influence of a series of obstacles by the experts (41 items) and an open question to explain if they thought there might be another obstacle not indicated in the items. The questionnaire was applied via the web. In Table 3, the reliability indices obtained are presented both for the overall nature of the instrument and for each of the dimensions. Indicating that these show high levels of reliability of the instrument to be indicated (Dwyer & Bernauer, 2014), although in the “Missing student training” and “Attitude/pupils belief for incorporation” dimensions, the values were relatively moderate, possibly due to the fact that these were the dimensions on which we formulated the smallest number of items. It should be noted that in order to identify if the elimination of any item would increase the degree of reliability, we perform the JournalofNewApproachesin EducationalResearch,8(2) |2019 |https://doi.org/10.7821/naer.2019.7.409 131 Julio,Barroso-Osuna;etal. DifficultiesintheIncorporationofAugmented RealityinUniversity Education: VisionsfromtheExperts item-total correlation, pointing out that the data obtained did not indicate that the elimination of any of them would increase the degree of reliability, hence we made the decision to keep the 41 items. Table 3 Reliability index of the instrument’s overall nature and its dimensions Dimension Cronbach’s Alpha Instrument 0,944 IE (ítems: 11, 18 and 22) 0,787 DUT (ítems: 4, 13, 14, 15, 16, 26 and 27) 0,801 FFA (ítems: 31 and 37) 0,635 ACA (ítems: 19 and 28) 0,56 ACP (ítems: 20 and 24) 0,875 FEE (ítems: 1, 3 and 21) 0,661 FFC (ítems: 2, 9 and 25) 0,65 FFP (ítems: 5, 30 and 32) 0,788 PTE (ítems: 7, 6, 10, 12, 17, 23 and 29) 0,787 FIE (ítems: 8, 33 and 39) 0,812 DI (ítems: 33, 34 and 35) 0,836 CCE (ítems: 38, 40 and 41) 0,904 The instrument was applied via the Internet, and the study was carried out at the end of 2017. 3RESULTS We will begin our analysis by presenting the average scores and standard deviations reached in the different items that constituted the instrument (Table 4). The ten items that obtained lower than average score, and thus indicating to be obstacles of few or no significance, were the following: • It can only be used by adults and at higher levels of education (1.31). • It only serves to distract, not for students to learn and acquire knowledge (1.33). • Its use can only mobilize traditional methodologies and nothing innovative (1.39). • Students feel bored and confused when they are immersed in situations of using learning objects in AR (1.40). • It cannot be used in all disciplines (1.42). • The attitude shown by students towards these technologies is not positive for its incorporation into teaching (1.56). • Students believe that AR is not a significant technology for learning (1.66). • Students are not trained for its use (1.82). • These are very rigid systems that do not allow their adoption by the teacher in their class context (1.95) JournalofNewApproachesin EducationalResearch,8(2) |2019 |https://doi.org/10.7821/naer.2019.7.409 132 Julio,Barroso-Osuna;etal. DifficultiesintheIncorporationofAugmented RealityinUniversity Education: VisionsfromtheExperts Table 4 Obstacles for the incorporation of RA in university training contexts Ítem M SD 1) There are more technological developments than educational practices and experiences 4,49 1,52 2) Its novelty prevents the lack of conceptual and theoretical reflection for its incorporation into educational practice 3,87 1,48 3) The incorporation experiences carried out are more specific actions, than planned and continued actions for their educational adoption 4,56 1,22 4) Lack of educational materials for its incorporation into teaching 4,54 1,49 5) The lack of teacher training for its use, and especially for its application in innovative teaching strategies 5,07 1,27 6) The technological devices that are used for interaction have different operating systems, which calls for the realization of different versions of objects, and this makes it difficult to incorporate them into teaching 4,04 1,59 7) The difficulty and cognitive disorientation that supposes for some students, interacting in a context formed by the mixture of the real and virtual 2,8 1,65 8) The lack of educational research 4,37 1,37 9) The lack of a theoretical foundation to make decisions regarding its use and incorporation into teaching 4,09 1,57 10) Expensive equipment is needed for its use 2,96 1,84 11) It only serves to distract, not for students to learn and acquire knowledge 1,33 1,61 12) The educational centers do not have the technology for their use 3,52 1,84 13) It is difficult to produce by the teaching staff 3,86 1,56 14) Objects produced in AR are difficult to use by students 2,08 1,64 15) It can only be used by adults at higher levels of education 1,31 1,57 16) It cannot be used in all disciplines 1,42 1,61 17) For its use, it is necessary to have state-of-the-art technological devices 2,62 1,93 18) Its use can only mobilize traditional methodologies and nothing innovative 1,39 1,78 19) The attitude shown by students towards these technologies is not positive for its incorporation into teaching 1,56 1,84 20) The attitude shown by teachers towards these technologies is not positive for their incorporation into teaching 3,43 1,62 21) We do not have a volume of ”good practices” that indicate how to incorporate it into teaching 4,14 1,37 22) There are very rigid systems that do not allow their adoption by the teacher in their class context 1,95 1,62 23) Although it has evolved in recent times, its technical operation is imprecise 2,37 1,54 24) The resistance of teachers to the incorporation of ICT in education 4 1,57 25) The lack of instructional design for incorporation 4,11 1,44 26) Cognitively overload students because of the diversity of information they can offer 2,1 1,63 27) Students feel bored and confused when they are immersed in situations of using learning objects in AR 1,4 1,58 28) Students believe that AR is not a significant technology for learning 1,66 1,58 29) Industrial houses that do not decide to establish a technological standard for the use of AR 3,38 1,8 30) The lack of knowledge that teachers have about this technology 4,7 1,3 31) The ignorance that students have of this technology 3,26 1,84 32) Teachers are not trained for its use in teaching 4,79 1,36 33) There are no educational researches that allow analyzing their educational possibilities 3,71 1,74 34) The educational institution does not favor the use of ICT in general, and AR in particular 4,15 1,6 35) In the educational centers, there is no technology for its use 3,45 1,81 36) No institutional support is received for the incorporation of AR 4,08 1,64 37) Students are not trained for its use 1,82 1,78 38) The rigidity of the educational curriculum does not facilitate the incorporation of AR technology 3,41 1,82 39) There is scarce research on their educational possibilities 3,74 1,74 40) The educational curriculum is usually not very flexible to make educational technological innovations with the AR 3,27 1,82 41) The curriculum does not facilitate the incorporation of AR 3,13 1,87 JournalofNewApproachesin EducationalResearch,8(2) |2019 |https://doi.org/10.7821/naer.2019.7.409 133 Julio,Barroso-Osuna;etal. DifficultiesintheIncorporationofAugmented RealityinUniversity Education: VisionsfromtheExperts • Objects produced in AR are difficult to use for students (2.08). Contrastively, the ten items that obtained the highest average score, and which suggest very significant obstacles for their incorporation were the following: • The lack of instructional design for incorporation (4.11). • We do not have a volume of “good practices” that indicate how to incorporate it into teaching (4.14). • The educational institution does not favor the use of ICT in general, and AR in particular (4.15). • The lack of educational research (4.37). • There are more technological developments than educational practices and experiences (4.49). • Lack of educational materials for its incorporation into teaching (4.54). • The incorporation experiences carried out are more specific actions, than planned and continued actions for their educational adoption (4.56). • The lack of knowledge that teachers have about this technology (4.70). Teachers are not trained for its use in education (4.79). • The lack of teacher training for its use, and above all for its application in innovative teaching strategies (5.07). With regard to the mena scores and standard deviation reached, both in the overall nature of the instrument and in the different dimensions that make it up, the results are presented in Table 5 . Table 5 Mean and standard diviation obtained in the dimensions Dimension M SD Educational futility (EF) 1,55 1,44 Difficulty using technology (DUT) 2,39 1,13 Missing student training (MST) 2,54 1,53 Student attitude / belief for incorporation (SAI) 1,61 1,45 Teacher attitude / belief for incorporation (TAI) 3,71 1,48 Lack of educational experience (LEE) 4,4 0,97 Lack of conceptual foundation (LCF) 4,02 1,15 Lack of teacher training (LTC) 4,85 1,07 Technological / economic problems (TEP) 3,1 1,17 Lack of educational research (LER) 3,94 1,4 Institutional difficulty / lack of institutional support (ID) 3,89 1,46 Characteristics of educational curriculum (CEC) 3,27 1,64 Five have been the dimensions, to which the experts pointed out that might present the most difficulty: • Lack of teacher training JournalofNewApproachesin EducationalResearch,8(2) |2019 |https://doi.org/10.7821/naer.2019.7.409 134 Julio,Barroso-Osuna;etal. DifficultiesintheIncorporationofAugmented RealityinUniversity Education: VisionsfromtheExperts Education: Advantages and Applications. International Education Studies,8(13), 1–8. https:// doi.org/10.5539/ies.v8n13p1 Santos, M., Wolde, A., Taketomi, T., Yamamoto, G., Rodrigo, M., Sandor, C., & Kato, H. (2016). Augmented reality as multimedia: the case for situated vocabulary learning. Research and Practice in Techology Enhanced Learning,11(4), 1–23. https://doi.org/10.1186/s41039-016 -0028-2 Seifert, T., Hervás, C., & Toledo, P. (2019). Diseño y validación del cuestionario sobre percepciones y actitudes hacia el aprendizaje por dispositivos móviles. Pixel-Bit. Revista de Medios y Educación,54, 45–64. https://doi.org/10.12795/pixelbit.2019.i54.03 Serio, A. D., Ibáñez, M. B., & Delgado, C. (2013). Impact of an augmented reality system on students’ motivation for a visual art course. Computers y Education,68, 586–596. https://doi.org/10 .1016/j.compedu.2012.03.002 Tekedere, H., & Göker, H. (2016). Examining the Effectiveness of Augmented Reality Applications in Education: A Meta-Analysis. International Journal of Environmental & Science Education, 11(16), 9469–9481. Telefónica, F. (2011). Realidad Aumentada: una nueva lente para ver el mundo. Madrid: Fundación Telefónica-Ariel. Tsai, M., Liu, P., & Yau, J. (2013). Using electronic maps and augmented reality-based training materials as escape guidelines for nuclear accidents: An explorative case study in Taiwan. British Journal of Educational Technology,44(1), E18–E21. https://doi.org/10.1111/ j.1467-8535.2012.01325.x Wang, X., Jeong, M., Love, P., & Kang, S.-C. (2013). Augmented Reality in built environment: Classification and implications for future research. Automation in Construction,32, 1–13. https://doi.org/10.1016/j.autcon.2012.11.021 Wang, Y. (2017). Using augmented reality to support a software editing course for college students. Journal of Computer Assisted Learning,33, 532–546. https://doi.org/10.1111/jcal.12199 Wojciechowski, R., & Cellary, W. (2013). Evaluation of learners’ attitude toward learning in ARIES augmented reality environments. Computers & Education,68, 570–585. https://doi.org/10 .1016/j.compedu.2013.02.014 Wu, H.-S., Wen-Yu, S., Chang, H.-Y., & Liang, J. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education,62, 41–49. https://doi.org/10 .1016/j.compedu.2012.10.024 Yilmaz, R. M., & Goktas, Y. (2017). Using augmented reality technology in storytelling activities: examining elementary students’ narrative skill and creativity. Virtual Reality,21(2), 75–89. https://doi.org/10.1007/s10055-016-0300-1 Zartha, J., Montes, J., Toro, I., & Villada, H. (2014). Método Delphi - Propuesta para el cálculo del número de expertos en un estudio Delphi sobre empaques biodegradables al 2032. Espacios, 35(13). JournalofNewApproachesin EducationalResearch,8(2) |2019 |https://doi.org/10.7821/naer.2019.7.409 141