Adoption of augmented reality technology by university students

Adoption of augmented reality technology by university students Julio Cabero-Almenara, Jos e María Fern andez-Batanero * , Julio Barroso-Osuna Department of Teaching and Educational Organization, University of Seville, Seville, Spain ARTICLE INFO Keyword: Education ABSTRACT In recent times, Augmented Reality has gained more relevance in the field of education. This relevance has been enhanced due to its ease of use, as well as the availability of the technical devices for the students. The present study was conducted with students enrolled in the Pedagogy Degree in the Faculty of Education at the University of Seville. The objective was to understand the degree of technological acceptance of students during their interaction with the AR objects produced, the performance achieved by the students, and if their gender affected their acquisition of knowledge. For this, three data collection instruments were utilized: a multiple choice test for the analysis of the student's performance after the interaction, the Technology Acceptance Model (TAM) diagnostic instrument, created by Davis (1989), and an “ad hoc”instrument created so that the students could evaluate the class notes enriched with the AR objects created. The study has allowed us to broaden the scientific knowledge of the TAM by Davis, to understand that AR objects can be utilized in university teaching, and to know that the student's gender does not influence learning. 1. Introduction 1.1. References to augmented reality “Augmented Reality”(AR) has become an emergent technology with great possibilities for its use in education (Akçayır and Akçayır, 2017; Tecnol ogico de Monterrey, 2017). It allows for the combination of digital and physical information in real time through different technological formats such as tablets or smartphones to create this new reality, which has had repercussions in the number of applications created for AR (Fombona et al., 2018). This mixed integration of the physical and digital realities has been performed at different levels, with the use of QR codes, images, 3D objects, the deployment of coordinates through GPS, to the use of heat signatures (Schmalstieg and H€ ollere, 2016). Its significance for education has been determined, on the one hand, by its specific characteristics, such as: being a mixed reality, the possibility that it offers for integrating different layers of information in real time, such as information in different types of formats (text, url, videos…), it is an interactive technology, its ease of use, and through it use, one can enrich or alter the information of reality, adding additional information (Cabero and García, 2016). Also, the device that is usually used for its viewing, such as the smartphone, is an easily-available technology for university students, and is a technology that has a high degree of acceptance among them (Y a~ nez-Luna and Arias-Oliva, 2018). AR differentiates itself from mixed and virtual reality, in that the environment and the objects are visible (Brigham, 2017). To facilitate understanding of the concept of Augmented Reality (AR), this term has been differentiated from Virtual Reality (VR) and Mixed Reality (MR). Thus, in this distinction we can follow different paths. One of them is locating both within the reality-virtuality continuum, in which the AR would be closer to the real context, while the VR would be located in one of the extremes, being in the middle of these two "Augmented Virtual Reality"or "Mixed Reality". This includes an element of the AR and VR. We can also contrast them both. We can say that AR combines reality with informative elements placed in technological gadgets in order to achieve a new reality. Whereas VR the individual is placed in a technological immersive context and not mixed with reality (Díaz, 2016; Johnson and Adams, 2016). The first one, the aim is mixing reality with virtual reality. This means that the user can interact with the physical and digital worlds; while the second one, facilitates the user's transition to an immersion alternative world, simulated by the computer where there are different sensorial experiences, in which the user can interact with the environment as one is in it. Its use in education, as pointed out by different authors (Han et al., 2015;Santos et al., 2016;Akçayır and Akçayır, 2017;Aguayo et al., 2017;Pedraza et al., 2017;Pejoska-Laajola et al., 2017;Chang and Hwang, 2018;Iba~ n ez &Delgado, 2018;Rauschnabel et al., 2018), could * Corresponding author. E-mail address: [email protected] (J.M. Fern andez-Batanero). Contents lists available at ScienceDirect Heliyon journal homepage: www.heliyon.com https://doi.org/10.1016/j.heliyon.2019.e01597 Received 29 January 2019; Received in revised form 25 March 2019; Accepted 25 April 2019 2405-8440/©2019 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/bync-nd/4.0/). Heliyon 5 (2019) e01597 provide different possibilities such as: a) The elimination of information that could hinder the capturing of significant information by the student. b) The increasing or enriching of the information from reality to make it more comprehensible for the student. c) To be able to observe an object from different points of view, with the student selecting when and the point of views themselves. d) To promote ubiquitous learning. e) The creation of safe “artificial”scenarios for students, such as laboratories or simulators. f) Enriching the printed material for the students with additional information in different formats. g) Turning the students into “pro-consumers”of learning objects in AR format. h) Can be used in training activities based on the Flipped Classroom methodology. i) Informal learning is promoted. j) Can be used in different disciplines and educational levels. As for its impact on education, and although the research on this topic has been limited (Alkhattabi, 2017), some aspects have been evidenced, such as the students showing favorable attitudes towards it, with its use increasing their motivation towards learning (Tekedere and G€ oker, 2016; Barroso, 2018). These aspects could favor the creation of a constructivist context of learning (Alkhattabi, 2017), which could promote an active environment of learning (Akçayır and Akçayır, 2017;Cheng, 2017), resulting in a high degree of satisfaction of the students (Cheng, 2017; Hwang and Zo, 2016;Díaz-Noguera et al., 2017;Marín, 2017;Martínez and Fern andez, 2018), so that its use improves the results of learning (Tekedere and G€ oker, 2016;Barroso, 2018). As for its limitations, the following has been mentioned: there have been more technological developments that educational ones; the novelty has resulted in a lack of theoretical reflection, lack of theoretical substantiation for its incorporation, small number of educational materials, limited training of the teachers, and scarcity of research studies (Saidin et al., 2015;Alkhattabi, 2017;Aguayo et al., 2017;Wang, 2017). 1.2. TAM model of technology acceptance The “Technology Acceptance Model”or TAM was formulated in 1986 by Davis (1989), and was based on the psychological theory of “Reasoned Action”(Ajzen and Fishbein, 1980), which sought to predict the individual's behavior as a function of their intentions and attitudes, although it also showed elements related with the theory of “perceived self-efficacy”, formulated by Bandura (1990). It suggested that the attitude or pre-disposition one had with respect to the intention to use a technology was determined by two variables: the perceived usefulness, and the perceived ease of use. The first was understood as the subjective probability that a person perceives that using a specific technological system would improve the person's performance within a specific context; so that it represented the degree in which an individual believed that the use of a specific technology would contribute with the increased performance in all the actions the person conducted. The second referred to the degree in which the individual hoped that the use of a technological system would imply little effort, and as a result, the person would find little difficulty and would require little energy for its use. The initial model was also directed to others such as the “Unified Theory of Acceptance and Use of Technology”(UTAUT) (Cabero et al., 2016a;Guzm an et al., 2019). Also, relations have been established with the theories of self-determination (Fathali and Okada, 2018) and planned behavior (Cheng, 2018). In agreement with the proposal originally formulated by Davis (1989), these variables would have repercussions on the “attitude towards its use”and on the “Intention to use”of a person towards a specific technology. All of this would be determined by different external variables, such as: type of user, gender, age, experience in the use of technologies, degree of training, professional level, and personal tendency towards innovation (Cabero et al., 2016b). In our case, the materialization we have done of the model for our research study is presented in Fig. 1. Different hypotheses stem from it, and these will be presented below. The different research studies conducted, as well as their metaanalysis (He and King, 2008), have evidenced that it is a valid and robust model that could be used to explain the intention to use of people in any technological environment. It has also been recently confirmed when it was analysed through a structural equation model (Cabero and P erez, 2018). Research has been conducted on different technologies such as: portfolios (Wai-tsz, Chi-kin, Chang, Zhang and Chiu, 2014), e-learning, b-learning and m-learning (Mohammadi, 2015;Al-Azawei et al., 2017;Al-Emran et al., 2018), electronic mail (Oyeleye et al., 2015), virtual and social learning communities (Akman and Turhan, 2017), tele-training platforms (Alharbi and Drew, 2014), wiki (Altanopoulou and Tselios, 2017), videogames (Cheng et al., 2013), or mobile devices (Villani et al., 2018;Y a~ nez-Luna and Arias-Oliva, 2018). Nevertheless, we should not forget that the model possesses a series of limitations, which according to proposals by different authors (L opez-Bonilla and L opez-Bonilla, 2017;Cheng, 2018), they could be defined as the following: the results are conditioned by the context in which the technology is utilized; for example, the different contexts of Fig. 1. Technology acceptance model (TAM). J. Cabero-Almenara et al. Heliyon 5 (2019) e01597 2 obligatory and voluntary use; review if the concept of attitude itself should be or not incorporated into the model, the confusion with the theory of planned behavior; the information collection instrument is based on self-reports, and these have their own limitations; the difficulty in searching for objective metrics of the degree of acceptance of this technology, and the conceptual simplicity of the model, overall when referring to the relationship between the intention to use and the real use, as the latter one, as shown, can be conditioned by different types of variables. 2. Hypothesis The questions we aim to provide an answer to with this research study, are presented in the following terms: What is the degree of technological acceptance that the enrichment of class notes through the use of AR awakes in university students? Are there significant differences on the performance achieved by the students after their interaction with class notes enriched with AR? Does the student's gender have repercussions on their acquisition of knowledge? These questions will allow us to understand and delve into different aspects needed for the addition of AR within the contexts of learning; on the one hand, when broadening the studies from an educational perspective about them, which, as pointed out by different authors (Saidin et al., 2015;Alkhattabi, 2017;Aguayo et al., 2017), tend to be very limited; and on the other hand, to delve on the question of if this technology awakens a certain degree of acceptance in the students, and if they, when interacting with it, learn the content presented. At the same time as these questions are answered, the degree of knowledge on the TAM model and its validation with AR technology will be incremented. On the other hand, one of the external variables that has been considered in the studies on the TAM, as a predictive variable, has been the student's gender, with contradicting results found as respect to their significance (Ho et al., 2013;Hohlfeld et al., 2013;Olaoluwakotansibe, 2013;Tarhini et al., 2014;Cabero et al., 2018b). On the other hand, it should be forgotten that the degree of significance of the results that will be found will be determined by the reliability and validity of the diagnosis instruments utilized. The hypotheses (H) posited were the following: H1-H2-H3. The perception of the technical quality of the object produced in AR can positively and significantly affect the perception of enjoyment, the ease of use and the usefulness perceived of the use of learning objects in AR. H4-H5-H6. The subject's gender can positively and significantly affect the perception of enjoyment, the ease of use and the perceived usefulness of the use of learning objects in AR. H7-H8-H10. The perception of ease of use can positively and significantly affect the perception of enjoyment, the perceived usefulness and the attitude towards the use of learning objects in AR. H9-H14-H15. The perceived usefulness of the use of learning objects in AR can positively and significantly affect the perception of enjoyment, the attitude towards its use and the intention to use the learning objects in AR. H16. The perceived usefulness can positively and significantly affect the academic performance achieved by the students on the use of the AR learning objects. H11-12. The perception of enjoyment can positively and significantly affect the attitudes and intention to use of the AR learning objects. H13. The perception of enjoyment can positively and significantly affect the academic performance achieved by the students on the use of the AR learning objects. H17. The attitude towards its use can positively and significantly affect the intention to use of the AR learning objects. H18. The intention to use of the AR objects can positively and significantly affect the academic performance achieved by the students on the use of AR learning objects. 3. Materials &method The research design utilized was experimental, with a single pretestposttest group (Sans, 2004). Within it, the independent variable was the object produced in AR, and the dependent ones were the academic performance, the subject's gender, the degree of acceptance of the AR and the evaluation of the object performed. The data described in this article contains no personal, or personally identifiable information and are not accessible to other researchers as per written agreement with participants and ethical approval. 3.1. Sample The research was conducted with students enrolled in the courses “Educational Technology”in the second year of the Pedagogy Degree and “Educational application of the Information and Communications Technology (ICT)”in the Children and Primary Education Degrees, both of which were taught at the Faculty of Education from the University of Seville. The total number of participants was 396 students, of which 34.85% (f ¼138) were men and 65.15% (f ¼258) women. The proportion of the men and women is the one commonly found for these types of university degrees, as they tend to have a greater female population. It should be pointed out that all the students enrolled in the course at the University participated in the study. The sample used was non-probabilistic, using convenience sampling, based on the ease of access to the subjects who participated in the study by the researchers. 3.2. Material produced The material produced for the study revolved around the contents from “Ways of using videos in teaching”and dealt with different aspects, from the different proposals that different authors had suggested on the ways of using videos for teaching, the analysis of the different uses (as a provider of information, as an instrument of knowledge, evaluator of learning,…), to the bibliography on the subject and references, in order to become more knowledgeable. We selected this criteria because it refers to the use of audio-visual tools applied to educational contexts; it addressed different issues, ranging from proposals made by different authors on the ways to use videos in teaching. The material was prepared under the modality of books or notes enriched with objects produced in AR. Through QR codes or images, the information presented is enriched with different types of resources (multimedia, video clips, audio podcast,…), and according to different studies, these enriched materials improve learning and motivation, and it is a technology that has been highly accepted by students as well, as it does not require technological resources for its use (Nadolny, 2016;Cheng, 2017;Hung et al., 2017; Garay et al., 2017;Barroso, 2018). The object produced could be classified as RA type II (Cabero and García, 2016)in"rich notes"format with AR artefacts, transferring resources in video format. This kind of AR output has been demonstrated effective in different investigations (Martín-Guti errez et al., 2015;Wei et al., 2015;Cheng, 2017). We should point out it was the first time that students have worked with this kind of equipment. The information from the class notes was enriched with two video clips of the professor's explanations that could be viewed by the students through different mobile devices. The video clips were inserted into the class notes through the SDK (Software Development Kit) from Metaio, and later apps for Windows, IOS and Android were created, to promote J. Cabero-Almenara et al. Heliyon 5 (2019) e01597 3 their download and use. Fig. 2 shows images of the material created. At the same time, a guide was created that contained additional information, indications about where the different apps could be downloaded from, an explanation of the procedure for using the object, and a presentation of additional information to delve into the subject matter. Fig. 3 shows images of the guide. The procedure followed for its addition to the teaching task was composed of four phases: 1) In the first phase, the pretest was applied. 2) In the second phase, the functioning of the objects was explained to the students, and different comments on the use of AR, the place where they had to go to download the app, to install it in their mobile devices, as well as the class notes, were provided. 3) The third phase lasted two weeks, and the students worked independently with the objects and in the practical classes; their doubts they had on the content presented, AR or the manipulation of objects were resolved, 4) In the last phase, the students completed the instruments of motivation diagnostics, performance and quality of the object produced. It should be noted that the students were aware that the grades received in the experience would be used to compute the course's final grades. 3.3. Information collection instruments There were three information collection instruments: multiple choice test for the analysis of the performance achieved by the students after their interaction, the technology acceptance model (TAM) diagnostic tool created by Davis (1989), and an instrument created “ad hoc”, used for the students' evaluation of the class notes enriched with the AR objects created for the course. The analysis of the academic performance achieved by the students was conducted with multiple choice tests, administered as a “pretest”and “posttest”. These were constructed with 15 items that sought to collect information on the following categories according to the taxonomy by Bloom: apply (4 items), remember (6 items) and understand (5 items). The items in both tests were the same, although the order of presentation was altered. As for the instrument used to diagnose the acceptance (TAM) of the class notes enriched with AR, the one by Davis (1989) was used. It was composed of 15 items, which collected information on the different dimensions: perceived usefulness (4 items), perceived ease of use (3 items), perceived enjoyment (3 items), attitude towards its use (3 items), and intention to use (2 items). This instrument, as the previous one, had a Likert format, with 7 response options (1 ¼extremely improbable/in disagreement to 7 ¼extremely probable/in agreement). The reliability reached by the instrument, analysed with Cronbach's Alpha, was 0.940 for the instrument overall, and the following values were found for the different dimensions that comprised it: Perceived utility (PU) ¼0.895; Perceived Ease of Use (PEU) ¼0.823; Perceived enjoyment (PE) ¼0.907; Attitude towards its use (AU) ¼0.669; and Intention to use (IU) ¼0.873. These values indicated a high degree of reliability (O'Dwyer &Bernauer, 2014). The instrument for the evaluation of the quality of the class notes enriched with AR was created “ad hoc”for the present study. The instrument was composed of 12 items that sought to collect information from different dimensions: technical and aesthetic aspects (4 items), ease of navigation and movement in the environment (6 items) and two items Fig. 2. Objects produced. Fig. 3. Images of the guide. J. Cabero-Almenara et al. Heliyon 5 (2019) e01597 4 destined for the evaluation of the program's guide/tutorial. The instrument was constructed using a Likert-type format with 6 response options (1 ¼very negative/in strong disagreement and 6 ¼very positive/very much in agreement). The instrument received a reliability score of 0.930 for the instrument as a whole, and 0.888 for the dimension “Technical aspects”, 0.860 for “Ease of navigation”and 0.843 for the dimension “Program tutorial”. These values indicated high levels of reliability. All of these instruments were administered via the Internet. Tables 1 and 2show the scores of the items that comprised the instrument used for the evaluation of the material and the evaluation of the TAM. 4. Results The results of the academic performance of the students showed that the median score obtained in the pretest was 4.56, with a standard deviation of 3.21; in the case of the posttest, the mean was 9.95 with a standard deviation of 2.20. In order to analyse if the differences found between the posttest and the pretest scores were significant from the statistical point of view, which would allow us to predict if the students had acquired the in-person information, we formulated the following hypothesis: H0 (null hypothesis): There are no significant differences with an Alpha risk of rejecting the hypothesis of 0.05, with respect to the scores received by the students in the pretest and the posttest. H1 (alternative hypothesis): There are significant differences with an Alpha risk of rejecting the hypothesis of 0.05, with respect to the scores received by the students in the pretest and the posttest. For this, Student's t-test was used, resulting in a value of t ¼18.733, for 197 degrees of freedom, which resulted in the rejection of the H0 formulated, with a risk of less than 0.01. As a result, we concluded that the participation of the students in the experience with enriched class notes served the students to acquire the contents that were developed in the document that referred to “ways of using video for teaching”. Therefore, they were able to increase their knowledge on this subject. Table 1 shows the mean scores and standard deviations of the students' evaluation of the object produced. The mean value of the instrument was 3.60 with a standard deviation of 0.82. The average evaluations received for the overall instrument, as well as for the dimensions that composed it and the items that comprised them, were found to be slightly higher that the central value of the scale used, which was between 1 and 6. Thus, a certain degree of positive evaluation of the AR object used was found, as referring to the technical aspects, the ease of use and the model guide created. In order to analyze if relationships could be established between the student's evaluations of the object produced and the academic performance achieved, Pearson's correlation coefficient was used, obtaining a value of 0.169. This value, in agreement with Etxeberria and Tejedor (2005), indicates that there is significant and positive relationship between the performance achieved and the evaluation of the object provided, although the relationship was moderate. As for the scores of the TAM diagnostic instrument, Table 2 shows the means and standard deviations of the instrument's dimensions as well as their items. The mean value obtained for the overall instrument was 4.97, with a standard deviation of 1.03, which surpassed the scale's mean of 3.5. Thus, this indicated a certain degree of acceptance of the AR technology by the students, after their participation in the experience. As shown by the values in Table 2, the students perceived the system as useful, although what was more interesting was that they mentioned “having enjoyed the system”(5.13) and showing a high “intention for use”(5.12) in the future. One of the hypothesis posited in this research study was destined for the analysis of whether or not the student's gender had a significant influence on the perception of enjoyment, in the ease of use and the perceived usefulness of using class notes enriched with AR. Table 3 shows the mean values of the results. Table 1 Average scores provided. Dimension Mean SD 1. Technical and aesthetic aspects 3.66 0.86 1.1. The functioning of the resource with AR that we have presented was: 3.67 1.010 1.2 In general, you consider the aesthetic of the resource produced in AR as: 3.64 0.993 1.3 In general, you would rate the technical functioning of the resource produced in AR as: 3.46 1.054 1.4. In general, how would you evaluate the presentation of the information on the screen: 3.66 1.086 2. Ease of use 3.46 0.87 2.1. How would you rate the ease of use and handling of the AR resource that we have presented? 3.58 1.056 2.2. How would you rate the ease of comprehension of the technical functioning of the AR resource we have presented? 3.47 1.130 2.3. From your point of view, how would you evaluate the general design of the AR resource we have created? 3.48 1.064 2.4. From your point of view, how would you evaluate the accessibility/usability of the AR resource we have presented? 3.37 1.164 2.5. From your point of view, how would you evaluate the flexibility of use of the AR material we have presented? 3.38 1.086 2.7. The use of the AR resource produced with AR was fun: 3.55 1.288 3. Guide/program tutorial 3.66 0.95 3.1. In general, how would rate the efficacy and the comprehensibility of the information offered for the use of the AR resource we have presented? 3.60 1.064 3.2. The information offered for using the AR resource was simple and comprehensible to you. 3.50 1.148 Table 2 Mean values and standard deviations of the TAM instruments. Dimension Mean SD Perceived usefulness (PU) 4.77 1.13 The use of this AR system will improve my learning and performance in this course (PU1) 4.68 1.327 The use of this AR system during the classes will ease my comprehension of certain concepts (PU2) 4.79 1.346 I believe the AR system is useful when one is learning (PU3) 4.93 1.265 My performance will increase with the use of AR (PU4) 4.68 1.253 Perceived ease of use (PEU) 4.87 1.09 I believe the AR system is easy to use (PEU1) 4.64 1.313 Learning how to use the AR system is not a problem for me (PEU2) 5.01 1.211 Learning how to use the AR system is clear and comprehensible (PEU3) 4.96 1.294 Perceived enjoyment (PE) 5.13 1.35 Using the AR system is fun (PE1) 5.04 1.539 I enjoyed using the AR system (PE2) 5.08 1.439 I believe the AR system allows learning while playing (PE3) 5.28 1.415 Attitude towards it use (AU) 5.07 1.19 The use of a AR system makes learning more interesting (AU1) 5.22 1.388 I was not bored while using the AR system (AU2) 4.67 1.765 I believe the use of an AR system in the classroom is a good idea (AU3) 5.33 1.400 Intention to use (IU) 5.12 1.34 In the future, I would like to use the AR system if I had the opportunity (IU1) 5.23 1.388 I would like to use the AR system to learn other subjects (IU2) 5.00 1.453 J. Cabero-Almenara et al. Heliyon 5 (2019) e01597 5 To know if the differences found were significant from the statistical point of view, the following hypotheses were formulated: H0 (Null hypothesis): There are no significant differences as a function of the student's gender on the degree of acceptance of the technology (perceived enjoyment, perceived ease of use and perceived usefulness) of the class notes enriched with AR objects, with an Alpha risk of rejection of 0.05. H1 (Alternative hypothesis): There are significant differences as a function of the student's gender on the degree of acceptance of the technology (perceived enjoyment, perceived ease of use and perceived usefulness) of the class notes enriched with AR objects, with an Alpha risk of rejection of 0.05. For their analysis, Student's t-test was used to accept or reject the H0 with an Alpha risk of rejection. Table 4 shows the values received. As observed, none of the values obtained allowed us to reject the H0 formulated; thus, we could point out that there were no significant statistical differences with an Alpha risk or rejection below 0.05 on the degree of acceptance of the technology (perceived enjoyment, perceived ease of use and perceived usefulness) and the student's gender. To analyse the hypothesis that arose from the dimensions of the TAM formulated (Fig. 1), it should be indicated that the H0 referred to the significance of the relationship cited (Table 5). The Person's correlation coefficients obtained allowed us to obtain different conclusions. In first place, the establishment of significant relationships between the different dimensions related to the evaluation of quality and the dimensions from the TAM, with an alpha risk of rejection of less than 0.01. As a result, we concluded that: - The perception of the technical quality of the class notes enriched with AR and the perception of ease of use of the AR class notes positively and significantly affected the perception of enjoyment, the perceived ease of use and the perceived usefulness. - The perceived enjoyment of the class notes enriched with AR had a positive and significant effect with respect to the perceived ease of use, perceived utility, attitude towards its use and intention of use - The perceived ease of use of the class notes enriched with AR had a positive and significant effect with respect to the perceived usefulness and the attitude towards its use. - The perceived usefulness of the class notes enriched with AR positively and significantly affected the attitude towards its use and the intention to use it. - The attitude towards the class notes enriched with AR positively affected the intention to use. The relationships between the variables cited oscillated between the 0.440 found between the perception of the “technical quality”of the class notes enriched with AR objects, and the “perceived ease of use”; and the relationship of 0.779 between the “perceived enjoyment”and the “attitude towards its use”. These results led us to conclude that there were strong relationships between the different variables studied. In second place, we can point out that there were not significant relationships between the TAM dimensions “perceived enjoyment”, “perceived usefulness”and “intention to use”, and the performance achieved by the students. Our last comparison was destined for the analysis of whether or not the relationships could be established between the TAM dimensions and the subject's perception of quality of the object produced. Table 6 shows the results after the application of Pearson's correlation. The data found allowed us to demonstrate relationships that were moderate (.330) between the “ease of use”and the “attitude towards its use”, and high (.828) between the overall evaluation of the “quality of the object”and the “intention to use”. Once the analysis that established the reliability and the internal consistency of the items within the different dimensions that comprised the TAM created for AR were finished, a structural model was analysed, as formulated by obtaining path coefficients, the values of Student's t-test and the R 2 ; a coefficient that determined the percent of variance of a construct that was explained by its predictive variables. Fig. 4 shows the values obtained, and for this, the Structural Equation Model (SEM) was used, as it allows for the estimation of the effects and the relationships between multiple variables. Its great potential comes from it being able to represent the causal effect between the existing variables in a given Table 4 Student's t-test for the significance of the TAM scores as a function of the student's gender. Dimension Student's t-test Two-tailed signif. Perceived enjoyment 1.342 0.181 Perceived ease of use 0.668 0.505 Perceived usefulness 1.232 0.219 Total 0.763 0.568 Table 3 Mean values and standard deviations found for the TAM and its dimension as a function of the student's gender. Dimension Student's gender Mean SD Total Men 4.8660 1.63765 Women 4.5068 1.30482 Perceived enjoyment Men 5.0980 1.76684 Women 4.6034 1.36005 Perceived ease of use Men 4.8824 1.67474 Women 4.5460 1.27305 Perceived usefulness Men 4.6176 1.47139 Women 4.3405 1.28136 Table 5 Pearson's correlations between “Technical quality”,“Perceived enjoyment”,“Perceived ease of use”,“Perceived usefulness”,“Attitude towards its use”,“Intention to use”and “Performance”. Dimension Technical quality Perceived enjoyment Perceived ease of use Perceived usefulness Attitude towards its use Intention to use Performance Technical quality Pearson's C. —— 0.486 0.440 0.484 Two-tailed Sig. 0.000 (**) 0.000 (**) 0.000 (**) Perceived enjoyment Pearson's C. ——— 0.588 0.693 0.779 0.743 0.500 Two-tailed Sig. 0.000 (**) 0.000 (**) 0.000 (**) 0.000 (**) 0.488 Perceived ease of use Pearson's C. ——— 0.548 0.541 Two-tailed Sig. 0.000 (**) 0.000 (**) Perceived usefulness Pearson's C. ———— 0.701 0.640 0.700 Two-tailed Sig. 0.000 (**) 0.000 (**) 0.328 Attitude towards its use Pearson's C. ——— 0.759 Two-tailed Sig. 0.000 (**) Intention to use Pearson's C. ——— 0.013 Two-tailed Sig. 0.857 Performance Pearson's C. ——— Two-tailed Sig. J. Cabero-Almenara et al. Heliyon 5 (2019) e01597 6 model. One of the main contributions of the SEM is that it allows researchers to evaluate and analyse theoretical models formulated (Kerlinger and Lee, 2002). Along this line, Cupani (2012) specified its use for three tasks: as a strategy for the confirmation of established theoretical models, to contrast rival theoretical models, and for the development of a specific model. The SEM was used for the first case. With the Smart PLS () program used for this. As for the latent variables referred to in the TAM model formulated by Davis (1989), and after the application of the SEM, we found that 72.69% of the variance of the latent variable “Attitude towards its use”was explained by the latent variables “Perceived ease of use”,“Perceived usefulness”and “Perceived enjoyment”; 68.74% of the variance of the latent variable “Intention to use”was explained by the latent variables “Perceived usefulness”,“Attitude towards its use”and “Perceived enjoyment”; 55.95% of the variance of the latent variable “Perceived enjoyment”was explained by the latent variable “Perceived usefulness”, “Perceived ease of use”and “Technical quality”; 38.55% of the variance of the latent variable “Perceived usefulness”was explained by the latent variables “Perceived ease of use”and “Technical quality”, and only 24.3% of the variance of the variable “Perceived ease of use”was explained by the latent variable “Technical quality”. As for the results referring to the performance, only 2.37% of the variance was explained by the latent variables “Perceived enjoyment”,“Intention to use”and “Perceived usefulness”. 5. Discussion &conclusion One of the main conclusions of the study refers to the reliability of the diagnostic instruments utilized, in the case of the TAM, as well as the instrument created for the present study, which was used by the students to evaluate the quality of the AR object-enriched class notes. In the case of the TAM instrument, the results were similar to those found by other authors (Ho et al., 2013;Cheng et al., 2013;Tarhini et al., 2014;Barroso, 2018;Cabero and P erez, 2018;Fern andez, 2018). This allowed us to conclude that the diagnostic instruments utilized possessed acceptable levels of reliability, as a group and according to the different dimensions that comprise it as well. With respect to the first hypothesis (H1-H2-H3) point out that the class notes enriched with AR are perceived by the students as being easy and flexible to use, with the students showing at the same time a true intention to use them for their future education. This leads us to suggest the possibility that within the university training context, a tendency must emerge for the production of multimedia materials under this format, which will enrich the possibilities of interaction of the students. In agreement with other works carried out with different technologies, in our case with AR, no significant differences were found between the student's gender and the degree of acceptance of this technology (H4- H5-H6). This indicates that the gender-based digital divide within the university context is being reduced even more, and the digital competence of the students are not shown to be related to gender (Hohlfeld et al., 2013). The results from the use of Pearson's correlation coefficient and through the use of structural equations evidence the internal consistency between the different dimensions from the TAM formulated by Davis (1989) and at the same time, verifies the findings by other authors with respect to the significance of the model (Cabero et al., 2016b;Cabero and P erez, 2018). This suggests that the TAM model could be considered adequate for understanding the degree of acceptance that a technology awakens in its potential users and their future intention of use as well (H7-H8-H10). As pointed out in the first part of the present work, this model has already been utilized with many types of technologies, and our use with AR re-enforces it. Another of the questions formulated in the present study was related to the analysis of the degree of acceptance that this AR technology awoke in the students. In this sense, the work allowed us to point out that the AR-enriched notes were perceived by the students as easy to use, with the students showing a true intention of using them for their training, as well Table 6 Pearson's correlation coefficient. Dimension Perceived ease of use Perceived enjoyment Attitude towards its use Intention to use Quality of the object Pearson's C.C. .440 (**) .486 (**) .498 (**) .828 (**) Twotailed Sig. .001 .000 .000 .000 Technical and aesthetic aspects Pearson's C.C. .402 (**) .446 (**) .458 (**) Twotailed Sig. .004 .001 .001 Ease of use Pearson's C.C. .330 (*) .612 (**) Twotailed Sig. .019 .000 Guide/ tutorial Pearson's C.C. .498 (**) Twotailed Sig. .000 Note (**) significant at alpha ¼0.01. Fig. 4. Structural equation model of the TAM formulated. J. Cabero-Almenara et al. Heliyon 5 (2019) e01597 7 as having a high degree of acceptance (H9-H14-H15). This conclusion is similar to other researches such as Jahnke and Kroll (2018) who also observed an interest in its use as a learning tools in education. Thus, the creation of this type of learning objects should be promoted for their use in university teaching settings. This type of production is not complex, and production activities have already been conducted by university students (Cabero et al., 2018a), who had a great number of easy-to-use programs available, such as “Augmented Class”,“HP Reveal”, or “Blippar”. Besides the ease of production of the AR objects for their use with enriched notes, their addition also counts with the advantage of the presence of mobile devices owned by students at this educational level (Lagunes-Domínguez et al., 2017). The results found after the application of Pearson's correlation and the use of structural equations showed the internal consistency of the different dimensions of the TAM model formulated by Davis (1989), and at the same time, confirmed the findings by other authors related to the model's significance (Cabero et al., 2016b;Cabero and P erez, 2018). Another of the research questions asked if the addition or incorporation of this technology to teaching activities would serve for the students' learning of the content presented, and the results clearly showed that this was indeed the case. The notes enriched with AR were shown to be powerful objects for learning (H11-H12-H17). This allowed us to suggest, as other authorshave (Alkhattabi, 2017;Joo et al., 2017), that they could perfectly be used for e-learning activities to favor the practical activities of the students and to ease learning in real contexts within the virtual training environments. Therefore, the perception of enjoyment and the intention to use AR objects positively and significantly affect to academic performance (H13- H16-H18). Not only is the production of AR objects needed, but the creation of guides that indicate how the objects should be utilized, guidelines for their use, etc., as well. It was notable that no significant relationships were found between the student's assessment of the object and their academic performance, or the degree of technological acceptance shown. The inexistence of significant relationships could be explained by the fact that the students, once they knew that the score obtained in the performance test would be used for their final grade, cognitively forgot about the perception they had about the media, and made efforts to capture the information and its meaning. It should be pointed out that the enrichment of the materials proposed in this study resulted in the students receiving information through different types of symbolic codes, and this could activate their different types of intelligence. Nevertheless, different aspects should be considered for their addition to university training, such as: training of the teachers for their use and design, and the need for universities to create centers that support the production of technological documents for teaching. In summary, the study has allowed us to broaden the scientific knowledge on the TAM model by Davis, to understand that AR objects can be used in university training, to verify that the interactions established by the subjects with the object are not influenced by the student's gender, that it is a technology that awakens a true interest in the students so that it will be used in the future, and that their addition to education should be accompanied by a usage guide. As future studies, which could serve to eliminate some of the limitations of the present work, the need to work in different directions is suggested: to perform the experiment with disciplines that are different from the ones used in the study, which were clearly related to technologies; to work with content proposals that are not so open-ended as the ones used in the study; to use other diagnostic instruments that are not the self-reports of the study subjects, but in-depth interviews, for example, and to bring the two use strategies closer to university degrees that are different from the ones used in the present study. Lastly, it should be pointed out that the study increases the knowledge needed for the incorporation of AR technology, which is so in need of the creation of theoretical models and research studies. Declarations Author contribution statement Julio Cabero-Almenara, Jos e María Fern andez-Batanero, Julio Barroso-Osuna: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper. Funding statement The work is framed within the R&D research project financed by the Ministry of Economy and Competitiveness of Spain, name: “Augmented reality to increase training. Design, production and evaluation of augmented reality programs for university training”(EDU-5746- P–Proyecto Rafodiun). Competing interest statement The authors declare no conflict of interest. Additional information No additional information is available for this paper. References Aguayo, C., Cochrane, Th., Narayan, V., 2017. Key themes in mobile learning: prospects for learner-generated learning through AR and VR. Australas. J. Educ. Technol. 33 (6), 27–40. Ajzen, I., Fishbein, M., 1980. Understanding Attitudes and Predicting Social Behavior. Prentice-Hall, Englewood Cliffs. NJ. Akman, I., Turhan, C., 2017. User acceptance of social learning systems in higher education: an application of the extended Technology Acceptance Model. Innov. Educ. Teach. Int. 54 (3), 229–237. Akçayır, M., Akçayır, G., 2017. Advantages and challenges associated with augmented reality for education: a systematic review of the literature. Educ. Res. Rev. 20, 1–11. Al-Azawei, A., Parslow, P., Lundqvist, K., 2017. Investigating the effect of learning styles in a blended e-learning system: an extension of the technology acceptance model (TAM). Australas. J. Educ. Technol. 33 (2), 1–23. Al-Emran, M., Mezhuyev, V., Kamaludin, A., 2018. Technology Acceptance Model in M- learning context: a systematic review. Comput. Educ. 125, 389–412. Alharbi, S., Drew, S., 2014. Using the technology acceptance model in understanding academics’behavioural intention to use learning management systems. Int. J. Adv. Comput. Sci. Appl. 5 (1), 143–155. Alkhattabi, M., 2017. Augmented reality as E-learning tool in primary schools’education: barriers to teachers’adoption. Int. J. Emerg. Technol. Learn. 12 (2), 91–100. Altanopoulou, P., Tselios, N., 2017. Assessing acceptance toward wiki technology in the context of higher education. Int. Rev. Res. Open Distance Learn. 18 (6), 127–148. Bandura, A., 1990. Multidimensional Scales of Perceived Self-Efficacy. University Stanford, C.A, Stanford. Barroso, J., 2018. The technological scenarios in Augmented Reality (AR): educational possibilities in university studies. Aula Abierta 47 (3), 327–333. Brigham, T., 2017. Reality check: basics of augmented, virtual, and mixed reality. Med. Ref. Serv. Q. 36 (2), 171–178. Cabero, J., García, F., coords., 2016. Realidad aumentada. Tecnología para la formaci on. Síntesis, Madrid. Cabero, J., P erez, J.L., 2018. TAM model validation adoption of augmented reality through structural equations. Estud. Sobre Educ. ESE 34, 129–153. Cabero, J., Barroso, J., Llorente, M.C., 2016a. Technology acceptance model & augmented reality: study in progress. Rev. Lasallista Invest. 13 (2), 18–26. Cabero, J., Sampedro, B., Gallego, O.M., 2016b. Assessments of the “Acceptance of virtual training technology”by university teachers attending to a virtual training course. Revista Electr onica de Tecnología Educativa 56, 31–47. Cabero, J., Barroso, J., Gallego, O., 2018a. The production of objects of learning in augmented reality by the students. The students as information prosumers. Tecnología, Ciencia y Educaci on 11, 15–45. Cabero, J., Marín, V., Sampedro, B.E., 2018b. Technology acceptance model in higher education. Rev. Invest. Educ. 36 (2), 435–453. Chang, S., Hwang, G., 2018. Impacts of an augmented reality-based flipped learning guiding approach on students’scientific project performance and perceptions. Comput. Educ. 125, 226–239. Cheng, K.-H., 2017. Reading an augmented reality book: an exploration of learners’ cognitive load, motivation, and attitudes. Australas. J. Educ. Technol. 33 (4), 53–69. J. Cabero-Almenara et al. Heliyon 5 (2019) e01597 8 Cheng, E., 2018. Choosing between the Theory of Planned Behavior (TPB) and the Technology Acceptance Model (TAM). Education Technology Research Development. Cheng, Y., Lou, S., Kuo, S., Shih, R., 2013. Investigating elementary school students' technology acceptance by applying digital game-based learning to environmental education. Australas. J. Educ. Technol. 29 (1), 96–110. Cupani, M., 2012. An alisis de Ecuaciones Estructurales: conceptos, etapas de desarrollo y un ejemplo de aplicaci on [Analysis of Structural Equations: concepts, stages of development and an example of application]. Revista Tesis 1, 186–199. Davis, F., 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 13 (3), 319–340. Tecnol ogico de Monterrey, 2017. ReporteEduTrends. Radar de Innovaci on Educativa 2017 [EduTrends Report. Educational Innovation Radar 2017]. Monterrey: Tecnol ogico de Monterrey. Díaz, R., 2016. Augmented Reality versus Virtual Reality: the Battle Is Real. Techcrunch. Díaz-Noguera, M.D., Toledo, P., Herv as, C., 2017. Augmented reality applications attitude scale (ARAAS): diagnosing the attitudes of future teachers. New Educ. Rev. 50 (4), 215–226. Etxeberria, J., Tejedor, F.J., 2005. An alisis descriptivo de datos en educaci on [Descriptive analysis of data in education]. La Muralla, S.A, Madrid. Fathali, S., Okada, T., 2018. Technology acceptance model in technology-enhanced OCLL contexts: a self-determination theory approach. Australas. J. Educ. Technol. 34 (4), 138–154. Fern andez, B., 2018. The use of objects of learning of augmented reality in the university education of primary education. Int. J. Eng. Res. Innov. 9, 90–104. Fombona, J., V azquez-Cano, Del Valle, M.E., 2018. Analysis of geolocation and augmented reality on mobile devices, social and educational proposals related to the environment and field trips. Profr. Rev. Curríc. Form. Profr. 22 (4), 197–222. Garay, U., Tejada, E., Maiz, I., 2017. Valuation of the educational objects with augmented reality: experience with postgraduates students. Píxel-Bit. Revista de Medios y Educaci on 50, 19–31. Guzm an, M.D., Duarte, A., Mojarro, A., Aguaded, I., 2019. Mobile learning in university contexts based on the unified theory of acceptance and use of technology (UTAUT). J. N. Approaches Educ. Res. 8 (1), 7–17. Han, J., Jo, M., Hyun, E., So, H., 2015. Examining young children’s perception toward augmented reality-infused dramatic play. Educ. Technol. Res. Dev. 63 (3), 455–474. He, J., King, W.R., 2008. The role of user participation in information systems development: implications from a meta-analysis. J. Manag. Inf. Syst. 25 (1), 301–331. Ho, L.H., Hung, C.L., Chen, H.C., 2013. Using theoretical models to examine the acceptance behavior of mobile phone messaging to enhance parent–teacher interaction. Comput. Educ. 61, 105–114. Hohlfeld, T., Ritzhaupt, A., Barron, A., 2013. Are gender differences in perceived and demonstrated technology literacy significant? It depends on the model. Educ. Technol. Res. Dev. 61 (4), 639–663. Hung, Y., Chen, C., Huang, S., 2017. Applying augmented reality to enhance learning: a study of different teaching materials. J. Comp. Assist. 33, 252–266. Hwang, K., Zo, H., 2016. Understanding users' continuance intention toward smartphone augmented reality applications. Inf. Dev. 32 (2), 345–367. Iba~ n ez, M., Delgado, C., 2018. Augmented reality for STEM learning: a systematic review. Comput. Educ. 123, 109–123. Jahnke, I., Kroll, M.M., 2018. Exploring students’use of online sources in small groups with an augmented reality-based activity e group dynamics negatively affect identification of authentic online information. Heliyon 4 (6), e00653, 1–2. Joo, J., Martínez, F., García, J., García, F., 2017. Augmented reality and pedestrian navigation through its implementation in m-learning and e-learning: evaluation of an educational program in Chile. Comput. Educ. 111, 1–17. Kerlinger, F., Lee, H., 2002. Investigaci on del comportamiento. M etodos de investigaci on en las ciencias sociales [Behavioral research. Research methods in the social sciences]. McGraw-Hill, M exico. Lagunes-Domínguez, A., Torres-Gastelú, C., Angulo-Armenta, J., Martínez-Olea, M., 2017. Prospectiva hacia el Aprendizaje M ovil en Estudiantes Universitarios. Formaci on universitaria 10, 1. L opez-Bonilla, L., L opez-Bonilla, J., 2017. Explaining the discrepancy in the mediating role of attitude in the TAM. Br. J. Educ. Technol. 48 (4), 940–949. Marín, V., 2017. The augmented reality in the educational sphere of student of degree in chilhood education. Case study. Píxel-Bit. Revista de Medios y Educaci on 51, 7–19. Martín-Guti errez, J., Fabiani, P., Benesova, W., Meneses, M.D., Mora, C.E., 2015. Augmented reality to promote collaborative and autonomous learning in higher education. Comput. Hum. Behav. 51, 752–761. Martínez, S., Fern andez, B., 2018. Objects of augmented reality: perceptions of Pedagogy students. Píxel-Bit. Revista de Medios y Educaci on 53, 207–220. Mohammadi, H., 2015. Investigating users’perspectives on e-learning: an integration of TAM and IS success model. Comput. Hum. Behav. 45, 359–374. Nadolny, L., 2016. Interactive print: the design of cognitive tasks in blended augmented reality and print documents. Br. J. Educ. Technol. 48 (3), 814–823. O'Dwyer, L., Bernauer, J., 2014. Quantitative Research for the Qualitative Researcher. Sage, California. Johnson, L., Adams, S., 2016. NMC Horizon Report: 2016 Higher Education Edition. The New Media Consortium, Austin. Texas. Olaoluwakotansibe, A., 2013. Interactive digital technologies’use in Southwest Nigerian universities. Educ. Technol. Res. Dev. 61 (2), 333–357. Oyeleye, O., Sanni, M., Shittu, T., 2015. An investigation of the effects of customer's educational attainment on their adoption of ebanking in Nigeria. J. Internet Bank. Commer. 20 (3), 133. Pedraza, C., Amado, O., Lasso, E., Mun evar, P., 2017. The experience of augmented reality (AR) in teacher training at the Universidad Nacional Open and Distancia UNAD Colombia. Pixel-Bit. Revista de Medios y Educaci on (51), 111–131. Pejoska-Laajola, J., Reponen, S., Virnes, M., Leinonen, T., 2017. Mobile augmented communication for remote collaboration in a physical work context. Australas. J. Educ. Technol. 33 (6), 11–25. Rauschnabel, P., He, J., Ro, Y., 2018. Antecedents to the adoption of augmented reality smart glasses: a closer look at privacy risks. J. Bus. Res. 92, 374–384. Saidin, N., Halim, N., Yahaya, N., 2015. A review of research on augmented reality in education: advantages and applications. Int. Educ. Stud. 8 (13), 1–8. Sans, A., 2004. M etodos de investigaci on de enfoque experimental. En Bisquerra, R. (cords.). Metodología de investigaci on educativa, Madrid: La Muralla. Santos, M., Wolde, A., Taketomi, T., Yamamoto, G., Rodrigo, M., Sandor, Ch., Kato, H., 2016. Augmented reality as multimedia: the case for situated vocabulary learning. Res. Pract. Technol. Enhanc. Learn. (RPTEL) 11 (4), 1–23. Schmalstieg, D., H€ ollere, T., 2016. Augmented Reality. Principles and Practice. Addison- Wesley, Boston. Tarhini, A., Hone, K., Liu, X., 2014. Measuring the moderating effect of gender and age on e-learning acceptance in England: a structural equation modeling approach for an extended technology acceptance model. J. Educ. Comput. Res. 51 (2), 163–184. Tekedere, H., G€ oker, H., 2016. Examining the effectiveness of augmented reality applications in education: a meta-analysis. Int. J. Environ. Sci. Educ. 11 (16), 9469–9481. Villani, C., Morganti, L., Carissoli, C., Gatti, E., Bonanomi, A., Cacciamani, S., Confalonieri, E., Riva, G., 2018. Students’acceptance of tablet PCs in Italian high schools: profiles and differences. Br. J. Educ. Technol. 49 (3), 533–544. Wai-tsz, R., Chi-kin, J., Chang, Ch., Zhang, Z., Chiu, A., 2014. Digital teaching portfolio in higher education: examining colleagues' perceptions to inform implementation strategies. Internet High Educ. 20, 60–68. Wang, Y., 2017. Using augmented reality to support a software editing course for college students. J. Comput. Assist. Learn. 33, 532–546. Wei, X., Weng, D., Liu, Y., Wang, Y., 2015. Teaching based on augmented reality for a technical creative design course. Comput. Educ. 81, 221–234. Y a~ nez-Luna, J.C., Arias-Oliva, M., 2018. M-learning: technological acceptance of mobile devices in online learning. Tecnología, Ciencia y Educaci on (10), 13–34. J. Cabero-Almenara et al. Heliyon 5 (2019) e01597 9