Effects of Gamified Augmented Reality in Public Spaces

Received August 30, 2019, accepted September 24, 2019, date of publication October 7, 2019, date of current version October 23, 2019. Digital Object Identifier 10.1109/ACCESS.2019.2945819 Effects of Gamified Augmented Reality in Public Spaces MARIUS NOREIKIS 1, (Member, IEEE), NINA SAVELA2, (Member, IEEE), MARKUS KAAKINEN3, (Member, IEEE), YU XIAO 1, (Member, IEEE), AND ATTE OKSANEN 2, (Member, IEEE) 1Department of Communications and Networking, Aalto University, 02150 Espoo, Finland 2Faculty of Social Sciences, Tampere University, 33100 Tampere, Finland 3Institute of Criminology and Legal Policy, University of Helsinki, 00100 Helsinki, Finland Corresponding author: Yu Xiao ([email protected]) ABSTRACT Advancements in smartphone technology have resulted in the proliferation of Augmented Reality (AR) applications and games. Researchers have acknowledged the great potential of AR applications to enhance entertainment and improve learning experiences. In this study, we examined the potential effects of gamified AR in public places. We developed ARQuiz, an AR-based quiz game, for a public exhibition space and conducted a user study with respondents via survey (N =176; 55.68% female, mean age 35.94 and SD =11.89) and face-to-face interview (N =28; 57.14% female, mean age 31.07 and SD =7.42). We analyzed the relationship between perceived application usefulness, perceived application enjoyment, perceived exhibition enjoyment, and perceived quiz enjoyment. In addition, we examined perceived sociability before and after the quiz, quiz score, and user behavior in the exhibition space. The results indicate that visitors who enjoyed playing the ARQuiz game enjoyed the exhibition more, obtained better quiz results and felt more social after visiting the exhibition. Furthermore, the ARQuiz was regarded as a possible platform for improving visitors’ learning and overall experiences in public exhibitions. Although some players expressed concerns about the privacy and intrusiveness of AR, our results indicate that a well-designed AR game may boost the overall satisfaction of an exhibition visit and increase players’ sociability. INDEX TERMS Augmented reality, museum experience, user behavior. I. INTRODUCTION Augmented reality (AR) has been an emerging technology for the last decade. With the proliferation of high-performance smartphones and smart wearables such as smart glasses, an increasing number of AR applications are developed every year. The top categories of AR applications consist of industrial, learning and entertainment applications. Augmented learning applications allow students to associate real world content with virtual teaching material to boost their results and interest in the learning process [2]. Meanwhile, AR-based games have achieved tremendous interest among players globally after the release of games such as Ingress and Pokémon GO! [26]. The games have enabled gameplay in large outdoor areas and have engaged millions of people. Recent advancements in AR technology have enabled the deployment of AR experiences in large indoor environments [25]. The environments include areas such as The associate editor coordinating the review of this manuscript and approving it for publication was Eyuphan Bulut . shopping malls, concert halls and museums, thus empowering an expansion of AR applications to places where they have never been used before. Previous studies have investigated user behavior and the effects of AR by deploying AR applications that provide additional information about exhibits within museums [15] and other cultural sites [8]. Other studies have surveyed gamified experiences within cultural heritage sites and museums [22], indicating the positive effects of games on museum exploration and informal education. The potential for gamification or AR applications to improve the experiences of visiting cultural sites has been noted in research literature; however, little has been done to study AR games and their impact on cultural sites’ visitors, with studies only limited to rudimentary AR experiences [29]. In this study, we investigated AR user experiences in indoor public exhibitions. Our aim is to examine the user experience (i.e. perceived usefulness and enjoyment of the application), to study the relationship between perceived usefulness, enjoyment of an AR application, and enjoyment 148108 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/ VOLUME 7, 2019 M. Noreikis et al.: Effects of Gamified Augmented Reality in Public Spaces of an exhibition and to investigate the relationships between AR gaming and learning experiences and the sociability of venue visitors. The research questions of this study are as follows: 1) What factors contribute to a positive AR user experience? 2) How does AR gamification influence user behavior? 3) How does the perceived experience with the AR game affect the overall visitor experience in a public exhibition? To answer these research questions, we developed and deployed an AR game to be played by visitors of a public exhibition. The game, named ARQuiz, allows people to participate in a virtual quiz while they are visiting an exhibition at Heureka, the Finnish Science Centre. Using an AR application installed on a smartphone, players must find questions scattered around the exhibition and earn points by correctly answering them. The content of the quiz questions is related to the nearby exhibits. Additionally, users may interact with other players by leaving anchored messages at various locations and by replying to the messages left by others. A user study was conducted in the ‘‘Heureka’’ Science Centre in autumn 2018 with 176 participants, including 28 face-to-face interviews. We designed survey questionnaires to be answered before and after playing the game and a semistructured interview questionnaire to further explore the users’ experiences. In addition to collecting quantitative survey and application data and qualitative follow-up interviews, we identified types of real-time data to be collected during gameplay. All three types of data were used to explore user behavior and to obtain a comprehensive view of participants’ experiences. II. BACKGROUND AND RELATED WORK In this section, we will briefly introduce AR and the different types of AR-based games. We will review the recent work on applying AR as a learning tool and as a way to enhance visitor experiences in cultural places. A. AUGMENTED REALITY AR is defined as an interactive computer system that superimposes virtual objects onto a real world view, seen through a head-up display or a mobile device, in real time [23]. AR applications should seamlessly render artificial objects so that they appear as a part of the real world to a user, even when an AR device is being moved. Advancements in technology of hand-held devices (i.e., tablets and smartphones) have resulted in the development of numerous AR games played by millions of people worldwide. AR games can be divided into two categories: locationindependent and location-aware. Location-independent games can be played at an arbitrary location where the con-tent of the game does not depend on a spatial context [12], [18]. Location-aware games, on the other hand, make use of a known location of a player in their gameplay. Examples of location-based AR games include Ingress and Pokémon GO. AR games are typically marker-based or marker-less. In a marker-based AR game, a user must point a mobile device to a particular visual marker (picture) or a quick response (QR) code that will act as an anchor for the augmented objects and where the gameplay will take place [4]. Regarding the marker-less approach, the locations of augmented game objects are determined by using an accurate global localization system, e.g., a Global Navigation Satellite System (GNSS) for outdoors or a computer vision-based positioning system for indoors [10]. B. AR AS A LEARNING TOOL The applicability of AR has been extensively studied in various educational settings [2], [3]. Improved learning, increased motivation, better interaction and improved collaboration are among the key advantages of AR, as discovered by previous works. Although AR has mostly been used for providing additional information on a certain topic and AR lab experiments, gamified AR learning experiences have also been studied [17], [27]. Fonseca et al. [11] examined the effects of using a collaborative AR tool to visualize building information models during lectures in a department of architecture. The researchers analyzed the relationship between usability of the AR tool, class participation and improvements in academic performance of architecture students. A high correlation between the aforementioned factors was highlighted by the researchers. Most previous works also discuss potential challenges of using AR. The identified drawbacks of applying AR as a learning tool include challenges to implementing technology [6], intrusiveness of AR, and a risk of users paying too much attention to the augmented information rather than to the surrounding environment [28], [30]. C. AR IN CULTURAL SPACES Previous research has recognized the potential of AR applications to improve the interaction in and engagement with cultural sites [8] and museums [9], [15]. Díaz et al. [9] investigated how social interaction is affected by user-generated AR content within a museum exhibition. Visitors could record videos related to a particular exhibit, share the videos within a close proximity to the exhibit and watch videos left by the other visitors. These videos allowed visitors to maintain a physical connection to the actual exhibits. The researchers highlighted the potential of these videos to enrich the museum experiences for visitors. Jung et al. [15] analyzed the effects of Virtual Reality (VR) and AR on enhancing the overall museum visitor experience and improving the likelihood of revisiting the museum. Their results indicate strong relationships between social presence and factors of experience economy [24], such as enjoyment, escape and entertainment. Attitudes and behavioral intentions towards cultural sites were also analyzed by Chung et al. [8]. By analyzing VOLUME 7, 2019 148109 M. Noreikis et al.: Effects of Gamified Augmented Reality in Public Spaces perceived advantage, aesthetic experience, perceived enjoyment and satisfaction of AR, researchers studied the extra value that AR can bring to the perceptions of the visitors. Their results indicate that satisfaction with AR leads to a positive perception of a site and an intention to revisit the place. D. GAMIFICATION IN CULTURAL SPACES In recent years, several researchers have focused on providing gamified experiences and analyzing their effects within cultural heritage sites, such as historical places and museums [22]. However, little research has been conducted on AR-based games in such environments in order to study their effects on exhibition enjoyment and improved learning. Rubino et al. [29] developed a location-based mobile game targeted at teenagers that combines entertainment with educational objectives. The aim of the game was to motivate teenagers to explore the museum in Turin, Italy and broaden their understanding of cultural aspects in an informal style. The researchers found a positive correlation of gamification towards entertainment within the venue, especially among younger participants, and observed effective communication of cultural information to adult audience. They concluded that finding and scanning visual markers within the venue fostered engagement in exploring the exhibition; however, in their study, the AR interaction was limited only to augmenting content onto panoramic photographs. III. GAME DESIGN The aim of our research is to investigate the effects of augmented reality games on user experiences while they are visiting public exhibitions. Thus, we have developed an AR-based quiz game for mobile phones to be played at Heureka, the Finnish Science Centre. A. ARQUIZ GAME We developed a game called ARQuiz that runs on smart phones. ARQuiz enables guests of the Science Centre to participate in a virtual quiz while they are visiting the main exhibition. The goal of the game is to find and correctly answer questions about biotechnology that are related to the exhibits within the exhibition area. By looking around through a phone’s screen, a player can see hints in AR of where unanswered quiz questions are located. After approaching a hint, a related quiz question is automatically shown in place of the hint (see Figure 1). There were 15 questions located in 8 different places within the exhibition. Players answer the questions by tapping on one of the given answers from multi-choice options. Each question contains 4 possible answers (see Figure 1b), out of which one answer is correct. There may be several questions at the same position, in which case a player answers them one-by-one. Once all the questions at that position are answered, the hint for that position disappears. A user may answer questions in any particular order and can track the current score throughout the quiz. The content of ARQuiz questions is always FIGURE 1. Screenshots of ARQuiz gameplay. related to nearby exhibits, and investigating the exhibits may help in answering the quiz questions. In this way, the game makes users wander around the exhibits and to solve the quiz questions while being close to the exhibits mentioned in the questions. B. SOCIAL ASPECTS OF ARQUIZ Solving the quiz questions is typically a single user experience. However, we aim to gain insight into the social activity of ARQuiz players by allowing them to leave text messages that are later shown as augmented content within the exhibition (see Figure 1b). The messages are seen by all other players at the same locations where they were created. Players can read and reply to them. In fact, in ARQuiz, users may leave any kind of comments in the AR space by simply choosing a point on the ground where the message will be anchored and by entering a desired text. Other users can immediately see a message symbol in the exact same place where the first user left the comment. By tapping on the symbol, a user can see the message. It is also possible to reply to the message, initiating a live or offline conversation. Since messages are created at any user-chosen physical place, visitors may choose to comment on nearby quiz questions, exhibits of interest, or arbitrary points within the exhibition. Allowing anyone to comment may lead to inappropriate or harmful comments being posted to the public. However, we allowed showing the messages to other visitors immediately, later observing that no inappropriate texts were inserted by the players. Because of the accumulative nature of the commenting functionality, the users of the first test days saw less comments in the space than the users of the last test days. 148110 VOLUME 7, 2019 M. Noreikis et al.: Effects of Gamified Augmented Reality in Public Spaces FIGURE 2. AR system architecture and data flow between the mobile application (in the center) and server components. To bootstrap messaging and to show example comments for the players to read, we created 2 entries ourselves and placed them within the exhibition. The starting messages were as follows: ‘‘When you stand in a certain point of the room, you can see the Big Dipper’’ ‘‘You can leave comments and reply to comments left by others!’’ C. DESIGN AND IMPLEMENTATION ARQuiz combines concepts of location-based gaming and AR. However, there are a few important technical challenges that must be solved to enable the previously described functionality of the ARQuiz at the Finnish Science Centre. First, the Science Centre is an indoor environment, where GNSS is not available, and installing additional radio devices for mobile phone positioning, such as Bluetooth beacons [19], is not allowed. Additionally, the exhibits are rather small, with sizes ranging from several meters to submeter radius objects. Therefore, to place the AR questions near the exhibits, a reliable submeter accuracy positioning system must be in place. Second, the mobile application must cope with real-time data. It must display text messages created by other users in real time and must collect application usage data that are obtained by recording real-time events, such as discovering and answering quiz questions, interacting with messages and answering survey questions. Finally, the application must be fun and easy to use and, most importantly, suitable for visitors with diverse backgrounds and of different age groups. To address the technical challenges, we designed and implemented a system architecture presented in Figure 2. The main parts of the system include an image-based localization module, databases for quiz questions, user messages, and telemetry, an AR-enabled mobile application. 1) IMAGE-BASED LOCALIZATION Before starting the quiz, a user must be located within premises with the help of an image-based localization module. To do so, a user opens the ARQuiz application on a smartphone and aims the camera towards the premises. A photograph is automatically captured and sent to the back-end localization server for processing. The server then supplies the phone with an accurate 3D position and facing direction within the premises. Once the location is obtained, the application can download ARQuiz questions and a list of messages and place them in their designated positions within the AR space. To implement a reliable indoor localization system, we developed a positioning system based on ideas presented in a previous study [10], which proposed a localization method that can locate a user within a meter level accuracy 90% of the time. First, using smartphones, we captured videos of the venue that covered all the places within the premises. The video data are used for venue modeling. Then, we collected 4 sets of photographs at known locations within the venue, which are used for a process called ‘‘geo-calibration’’. We then processed the videos and the photos with Structurefrom-Motion (SfM) algorithms [31] by using an open source SfM pipeline OpenMVG1. The process created a 3D point cloud of the area, and by using the geo-calibration photos, we scaled the model to a real world coordinates system2. Each 3D point within the reconstructed point cloud contains a set of unique image features [20]. To locate a user, we first extract image features from a photograph captured with the mobile application. We then use the previously generated point cloud and the captured photograph with the extracted features as an input for the SfM algorithm. The algorithm calculates the 3D position and facing direction (also known as a pose) of the photograph. Since the point cloud was scaled to real-world coordinates, the obtained pose correctly represents the current location of the mobile device within the premises. There are a couple of limitations of such a localization approach: the delay of obtaining the location and the location accuracy. Feature extraction and SfM are compute intensive algorithms; therefore, there is a delay of several seconds to compute a phone’s location from an image. For that reason, between two subsequent image-based localization requests, we use ARCore3-based position tracking on a mobile device to update its 3D position and the alignment between real and artificial worlds in real time. Some estimated user locations may have errors that cause inaccurate placement of AR objects. Therefore, ARQuiz places hints and questions close to the museum exhibits rather than trying to attach the questions to the exhibits. In this way, an exhibit that was referenced in a quiz question can always be found near 1https://github.com/openMVG/openMVG 2https://openmvg.readthedocs.io/en/latest/software/ui/SfM/control_ points_registration/GCP/ 3ARCore overview - Goggle Developers https://developers. google.com/ar/discover/ VOLUME 7, 2019 148111 M. Noreikis et al.: Effects of Gamified Augmented Reality in Public Spaces the question; thus, the localization error does not influence the intended user experience. 2) REAL-TIME DATA TRANSFER We developed a client-server architecture to enable real-time data transfers and utilized the back-end server to also collect ARQuiz scores and application usage data from the phones. The client server approach allows multiple users to interact with the AR space and create shared content (messages); however, it imposes a requirement of persistent internet connection. In the case of the Science Centre, the venue provides free and fast Wi-Fi networking for their visitors. The collected data were stored in a MongoDB4database for further analysis. 3) APPLICATION USER INTERFACE During the game, the user observes and interacts with the augmented content through the phone’s touchscreen. Interactions with AR objects are performed by tapping on the objects through the phone screen, leaving the UI simple with only a score indicator, and using buttons to place messages and to quit the game (see Figure 1). The AR questions are displayed as the user walks close to them. All interactions with game objects and other real-time data (telemetry) collected by the application are sent to the usage database. We used Unity5to develop nice looking, animated 3D artifacts that seamlessly blend in with the real world view. We provided simple textual and visual instructions to make it easier for players to start and control the game. IV. USER STUDY DESIGN To study user experience, quiz and exhibition experience, and sociability and behavior of users of the AR application, we designed and carried out a user study at Heureka, the Finnish Science Centre. The study can be divided into two parts: 1) quantitative data collection, when users are asked to fill out surveys, immediately before and after using the ARQuiz, and 2) qualitative data collection that we carried out using face-to-face interviews after participants finished the quiz and the surveys. As part of the quantitative data, we also collected application usage data that included recorded walking traces within the exhibition, read and published messages, and quiz scores. Throughout this research, we refer to 1) Survey and Application data and 2) Interview data. Since weekend days were the busiest days for the venue with respect to the number of visitors, we collected data over the course of five Saturdays. A. PROCEDURE The ARQuiz game was set up in one of the exhibition halls of the Science Centre. We asked every visitor arriving at the exhibition whether they would agree to participate in the research study. Participation was completely voluntary. 4Open Source Document Database https://www.mongodb.com 5Unity game engine https://unity.com/ Participants received chocolates after completing the survey. They could choose to use their personal phones by installing ARQuiz on their phones or borrowing one of the phones with a preinstalled application. Before completing a survey, each participant was given a short introduction about the study and told that they would be playing an AR-based quiz game and answer survey questions before and after the quiz. When players start ARQuiz, they are presented with an informed consent text that they must agree to in order to participate in the study. Upon agreeing, the participant is presented with the first part of the survey. Once the survey is completed, the ARQuiz game starts. Players are free to explore the exhibition and play the game at the same time. In general, there was no time limit, and participation was only limited to the working hours of the Science Centre. We stopped recruiting participants at least a half an hour before the closing time of Heureka to ensure sufficient testing time for the participants. After a player finished the quiz, either by finding and answering all the hidden questions or by quitting the application, the player was presented with a last set of survey questions. Upon answering these survey questions, the participant returned the phone (if borrowed). After the experiment, we noted some of the comments made by participants when returning the borrowed smartphones. At that point, we would also ask if a participant was willing to participate in a follow-up interview. During the experiment, we did not observe individual players and did not interact with them during the test unless there was a technical issue (e.g., a malfunctioning phone). However, since an automatic application data collection was used, we could later analyze the data to reveal the walking paths of the players and discover that at most 9 players were playing the game at the same time. However, we did not observe how or if the concurrent players interacted with each other. B. PARTICIPANTS In total, 231 participants completed the survey and played ARQuiz. However, some observations were eliminated due to technical problems, such as application malfunction, loss of internet connectivity, or due to participants quitting the study before starting the ARQuiz game. A common reason for not completing the study was, e.g., when parents were not able to look after their children and play ARQuiz at the same time. Consequently, in this research, we analyzed results from 176 participants. Of these, 98 were female (55.68%) and the mean age was 35.94 (SD =11.89; range =15 −73). Before the test, 21.59% of the participants had previous experience with AR-based games. Participants who completed the survey and played ARQuiz were asked to participate in a 5-10 minute follow-up interview. Of those who agreed to participate (N =28), 16 were female (57.14%), and the mean age was 31.07 years (SD = 7.42; Range =16 −43). Each participant was interviewed in a face-to-face format with the conversation recorded with an audio recorder and notes made on paper. 148112 VOLUME 7, 2019 M. Noreikis et al.: Effects of Gamified Augmented Reality in Public Spaces TABLE 1. Descriptive statistics of our study variables (N =176): Frequencies (n), percent (%), range (Min, Max), mean (Mean), standard deviation (SD), number of survey questions in a variable (No. of items), and cronbach’s alpha [5] (α) for sum variable indicators. C. MEASURES AND INSTRUMENTS In this research, we identified 14 study variables to be measured and analyzed with survey and application data and 9 questions to be examined with interview data. We executed the interviews using a semistructured method [1] to allow new ideas and other thoughts of the respondents to be brought up in the analysis. 1) SURVEY AND APPLICATION DATA The study variables measured in this research are presented in Table 1. The first four variables are the main variables studied, and the others form a set of background variables. The main dependent variables of interest are perceived enjoyment of the application usage, perceived usefulness of the application, perceived enjoyment of the quiz, and perceived enjoyment of the exhibition. Each of these 4 variables was measured by asking the participants to rate a particular statement on a scale of 1 (Strongly disagree) to 7 (Strongly agree). The statements were as follows: 1) The application I tested was fun to use 2) The application I tested helped me to become familiar with the exhibition 3) I liked the quiz about the exhibition 4) I enjoyed the exhibition For sociodemographic background variables, we asked participants to provide their age and gender (male or female). Other background variables included mobile use activity, mobile applications self-efficacy, and extroversion. Mobile use activity was measured by asking participants whether they used mobile phones for the following activities: making phone calls, sending messages, using social media, watching videos, playing games, and playing games that use AR. We created a new sum variable from the answers to these dummy variables to measure the activity of mobile phone use for different functions. In the new variable, a respondent who uses a mobile phone for all activities would obtain a value of 6, and a respondent who does not use a phone for any of the activities would obtain a value of 0. To measure mobile app self-efficacy, we used a single item measure [13] adapted to a mobile application context by asking respondents to rate on a scale from 1 (Strongly disagree) to 7 (Strongly agree) a statement of ‘‘I am confident that I will learn to use new smartphone applications’’. Extroversion was measured using three items (α=0.90) taken from the 15-item Big Five Inventory [16]. Participants had to evaluate the following statements on a scale from 1 (Strongly disagree) to 7 (Strongly agree): •I am talkative •I am usually quiet and reserved •I am sociable and outgoing In addition, we examined the relationship of our main study variables to players’ perceived sociability before and after the quiz, quiz score, length of walking distance, and comments read, made, and replied to by the players. Sociability was measured before (α=0.86) and after (α=0.91) the game with the same 4-item instrument [32], including the following statements: •Right now I feel social •Right now I feel unsocial •Right now I feel talkative •Right now I feel quiet With the utilized pretest-posttest design, we can analyze the relationship between AR use and sociability, both before and after the exhibition experience. A variable representing the quiz score includes the total score participants obtained from the quiz, as calculated by the Science Centre’s designed scoring system. A user gets 3 points if the correct answer to the question was ‘‘All of the above’’, 1 if only a single VOLUME 7, 2019 148113 M. Noreikis et al.: Effects of Gamified Augmented Reality in Public Spaces TABLE 2. Pearson correlation coefficients of our study variables (N=176). Statistically significant correlations (p< .05) highlighted in bold. choice was selected when ‘‘All of the above’’ was the correct answer, 1 point for every other correct answer and 0 points for incorrect ones. Walking length represents the total physical distance in meters that a player traveled while playing ARQuiz. We used points obtained from the localization system and position tracking modules, and after connecting subsequent points with straight lines, we measured the distance of the whole path. The application also calculated the number of comments read, made, and replied to by every player. 2) INTERVIEW DATA We used a semistructured interview method to examine user experiences in more detail. We designed an interview with 9 questions. The answers to the questions provided the necessary content to obtain a more thorough analysis of user experiences after they used the ARQuiz application. D. ANALYSIS We used two different methods for the analysis of the collected survey and interview data. 1) SURVEY AND APPLICATION DATA We used descriptive statistical methods to analyze the survey data and the application usage data. The measured variables are presented in Table 1. In addition to descriptive information of the variables, we calculated Pearson correlation coefficients (see Table 2). Statistically significant correlations (p < .05) are highlighted in the table. Analysis was performed using Stata 12.0 MP statistical software6. The analysis was performed by three authors of the research team. Heat map analysis was performed by the first author. 2) INTERVIEW DATA We used a content analysis method to study interview data to examine the experiences of using augmented reality applications, such as ARQuiz. A summative approach of content analysis was used to summarize the information on the interview data [14]. The analysis of the interview data was conducted by the second author. V. RESULTS In this section, we will present the important findings obtained during data analysis. First, we will present the results of the Pearson correlation analysis for the survey data and the application usage data. Then, we will describe the findings from the analysis of the interview data. A. RESULTS FROM SURVEY AND APPLICATION USAGE DATA Table 2presents the correlation analysis of our study variables. According to the analysis, the enjoyment of the exhibition was highly correlated with exhibition quiz satisfaction (r =.33), perceived usefulness of the application (r =.32), 6https://www.stata.com/stata12/ 148114 VOLUME 7, 2019 M. Noreikis et al.: Effects of Gamified Augmented Reality in Public Spaces FIGURE 3. Heat map of the players’ positions who played ARQuiz, overlayed onto a top-down view of the museum. A higher density of player presence was recorded close to the locations of hidden questions. Players started the game close to the exhibition entrance. and application use enjoyment (r =.29). Furthermore, satisfaction with the exhibition quiz was strongly correlated with perceived usefulness of the application (r =.51) and application use enjoyment (r =.58). The analysis also shows that those participants who enjoyed using the application also perceived it as more useful (r =.63). The results suggest that gamified AR applications, such as ARQuiz, have a considerable potential to increase visitor satisfaction with the exhibition. Posttest sociability was correlated with exhibition quiz satisfaction (r =.37), perceived usefulness of the application (r =.35), and application use enjoyment (r =.35). Those participants who enjoyed the exhibition rated themselves as more sociable before (r =.22) and after (r =.37) the quiz, which indicates that ARQuiz has the potential to improve the sociability of visitors after the exhibition visit. It should be noted that satisfaction with the application was not correlated with sociability before the exhibition. As additional findings, the correlation analysis indicates that the highest scores in the exhibition quiz were achieved by those who liked the quiz (r =.22), found the application useful (r =.16), and enjoyed the application use (r =.42). Higher application use enjoyment was reported among those participants with higher confidence in their mobile application use (r =.16). B. RESULTS FROM SPATIAL MOVEMENT ANALYSIS As shown in Table 2, obtaining a high quiz score correlates with the distance travelled in the exhibition (r =.33). This comes from the fact that one has to explore the exhibition area in order to find all the questions. The size of the exhibition, where the ARQuiz was deployed, accounts for roughly 800m2. Figure 3shows the heat map of all recorded player positions during the study. Intuitively, areas where the hidden questions were located were visited more often than the other areas. The heat map also reveals that players covered the entire area of the exhibition while playing the game. This indicates an exploration factor when visitors were keen on exploring all places within the exhibition in search for the quiz questions. However, since there were no data available for mobility patterns of users who did not play the ARQuiz, it is not possible to directly compare heat maps of the positions of the two groups. C. INTERVIEW RESULTS The results of the content analysis suggest that even though the complexity of the AR user interface decreases the positive experiences for some users, for many players, the AR application offers enjoyment and excitement. Nine interviewees (9/28 32%) had not used AR applications before, but 20 respondents (20/28 71%) believed that AR applications were suitable for them. Most interview respondents (20/28, 71%) thought the exhibition experience differed from previous experiences, and some (8/28, 29%) respondents said that the application was a positive addition. When asked about the experiences of using the application, half of the interviewees considered the application to be addictive or immersive and fun to use. One of the participants said: ‘‘It was fun, and I immediately thought that this is contemporary. Precisely, this augmented reality stuff. It was fun to search for the [question] marks from there and answer the questions.’’ Some respondents (6/28, 21%) reported neutral attitudes towards the application, including appraisals of it being new and modern, different and unexpected, or puzzling before learning how the application worked. The rest of the respondents (8/28, 29%) described experiences of confusion and uncertainty or difficulties in using the application. As noted by one of the interviewees: ‘‘A bit difficult at first...I probably accidently pushed the comments with my thumb, I wasn’t quite sure how to open the question marks. But then I understood that you need to walk towards them.’’ When asked about improvements for the application experiences, half of the interviewees responded that they wanted the functions of the application to be more intertwined with the surrounding environment. Similar comments were made on multiple occasions when answering other questions. The feelings of conflict between the physical environment and the reality seen through the application were also visible in the answers to the questions about the influence of the application on the participants’ behavior in the exhibition space. Approximately half of the respondents (15/28, 54%) reported focusing on the task at hand and ignoring the environment and other people. One participant noted the following: ‘‘I did not notice other people. I only looked at my phone. I did not see or pay any attention to the exhibition itself. I only searched for the question marks.’’ Only four interviewees (14%) put the application on hold and chose to explore the exhibition as they would without the application. The rest did not specify their course of action. VOLUME 7, 2019 148115 M. Noreikis et al.: Effects of Gamified Augmented Reality in Public Spaces In addition, some respondents (4/28, 14%) reported privacy concerns unprompted, describing feelings of anxiety about holding a phone near other people who might think they are videotaping them. More than half of the interviewees (18/28, 64%) described positive learning experiences. When asked about the influence of AR on their learning, most of the respondents actually commented on AR’s enjoyment value on learning in general. ‘‘‘I believe that augmented reality would be better compared to long texts, and is maybe especially suitable for the young generation.’’ Twelve interviewees (12/28 43%) said they believed that AR has the potential to improve the learning experience by being a more enjoyable way to learn, and one of them explicitly said that while an enjoyable way of learning, it would not necessarily enhance learning. Seven participants (7/28 25%) responded with negative attitudes about learning through the AR application, and the rest of the interviewees did not know how they felt about AR and learning. Instead of learning itself, the answers about the motivation to finish the quiz suggest that the value of the AR-based quiz relates to gamification. More than half of the interviewees (17/28, 61%) believed that augmented reality motivated them to finish the quiz and reported feelings of ambition to find and answer all the questions. As one interviewee noted: ‘‘I finished it. Yes, there comes this feeling that ‘‘let’s complete it to the end!’’. It wouldn’t occur to me to drop out.’’ Descriptions of execution appeared in the answers more often than descriptions of content. Only five interviewees described learning motivations related to the actual content of the quiz. This indicates the potential of AR applications to further motivate learning experiences in public exhibitions. VI. DISCUSSION This user study revealed that visitors’ enjoyment of the exhibition is correlated with how much they enjoyed the quiz and AR application and how useful they thought it was. This implies that well-made AR applications have the potential to increase the satisfaction of public place visits. In the same manner, the results suggest that satisfaction with the content of an AR application, in this case, the satisfaction of the content of the Science Centre quiz, increases when people find the application useful and enjoyable. The results also indicated that those who are more satisfied with the application are more likely to have higher quiz scores and increased sociability after visiting the exhibition. Further analysis of the interview data revealed positive attitudes towards the potential of AR, with more than two-thirds of the respondents seeing themselves using AR applications and half of the respondents noting that AR games are addictive and fun to use. In fact, due to the aspect of AR gamification, almost two-thirds of ARQuiz players were motivated to finish the quiz driven by the curiosity to discover all the hidden quiz questions. Several respondents described how AR helped to create a novel museum experience, and some stated that it was a positive addition to the visit experience. These statements agree with Diaz et al. [9] and Chung et al. [8] findings that AR can induce positive emotions during a museum visit and may encourage guests to revisit the exhibition in the future. While previous studies [7], [21] asserted that AR applications can improve students’ learning experiences in schools, our findings indicate that this may also be the case in public spaces, such as museums. Interview analysis revealed that 64% of the respondents mentioned positive learning experiences, and almost half of the respondents agreed that AR has the potential to improve learning experiences. However, one-fourth of the interviewees were skeptical about that. In addition to AR functionality, the quiz as a format of learning was also given as a reason for a positive learning experience. A. CHALLENGES OF USING AR Several important challenges of AR were pointed out during the interviews, including excessive engagement in the augmented world, difficulties in UI, and concerns about privacy violation. Earlier studies indicate concerns related to too much distraction from the real world while using AR applications [28], [30]. In our study, we observe a similar challenge caused by players constantly focusing on the smartphone display rather than on the real world. Associating the content of ARQuiz questions with real world exhibits may help reduce the time spent interacting with only AR; however, novel AR experiences must be designed with extreme care to effectively ration the time between AR and the real world. The analysis of the ARQuiz usage data and transcriptions from the interview responses indicated difficulties in understanding the application UI and problematic interactions with augmented objects. Difficulty in using the application may have notably influenced several players’ experiences. Some AR users did not finish the quiz, while several visitors did not even start the quiz. Most of the visitors who did not start the quiz experienced problems with the AR application, mostly due to a misunderstanding of how to use the application. This indicates that the user interface must be made simpler and more self-explanatory, which motivates us to improve the AR user interface and user experience in future work. Usability problems also hinted that mobile AR games are still a maturing technology. Several respondents noted having troubles with image-based localization when the localized position of a person would be several meters away from their real world position, making some of the virtual questions appear in incorrect places. The most likely cause of such system behavior is problems with ARCore-based position tracking if a user accidentally covers a large portion of the device’s camera. While the camera is covered with neither ARCore tracking nor image-based localization, if a device is moved, the misalignment between the real and artificial worlds increases before the camera is unblocked, and a new 148116 VOLUME 7, 2019