Gaming as a Medium for the Expression of Citizens' Views on Environmental Dilemmas

- 93 - Effects of Culture on Open Science and Artificial Intelligence in Education Please cite this article as: D. Griffiths, J. Ower, P. Hollins, A. Garg. Gaming as a Medium for the Expression of Citizens' Views on Environmental Dilemmas, International Journal of Interactive Multimedia and Artificial Intelligence, vol. 9, no. 2, pp. 93-103, 2025, http://dx.doi.org/10.9781/ijimai.2025.02.006 Keywords Citizen Engagement, Games, Gamification, Policymaker, Survey. Abstract The decline of traditional media and channels of communication has led to policymakers experiencing difficulty in understanding public sentiment. A case study was conducted to explore how games-based activities can be used to provide a link between citizens and policy makers. A system developed by PlanetPlay, and extended in the GREAT project, was used to embed a survey in the game SMITE. The intervention and survey questions were designed in collaboration with the United Nations Development Programme (UNDP) and the Hi-Rez game studio. The effectiveness of the infrastructure and the collaborative approach were demonstrated. The results revealed some significant differences in views on climate change between different age groups, genders, and education level. However, the data was heavily skewed towards males in the 18-35 age group, and to respondents in the United States, which limited the generalizability of the findings. It was concluded that in-game placement in collaboration with games studios is more effective than paid placement, and that a wider variety of games is needed to ensure that a study has an adequate range of respondent profiles. Finally, reflections are offered on the possible role of artificial intelligence in gathering such data. DOI: 10.9781/ijimai.2025.02.006 Gaming as a Medium for the Expression of Citizens' Views on Environmental Dilemmas Dai Griffiths1*, Jude Ower2, Paul Hollins3, Anchal Garg3 1 Research Institute for Innovation & Technology in Education (UNIR iTED), Universidad Internacional de La Rioja, Logroño, La Rioja (Spain) 2 PlanetPlay (UK) 3 The University of Bolton (UK) * Corresponding author: [email protected] Received 8 July 2024 | Accepted 18 December 2024 | Published 10 February 2025 I. Introduction THE research reported here was carried out in the context of two interconnected cultural trends. The first, the increasing creative range and reach of the games industry has been generally welcomed. In contrast the second trend, the decline of traditional media and channels of communication has given widespread cause for concern, particularly as regards channels of communication linking citizens with civic authorities and policymakers. There may or may not be a degree of causal relationship between the two processes, but that is not our concern here. Rather, we identify and explore an opportunity to make use of the former in addressing some of the concerns raised by the latter. Before describing our study, we briefly introduce these two trends. A. The Decline of Traditional Communication Channels There is extensive evidence to support the statement by ContrerasEspinosa and Blanco [1] that “many democracies are facing, as a growing problem, a breach of communication between citizens and their political representatives”. Since the 1990s, the proportion of citizens who are “dissatisfied” with democracy in their countries has risen by almost 10 percentage points globally, and the deterioration has been particularly marked in high-income, “consolidated” democracies, where the proportion has risen to a third to half of all citizens [2]. Similarly, the United Nations [3] considers that distrust of news sources and scientists is at an all-time low. Dissatisfaction and mistrust correlate with skepticism, for example concerning vaccines and covid19 [4] [5]. As Morelli has argued, these lower levels of trust in markets, governments, and political institutions have led to a crisis among traditional parties, and to an associated rise of populist rightwing parties [6]. The causes of this change are complex and contested, but it is relevant that there has been a marked decadence of institutions which have traditionally served to channel citizens’ views to policymakers. The International Labor Organization has reported that the past thirty or forty years have been marked by the replacement of older unionized workers with less unionized but better educated younger workers [7]. Similarly, and despite high-profile fundamentalist exceptions, “most high-income countries show declining emphasis on religion” [8] (p.79, emphasis in the original). Perhaps most dramatic is the world-wide transformation of the news media landscape. For example, in the United States newspaper circulation has fallen by about two thirds since 1990 International Journal of Interactive Multimedia and Artificial Intelligence, Vol. 9, Nº2 - 94 - [9] and, correspondingly, 86% of US adults often or sometimes get news from a smartphone, computer or tablet, with 56% who doing so often [10]. This transformation no doubt reflects the greater convenience of digital distribution but is also impacted by two additional factors. Firstly, some politicians have undermined news media which they see as opposed to their interests, most notably Donald Trump, whose “antagonistic tweets are a systematic approach to delegitimize the news as an institution” [11]. Secondly, as Kreps et al. [12] observe, generative AI is flooding the media with enormous volumes of content, which is usually of little value and may constitute misinformation. They argue that “this undermines efforts to understand constituent sentiment, threatening the quality of democratic representation”. It is precisely these difficulties in understanding public sentiment, flowing from the range of factors that we have mentioned, which motivate the research reported here. B. The Emergence of Game Culture In parallel with the decline of traditional news media, there has been a radical transformation in entertainment media, with the digital games industry having become a larger economic sector than either music or film, and which has been estimated to have generated $227 billion worldwide in 2023 [13]. It can hardly be doubted that this major cultural change has had a major impact on society, but there is no consensus on the consequences, nor even if these are positive or negative. According to Adrienne Shaw [14], the term ‘video game culture’ became common during the first decade of the 21st Century. However, she also warns that games players are highly varied in age, genders, sexualities, races, religions and nationalities, and that “Not all of these types of play and players can be encompassed in a study of an isolated gamer community” [14]. What, then constitutes the game (or games, or gaming) culture that has been a focus of so much discussion over the past fifteen years? Participants in games culture cannot readily be identified, as they do not usually wear distinctive clothing, despite the popularity of ‘cosplay’ on special occasions, as described by Lunning [15]. Mia Consalvo argues that membership of games culture is not only, or even mainly, concerned with playing games, it is marked by being knowledgeable about games, passing that information on, and having opinions about games [16]. Games culture has often been critiqued for being male dominated, aggressive and sexist, and, for example, Vergel et al. conclude that cybersexism and its manifestations “are a harsh reality for women who want to play digital and online games” [17]. Be this as it may, it is noteworthy that many gamers are women (though it is not clear if this undermines the conclusion of Vergel et al., or if it makes it still more concerning). In Europe 2022 46.7% of video game and console players were women, while for smartphone and tablet games they were in the majority (51%) [18]. There has also been widespread concern that aggressive video games are fomenting violence in society, and especially among people, but there is continuing doubt about the reality of this impact. For example, in 2020 Drummond et al. reported that “meta-analytic studies now routinely find that the long-term impacts of violent games on youth aggression are near zero” [19], while in a meta-analysis of 2021 Burkhardt and Lenhard identified “a significant and meaningful positive effect of VVG on subsequent physically aggressive behavior” [20]. From a more practical perspective, gaming, together with social media, web browsing, occupies a large amount of young people’s time, with one study estimating U.S. teens’ screen time at 8.39 hours per day, excluding educational activities [21]. There is concern that this highly competitive attention economy [22] may result in ‘attentional serfdom’ [23], and a paralysis of political participation in the face of the dilemmas which face society. II. Motivation, Objectives and Research Questions A. The Motivation for This Study As Kroger et al. argue [24], the prevailing business model of the games industry is increasingly dependent on harvesting and making use of personal data for competitive advantage, often without players being aware of what data is being collected or for what purposes. The rapid introduction of artificial intelligence (AI) tools serves to accelerate this trend and make it still more opaque. Without entering into the rights and wrongs of this practice, we seek to show that another approach to data gathering through games is possible, with players choosing to provide data relating to issues which are relevant to them. Moreover, we position this data-gathering in terms of open science, proposing a methodology within which the participation of stakeholders can be maximized, and data can be made widely available for social benefit, in our case the expression of citizens’ views on policy dilemmas. The strength of games culture has long been seen as an opportunity to communicate ideas, promote attitudinal change, and enhance educational processes. However, there is a mismatch between the promise of these approaches and the disappointing scale of practical achievements. Moreover, while the Council of Europe has identified “great potential of video games in promoting positive cultural and social changes” [25], social and educational applications of games usually involve reception by the player of ideas or knowledge, and do not engage players in building an inquiry or making a contribution to society. Within this context, the work reported here addresses a gap in the research literature: methods are not described whereby participation in gaming culture can enable citizens to express their attitudes and preferences, and so address the problem of understanding constituent sentiment, identified by Kreps et al., above. The case study described here leans heavily on the infrastructure and business processes of PlanetPlay (originally developed by Playmob, who were acquired by PlanetPlay in 2024). An additional motivation for this study is to examine the potential of these tools in the context of an academic case study, for the first time. B. Objectives and Research Questions In line with this motivation, our overarching research objective was to understand how games-based activities can be used to provide a link between citizens and policy makers. To this end, the practical objective of the work carried out in the case study was to provide insight to the participating policy stakeholders about citizens’ views on a range of climate goals. In addressing our research objective, we sought to answer three of the wider research questions which have been defined for the GREAT project within which this research was embedded: RQ1 Which games-based activities can be used to elicit, represent and communicate citizens’ views on policy dilemmas? RQ2 How effective are games-based activities in eliciting, representing and communicating citizens’ views on policy dilemmas? RQ3 How efficient is the use of games-based activities in eliciting, representing and communicating citizens’ views on policy dilemmas? III. Related Work Games have been used for many years as an educational resource, and a substantial body of research has investigated its impact on different fields, as summarized, for example the systematic reviews offered by Yu et al. [26] for online education, Guan et al. [27] for primary education, and Vlachopoulos and Makri [28] for schools. More - 95 - Effects of Culture on Open Science and Artificial Intelligence in Education specifically, games have long been used to enhance awareness and understanding of environmental and other social issues. For example a systematic review by Janakiraman et al., [29] argues that games have demonstrated the potential for producing attitudinal change, while Dhiman [30] concludes that games can “educate, advocate, create empathy, and build communities around social issues”. This large body of research forms the background to our work but does not directly inform our research objective. However, the use of ‘gamification’ to gather citizens’ views is more immediately relevant. Karl M. Kapp merged a number of definitions of gamification in describing it as “...using game-based mechanics, aesthetics and game thinking to engage people, motivate action, promote learning, and solve problems” [31] (p.10). However, this can refer to a wide variety of approaches. In their systematic review, Keusch and Zhang [32] observe that forms of gamification in surveys vary widely, including “simply rephrasing questions to sound more game-like … virtual badges and other intangible social rewards, and embedding the entire survey experience into a game where respondents are assigned to avatars and adventure through a fantasy land as they answer survey questions”. Following Hamari, Koivisto, and Sarsa [33] they refer to such game elements as ‘motivational affordances’. Building on a widely adopted classification proposed by Yee [34], Blanco et al. reviewed the use of such elements in e-government services, and distinguish three relevant categories of gamification mechanics and game-design features: immersion (e.g. storytelling, avatars or role-play); achievement-related (e.g. challenges, badges, leaderboards or progression metrics); and social (e.g. social interaction and collaboration) [35]. In addition to the different categories of gamification that can be applied, it is important to consider the types of citizen engagement which they can support. Arnstein [36] made an early contribution to this discussion, conceiving of a ladder of participation consisting of three stages: non-participation, tokenism and genuine participation, each of which has multiple rungs. Mayer [37] also identifies three levels of relationship between citizens and political entities. E-enabling is about supporting those who would not typically access the internet and take advantage of the large amount of information available. … E-engaging with citizens is concerned with consulting a wider audience to enable deeper contributions and support deliberative debate on policy issues. The use of the term ‘to engage’ in this context refers to the topdown consultation of citizens by government or parliament. … E-empowering citizens is concerned with supporting active participation and facilitating bottom-up ideas to influence the political agenda. … Here there is recognition that there is a need to allow citizens to influence and participate in policy formulation. In a similar vein, Thiel et al. [38] distinguish between one-way and two-way communication in gamified participation approaches, corresponding roughly to the second and third of Mayer’s three categories. Three challenges for gamified surveys can be distinguished in the literature. Firstly, the expected improvement in engagement levels has not materialized, and Gastil and Broghammer write that “Unrealistic expectations are common when government and civic organizations adopt digital technologies to improve public engagement” [39]. This is true for the gamification of surveys, where evidence for a transformative impact is scant. A systematic review by Oliveira and Paula on this topic concludes that “it is not possible to say whether gamification stimulates engagement”, but adds that there are indications that gamified surveys are more attractive and easier to answer [40]. Secondly, methodological concerns have been raised. In their review of gamified surveys Keusch and Zhang discuss the risk of “potential bias as a result of making surveys fun”: The biggest issue about survey gamification still concerns the influence of gamification on measurement error. One major challenge is that gamifying surveys often involves using techniques, such as rewording a question, changing response format (e.g., drag and drop), and adding additional visual elements, all of which inherently affect traditional data quality measures, such as response times, straightlining, and length of open-ended questions. [32] Thirdly, as Harms et al. commented “survey gamification requires a lot of effort” [41], and it is not clear that the benefits are commensurate with this effort. We return to these three challenges when we discuss our conclusions. IV. Method and Tools A. Study Design The GREAT project has developed a case study methodology for use in a series of case studies linking citizens and policy stakeholders through games-based activities. This is designed as a cycle of steps, shown in Fig. 1. The expected activities and outputs of each step in the cycle are described in detail in GREAT deliverable D4.2 [42], together with templates for planning and documentation and guidance for the leaders of case studies. An evaluation framework has been developed, with a set of instruments for use in the design of individual case studies [43]. The steps are not mandatory, given the range of requirements created by the use of two contrasting platforms (PlanetPlay surveys and in-depth exploration of dilemmas in serious games using SGI’s DiBL platform1), to address a wide range of institutional and geographical contexts. In the present paper we report on steps one to six, which correspond to our focus on the design, implementation and analysis of the embedded survey. The study is exploratory, in the sense that it investigates the potential of the approach and seeks to clarify the most effective use that can be made of the infrastructure in planned GREAT studies at a larger scale. Create and prioritise research topics Collaborate in study design Collaborate in game activity design Piloting and gathering data Data interpretation and outcome Conclusions and outputs Public engagement, disseminate outcomes Advocate for change at policy level 1 2 3 4 5 6 7 8 GREAT Case Study Cycle Fig. 1. The GREAT Case Study Cycle. 1 https://dibl.eu/about-us/ International Journal of Interactive Multimedia and Artificial Intelligence, Vol. 9, Nº2 - 96 - PlanetPlay worked closely with the collaborating studio to design introductory messaging, to encourage as many players as possible to click on an in-game news item. This is important, to avoid inadvertently pre-selecting a certain profile of person with specific opinions on the topic. B. Tools and Instruments Used This study makes use of the PlanetPlay system and infrastructure, which is designed to collect insights from video and players of highly popular mobile, computer or console games through a short set of questions that are promoted by game studios at scale to their player base. The PlanetPlay survey system has been developed incrementally since 2020 through a series of practical implementations to meet the information needs of clients. 1. The PlanetPlay System The PlanetPlay system consists of the survey web application, the infrastructure to host surveys and collect and process data, a ‘LiveOps’ (live operations) dashboard to monitor activity, and internal tools to assist in building and deploying them. In a typical deployment, players are asked about 7-10 questions centered around a single topic. After a brief introduction, questions either ask about the respondent’s sentiments (opinions on a topic) or knowledge (where there is a clear right answer), as shown in Fig. 2. Questions are usually shown one at a time, with fixed answer options given via buttons. There is no technical limit to the number of alternative answers, but to limit scrolling and maximize comprehensibility there are usually between four and six. Questions can also be marked as multiple choice. A small number of demographic questions are added at the end, usually gathering data about age, gender and education level. The interface design emphasizes simplicity, prioritizing respondents focus on a clearly delimited task, and so to maximize completion rates. Accordingly, additional features, such as branching, external links, and branching according to users’ responses, have to date been consciously avoided. Fig. 2. Global Climate Insights survey introduction and opening question. 2. Infrastructure The survey system uses a single Next.js app capable of handling multiple sets of survey content, which is hosted on Amazon Web Services via SST2. Survey pages are statically generated to improve load times. The preferred language is identified based on web browser 2 https://sst.dev settings and used if localized content is available. An example survey is available online3. Data is collected and previewed with a specialist real-time database platform, Tinybird, that makes it possible to quickly create an infrastructure that scales well. The Tinybird managed Events API receives data directly from the survey web app, continuously aggregating responses to individual questions, so data is not lost if the survey is not completed. To track responses, surveys are tagged with up to 4 identifiers: • survey: the specific set of content (questions/answers) for a survey. • source: the name of the game studio and/or their game promoting a survey. • distribution: one or more specific distribution methods (social media, in-game etc.) used for a survey/source. • variant: used when A/B testing is employed, e.g. to test response rate changes for UI variations. A ‘LiveOps’ dashboard has been developed for the GREAT project using Next.js and the Tremor4 framework to monitor survey campaigns which form a part of case studies. It is used by PlanetPlay staff and GREAT Project partners too: • Review recent survey activity and top-level performance indicators for surveys. • View aggregate answers given to survey questions. • See geographical and language distribution of responses. • Trigger data exports (summaries or full data sets). • List links to the administration panels of GREAT serious games, using the DIBL platform of Serious Games Interactive5. • Monitor Tinybird infrastructure resource usage/costs. The LiveOps dashboard, shown in Fig. 3, also offers a quick way to copy summaries for a particular studio’s distribution, so that high level response breakdowns can easily be shared with a game studio without the need for a data analyst. Fig. 3. LiveOps dashboard. It should be stressed that the LiveOps dashboard is not a tool for data analysis, but rather a quick way of sharing a view of the dataset in a way which is comprehensible to non-technical users. When the survey is complete, all the data is exported in CSV format, using three tables to minimize file size, then processed via the preferred 3 https://survey.planetplay.com/survey/ gci_2?source=casestudy&distribution=sampleurl 4 https://tremor.so 5 https://www.seriousgames.net/en/ - 97 - Effects of Culture on Open Science and Artificial Intelligence in Education data analysis tools of the analyst, in this case SPSS. Aggregate survey responses can also be made available to the public on the PlanetPlay website in a data panel, as well as being published as open data in line with GREAT project policy. 3. Survey Questions The questions to be included in the embedded survey were developed in close collaboration with the United Nations Development Programme6 (UNDP), who were the policy stakeholders involved in the study. The final set of questions was as follows: • What should the world do about climate change? • To address the climate crisis, how should your country improve transport? • To address the climate crisis, what should your country do about energy? • To address the climate crisis, what do you think your country should do about nature? • To address the climate crisis, what should governments do about farms and food? • To address the climate crisis, what should governments do about the economy? • How can your country better protect people from extreme storms, flooding, droughts, forest fires and other climate impacts? • Do you think games can contribute to resolving climate change? • In your view How do you think games could best help tackle climate change? • What climate topics do you think games can best cover? • Age. • Gender. • How old were you when you left education? C. Participants Policy stakeholders and other case study partners engage with the GREAT project for a variety of different reasons. The primary purpose of the stakeholders in this case study, UNDP, was to explore methods and approaches that could engage the global community in their Climate change policy discussions, as framed by their organization objectives, which include the engagement of citizens in support of the achievement Nationally Determined Contributions (NDC). Each country which is party to the Paris Agreement [44] is required to establish a NDC plan to adapt to climate impacts. and update it every five years. PlanetPlay works with game developers and publishers to reach survey respondents from their player base in several ways, including through, paid placement of advertisements, directly in-game, or through social media channels, QR codes and other channels such as a newsletter or homepage link. The case study explored two approaches to reaching participants through embedded content in the game. The first involved exploring the use of paid placement using the Meta ads platform on Facebook and Instagram. A/B testing was carried out to see if a better response was obtained when showing the first question directly, or when showing an introductory screen. The second approach was in-game roll out. This access was not paid for but was the result of direct negotiation with Game studios to allow the incorporation of the activity within the game. Prior experience had indicated that in-game promotions implemented in collaboration with a publisher tended to get the highest volume of 6 https://www.undp.org/ responses and response/completion rates. Candidate games usually have a ‘live service’ model (also known as Games as a Service) and are designed for a long lifespan with a continuous release schedule of new content. To support this model, they usually have existing in-game news/messaging systems that can be leveraged to promote a survey. In collaboration with UNDP stakeholders, the game studio Hi-Rez was selected, and the embedded QR code is shown in Fig. 4. SMITE, originally published in 2014, is a free-to-play third person Multiplayer Online Battle Arena (MOBA) digital game published on multiple platforms including the Microsoft X Box, Sony PlayStation 4, Nintendo Switch and Amazon Luna. Players control a ‘god’ ‘goddess’ or other mythological figures to participate in team-based combat activities with other players and non-player characters (NPC) ‘minions’. Fig. 4. SMITE in-game promotion on a console platform, with Global Climate Insights survey QR code. The game has multiple modes, supports an active e-sports community, and currently has over ten million global players. This game was chosen as having a high number of users but not so many that data management would be problematic for an exploratory study. The graphics shown in Fig. 5 were used to link to the embedded survey. Fig. 5. The graphics used to link to the survey. Delivery of the survey can be tightly targeted, down to the level of a town or part of a city. However, it was decided to gather data at a global level, to explore the responses across different countries. D. Procedure The procedure followed the overall structure of the GREAT case study steps 1 to 6. Step 1 of the case study cycle was very brief, as UNDP already had a clear view on the topic it wanted to address, i.e. perspectives of citizens on the appropriate actions to address climate change which could inform NDCs. Corresponding to step 2 of the case study cycle, the project team worked with UNDP and Hi-Rez Studios to design the intervention. It was decided to combine paid placement and in-game roll out, with International Journal of Interactive Multimedia and Artificial Intelligence, Vol. 9, Nº2 - 98 - Hi-Rez studios providing the case study with free access to the game SMITE through a QR code linking to the survey in their main menu. The design of activities, step 3 of the case study cycle, was limited to the collaborative authoring of questions (see previous section) with the policy stakeholder UNDP. In step 4, data was collected anonymously, without the respondent needing an account or being tracked or identified via cookies. This streamlined the user experience and allowed a full survey to be completed in under a minute for about 10 questions. Answers were submitted to the survey system when the respondent proceeded to a new question, ensuring that even if a survey instance was not completed, as many answers as possible were collected. Along with the language used, the respondent’s country and city were deduced and logged on the basis of the HTTPS headers from Cloudfront, together with the browser’s user agent string which includes their device and operating system type. Identification at this level maintains anonymity but offers additional dimensions for segmentation during data analysis. Efforts were made to discourage multiple responses from the same person, by showing a message reminding the player if they have already participated in a particular survey rather than taking them to the questions. However, this is browser dependent and can be circumnavigated by a technically aware respondent. Additionally, the hashed IP address of the respondent was also logged, and this can be taken into consideration during analysis to flag potential repeated responses. As with the interaction design, this avoidance of all identity management reflects the priority given to a streamlined user experience in order to achieve large scale responses. Finally, the survey/source/distribution/variant identifiers used during the survey promotion were also logged per session. In addition to the answers given, the dwell time of a question before an answer was captured, giving an indication of how much time someone might have thought about their response. The survey system also tracked ‘events’, e.g. when a survey is first loaded up, the initial engagement, when questions are shown, and if a link at the end of the survey is followed. This allowed engagement and completion rates to be calculated to measure the effectiveness of a survey instance with a specific partner and distribution method. Step 5, Data Interpretation and Outcomes, and Step 6, Conclusions and Outputs, are discussed in the following two sections. V. Results A. Response Rate by Distribution Method TABLE I. Response Rates by Distribution Method Paid placement (advertisement) In-Game Roll out Reach/First page load 7,257 4,352 Engagement 398 (5%) 2,539 (58%) Completion 179 (45%) 2,148 (84%) Community Sign Up 18 (10%) 282 (13%) The categories in Table I, above, are defined as follows: • Reach: people who saw the initial advertisement or survey screen page. • Engagement: people who performed an action on the initial page. • Completion: people who completed the survey. • Community sign up: people who responded to the prompt “Join us and your favorite games, to fight against climate change and save our planet! Track our collective progress and take action now!” by creating a PlanetPlay account. Given the poor response rate for paid placement, this data was discarded in further analysis. Not only was it considered that the low level of engagement might be associated with poor quality data, but also this enabled us to focus clearly on evaluating the results of the in-game roll out. There was a small increase in engagement when an introductory page was shown (58.1% vs 61.6%). Similarly, there was slight preference for the graphic link showing a woman at work rather than a burning planet. However, both effects were small, and are not considered to be statistically relevant. B. Geographical Distribution The case study was open to participants from around the world (see Table II). Every time a player visited the survey a ‘session’ was created, including people who simply visited the introductory page. A session is the parent of all the answer and event data from a specific user in a browser. TABLE II. Sessions by Country Country Sessions United States 2119 Canada 289 Brazil 225 United Kingdom 184 Mexico 157 Spain 156 Argentina 138 Germany 126 France 114 Russia 93 Colombia 50 TOTAL 3651 From the total of 3651 sessions, 2200 completed responses were obtained. Response rates can also be tracked by city. As can be seen in Table III, these were widely distributed. TABLE III. Response Rates by City City Country Sessions Buenos Aires Argentina 27 Chicago United States 27 Lima Peru 26 Los Angeles United States 22 Moscow Russia 21 Bogotá Colombia 21 Montreal Canada 21 Houston United States 19 C. Data Analysis Data analysis corresponds to Step 5 of the case study cycle. The full data set from the in-game placement has been made available as open data for inspection or further analysis by interested parties [45]. Several interesting trends were identified, as we discuss below. However, as shown in Fig. 6, the data was strongly skewed to the 18-35 age group, and the male gender. In view of the small number of respondents in some age groups, and the even smaller number of female respondents - 99 - Effects of Culture on Open Science and Artificial Intelligence in Education within those age groups, the results should be seen as indicative what can be achieved using this approach and should not be generalized to a wider population. It is not surprising that there should be an imbalance in the gender distribution, but it was not anticipated that this would be so extreme. It is often reported that the male-to-female ratio of game players is close to 50:50, but we obtained far fewer female respondents. We examined the data to see if this could be due to over-representation of console players (a platform that is often seen as being more male dominated) but the response rates were similar for PlayStation (male 85.15%, female 8.25% and other 6.60% (n=303)) and for the data overall (male 83.72%, female 8.78%, other 7.50% (n=2107)). It is therefore concluded that the imbalance reflects an unexpectedly large gender divergence among players of SMITE. In responses to questions on specific climate change strategies, age was not a significant factor in views on the conservation of forests and land or promotion of plant-based diets. However, the over 60s diverged significantly in their responses for transport, energy, the economy and protecting people. Gender differences were observed across variables, with statistical differences related to improving transportation of goods (X2 (2, 2107) = 12.263, p = 0.002), design and planning of cities and communities (X2 (2, 2107) = 6.579, p = 0.037), supporting communities (X2 (2, 2107) = 29.186, p = 0.000), using renewable power (X2 (2, 2107) = 7.696, p = 0.021), reducing food waste (X2 (2, 2107) = 10.808, p = 0.004), building infrastructure and conserving nature (X2 (2, 2107) = 15.828, p = 0.000) as well all on all economical interventions (p<0.05). However, there was agreement on several topics, such as using electric vehicles, wasting less energy, stopping fuel burning, conservation of forests and land, and promoting plant-based diets. All the respondents irrespective of gender agreed that games can support climate change initiatives in areas like transport, food and farms, and protecting people. An interesting result is that views on the appropriate response to climate change vary substantially with age, as shown in Fig. 7. Respondents under 60 were strongly in favor of “Do everything necessary urgently” whereas the majority of those over 60 chose “Do nothing”. The relatively small number of respondents over 60 means that the reliability of this result should be treated with caution. It could also be argued that, because of their small numbers, players of SMITE who are over 60 may be a niche population with characteristic attitudes, whereas this is less likely for age groups where the game is more widely played. Consequently, the over 60s may be less typical of citizens as a whole than are those age groups which are more strongly represented. Regarding responses to questions on specific climate change strategies, the chi-square results indicated that age was not a significant factor in views on the conservation of forests and land (X2 (3, 2107) = 7.655, p = 0.054) or promotion of plant-based diets (X2 (3, 2107) = 3.075, p = 0.380). However, significant differences were seen in the areas of transport, energy, the economy, protecting people (p<0.05) but not for nature (X2 (3, 2107) = 7.212, p = 0.065) and food and farms (X2 (3, 2107) = 4.976, p = 0.174). Regarding level of education (see Fig. 8), those respondents who left school after the age of 17 were strongly convinced that games can contribute to solving the climate crisis, whereas those who had left before 16 were largely unsure. Those who had never been to school were predominantly negative or unsure about this topic, but their numbers were too low to be reliable. 500 400 300 200 100 How old were you when you left education? Under 12 12-16 17-19 20 or over Still in education Never went to school Count Do you think that games can contribute to solving climate change? Yes No Don’t know Fig. 8. Educational level of respondents / “Do you think that games can contribute to solving climate change”. The statistical chi-square test results revealed significant differences in views on climate change across different education levels for most variables, as all p-values were below the threshold of 0.05. However, there was no significant difference for variable concerning Do everything necessary urgently Age 60+ 36-5919-35Under 18 Count 200 400 600 800 1000 1200 What should the world do about climate change? Act slowly while we learn more about what to do Do nothing Fig. 7. Age / “What should the world do about climate change?”. Gender Count Age 1500 1000 500 Under 18 18-35 36-59 60+ X Male Female Fig. 6. Age and gender of respondents. International Journal of Interactive Multimedia and Artificial Intelligence, Vol. 9, Nº2 - 100 - conservation of forest and land as indicated by chi-square value (X2 (6, 2255) = 10.023, p = 0.124) indicating that the respondent views do not vary significantly by education level. VI. Discussion We now discuss our results in terms of the research questions set out in Section II. RQ1 Which Games-based Activities Can Be Used to Elicit, Represent and Communicate Citizens’ Views on Policy Dilemmas? The results show that embedding questions within a popular commercial game is a viable strategy for gathering citizens’ views on policy dilemmas, and, in particular, with regards to climate change. The data collection, data formats and procedures tested in this case study proved to be fit for purpose. It is concluded that the approach does enable a link between policy makers and the targeted sector of games culture, but concerns are raised about potential distortion, which need to be addressed (see RQ2, below). More specifically, the results show that in-game roll out in collaboration with a game studio is far more effective than paid placement, as shown in Table I. One factor which may influence this result is that in-game roll out enables a respondent to scan a QR code on their mobile phone, and to answer the questions there while leaving the state of the game unchanged on their PC or console. Our survey was delivered through a QR code link on the main menu of the game. Other games may have a news system that shows regular updates and a survey can be promoted there, with a link button opening an external browser window. These existing news (or player messaging) systems are primarily designed to promote new game content and events, which keep players engaged and eventually monetized, so they could also be a very effective and visible way to reach players with a survey. Our experience of collaboration suggests two motivations which explain why game studios might be willing to offer access to organizations such as UNDP. Firstly, this shows that a studio is using its power as a media provider in a socially responsible way, contributes to meeting their Corporate and Social Responsibility commitments, and improves their image and brand association. Secondly, it helps the studio to better understand the issues that are important to their player base, which assists them in game design and content decisions. RQ2 How Effective Are Games-based Activities in Eliciting, Representing and Communicating Citizens’ Views on Policy Dilemmas? The results show that our approach is an effective alternative to gamified surveys. Three challenges to be addressed were identified in Section III above. Regarding challenge one ‘engagement levels’, the method and tools were effective in obtaining a large sample size with relatively little technical effort, once agreement had been reached with the studio. The response rate was good, as was the quality of the engagement, particularly in the in-game roll out, where 84% completed the survey once they had started, and 13% took the additional step of enrolling in the PlanetPlay climate change community. We conclude that the approach is effective in terms of engagement. Challenge two was ‘measurement error’ caused by the gamified elements of a gamified survey. Our separation of the survey from the mechanics and interactions of the host game worked well, and while we did not directly evaluate comparative measurement error, this strategy addresses some of the underlying concerns of this critique. However, the very large skewing of the data towards males between 18 and 25 years old is a concern, as it may misrepresent the views of all citizens. Information about the profile of players of particular games is published for different genres [46] [47], but the skew in our data is much more substantial than we anticipated from such high level analyses of player profiles. RQ3 How Efficient Is the Use of Games-based Activities in Eliciting, Representing and Communicating Citizens’ Views on Policy Dilemmas? Challenge three identified in Section III was ‘effort’, which relates strongly to our RQ3. In this respect we can offer some encouraging initial results. The PlanetPlay infrastructure enabled the intervention to be designed and delivered and the data managed with a relatively low level of skilled technical input, in the order of person days rather than person months. The system was easily able to handle the number of respondents, and there were no indications that the system would not be scalable to very large numbers. Our experience in this case study, therefore, was that the system used is highly efficient in technical terms. This does not take into consideration the very considerable effort involved in creating the system, which would need to be replicated by anyone adopting the approach who did not partner with PlanetPlay. As discussed in relation to RQ2, in-game rollout was a far more effective way of engaging with gamers than paid placement. However, the case study underlined the essential role played by the GREAT project, and specifically partner PlanetPlay, in mediating between the policy stakeholder UNDP and the games industry. This effort required to establish collaboration with games studios to obtain access to their platforms also needs to be taken into consideration when assessing the efficiency of the approach. However, this is hard to quantify, as it varies greatly from case to case, and depends strongly on the strength of the existing connections of the team carrying out the case study with appropriate sections of the games industry. VII. Reflections and Future Work A. Limitations An important limitation of this study is the highly skewed age and gender of the respondents. This constrains the generalizability of interesting results, such as the preference of over 60s for doing nothing to address climate change, in contrast to the views of younger people. In terms of the overall GREAT case study methodology, this study is limited by focusing only on the first five steps of the cycle. It therefore does not consider the value to stakeholders of the information obtained, nor its use to inform their decision making. For both of the above reasons, the value of the study lies more in the validation of the method, rather than in the impact or value of the specific data which was generated about citizens views. It should also be recognized that our approach corresponds to the second of Mayer’s three categories cited in Section III, e-engaging, and therefore involves “top-down consultation of citizens by government or parliament”. This is not presented as an alternative to e-empowering, but rather as the provision of a tool for linking citizens and policymakers at a scale which would not be possible for approaches which involve games players in collaborative policy making. B. Implications This case study has demonstrated that the in-game roll out approach to obtaining the views of citizens is effective and can generate information which is of value to policy stakeholders. This provides evidence in support of the use of the approach by policymakers and other policy stakeholders who experience difficulties in establishing a full picture of the views of citizens with regard to policy dilemmas. If the policy stakeholder is interested in the views of citizens over large geographic areas, then in-game roll out is appropriate, as the intervention is delivered in the regions where a game is marketed, - 101 - Effects of Culture on Open Science and Artificial Intelligence in Education and indeed this is often global. In collaboration with studios, an ingame roll out study can select games for the study with contrasting age, gender and geographic profiles. For example, Table II shows that SMITE would be a particularly good choice for an inquiry centered on the United States. Alternatively, if policy stakeholders have a need to work with a highly focused population, paid placement of playable advertisements is more appropriate, as this can be restricted to a particular city or region. Very specific targeting can be achieved by making use of more detailed data from a paid service provided by companies such as Ironsource and Loopme. Consequently, paid placement may still be a valuable option for more local inquiries, despite the lower response rate we have reported. The work reported here may be seen as a case study which examines the potential of maximizing scale and ease of participation at the cost of the depth of games players’ engagement or collaboration. This strategy has been shown to be effective, in as much as it has been well received by the policy stakeholder, UNDP, who have committed to a larger scale study informed by the results and limitations of this case study, being implemented at the time of writing. However, other approaches, with different trade-offs between these aspects, would potentially be equally effective. These would result in different benefits and costs, which may be more suitable for other contexts. Indeed, in parallel with the work reported here, the GREAT project is working with serious games to explore citizens’ views on policy dilemmas in intensive interactions between small numbers of participants. In future studies, greater attention needs to be paid to the selection of the game or games which host the survey, in order to achieve a better balance of gender and age. This can be done through selection of appropriate games, with the possibility of adapting the distribution of the survey in the light of data collected in the ‘LiveOps’ dashboard. This approach will not guarantee equal numbers of respondents from all genders and age groups but should provide sufficient responses from all genders to ensure the validity of the results. C. The Potential Use of Artificial Intelligence A possible strategy which could change the equation between the scale of an intervention and the depth of the interactions would be to make use of artificial intelligence (AI). In raising this possibility we are not referring to the automated generation of surveys or survey questions, as proposed by Gonzalez Bonorino [48] and by numerous websites such as responsly.com. Rather we see two potential techniques. Firstly, as suggested by Xiao et al. [49], AI-based chatbots could be used to generate interactions with players. This could act as a virtual equivalent of the familiar researcher with a clipboard who engages with a respondent. While one might associate chatbots with text, perhaps the most compelling application would be to generate conversational spoken language interactions between an app and a game player, which would provide richer data about players views than is available with text. The interactions could be as short or extended as the designer wished or could adapt to the respondent’s input. To address privacy concerns, it would be advisable to transform the speech into text, and to store the results of sentiment analysis, in as close to real-time as possible, and to avoid storing a recording of the player’s voice. Such a use of chatbots could, in principle, be compatible with the broad approach that we have discussed. Secondly, it would be possible to implement AI based non-player characters which could interact with players of a game. Johnson’s description of the goal of AI in commercial video games, from 2014, remains relevant: ...to produce believable behavior that is predictable and unpredictable and feels as if the player is being challenged and given interesting decisions to make in relation to an intelligent agent or character. [50] It is easy to see how giving players “interesting decisions to make” in relation to policy dilemmas could provide rich data about players views. Such an approach moves away from the highly focused largescale interactions in the present case study. It would also require substantial investment for each game in which it was included and would engage players in extended interactions. Moreover, it is doubtful that games studios would generally be willing to provide the access to their games which would be needed to implement this kind of interaction, as it would interfere directly in the gameplay. Consequently, it seems more likely that this approach would require the creation of a specific project to create a purpose-built game, with an alliance between researchers and industry. Acknowledgments The authors would like to acknowledge the essential collaboration of the United Nations Development Programme, without whom this work would not have been possible. They would also like to acknowledge the valuable contributions of Katharina Koller in identifying related work, of Joost Schuur in cleaning and formatting the data and clarifying the technical infrastructure, and of Pradeep Hewage in supporting the data analysis. This research was partially funded by the European Commission and by UK Research and Innovation (UKRI), through the Horizon Europe research project GREAT, grant agreement 101094766. Views and opinions expressed are, however, those of the authors only and do not necessarily reflect those of the European Union or UKRI. Neither the European Union nor UKRI can be held responsible for them. References [1] R. S. Contreras-Espinosa and A. Blanco-M, “A Literature Review of E-government Services with Gamification Elements”, International Journal of Public Administration, vol. 45, no. 13, pp. 964–980, 2021, doi: 10.1080/01900692.2021.1930042. [2] R. S. Foa, A. Klassen, M. Slade, A. Rand, and R. Collins, “The Global Satisfaction with Democracy Report 2020”, Bennett Institute for Public Policy, Cambridge, UK, 2020. Accessed: Aug. 08, 2024. [Online]. Available: https://www.cam.ac.uk/system/files/report2020_003.pdf [3] United Nations, “Trust in public institutions: Trends and implications for economic security (Policy Brief 108)”, United Nations Department of Economic and Social Affairs, 2021. Accessed: Aug. 08, 2024. [Online]. Available: https://www.un.org/development/desa/dpad/wp-content/ uploads/sites/45/publication/PB_108.pdf [4] D. Freeman et al., “Coronavirus Conspiracy Beliefs, Mistrust, and Compliance with Government Guidelines in England”, Psychological Medicine, vol. 52, pp. 251–263, 2020, doi: 10.1017/S0033291720001890. [5] A. Caplanova, R. Sivak, and E. Szakadatova, “Institutional Trust and Compliance with Measures to Fight COVID-19”, International Advances in Economic Research, vol. 27, no. 1, pp. 47–60, 2021, doi: 10.1007/s11294021-09818-3. [6] M. Morelli, “Sad populism and the policies of hope”, EconPol Forum, ISSN 2752-1184, CESifo GmbH, Munich, vol. 25, no. 2, pp. 40-42. [7] J. Visser, “Trade Unions in the Balance, ILO ACTRAV Working Paper”, 2019, Accessed: Aug. 05, 2023. [Online]. Available: https://etufegypt.com/ wp-content/uploads/2019/10/wcms_722482.pdf [8] R. F. Inglehart, Religion’s Sudden Decline. Oxford University Press, UK, 2021. doi: https://doi.org/10.1093/oso/9780197547045.001.0001 [9] N. Forman-Katz, “Americans are following the news less closely than they used to”, Pew Research Center, Washington DC, USA. Accessed: Jun. 14, 2024. [Online]. Available: https://www.pewresearch.org/shortreads/2023/10/24/americans-are-following-the-news-less-closely-thanthey-used-to/ [10] Pew Research Center, ‘News Platform Fact Sheet’. Accessed: Jun. 14, 2024. [Online]. Available: https://www.pewresearch.org/journalism/factsheet/news-platform-fact-sheet/ [11] J. A. Lischka, “A Badge of Honor?: How The New York Times discredits President Trump’s fake news accusations”, Journalism Studies, vol. 20, no.