How to design gamification? A method for engineering gamified software

How to design gamification? A method for engineering gamified software Benedikt Morschheuser Institute of Information Systems and Marketing, Karlsruhe Institute of Technology, Germany Corporate Research, Robert Bosch GmbH, Germany [email protected] Lobna Hassan Information Systems Sciences, Hanken School of Economics, Finland Gamification Group, University of Tampere, Finland [email protected] Karl Werder Paluno – The Ruhr Institute for Software Technology, University of Duisburg-Essen, Germany [email protected] Juho Hamari Gamification Group, Tampere University of Technology, Finland University of Turku, Finland [email protected] DOI: Reference: https://doi.org/10.1016/j.infsof.2017.10.015 To appear in: Information and Software Technology Cite: Morschheuser, B., Hassan, L., Werder, K., & Hamari, J. (2017). How to design gamification? A method for engineering gamified software. Information and Software Technology, in press. doi:https://doi.org/10.1016/j.infsof.2017.10.015 This is the accepted manuscript of the article, which has been published in Information and Software Technology. 2017, 95, 219-237. https://doi.org/10.1016/j.infsof.2017.10.015 How to design gamification? A method for engineering gamified software Abstract Context Since its inception around 2010, gamification has become one of the top technology and software trends. However, gamification has also been regarded as one of the most challenging areas of software engineering. Beyond traditional software design requirements, designing gamification requires the command of disciplines such as (motivational/behavioral) psychology, game design, and narratology, making the development of gamified software a challenge for traditional software developers. Gamification software inhabits a finely tuned niche of software engineering that seeks for both high functionality and engagement; beyond technical flawlessness, gamification has to motivate and affect users. Consequently, it has also been projected that most gamified software is doomed to fail. Objective This paper seeks to advance the understanding of designing gamification and to provide a comprehensive method for developing gamified software. Method We approach the research problem via a design science research approach; firstly, by synthesizing the current body of literature on gamification design methods and by interviewing 25 gamification experts, producing a comprehensive list of design principles for developing gamified software. Secondly, and more importantly, we develop a detailed method for engineering of gamified software based on the gathered knowledge and design principles. Finally, we conduct an evaluation of the artifacts via interviews of ten gamification experts and implementation of the engineering method in a gamification project. Results As results of the study, we present the method and key design principles for engineering gamified software. Based on the empirical and expert evaluation, the developed method was deemed as comprehensive, implementable, complete, and useful. We deliver a comprehensive overview of gamification guidelines and shed novel insights into the nature of gamification development and design discourse. Conclusion This paper takes first steps towards a comprehensive method for gamified software engineering. Keywords: Gamification, software engineering, design science research, persuasive technology, gameful design, playfulness, game design 1. Introduction During recent years, the enhancement of software via design features borrowed from (video) games, also known as “gamification” [1], has become a notable development in many software engineering projects [2,3]. Gamification primarily aims at increasing users’ motivations towards activities or use of technology, and thereby, increasing the quantity and quality of the output of these activities [1,3,4]. Gamification has since been employed in a variety of fields such as in education [5–8], health management [9–11], enterprise systems [12–14] and governmental services [15,16]. While literature reviews and practitioner reports signal optimistic stance towards gamification and what it can achieve (see e.g. [2,17] for reviews), the understanding of how to successfully design gamified software is yet in its infancy [16,18]. Business analysts suggest that more than half of all organizations would have had gamified parts of their organizational software and internal practices by 2015 [19,20]. However, it has also been predicted that a majority of these gamification implementations are doomed to failure due to poor understanding of the gamification design process [18]. This gap canonically often manifests as modest gamification software that commonly only consist of simple, superficial introduction of game mechanics (such as points, badges, and leaderboards) to pre-existing software [2,3,17]. Designers engaging in these practices pay perhaps too little attention to the underlying psychological dynamics that primarily make games and gamification engaging to users [16,21] thus risking the success of the software they develop. Gamification is difficult to design for a variety of reasons, the most prominent of which is that: 1) the inspirational source of gamification design; games, are complex, multifaceted, and therefore, difficult to generally design and let alone transfer to other environments [1,21,22]; 2) the goal of gamification is to affect behavior and not only to entertain – as it is primarily the intention of games [1,2]. Thus, designing gamified software should not be equaled with developing games. Otherwise, transferring game design features to the engineering of serious software may lead to the design of software that provides a level of entertainment, but might not lead to a behavioral change as is intended from gamification; 3) the serious context in which gamification is applied provides requirements, which may limit the design space drastically compared to games [22,23] and thus adds another level of complexity; 4) to affect behavioral change, gamification involves motivational information system engineering [1,4] which entails the understanding of a host of (motivational) psychology and requires appropriate competencies in the development team. These four design challenges collectively along with many others add layers of software engineering complexities into the scope of gamification design. These engineering challenges along with the relative novelty of the research field and the reported lack of understanding as to how to successfully gamify software as discussed, inhibit organizations from designing and adopting effective gamified software. Thus far, only a few sources exist that provide methodological insights into how to gamify (e.g. [22,24–26]) or practical guidance on designing gamification (e.g. [23,27–30]). However, most of these frameworks have been developed in a vacuum, and, very few of them pay comprehensive attention to the previously outlined challenges of engineering gamification as is detailed in coming sections of this study. In this sense, the frameworks do not draw on each other but rather inhabit separate areas. As the theoretical and practical field of gamification continues to grow, there is a constant need to develop gamification engineering methods that comprehensively tackle gamification challenges as they grow. Therefore, in this paper, we seek to advance the understanding of the best practices related to the engineering of gamified software. We approach the research problem via a design science research approach [31,32]; firstly, by synthesizing the current body of literature on gamification design methods and by interviewing 25 gamification experts thereby producing a comprehensive list of design principles for developing gamified software. Secondly, and more importantly, we develop a detailed method for engineering of gamified software, based on the gathered knowledge and design principles. Finally, we conduct an evaluation of the artifacts via interviews of 10 gamification experts and via implementation of the engineering method in a gamification project. 2. Background 2.1 Gamification Gamification refers to the enrichment of software with design features known from games in order to invoke similarly engaging experiences as games do [1,22,33]. The software we use in our lives are developed for many purposes, the most dominantly either to be utilitarian or hedonic [34]. Recently, however, utilitarianism and hedonism have increasingly become interwoven in modern software, as users increasingly expect that software is not only useful but is also enjoyable to use [2,4,33]. Therefore, designers increasingly apply gamification in software development projects; turning to games for inspiration on how to enrich utilitarian software with hedonic elements. Taking the example of enterprise systems, game design elements including ranking lists, points, badges, leaderboards, challenges, and progress evaluations have been introduced to various forms of intranet systems and enterprise social software with the intention to increase knowledge sharing, usage of these systems and productivity within organizations [12–14]. Similar game design elements were introduced in educational environments and systems [5–8] to increase the motivation of learners and their learning performance or in fitness software, to support people doing exercise [9,11]. Various studies report positive psychological and behavioral outcomes of using gamification, for instances on motivation, social interaction and performance [2,3,4,5-8,9,17]. With the aim to identify the game features designers might employ when engineering gamified software, several studies produced lists of the most commonly used game elements in gamification of software (see [2,3,17] for overviews). All of these reviews revealed a recurring use of the same game elements, such as points, badges, and leaderboards. However, by drawing on these typical elements many gamification projects fail to invoke gaming-characteristic hedonic experiences [16,22,35], since the gameful experience in games emerge from not only singular game elements, but rather from the dynamics that the more holistic assemblage of game features gives rise to [1,36]. Comprehensive challenges in the design process, little research on methods for designing gamified software and missing guidelines as for how to ensure the behavioral impact of the gamification design may be reasons that discourage designers from using the full potential of games and thus failing to successfully engineer gamified software [22]. Consequently, further research is needed to address the key challenges of designing gamified software and to provide guidance for the implementation of gamification projects. 2.2 Challenges of designing gamified software Game engineering is a complex process that involves multidisciplinary work across psychology, design, programming to name only a few disciplines, thus making games multifaceted artifacts that are not only hard to define and understand [1,21,22], but additionally hard to successfully design [37]. We have begun to understand that the stimulation of human needs [1,21] the application of goals, rule systems and challenges [14,22,38] are key characteristics of games and probably responsible for their rich motivational experiences. However, since successful game approaches commonly employ manifold game designs [17,38], by utilizing many of these components, it is mostly difficult to unambiguously relate psychological outcomes to specific game features. The interplay of such design features and psychological processes characterize games [1,21], but is also responsible for their complexity. Engineering of gamification aims to invoke similar engaging experiences as games to motivate users towards specific behavior through the employment of design features from games to other environments [1,2,21,22], it thus inherits the same design complexity of games. Adding to this complexity, the goal of gamifying a software is to affect behavior and not only entertainment as is the primary goal of games [1,4]. For example, if we consider gamified enterprise systems [12,13,39], we see that these systems have been enriched with gamification in order to make the use of such utilitarian software hedonic and more enjoyable. However, this is one side of the coin. Typically, designers that implement gamification want to achieve a more frequent utilization of a system thus ensuring better facilitation of the underlying workflows [2,17]. A gamified software thus has the double requirements of being 1.) operationally well designed to function as intended, 2.) facilitate engagement with the software so as to ensure manifestation of appreciated behaviors and behavioral change. Games typically achieve engagement by providing challenges matched to players’ skill level to provide opportunities for the experience of feelings, such as achievement or mastery that keep players engaged with the game for longitudinal periods of time [1,21,40]. The difficulty of the challenges may occasionally vary towards easier ones in order to ensure a continuous challenging and a diverse experience that keeps players in a “flow” state: an optimal experience in which the individual is fully immersed in the task they are performing that they are not aware of other externalities [40,41]. Gamification attempts to mimic these experiences by employing challenges that are matched in design and presentation to game challenges [1,40]. However, the context in which gamification is applied adds complexity on the design of engaging challenges, as the context provides operational requirements that limit the unlimited design space that typically games have. Gamification designers should thus be aware that the gamified software should meet these operational requirements for the software to have operational value to necessitate engagement with it, as is the aim of applying gamification. The prevailing opinion is that games invoke motivation and influence behavior because they satisfy user's intrinsic needs, such as the needs for relatedness, mastery, or autonomy [1,16,21,33]. The fulfillment of basic human needs has been highlighted as a key justification for the psychological and behavioral outcomes of games in many studies [1,14,16,21,42]. However, designing software that satisfies specific human needs is complex. Designers need to be aware of motivational psychology and motivational design. This adds another layer to the complexity of designing gamified software. 3. Artifacts development As discussed in the previous sections, engineering of gamified software is challenging, requires multidisciplinary knowledge and has extensively been conducted through methods that do not draw on each other, but rather have been developed in a vacuum by individual gamification experts. Thus, the aim of this research is to synthesize the current body of literature on gamification design methods, as well as the design principles to answer the following research question: RQ: How should gamified software be engineered? Given the study’s focus, we opt for a design science research (DSR) approach [31,32]. DSR emphasizes the systematic development and evaluation of artifacts intended to solve practical problems. Therefore, the research process consists of two primary modes of investigation and their interplay: 1) developing/ building theory-ingrained artifacts and 2) evaluation of the developed artifacts. More specifically, we developed and evaluated two artifacts that build on each other. The first artifact is a list of design principles for engineering gamified software. Design principles, according to Gregor and Jones [43], provide high-level design guidance. In a similar vein, Zhang suggests that design principles “remind designers of what issues may exist and why” [44]. However, since design principles still provide no answer to the question of how to design something [44], we developed a second artifact that incorporates the first; a method for engineering gamified software that provides comprehensive guidance to the process of how gamified software could be designed developed. To develop such a method to the engineering of gamified software, we employed method engineering within our DSR approach. In Information Systems (IS) research the method engineering methodology, which is defined as “the engineering discipline to design, construct and adapt methods, techniques and tools for the development of information systems” [45], has been established for developing methods in software engineering [46–48,49]. A common practice in method engineering is the assembly of situational methods for specific engineering projects based on fragments, synthesized from existing method knowledge [45,46,50,51]. Gamification provides a concrete software engineering scenario that requires situational adaptations of a standard approach unique to every software project. As discussed above, gamified software is not limited to operational requirements, but requires an in-depth understanding of human psychology. Gamification relies on games and game design in its engineering methods in order to affect user behavior. Hence, a general understanding of the gamification process as an extension of established engineering approaches is imperative to the development of functional and successful gamified software. Such a situated process of gamified software engineering requires situational aspects as dictated by the projects characteristics and its operational context. Thus, we are aiming to develop a situational method [45] for gamification projects that provide general guidance for engineering gamified software and could be used as method base for further developing situational methods for specific gamification projects at hand in an iterative method engineering process [52]. Figure 1 provides an overview of our method engineering procedure. According to Brinkkemper [45], essential aspects of method engineering are: i), the development of a comprehensive method base that includes all resources needed for the development of a new method; ii), the assembly of the so-called “method fragments” from the method base in order to construct a new situational method [45,48,52,53]; iii), evaluation of the method in a specific project that can provide knowledge for further developments of situational methods [52]. The initial knowledge and method base could come from interviews with experts on the phenomenon understudy or through literature reviews or preferably both (Figure 1). Figure 1. Situational method engineering approach followed for artifacts development 3.1 Knowledge base In order to develop a holistic perspective on the subject matter, the method base of this study relies on two aspects: i) scholarly experiences from the literature on gamification and ii) professional experiences from experts through interviews. The coming subsections describe the sources of data for both, deriving principles for engineering gamified software and developing a comprehensive gamification method. 3.1.1. Literature review In order to study the currently available gamification design methods, we conducted a hermeneutically-oriented iterative review [54] that is a literature review process that employs two cycles of review; the first involves the identification of relevant sources, keywords, and initial literature pieces, the second cycle involves interpretation and evaluation of the obtained results from to determine their relevance and identification of new sources. The familiarization towards the topic is essential for the correct operationalization and execution of method engineering. Thus, we aimed to review all relevant literature sources and included gray literature and practical outlets. The review process was conducted in November 2015 and included the following databases: ProQuest, ACM Digital Library, AIS Electronic Library, IEEE Xplore Digital Library. As outlined, the first step of a hermeneutically-oriented iterative review included the identification of design related keywords for a systematic literature search, resulting in the following search strings: (gamify OR gamification) AND (framework OR model OR design OR approach). Our systematic review next identified 468 items. In the following step, we removed duplicates and excluded results based on title, resulting in 247 items. A review of the abstracts reduced the number of articles to 35. Through a backward and forward search [55] of the identified paper on the same selected databases, we identified another 26 potentially relevant articles. We applied the same inclusion and exclusion criteria to focus on articles that present either a process model, articulate specific design principles or present other relevant information for the design of gamification. Consequently, another 6 articles were added to the literature pool. Thus, we consider a list of 41 articles (35 from the literature search +6 from back and forward searches) that include relevant information about gamification. From these articles, we extract the descriptions of methods, phases, activities, deliverables, and requirements. In total, we found 17 gamification methods in the identified sources. The examination of these methods pointed towards seven main phases of engineering gamified software; (1) Project preparation: All activities that have to be executed before the project starts; (2) Analysis: Activities that are used to identify the necessary knowledge of users, processes and the project itself; (3) Ideation: Activities to come up with ideas for gamification designs; (4) Design: Designing gamification and creation of prototypes; (5) Implementation: deployment of a gamified software; (6) Evaluation: Evaluation and testing of the software; (7) Monitoring: Monitoring the performance of the software after the release. These phases would be further expanded upon in the coming section of this study. A summary of the identified 17 methods to engineer gamified software according to these phases is presented in Table 1. Method Preparation Analysis Ideation Design Implementation Evaluation Monitoring Method evaluation [56] ✓ ✓ - ✓ ✓ ✓ - Case study [57] ✓ ✓ ✓ - ✓ ✓ No evaluation [22] ✓ ✓ ✓ ✓ ✓ - Developmen t workshops, case study [29] ✓ ✓ ✓ - ✓ - No evaluation [42] ✓ ✓ - ✓ Case study [58] ✓ ✓ ✓ - ✓ - No evaluation [24] ✓ ✓ ✓ ✓ Case study [23] ✓ ✓ ✓ ✓ ✓ ✓ ✓ No evaluation [59] ✓ ✓ ✓ ✓ ✓ Two case studies [60] ✓ ✓ ✓ ✓ ✓ - No evaluation [61] ✓ ✓ ✓ Case study [30] ✓ ✓ ✓ - ✓ No evaluation [25] ✓ ✓ ✓ - - - No evaluation [28] ✓ ✓ ✓ - ✓ - No evaluation [26] - ✓ ✓ - ✓ - No evaluation [62] ✓ ✓ ✓ ✓ ✓ ✓ - No evaluation [27] ✓ ✓ ✓ ✓ ✓ - No evaluation Table 1. Summary description of gamification methods identified through the literature review 3.1.2. Expert interviews In order to compare and comprehend the gathered knowledge for engineering gamified software, we also conducted interviews with gamification experts. Within this study, we consider an individual an expert based on their publicly available information about their occupation. In particular, an expert has real-world gamification experience and shows strong interest in the subject matter, as indicated by one of the following cues: i) speaks at an international gamification conference (e.g. the Gamification World Congress), ii) is a member of a gamification association, or iii) is an active gamification “influencer” on social media channels1). We contacted over 90 gamification experts, 25 of whom are located in 17 different countries (Table 2) participated in the study [I1-I25]. The interviews conducted were semi-structured in order to ensure the collection of the most relevant answers from the experts, and yet give room for further probing to reach rich answers to unscripted interview questions when the need was presented [63]. The first part of the interview focused on the extraction of design principles for engineering gamified software while the latter part focused on engineering methods of gamified software. The interviews were conducted via Skype in English and German. All interviews lasted an average of 30 to 45 minutes and were recorded and transcribed with the permission of the interviewees. The knowledge we gathered by conducting the literature review and analyzing the transcribed interviews was structured in tables and organized along reoccurring method activities and deliverables. Gender Frequency Locations Frequency Male 23 USA 5 Female 2 Germany 3 Australia 2 Switzerland 2 Profession Frequency Austria 1 Director of a gamification consultancy 13 Argentina 1 UX Designer / Gamification consultant 7 Canada 1 Gamification researcher 5 China 1 Great Britain 1 Finland 1 France 1 1 The social media activity was analyzed with the service “Rise” https://www.rise.global/gurus based on the data from October 2015. feasibility to identify user needs and to develop personas, as introduced above differs depending on the size and complexity of the gamification project. Therefore, designers should carefully evaluate whether the activities of this phase can be performed as outlined or need to be adapted to the respective context of their project [45]. Figure 3. Activities of the analysis phase 3.4.3. Ideation Once an analysis of user and context characteristics has been obtained, the next step is to develop a gamification design. Surprisingly, we found that several published methods as indicated in Table 1, do not describe this core activity in detail. Most of the gamification methods identified promote gamification through the creation of engaging challenges by the use of design feature known form games (e.g. [24,27,28,58,60,61]). Usually, game elements such as points, badges or leaderboards are considered as game features. In addition, some authors also consider game mechanics such as rules and dependencies that define the gameplay, game dynamics that form the interaction between a user and a game, as well as narrative aspects [1,36]. DP 8, drawn from the majority of identified methods, emphasizes the selection of game features - particularly game elements - which match previously identified user needs and promote desired user behavior as a core approach for engineering gamified software. Some argue for the use of game features, such as rewards, points, badges, leaderboards or storytelling, as building blocks [30], and assume that the combination of these building blocks with goals of the real context would invoke engaging challenges and motivate goal-oriented behavior (e.g. do activity X to unlock badge Y). Furthermore, some authors recommend aligning several game features, in order to promote repeated performance (“engagement loops”) along with a “player journey” [23,27,30]. However, the detailed process of selecting and combining gamification building blocks in order to design a gamified software often lacks descriptive details and only a few authors provide information on the mapping of game features to user’s needs [22,28]. However, as the majority of the interviews showed that in practice gamification is a creative and iterative design process, we believe that the use of frameworks that define strict guidelines for the use of gamification building blocks may harm the needed creativity for its design. The interviews, on the other hand, indicated that the design of gamification is a creative process that requires an ideation phase. The interviewees suggested that practice pays more attention to this creative process, and thus practitioner tend to develop comprehensive lists of gamification design ideas during their work. The interviews indicate that the first step to developing such lists is typically an iterative brainstorming activity (with the goal to come up with a large amount of ideas) [I17, I19] cf. [22,23,60]. Explorative brainstorming has been highlighted as an important approach to understand the so-called “design space” (i.e. the space of possible design alternatives) [I17, I19, 7]. Some experts stressed the importance of coming up with an epic theme or a narrative to guide brainstorming and glue design elements together ([I1, I7, I8, I11, I21, I24], cf. [28]). Some interviewees recommended focusing brainstorming on the fulfillment of user’s needs, desired behavior, and target outcome, rather than on the technology or game elements to be employed by the gamified software [I11, I14, I22, I24]. This view has also been adopted in current theoretical and conceptual views of gamification [1]. Eventually, ideas are usually consolidated in order to create a list of ideas for the upcoming design phase [I17, I19, I22] [22,62] (Figure 4). Figure 4. Activities of the ideation phase Nearly all interviewed experts reported that they follow frameworks, such as the User-Centered Design framework [25,68], Design Thinking [I2, I11, I15, I16-I22, I25] [23], the Octalysis Framework3 [I6, I11, I12, I21], the Playful Experience framework (PLEX) [69] [I19], Lazzaro's 4 Keys 2 Fun4 [I16, I21] or the Person-Artifact-Task (PAT) model [42,70] in order to guide ideation. Five experts additionally mentioned the importance of user involvement in the ideation phase, in order to ensure the focus on user needs (cf. DP13). Interviewees also mentioned the use of creative techniques, such as “brainwriting”, or “proxy thinking” and “bodystorming” where a prototype of gamified software is imagined to already have been implemented in order to perceive the implications of its use in its intended use context. Such practices could be carried out in workshops with users, designers and other stakeholders [I15I17, I20]. To support designer, we supplemented this phase of the engineering process with an overview of the techniques, tools, and frameworks that may support the ideation, presented in Table 5. For example, five experts mentioned that the playing of games and the discussion of mechanics in board and video games could stimulate ideation. The literature provides additional approaches. For instances, [22] proposes the use of “innovation stems” that inspire prompts to guide and engage brainstorming (e.g. “How might we spark a sense of pride in an assembling process?”). 3 http://octalysisgroup.com 4 http://www.nicolelazzaro.com/the4-keys-to-fun/ Tools Purpose Board and Video Games Playing of games and discussion of game mechanics can stimulate general ideation [I1, I2, I10, I14, I25], [62]. Design Lenses Design lenses [22] provide a special perspective on a design space to guide ideation and design in a particular direction [I2, I6, I11, I16]. Design Cards Design cards mostly contain design lenses, such as basic human needs that are to be fulfilled by the software under development. Designers usually utilize them randomly and playfully during brainstorming to come up with ideas [I2, I8, I11, I16, I17, I19], cf. [69]. Visualizations Visualizations (e.g. process models) are used to simplify, visualize and understand the relationships between users and their behavior in the considered environment [I11], cf. [67]. Game design patterns Commonly recurring designs in games are often used as a foundation to develop ideas for gamification [I7, I8, I9] (see [2,17,28,30]). Story Cubes Story dices are dices with different icons, which are typically used to support storytelling in general. The story narrated through these dices, in turn, can be used as starting point to develop gamification themes and subsequent design [I8, I11], cf. [28]. Canvases Canvases can help to communicate ideas, identify weaknesses and compare gamification designs in a systematic manner. [I5, I16, I17, I22] (e.g. [29]). Decision trees They provide decision support and guidance for the selection of game elements and mechanics [I18], cf. [25]. Best practice / gamification patterns Best practices and recurring gamification design patterns are used as starting points for ideation [I14, I22, I24], cf. [23,24,27,30,57]. Table 5. Ideation toolbox used in practice 3.4.4. Design After collecting ideas, concrete gamification designs for the software can be developed. This step is strongly related to the ideation phase and focuses on the elaboration of “playable” prototypes to evaluate the effectiveness of a particular gamification idea. DP3 reflects the literature and interviewees, recommending the rapid development of prototypes (e.g. in form of paper prototypes, sketches or wireframes [22,59,60]), [I7, I8, I14, I19, I21, I22] and several sources as reflected in DP4 highlight that successful gamification arises from an iterative design process (Table 3), in which ideas and designs are frequently tested and improved until they seem to be efficient and promising to reach the previously defined goals in accordance with DP13 [22,25,56]. While the type of the developed and evaluated prototypes in this phase can vary depending on the considered context, most experts preferred such an iterative proceeding for any kind of gamification project. The literature [42,62] and 3 experts [I3, I17, I22] suggested creating a development concept as the outcome of this phase (Figure 5). This development concept should contain all the relevant information for the implementation of the gamified software designed. The interviewees also reported that sometimes an engineering transition takes place at the end of this phase, during which, the gamification designer hands the project over to a team of developers to continue the process [59]. DP12 additionally dictates the consideration of legal and ethical constraints that affect the design. Due to the different activities in this phase (Figure 5), several experts mentioned that gamification designer requires interdisciplinary skills, such as communicated by DP5, indicating that gamification design requires a profound understanding of human motivation, game design, business processes and information system design in order for the designers to be able to understand all dimensions of this phase and to effectively communicate with all stakeholders coming from diverse backgrounds. As the activities in the ideation and design phase are multifaceted, collaboration in interdisciplinary teams is a common principle (Table 3), particularly in these phases. Figure 5. Activities of the design phase 3.4.5. Implementation of a design The majority of the methods in the reviewed body of literature contain an implementation phase (Figure 6). However, little information about the details of its execution is found. It can be summarized that the purpose and outcome of this phase are the carrying out of a pilot, which can be used for field evaluation of the gamified software [42], [I15, I22, I24]. The interviews suggest that proceeding within this phase is determined by the decision to either (A) develop the gamified software through an in-house team, (B) employ external developers or (C) adapt the design to an existing off-the-shelf gamification platform. While various platform solutions for different enterprise scenarios exist, such as innovation communities, enterprise social software or employee training, other cases may require the development of individual solutions. Most experts reported that they usually build gamification solutions within their own team. Some reported that they use external developers [I10, I17, I21, I22] or the developers of a client [I10, I13, I14, I17, I21, I22]. A few times, the use of available gamification platforms was mentioned ([I22], cf. [59,60]). The decision depends on the considered context; thus, it might be suitable to adapt the activities of this phase to a particular project scenario [45]. While platform solutions typically provide pattern and best practices for the implementation of gamification features in a specific scenario, several sources recommend an iterative procedure (DP4) in development cycles to ensure quick identification and assessment of technical issues when developing individual gamification solutions [27,60]. In the latter, continued user and playtesting after development cycles, is a recommended practice to evaluate and optimize the designed gamification, ensuring its effectivity and success (DP11). An interviewee additionally emphasized the importance of project management practices and recommended the involvement of gamification experts within the development process [I14]. Figure 6. Activities of the implementation phase 3.4.6. Evaluation The aim of the evaluation phase (Figure 7) is to investigate, whether the developed gamified software meets its predefined objectives from phase (1) and to use the set metrics for gamification evaluation in accordance with DP9. Several approaches to evaluate a gamification design can be found in the literature. These range from quantitative to qualitative approaches [24,42,58,61,62]. The interviewed experts reported that they typically conduct interviews [I9, I12, I19, I21, I22, I25], surveys [I1, I22, I25] of users, to analyze their perception and interpret usage data [I19] or conduct A/B-testing [I18, I23], to determine the differences between a gamified and a non-gamified version or to evaluate the effects of different gamification interventions. Moreover, playtesting was one of the most mentioned evaluation methods. Playtesting refers to the observation of users while undertaking a task in a game [22,42]. Several experts have highlighted that observing user behavior is more effective than interviewing them about their behavior, as users experience difficulties in accurately describing their lived experiences verbally [I7, I11, I18, I21]. These difficulties may include the inability to articulate experience or the unintentional communication of perceptions different from the reality of what the users have experienced. Another evaluation technique is the use of a service quality model to measure the effectiveness of the gamified software as a service provided to end users [58]. Unfortunately, the experts stressed that in commissioned work, evaluation is often done in a lean manner or omitted altogether, since often no budget is set aside for the evaluation phase [I15-I17, I21, I22]. In these cases, the pilot is just launched. Figure 7. Activity of the evaluation phase 3.4.7. Monitoring While gamification may be perceived as a (never-ending) iterative process of design, development, evaluation, monitoring, and adaption [28] in accordance with DP11, the reviewed gamification methods have largely omitted this aspect. Most simply recommend a launch and post-launch monitoring (see [22] for an overview). The interviews indicate that practitioners often see gamified software as classical software engineering projects with a clear start and end [I17]. Therefore, monitoring and management are often not planned or budgeted for in practice [I12, I14, I15, I21]. However, this is not an ideal practice as the impact of game features may vary, based on the user characteristics and the usage duration [71]. More than half of the experts emphasize that gamification projects should not be considered as typical software engineering projects with a clear end. “A successful gamification project should never end because it will become part of how the organization works” [I3]. Most experts recommend a monitoring phase during which system usage is investigated in regular intervals. The collected data is used to evaluate the implemented gamification design, to identify irregularities and to check whether the desired user behavior is achieved in accordance with DP9. Based on the gathered insights, gamification mechanics, rules and contents should be tweaked in order to keep the system engaging and adaptive to changing objectives and user needs. One way gamified software should be tweaked or adapted is to ensure that activities that lead to cheating the systems are controlled (DP10). A typical outcome of this phase (Figure 8) is a list of improvements (e.g. adaption of parameters in the implemented software [I4, I21] or a plan for a new release [I2, I14]). Furthermore, the use of A/B-testing has been mentioned and can be utilized to continuously optimizes the parameters of gamification [I11, I19]. Figure 8. Activities of the monitoring phase 4. Artifacts evaluation We evaluated the developed artifacts in a twofold manner: 1) we conducted expert interviews [31] and 2) we utilized the developed method in a software engineering project as a case study -oriented evaluation, following Runeson and Höst [72]. 4.1. Experts evaluation 4.1.1. Participants All the 25 interviewed gamification experts who participated in the interviews for the method development were invited to participate again in a method evaluation interview. Ten experts agreed to evaluate the developed method [E1-E10]. Table 6 gives an overview of the participants. Gender Frequency Locations Frequency Male 9 USA 2 Female 1 Australia 1 Switzerland 1 Argentina 1 Profession Frequency Canada 1 Director of a gamification consultancy 6 China 1 UX Designer / Gamification consultant 3 Israel 1 Gamification researcher 1 Saudi Arabia 1 Slovak Republic 1 Table 6. Overview of interviewed experts in the evaluation of the method 4.1.2. Procedure The interviews were conducted via Skype and lasted for approximately 30 minutes. The interviews commenced with a comprehensive introduction to the developed design principles, as well as the developed method and then, both artifacts were extensively discussed with the experts. The interview discussions contained both, structured questions to generally evaluate the method as a unit and semi-structured questions through which the developed gamification method and the design principles were discussed more in-depth through probing. The structured questions were based on established frameworks for the evaluation of method quality [73,74]. According to Kitchenham et al. [73], we focused the methods’ semantic quality (feasible completeness and validity), pragmatic quality (feasible comprehension and understandability) and practical utility. First we asked the interviewees if they believed that the method covers the whole gamification engineering process comprehensively (feasible completeness); second we checked whether the experts perceive the method and its statements as correct and valid (feasible validity); third we asked if the method procedures are easy to understand (feasible understandability); fourth we evaluated whether the method is presented in an understandable form (feasible comprehension); fifth we asked the experts if they would use the method in their work or for education (practical utility). Based on the transcribed interviews we comprehensively analyzed the participants’ overall agreement or disagreement with these questions and the more open discussion of the design principles, the method, and its fragments. 4.1.3. Evaluation results Overall, the experts who participated in the evaluation confirmed that the identified design principles in Table 3, approved them as best practices and agreed that they were incorporated into the developed method. Most discussions were centered around DP4 which argues that engineering gamified software should follow an iterative design process and DP7 that states that stakeholders and organizations must understand and support gamification. The experts agreed that design and development should be iterative, however, two experts highlighted that commissioned work does not always allow an iterative procedure [E8, E10] and instead, mainly recommended adherence to DP3 that advocates the testing of gamified software early. When discussing DP7, one expert highlighted that it is more important that stakeholders support gamification rather than understand it [E4]. This expert reported that they typically involve stakeholders in the design process (DP13) consequently providing an introduction to stakeholders as to what gamification is and what it is not [E4]. Two experts completely disagreed with DP7 and stated that “often the organizations don’t even want to know what will be done in detail, they are just interested in achieving specific goals with gamification” [E2; E9]. All other experts highly agreed that DP7 is important. Experts additionally agreed that it should be determined whether gamification is the appropriate design or not at the outset of a project (DP6). However, different opinions were expressed about the situation where this activity should be carried out. [E4] mentioned that in some cases the decision to gamify software is often made before a project starts hence the project is already established with that purpose. [E1] highlighted that the current position of the “assess if gamification is applicable” activity in the method is correct, but in some cases, context knowledge might be needed in order to assess if gamification is the right choice. Therefore, he noted that depending on the situation, this activity might also be carried out after the context analysis. Thus, situating the developed method to a particular project might be needed with regards to the sequencing of some procedures that are part of the method. Another expert shared the opinion that software can be gamified almost always [E2] as the creativity of the designer creates the borders and hence the expert did not perceive a need for an activity to determine whether gamification is the appropriate solution. The evaluation of the method via expert interviews revealed positive quality results regarding all three evaluated dimensions of Kitchenham et al. [73]. All interviewees confirmed that the developed method is understandable and the selected presentation format supports its comprehensibility. However, two experts highlighted that a fundamental understanding of gamification is required before following this method at hand [E4, E9]. Nine of ten experts agreed that the contributed method is complete and contains all relevant steps. Two experts criticized the method for providing little assistance in the choice of gamification elements [E4, E9]. However, as the majority of the interviews showed that in practice gamification is a creative and iterative design process, we suggest that the use of frameworks that define strict guidelines may harm the creative design process. If more support is sought, we refer to the overview of methods, tools, and frameworks collected in Table 5. Small recommendations from the experts included: i) the need to emphasize the importance of the user journey in order to invite designers to think about long-term engagement with the gamified software [E8], ii) the aspect that problems during implementation can lead to a new design iteration [E2], or iii) that budget should be considered during the ideation phase to ensure realistic ideation [E4]. In general, the artifacts received positive feedback with three experts willing to test them in their work or compare their typical proceeding with the developed method [E2, E6, E7]. Nine out of ten experts agreed that a gamification engineering project is likely to be successful using the method - assuming that it was executed correctly - and confirmed its high practical utility. 4.2. Case study evaluation of the method The method developed in this paper can be seen as a method of methods since it synthesizes prior frameworks and knowledge on gamification design that has accumulated during the past 5-7 years. This fact may not have necessitated the empirical validation of the method as it already is composed of elements and methods fragments validated in a prior stream of accumulated knowledge on designing gamification. However, the most crucial aspect that warranted a practical validation next to the expert evaluation is the need to evaluate the holistic practicality of the assemblage. 4.2.1. Case study design and objectives To conduct a practical evaluation of the method as described, we utilized it in a gamified software engineering project, that has been conducted in cooperation with a large German engineering company. In selecting the company and subsequently the case study, we aimed to identify an environment in which there was a need to develop a gamified software and allowed a research cooperation under which we could evaluate the developed method. The aim of the software development project was to develop a gamified crowdsourcing application that motivates people to share parking space information and create an interactive map of parking spots, allowing people to easily see the location and conditions of on-street parking spaces across a city. We chose this context since the sharing of parking information is a task that is not fundamentally engaging while very effective in city management and could facilitate an integral aspect of the daily life of drivers. We hence identified a need to employ motivating mechanics through gamification in order to positively influence the sharing of parking information. Consequently, the engineering of a gamified software for such a challenging context can thus easily show whether an approach to its engineering is effective or not by evaluating the extent to which it provided enough engagement for users to participate in such a service. 4.2.2. Procedures The project was conducted by an interdisciplinary team of seven members, including two designers, three programmers, one software architect, one marketing/ business development expert, and one project manager. One of the authors of this paper was involved in the project to examine the applicability of the method. According to the contributed gamification engineering method, the team started with the preparation phase. The team carried out a workshop with three stakeholders of the engineering company (two business development experts and a user-experience expert), two car drivers and a local retailer. In this workshop, the team has identified, prioritized and justified the project objectives. Subsequently, the team developed a project plan including an estimated budget, an overview of the required skills in the development team, and an initial milestone list. Further, the team clarified the requirements of personal data collection, operational success criteria, the possible use of open source tools and libraries and the possible employment of the app by the stakeholders of the case company. A closer look at the company culture was not relevant for the project. Next, the team conducted the context and user analysis phase. In that phase, the design context of on-street parking mapping in collaboration with city-planners of several large German cities was carefully analyzed. Existing solutions were analyzed by reviewing 13 existing parking apps and services, as well as by a survey of 117 potential users of the app. This investigation of the gamification context assisted in the identification of different user groups and in the characterization of potential contributors that might share parking information in the app (see e.g. [17,75] on gamified crowdsourcing). Further, the team conducted user surveys and interviews with focus groups to understand potential users of the app and in particular their needs and motivations. Based on the insights from the user and context analysis, the team developed four personas (altruists, drivers, gamers, tinkerer). As part of the ideation phase, the team conducted several workshop sessions. The playing and analysis of especially context-related board and video games, such as Monopoly® or SimCity, along with the examination of typical gamification design patterns helped the team to come up with a list of gamification ideas. Based on the consolidated ideas, the team iteratively developed prototypes following an agile scrum approach. In the first iteration, the team developed and evaluated a paper prototype with printouts of maps, post-its, paper money and lego-figures. Next, the team developed several sets of wireframes and an initial app prototype using HTML and JavaScript.5 Finally, the project was transferred to a professional team of game developers, who are currently working on a market-ready pilot of the app.6 According to the initially identified operational requirements for the application, the developed prototypes were tested regularly: first through a small test group with 10 users and qualitative feedback, and afterward through a larger field test. 4.2.3. Analysis and findings Overall, we found that the method developed through this research showed great promise in the gamification software project it was implemented in. The developed gamified application was successful in meeting its development objectives of creating a map of parking information through crowdsourcing. In a three-month period, a total of 372 users in several large German cities used the gamification application developed. Parking information (e.g. price information, location, restrictions) for more than 7000 street segments have been shared by the users of the app, which indicates that the proposed method indeed supported us to develop functional gamified software.5 With regards to the practicality, order, and structure of the method, the development team along with the business partners experienced a sense of ease and understanding of what is to take place next with regards to the management of the project. The procedures of the method were followed in order and a need was not uncovered to change the structure or flow of the method. The only exception being that at the first stages of the project and utilization of the method, it was already determined that gamification was a suitable solution as that was the criteria for selecting the case study. That meant that in line with some experts, activity parts of stage one of the method could take place before the start of the project or the utilization of the method. Holistically, the expert interviews and the practical evaluation of the method indicate that the method is useful, easy to use and implement in practice as well as that the assemblage and organization of the method can relevantly and appropriately guide a gamification software projects stages and proceedings. 4.3. Threats to validity The main objective of the present study was to bring validity, reliability, and rigidity to the emerging and sprawling discussion on how gamification should be designed and implemented. In order to rigorously ground this pursuit of unified knowledge in the field, this study conducted the most comprehensive review of prior design frameworks as well as interviewed recognized experts of the area. Through careful processes of qualitative analysis and design science (see Figure 1), a gamification engineering method was produced which combines aspects of the prior discussions that have been regarded as the most important aspects of gamification design. The method was further evaluated through both its use in a real gamification initiative as well as through more expert interviews. However, it is also important to assess the validity and reliability of the present study; its research procedures and results [76]. The study consisted of four points of gathering data (before assembling the method: 1. expert interviews and 2. systematic literature search as well as after assembling the method; 3. expert interviews and 4. a case study of applying the method in a real-world gamification design scenario) and five points of separable data analysis portions (A. analysis of expert interview data before the 5 See http://parking-app.de for more information about the first app prototype and its field test. 6 See https://parkineers.com more information about the pilot. [59] P. Herzig, M. Ameling, B. Wolf, A. Schill, Implementing gamification: Requirements and gamification platforms, in: T. Reiners, L.C. Wood (Eds.), Gamification Educ. Bus., Springer International Publishing, Cham, 2015: pp. 431–450. doi:10.1007/978-3-31910208-5_22. [60] K.M. Kapp, The gamification of learning and instruction: game-based methods and strategies for training and education, Pfeiffer, San Francisco, CA, 2012. [61] M. Klevers, M. Sailer, W.A. Günthner, Implementation model for the gamification of business processes A study from the field of material handling, in: Proceeding 46th Int. Simul. Gaming Assoc. Conf., Kyoto, Japan, 2015. doi:10.13140/RG.2.1.2313.2006. [62] R. Schmidt, C. Brosius, K. Herrmanny, Ein Vorgehensmodell für angewandte Spielformen, HMD Prax. Der Wirtschaftsinformatik. 52 (2015) 826–839. doi:10.1365/s40702-015-0180-y. [63] M.D. Myers, M. Newman, The qualitative interview in IS research: Examining the craft, Inf. Organ. 17 (2007) 2–26. doi:10.1016/j.infoandorg.2006.11.001. [64] E.N. Webb, Gamification: When it works, when it doesn’t, in: Proc. 2nd Int. Conf. Des. User Exp. Usability(DUXU), Springer, Las Vegas, NV, USA, 2013: pp. 608–614. doi:10.1007/978-3-642-39241-2_67. [65] K. Knaving, S. Björk, Designing for fun and play, in: Proc. 1st Int. Conf. Gameful Des. Res. Appl., ACM Press, Stratford, ON, Canada, 2013: pp. 131–134. doi:10.1145/2583008.2583032. [66] R. Bartle, Hearts, clubs, diamonds, spades: Players who suit MUDs, J. MUD Res. 1 (1996) 19. [67] B. Morschheuser, C. Hrach, R. Alt, C. Lefanczyk, Gamifizierung mit BPMN, HMD Prax. Der Wirtschaftsinformatik. 52 (2015) 840–850. doi:10.1365/s40702-015-0188-3. [68] S. Nicholson, A user-centered theoretical framework for meaningful gamification, in: Proc. Games+Learning+Society 8.0, Madison, WI, 2012: pp. 223–229. [69] J. Arrasvuori, M. Boberg, J. Holopainen, H. Korhonen, A. Lucero, M. Montola, Applying the PLEX framework in designing for playfulness, in: Proc. 2011 Int. Conf. Des. Pleasurable Prod. Interfaces, ACM Press, Milano, IT, 2011. doi:10.1145/2347504.2347531. [70] C.M. Finneran, P. Zhang, A person–artefact–task (PAT) model of flow antecedents in computer-mediated environments, Int. J. Hum. Comput. Stud. 59 (2003) 475–496. doi:10.1016/S1071-5819(03)00112-5. [71] J. Koivisto, J. Hamari, Demographic differences in perceived benefits from gamification, Comput. Human Behav. 35 (2014) 179–188. doi:10.1016/j.chb.2014.03.007. [72] P. Runeson, M. Höst, Guidelines for conducting and reporting case study research in software engineering, Empir. Softw. Eng. 14 (2009), 131. doi:10.1007/s10664-008-91028. [73] B. Kitchenham, S. Linkman, S. Linkman, Experiences of using an evaluation framework, Inf. Softw. Technol. 47 (2005) 761–774. doi:10.1016/j.infsof.2005.01.001. [74] O. Lindland, G. Sindre, A. Solvberg, Understanding quality in conceptual modeling, IEEE Softw. 11 (1994) 42–49. doi:10.1109/52.268955. [75] B. Morschheuser, J. Hamari, J. Koivisto, A. Maedche, Gamified crowdsourcing: Conceptualization, literature review, and future agenda, Int. J. Comput. Stud. 106 (2017), 26–43. doi:10.1016/j.ijhcs.2017.04.005. [76] C. Wohlin, P. Runeson, M. Höst, M.C. Ohlsson, B. Regnell, A. Wesslén, Experimentation in software engineering, Kluwer Academic Publishers, 2002. [77] T. Diefenbach, Are case studies more than sophisticated storytelling?: Methodological problems of qualitative empirical research mainly based on semi-structured interviews, Qual Quant 43 (2009), 875. doi: 10.1007/s11135-008-9164-0. [78] U. Schultze, M. Avital, Designing interviews to generate rich data for information systems research, Inf. Organ. 21 2011, 1–16. doi: 10.1016/j.infoandorg.2010.11.001. [79] C. Cassell, G. Symon, G. Essential guide to qualitative methods in organizational research, Sage, 2004. Appendix A: The full method as process-deliverable-diagram before evaluation LIST OF OBJECTIVES PROJECT CONDITIONS Project preparation Context analysis Ideation Design Evaluation Identify context Create protoype Evaluate prototype Design prototype Evaluate success Plan development DESIGN CONCEPT PROTOTYPE PROTOTYPE EVALUATION DEVELOPMENT CONCEPT SUCCESS EVALUATION Concept goal reached? GAMIFIED PRODUCT DESIGN DOCUMENT PROJECT PLAN - Objectives - Project variables PROJECT ANALYSIS - Context characteristics - Design C oncept - Prototypes User analysis Implementation SET OF CONCEPTS - Sketches - Wireframes Define target users Identify user needs Identify user motivations Understand the context Identify and list objectives Justify objectives Rank objectives Identify requirements PERSONAS / SEGMENTATION - Interests - Preferences - Player type - Behavior - Needs - Motivation - Age - Gender - Job level Define success metrics SUCCESS METRICS - mesurement method - KPIs Decide implementation TYPE OF IMPLEMENTATION Adapt design to platform - In-house development - Available platform Prepare in-house development Prepare external development Advise and manage implementation Implement gamification feature Play testing Release project Pilot Brainstorm ideas Consolidate ideas LIST OF CONSOLIDATED IDEAS LIST OF IDEAS - Budget - Legal & ethical matters - Success criterion - Rank - Justification Monitoring and management IMPLEMENTED PRODUCT Assess if gamification is applicable GO DECISION Create personas - Storyboards - Quantitative results - Qualitative results GAMIFICATION FEATURES FEEDBACK - Interview results - Survey results - Observation results - External development ADAPTED DESIGN - KPIs - Measuring method - Technology infrastructure - ... Go decision? no yes no yes Not finished finished no yes Monitoring requested? In-house/external/ platform? external In-house platform Target achieved? Gamification expert/ Domain expert/ Business expert Gamification expert/ Domain expert/ Business expert Gamification expert/ Domain expert/End users Gamification expert/ Business expert/ Domain expert/End user Gamification expert/ IT expert/ Domain expert/ Business expert IT expert/ Gamification expert/ Domain expert Gamification expert/ Domain expert/ Business expert/ End users - Development budget estimation - Specification sheet - Development concept - Desired actions - Deadline - Company culture Gamification expert Identify user needs [Simple] - Activities - Human resources - Constraints Define target users Gamification expert/End User List of ideas LIST OF IMPROVEMENTS CONTEXT CHARACTERISTICS - Libraries - Processes - Architecture - Platform - Technology constraints - ... - Personas Monitoring Business expert/ Marketing expert Is it possible to conduct a comprehensive user analysis? B: Activities table Phase Activity Description Project preparation Identify and list objectives Identify all the objectives of all involved stakeholder and list all of them in a LIST OF OBJECTIVES. Rank objectives The objectives have to be ranked and prioritized in LIST OF OBJECTIVES, as not all objectives can be reached in one project. Justify objectives To have a clear understanding of what the objectives are and how they benefit the organization and the stakeholders the objectives should be justified in LIST OF OBJECTIVES. Assess gamification applicability With the ranked and justified objectives, it can be assessed and documented in the GO DECISION document, whether Gamification is suitable for the project or another concept would be better. Identify requirements All the necessary PROJECT CONDITIONS have to be identified and documented. Context analysis Identify context Identify the context of the project and document the identified context in CONTEXT CHARACTERISTICS. Understand context Conduct further research to understand the context in greater depth and elaborate more details within CONTEXT CHARACTERISTICS. Define success metrics The metrics which will be used to measure the success of the project have to be defined as part of the CONTEXT CHARACTERISTICS. User analysis Define target users The target users have to be identified and information about them has to be gathered in PERSONAS / SEGMENTATION. Identify user needs The user needs and objectives should be identified to enrich the PERSONAS / SEGMENTATION with this information. Identify user motivations After identifying the needs of the users, it is important to translate the needs in user motivations and to specify them in PERSONAS / SEGMENTATION. Create personas The identified information is used to create PERSONAS / SEGMENTATION which include all necessary information to create an engaging design. Ideation Brainstorm ideas The identified information provides the basis for the brainstorming of ideas that are documented in LIST OF IDEAS. The identified frameworks and tools can help to create concepts for gamification. Consolidate ideas The initial broad LIST OF IDEAS has to be condensed to a reduced and possible prioritized LIST OF CONSOLIDATED IDEAS, which will be designed and evaluated. Besides cost and usefulness estimations, the fit between the idea and the user and context characteristics should be considered when consolidating the ideas. Design Create user journey Develops a USER JOURNEY in order to plan the long-term engagement with the gamified software and to be able to derive precise requirements. Design prototype The consolidated and selected ideas from the ideation phase have to be conceptualized in DESIGN CONCEPT (e.g. by through initial mock-ups and wireframes). Create prototype The DESIGN CONCEPTS form the bases for further development of (playable) PROTOTYPES (e.g. paper prototypes, app prototypes, gamification plugins). Evaluate prototype The PROTOTYPE should be evaluated using playtesting with actual users so feedback can be gathered resulting in PROTOTYP EVALUATION. Plan development When the design is fixed the DEVELOPMENT CONCEPT has to be created which includes the specification of the implementation, budget and possibly other project management information for the implementation. Implementation Decide implementation Before the implementation, a decision about the TYPE OF IMPLEMENTATION has to be made. The implementation can be developed externally, inhouse, or by utilizing an available platform. Prepare development Depending on the prior decision, further preparation needs to be conducted (e.g. granting permission to use the APIs and information of the company or acquiring additional knowledge about gamification and its application in the specific context). Advise and manage implementation The gamification expert has to check and advise the implementation to ensure that the gamification is done right. Implement design In several cycles, the implementation of the design leads to developed GAMIFICATION FEATURES. Playtesting The GAMIFICATION FEATURES are created in cycles with playtesting after each cycle to check if the desired results are achieved by the developed gamification elements. In this activity, FEEDBACK needs to be collected for evaluating the effectiveness and functionality of the developed GAMIFICATION FEATURES. Pilot When the development of GAMIFICATION FEATURES is finished, the finale gamified software will be piloted with a small group of users. If successful, this step leads to a GAMIFIED PRODUCT. Evaluate success In order to check whether the initially defined objectives are met by the GAMIFIED PRODUCT, a success evaluation is conducted. This evaluation using qualitative or quantitative methods leads to SUCCESS EVALUATION. Release project The project can be released if the evaluation was successful. There are also different ways to release the project like "Big Bang" or gradually expand the pilot. Monitoring and management After the project is released it should be monitored and also re-designed if necessary to attend for possible future changes that are captured in a LIST OF IMPROVEMENTS. C: Deliverables table Concept Description LIST OF OBJECTIVES List of ranked and justified project objectives. GO-DECISION Decision if gamification is fitting or the project should be aborted. PROJECT CONDITIONS Important project conditions along legal & ethical, budget, deadline, desired actions, company culture, human resources, constraints, and success criterion. PROJECT PLAN A detailed plan for the gamification project that summarizes the objectives of the project and identified project conditions. The project plan can be used to communicate the vision of the project to the project stakeholder. CONTEXT CHARACTERISTICS Detailed overview of the context to be gamified, including technical characteristics and limitations of the context, as well as a clear understanding of how people behave in this context. PERSONAS / SEGMENTATION Artificial character that represents a user group with specific attributes, such as age, gender, activities, job level, motivation, needs, interests, preferences, player type, behavior. SUCCESS METRICS Documents different key performance indicators (KPIs) and their measurement method. LIST OF IDEAS Broad list of ideas for gamification design features, including user interaction elements, rules, mechanics, narrations. CONSOLIDATED IDEAS List with selected ideas for further design. PROJECT ANALYSIS Synthesizes and analysis CONTEXT CHARACTERISTICS, SUCCESS METRICS, and PERONAS / SEGMENTATION. SET OF CONCEPTS Aggregates the LIST OF IDEAS and the CONSOLIDATES LIST OF IDEAS in one document. USER JOURNEY Documents the long-term user engagement from onboarding to mastery. Often in combination with a visualization of flow curves. DESIGN CONCEPT Specific concept for gamification which can be turned into (playable) prototype. PROTOTYPE Prototype as an early sample which can range from simple mockups till more complex prototypes that are used for early playtesting. PROTOTYPE EVALUATION Evaluation results of playtesting with the prototype with feedback on the usage and experience. DEVELOPMENT CONCEPT Document with information for the development of the product including a specification sheet and development budget. DESIGN DOCUMENT Summarizes the DESIGN CONCEPT and the DEVELOPMENT CONCEPT. Further, it includes wireframes, prototypical implementations and the documentation of the USER JOURNEY. TYPE OF IMPLEMENTATION Selection of the implementation type (external development, inhouse development, or based on a platform) that will be used in the project. ADAPTED DESIGN An adapted version of the design for the considered platform. GAMIFICATION FEATURES Parts of a gamified product with enough functionality to be tested and which will be improved in cycles. FEEDBACK Feedback of the playtesting session from actual users. IMPLEMENTED PRODUCT A releasable product, pilot or a minimum viable product that could be used and evaluated by potential users. The implemented product is gamified and thus contain several iteratively developed GAMIFICATION FEATURES. GAMIFIED PRODUCT End result of the project. SUCCESS EVALUATION Evaluation results of the gamified product which will determine the further course of the project. LIST OF IMPROVEMENTS List of improvements in the form of adaptation parameters or features for future releases. D: The full method as process-deliverable-diagram with feedback from the evaluation LIST OF OBJECTIVES PROJECT CONDITIONS Project preparation Context analysis Ideation Design Evaluation Identify context Create protoype Evaluate prototype Design prototype Evaluate success Plan development DESIGN CONCEPT PROTOTYPE PROTOTYPE EVALUATION DEVELOPMENT CONCEPT SUCCESS EVALUATION Concept goal reached? Create user journey USER JOURNEYS GAMIFIED PRODUCT DESIGN DOCUMENT PROJECT PLAN - Objectives - Project variables PROJECT ANALYSIS - Context characteristics - Design C oncept - Prototypes User analysis Implementation SET OF CONCEPTS - Sketches - Wireframes Define target users Identify user needs Identify user motivations Understand the context Identify and list objectives Justify objectives Rank objectives Identify requirements PERSONAS / SEGMENTATION - Interests - Preferences - Player type - Behavior - Needs - Motivation - Age - Gender - Job level Define success metrics SUCCESS METRICS - mesurement method - KPIs Decide implementation TYPE OF IMPLEMENTATION Adapt design to platform - In-house development - Available platform Prepare in-house development Prepare external development Advise and manage implementation Implement gamification feature Play testing Release project Pilot Brainstorm ideas Consolidate ideas LIST OF CONSOLIDATED IDEAS LIST OF IDEAS - Budget - Legal & ethical matters - Success criterion - Rank - Justification Monitoring and management IMPLEMENTED PRODUCT Assess if gamification is applicable GO DECISION Create personas - Storyboards - Quantitative results - Qualitative results GAMIFICATION FEATURES FEEDBACK - Interview results - Survey results - Observation results - External development ADAPTED DESIGN - KPIs - Measuring method - Technology infrastructure - ... Go decision? no yes no yes Not finished finished no yes Monitoring requested? In-house/external/ platform? external In-house platform Target achieved? Gamification expert/ Domain expert/ Business expert Gamification expert/ Domain expert/ Business expert Gamification expert/ Domain expert/End users Gamification expert/ Business expert/ Domain expert/End user Gamification expert/ IT expert/ Domain expert/ Business expert IT expert/ Gamification expert/ Domain expert Gamification expert/ Domain expert/ Business expert/ End users - Development budget estimation - Specification sheet - Development concept - Onboarding - Engagement loops - Desired actions - Deadline - Company culture Gamification expert Identify user needs [Simple] - Activities - Human resources - Constraints Define target users - In & Out conditions Evaluation successful Project abortion Re-Implementation necessary Gamification expert/End User List of ideas LIST OF IMPROVEMENTS CONTEXT CHARACTERISTICS - Libraries - Processes - Architecture - Platform - Technology constraints - ... - Personas New design interation needed. Monitoring Business expert/ Marketing expert Is it possible to conduct a comprehensive user analysis?