Lean User Research for Agile Organizations

Received June 8, 2020, accepted June 29, 2020, date of publication July 14, 2020, date of current version July 24, 2020. Digital Object Identifier 10.1109/ACCESS.2020.3009101 Lean User Research for Agile Organizations EVA-MARIA SCHÖN 1, JÖRG THOMASCHEWSKI2, AND MARÍA JOSÉ ESCALONA3 1Department Business, Faculty of Business and Social Sciences, Hamburg University of Applied Sciences (HAW), 20099 Hamburg, Germany 2Faculty of Technology, University of Applied Sciences Emden/Leer, 26723 Emden, Germany 3Department of Computer Languages and Systems, University of Seville, 41012 Sevilla, Spain Corresponding author: Eva-Maria Schön ([email protected]) This work was supported in part by the POLOLAS Project of the Spanish Ministry of Economy and Competitiveness under Grant TIN2016-76956-C3-2-R, and in part by the NICO Project of the Spanish Ministry of Science and Innovation under Grant PID2019-105455GB-C32. ABSTRACT In the context of Digital Transformation, organizations need to react in a flexible manner to rapidly changing markets and disruptive technologies. Customer and user satisfaction are important in order to succeed. Agile User Experience (UX) Design and Lean User Research enable the organization to focus on the needs of its customers and users. The objective of this paper is to investigate the best practices that exist for user research in agile product development. In particular, we want to answer the following question: What impact does Lean User Research have and how is it conducted in industry? We have carried out several studies to answer our questions. First, we created the pattern Lean User Research by means of a systematic pattern mining process. Next, we gathered quantitative data by means of a questionnaire study. This was followed by the collection of qualitative data in a panel discussion with experts from academia and industry. The results reveal that Agile has an impact on how user research is conducted because of the speed of agile product development. In addition, we provide best practices of Lean User Research and show how they are applied in industry by giving practical examples. Moreover, we found that the UX community values the aspects of Lean User Research as important, but do not constantly apply it in their practical work. We can conclude that people have to experiment and adapt the best practices of Lean User Research to their special contexts as well as develop an agile mindset. INDEX TERMS Agile software development, human-computer interaction, requirements engineering, user experience, user research. I. INTRODUCTION Digital Transformation has an impact on the design of an organization. Organizations must adapt in a flexible manner to rapidly changing markets as well as to disruptive technologies. To this end, agility becomes an important asset of an organization. In the light of this, organizations often apply Agile Software Development (ASD) by means of using agile methodologies like Scrum [1], Kanban [2], or Extreme Programming [3]. The survey State of Agile [4] reports that the most important benefits of adopting Agile are the management of changing priorities, project visibility, business/IT alignment, delivery speed, time to market, and team productivity. The survey State of Agile further indicates that customer and user satisfaction is one of the most important criteria for measuring success within an organization. The associate editor coordinating the review of this manuscript and approving it for publication was Orazio Gambino . This leads to the following question: how can customer and user satisfaction can be measured, or, basically, what is it? The research field of Human-Computer Interaction (HCI) has been investigating such questions for decades [5], [6]. In this context, the User Experience (UX) was defined as a ‘‘person’s perceptions and responses resulting from the use and/or anticipated use of a product, system or service’’ [7]. During the past few years, new research fields have emerged because of the integration of Agile and UX. The fields of agile UX design [8]–[10] as well as agile Requirements Engineering (RE) [11], [12], [13] study customer and user satisfaction in the context of Agile. In their paper, da Silva et al. [14] outline the evolution and current state of agile UX design. They found that processes and practices, which are used in terms of agile product development, do not separate Agile and UX, whereas the people and social dimension currently crosses this separation of the two fields. Moreover, da Silva et al. [14] state that there are VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ 129763 E.-M. Schön et al.: Lean User Research for Agile Organizations still some gaps in research related to technology and artifacts, meaning that there is no full understanding of the needs in respect of the integration of Agile and UX, and there is still room for improvement. Lean is a good practice for agile organizations. An essential part of Lean is measurement. Based on measurements, the next steps are defined and optimizations are implemented. In Human-Centered Design (HCD) [7], user research is very important for measurements because user research aims to figure out who the users and customers are as well as examine their usage behavior. The goal of user research is to increase knowledge about users, their needs, and their usage behavior. Therefore, different techniques like qualitative and quantitative research and evaluation methods are applied [15], [16]. User research is a complex field of knowledge and not everything is appropriate for use in ASD. The demands that Agile has on the speed of product development also affect the way user research is conducted. To this end, we will narrow down the term Lean User Research in order to make it clear that we investigate user research in an agile environment. However, for the application in ASD, we need a lean user research. That is why we have conducted several studies to investigate best practices in the industry. We now present our results in our pattern Lean User Research (see Table 3), since patterns are well suited to describe best practices. By applying the proposed best practices, wasteful activities are reduced to promote those activities which are valuable to the environment. According to the Economic Cost model for Lean software development of Anderson [2], we want to show with our pattern Lean User Research how organizations can reduce wasteful activities with regard to user research. Our pattern includes best practices that enable user research in an agile environment. We consider human and social components as well as artifacts and processes. For example, established methods of user research are adapted to the agile timing in order to gather faster feedback from users and customers. Applying Lean User Research helps the agile team continuously increase its own knowledge about users and the context of use of the product. In addition, short feedback cycles enable a quick reaction to changing requirements, which are recognized at a late stage using established methods. In this article, we aim to investigate the best practices that exist for user research in agile product development. Hence, this article will address the following research question: •RQ1: What impact does Lean User Research have in the industry? •RQ2: How is Lean User Research applied in industry? The paper is structured as follows: Section II outlines the related work and defines an Agile Organization. Section III presents our research objectives and research questions and deals with our iterative research approach, covering quantitative as well as qualitative data. Section IV summarizes the key findings of our study, and, so, offers an overview of the pattern Lean User Research. It also reports the findings from our empirical studies. Finally, Section V discusses the meaning of the findings and the limitations of this study. II. BACKGROUND AND RELATED WORK Every organization must be flexible in today’s world, on account of rapidly changing markets and disruptive technologies. Narayan [17] distinguishes between business agility and IT agility; the former serves the market, whereas the latter serves the business. Moreover, he states that Digital Transformation is lot more dependent on Agile Transformation of an organization, as it seems. Based on the concept of business agility and IT agility, we created an overview of how an Agile Organization could be designed in the era of digital transformation (see Figure 1). FIGURE 1. Digital Transformation enabled by Agile Transformation of the organization. Figure 1 shows that in an Agile Organization, people engaged in the business work closely together with the IT people in order to deliver outcome-oriented products to the markets that the organization serves. Outcome-oriented product development is focused on the value that the product delivers to its users and customers instead of focusing on quantitative measures (e.g. number of developed features, project plans). Therefore, teams are equipped in such a manner that they can deliver business outcomes instead of performing single activities, such as RE, programming, or testing. People in the organization are constantly improving themselves. On the one hand, they improve the product, and, on the other, they optimize the collaboration and the product delivery process. The agile product development is divided into the following parts: product discovery and product delivery, which have an interplay (see Figure 1). Product discovery comes up with new ideas and defines the right product [18]. It examines what the agile teams should deliver. For this, approaches such as agile UX design [14], Lean UX [19], or Lean User Research [20] are applied to product discovery. People develop hypotheses and test them in experiments to find out whether they are going in the right direction with product development. In comparison, the product delivery is about building the product right. It focuses on the way ‘‘how’’ the ‘‘what’’ is delivered. People strive for technical excellence using engineering practices like continuous delivery [21] 129764 VOLUME 8, 2020 E.-M. Schön et al.: Lean User Research for Agile Organizations and applying agile approaches like Scrum [1], Kanban [22], or DevOps [23]. Some studies can be found in connection with user research in agile product development. For instance, Bertholdo et al. [24] describe that there is little time for user research in agile product development and, so meetings with users are used for various activities like usability testing or requirements analysis. In addition, they recommend conducting collaborative and participative design with users. Therefore, it is beneficial to have a contact plan of users consisting of a list of users who meet the product’s profile, their contact data, and the possibilities of participation. Moreover, Kautz [25] investigated how customers and users participate in agile product development with participatory design. He reports that customers and users were involved indirectly and directly through different kinds of activities. Additionally, the role of an onsite customer was applied in order to have frequent feedback loops. Bellucci et al. [26] explored in a field study how users interact and work with a prototype. Hence, they combined Extreme Programming [3] with co-design sessions in order to develop a product with strong user involvement. Olsson and Bosch [27] created a conceptual model that emphasized the need for combining qualitative user feedback in the early stages of development with quantitative user observation in the later stages of product development. Although research has already been done on the subject of user research in agile product development, there are still many problems concerning the direct involvement of users and stakeholders in agile product development [13], [28], [29]. However, the approaches presented by related literature oftentimes show concrete applications in a specific context. They describe a combination of specific artifacts, methods, and procedures. Our objective with the pattern Lean User Research is to generalize these cases in order to achieve best practices without looking at a particular case by describing a problem and the corresponding solution on a meta-level. III. RESEARCH METHOD This paper aims to study the way product discovery is performed in the context of an Agile Organization. In particular, we will investigate how Lean User Research is conducted in agile product development as well as how the UX community in the DACH region (DACH refers to the territory of Germany, Austria, and Switzerland) performs it today. Based on the aforementioned objective, we formulate two research questions (RQs): •RQ1: What impact does Lean User Research have in the industry? •RQ2: How is Lean User Research applied in industry? In order to reach our goal, we have carried out empirical studies (see Figure 2). The benefit of this approach is that the learning from each step allows us to achieve an overall picture of the current status quo. FIGURE 2. Overview of research approach. Step 1 allows us to define our understanding of Lean User Research in product discovery (see Figure 1) by means of creating the pattern Lean User Research (see Figure 2). In addition, Step 2 and Step 3 comprise empirical studies within the UX community of the DACH region, which help us answer our RQs. A. STEP 1: SYSTEMATIC PATTERN MINING PROCESS A pattern is a reusable solution to a commonly recurring problem [30]. It does not state a concrete solution, but rather provides a description of best practices on how to solve a problem. Therefore, the solution is described at a high level, and needs to be adapted to the environment to which the pattern is applied. In a first step, we created the pattern Lean User Research. The pattern Lean User Research was discovered by means of a systematic pattern mining process (see Figure 3 and [31]). The systematic pattern mining process was conducted as part of a PhD thesis in respect of agile RE [32]. The study comprised three phases (see Figure 3): 1) We started with a systematic literature review (SLR), with which we identified agile practices. The detailed results can be found in [13]. The agile practices are used for stakeholder and user involvement, data gathering, integrating HCD and ASD, building shared understanding, requirements management, and documentation of requirements or non-functional requirements. In sum, we found 27 relevant papers and analyzed the data. The studies often investigated agile RE in real life context and, therefore, the approaches are very close to existing work practices in companies. In respect of our pattern, the identified practices are used as solutions to the pattern problem, as they represent best practices. FIGURE 3. Overview of systematic pattern mining process. VOLUME 8, 2020 129765 E.-M. Schön et al.: Lean User Research for Agile Organizations 2) Subsequent to this, we conducted an empirical evaluation with the aim to identify the most important problems companies face today in terms of agile RE. The detailed results can be found in [32]. The panel of our Delphi study consisted of 26 experts in agile product development. The experts work in 19 different organizations in Germany and Switzerland. The organizations are heterogeneous in terms of size (e.g. freelancers to large-scale companies), industry (e.g. e-commerce, consulting, cybersecurity, publishing, finance), and business model (e.g. service providers or product manufacturers). This heterogeneity leads to different perspectives being incorporated into the identification of the problems and has the advantage that the results of the study can be transferred to many organizations. In three consecutive rounds, we surveyed the experts on the most important problems in dealing with requirements in agile product development. A total of 20 problems were identified. Six of these problems have been rated by the experts as very important. In addition, we asked for possible solutions to these important problems. The experts stated the agile practices with which they solve the problems within their organization. 3) A mapping between problems and agile practices was then carried out to solve the problems. Hence, we distinguished between problem domain and solution domain according to [33]. The mapping process was carried out in several iterations. First, experts from our Delphi study made the first mapping between problem domain and solution domain during the empirical study (see Phase 2). We then mapped the identified agile practices known from our systematic literature review (see Phase 1) to the agile RE problems. Afterward, we carried out a cross check with agile practices proposed by agile methodologies (e.g. Scrum, XP, or Kanban) and related patterns. As a result, we discovered 41 agile RE patterns. The detailed results can be found in [31]. For the development of our patterns we have oriented ourselves with respect to established guidelines [33]. The pattern Lean User Research was one out of a catalog of 41 patterns. We refined the pattern in discussions with experts from industry and academia. For this purpose, we conducted a panel discussion (see III.C. and [20]). Table 3 shows the refined pattern. Moreover, Section IV.A outlines an application of the best practices in industry. B. STEP 2: QUESTIONNAIRE STUDY As a second step, we gathered quantitative data after conducting a questionnaire study among the UX community of the DACH region. Therefore, we introduced the pattern Lean User Research (see Table 3) in a session during a UX barcamp (UX Camp Hamburg, Germany, August 11, 2018). We asked the participants of this session to fill out an online survey, FIGURE 4. Practical experience with agile process models. which we set up using Google forms. For the development of our questionnaire, we used appropriate guidelines [34], [35]. In addition, we conducted pretests with three participants using the think aloud method. In sum, the questionnaire was composed of 16 items. Six of those items queried socio-demographic data (see Appendix, Table 5), whereas five item pairs analyzed the experts’ experience with Lean User Research. On the one hand, experts were asked how they rate a specific aspect of the solution of the pattern (see solution a-e in Table 3). On the other hand, we asked whether they applied this in their work. An example of an item pair which queried the experts’ experience is shown in Table 1. TABLE 1. Exemplary item pair of the questionnaire. We decided to use a 7-point Likert scale of items since this has been proven to be the best choice in terms of avoiding interpolations within related research fields [35]. The participants in the survey had between 0 to 10 years (mean 3.06) of practical experience with agile process models (see Figure 4). In addition, they had 0 to 20 years (mean 5.25) experience with User Research/UX Design (see Figure 5). Of the 32 participants (N=32), 22 work in the private sector, four in the public sector while six are self-employed. Figure 6 shows the process models which the participants have worked with so far. It is obvious that more than half the participants have experience with non-agile process models like the Waterfall model [36], which often follow a plandriven approach [13]. Otherwise, Scrum and Kanban are the most frequently used agile process models among the participants. 129766 VOLUME 8, 2020 E.-M. Schön et al.: Lean User Research for Agile Organizations FIGURE 5. Practical experience with User Research/UX design. FIGURE 6. Process models worked with by participants. C. STEP 3: PANEL DISCUSSION In the third step, we gathered qualitative data by means of a panel discussion with experts from academia and industry. The panel discussion was part of one of the largest HCI conferences in Europe (Mensch und Computer 2018, 2–5 September 2018, Dresden, Germany). The panel consisted of four experts from industry and academia, who represented different perspectives and functions (see Table 2). All the experts have experience with sequential approaches (e.g. Waterfall model) and with agile approaches (e.g. Scrum or Kanban). In the forefront of the panel discussion, the experts received a preparation package. This package included an introduction to the topic and a presentation of our pattern Lean User Research using a presentation and a written description [20]. Moreover, we held a meeting with all experts to clarify open questions and expectations. The experts then prepared statements on each aspect of the solution of the pattern (see solution a-e in Table 3) based on their experience. These statements were made available to the moderator before the panel discussion. During the panel discussion the experts discussed how UX design and user research can be integrated in agile product development. Moreover, they reported on how they apply the pattern Lean User Research (see Table 3) in their daily work by means of presenting best practices on agile UX and Lean User Research. The discussion was led by a moderator (first author of this paper) so that the results could be interpreted directly and fed back into the discussion. For this purpose, the single aspects of the solution of the pattern (see solution a-e in Table 3) were addressed. The 55 participants in the panel TABLE 2. Perspectives and experience of the experts. FIGURE 7. Example of a sketch note in respect of the discussion adaption of user research. discussion were able to ask questions of the experts in order to interact with the panel. In order to document the results, sketch notes were created (see example in Figure 7). These sketch notes summarize the most important statements of the experts. For the analysis we have backed up the statements of the experts as far as possible with further literature (see Section IV.C). IV. RESULTS This section describes the results of our research approach and shows the findings in respect of the UX community. VOLUME 8, 2020 129767 E.-M. Schön et al.: Lean User Research for Agile Organizations First of all, we will present the pattern Lean User Research (see Table 3). Moreover, we will illustrate a context-related application of the pattern in order to demonstrate how the pattern is applied in industry. Secondly, we outline the results of our questionnaire study, where we queried the impact and the application of the pattern in the UX community of the DACH region. Finally, we present the most important findings of our panel discussion. A. PATTERN LEAN USER RESEARCH A pattern describes a problem and the corresponding solution on a meta level. To apply a pattern in the field, the context in which the product development takes place must be considered. The best practices are then applied to the identified context. This results in a context-related application of the TABLE 3. Pattern lean user research. pattern. Our pattern can, therefore, be used together with any process model. It is not limited to agile process models but is optimized for this application. For the development of our pattern, we have used suitable guidelines [33]. We have developed a pattern template and refined it with the pattern community [32]. The development process of the pattern is described in Section III.A. Table 3 shows the pattern Lean User Research. Subsequently, a context-related application of the pattern is presented using the example of DATEV eG (a large-scale provider of software solutions in financial services, located in Germany) to illustrate the application of the described best practices in industry.The application of the concrete pattern above in the practical approach of user research and HCD describes procedures that are already in operation at DATEV. a) ADAPTION OF USER RESEARCH Qualitative evaluation procedures are supported by quantitative measurement methods [16]. Testing sessions provide the agile team with Key Performance Indicators (KPIs) for orientation on the basis of user behavior. The effort required for evaluation and report generation is reduced by abandoning classic reports. The evaluation is done by the agile team directly after each session. To this end, teams that are new to agile and user research are supported by an experienced user researcher. The duration of this coaching depends on the skills of the agile team in terms of HCD. The aim is to build up basic skills in user research among the whole team. b) AVAILABILITY OF PARTICIPANTS The recruitment of participants is supported by panels or a pre-recruitment for several user research testing sessions. The testing sessions are carried out in parallel to product delivery (e.g. at the end of each sprint). c) SELF-ORGANIZATION Agile teams are enabled to create guidelines based on user stories and to evaluate test sessions themselves under the guidance of a neutral person. Furthermore, they learn to assess the severity of the recognized problems themselves. d) TRANSPARENCY User research measures are made available as transparently as possible to all teams involved in the development of a product or product portfolio (e.g. via Community of Practices [37]). This promotes the learning of agile teams and avoids redundant user research measures. e) VALUE FOR CUSTOMERS OR USERS For the definition of customer or user benefit, explorative field research is carried out in a preliminary manner for agile development to identify problems and opportunities as well as to take a holistic view of the customer journey. Based on this, product visions are defined; these are used as a target picture for agile development. 129768 VOLUME 8, 2020 E.-M. Schön et al.: Lean User Research for Agile Organizations In summary, it can be stated that the pattern Lean User Research enables one to solve the issue of involving users and stakeholders directly into agile product development by means of regular, short feedback cycles. B. RESULTS OF QUESTIONNAIRE STUDY This section presents the results of the questionnaire study (see Section III.C). The aim of this study was to figure out the extent to which the pattern Lean User Research is already applied among the UX community. Hence, we queried each item (see solution a-e in Table 3). Appropriate best practices and hints with regard to the application of each item in industry are presented in Section IV.A as well as in Section IV.C. Figure 8 shows the results of item a (adaption of user research). Although 81% of the participants (N=31) rate this item as totally important or important, in contrast, only 37% (N=27) enable shorter feedback cycles by means of adapting user research to agile timing. Therefore, we can conclude that this is still an unsolved problem in industry that might be solved by applying our pattern. FIGURE 8. Adapt user research to agile timing (e.g. sprints, iterations) in order to enable shorter cycle times (Item a). Figure 9 presents the results of Item b (availability of participants). Item b shows similar results compared to Item a. 74% of the participants (N=31) rate this item as totally important or important. However, only 38% (N=26) of the participants enable frequent feedback cycles by means of ensuring the availability of participants for tests. In Figure 10, the results of Item c (self-organization) are highlighted. For this item, the answers of the participants (N=29) vary from totally important (24%) and important (31%) to rather important (28%) and neutral (17%). Moreover, 58% of the participants (N=24) enable the selforganization of agile teams when conducting user research. Figure 11 shows the results of Item d (transparency). In sum, 93% of the participants (N=30) rate this item as totally important or important. Compared to the other items, Item d has the highest rank in terms of importance. Furthermore, 54% (N=24) of the participants create transparency in respect of user research in a scaled environment where multiple teams work in parallel. In Figure 12, we present the results of Item e (value for customers or users). It is important for an agile team to recognize at an early stage whether the changes to the product are valuable for customers and users in order to avoid expensive FIGURE 9. Ensure the availability of participants for tests to enable frequent feedback cycles (Item b). FIGURE 10. Enable the self-organization of agile teams when conducting user research (Item c). FIGURE 11. Create transparency regarding user research when several agile teams work in parallel (Item d). undesirable developments. 84% of the participants (N=31) rate this item as totally important or important. In addition, 56% (N=25) support teams in identifying early on whether they deliver value for users by means of integrating user research methods into the agile development process. In addition, Table 4 shows additional statistical data (mean, standard deviation and confidence interval). The response options of the Likert scale were normalized for evaluation FIGURE 12. Integrate user research methods into the agile development process so that teams can identify early on whether they deliver value for users (Item e). VOLUME 8, 2020 129769 E.-M. Schön et al.: Lean User Research for Agile Organizations to the value range 1 (totally important) <x<7 (totally unimportant). It is obvious that Item d (transparency) and Item e (value for customers or users) achieved the highest ranks in terms of importance. TABLE 4. Statistical data regarding the items of our questionnaire related to the use of lean user research. Figures 8-11 show that the importance is close to each other in the range of ‘‘totally important’’ to ‘‘neutral’’. Hence, it is not surprising that the confidence interval (see Table 4) is small. Summarizing the findings of the questionnaire study, it is interesting to see that the aspects of Lean User Research have received high values in terms of importance by the UX community of the DACH region. Nevertheless, the participants in the survey do not consistently apply them yet in their practical work. C. RESULTS OF PANEL DISCUSSION a) ADAPTION OF USER RESEARCH Expert 1 (see Table 2) emphasized that it is important to have continuity in all activities of product development. This is also true for research, design, and evaluation of the UX [14]. As regards user research, it is helpful to conduct it one sprint ahead to the product delivery. This is a common behavior for UX activities in an agile environment [24], [38]. In addition, Expert 2 stated that the agile mindset is helpful in terms of clarifying questions with regard to the continuous involvement of users in product development. In this context, Expert 3 argued that the agile way of working means that time is reduced for planning and documenting user research, so that the focus is on conducting it. Expert 4 emphasized that reflecting is an important part of agile working, and is not always about speed. b) AVAILABILITY OF PARTICIPANTS Expert 2 referred to the limited availability of test persons, which is a well-known issue in the execution of user research in agile product development [24]. In this context, Expert 3 discussed the point that because of regularity (sprints or iterations), it is easier to build up a panel of test persons. Creativity plays an important role in recruiting and, so, she uses conferences, fairs, and exhibitions. c) SELF-ORGANIZATION Expert 1 pointed out that user research is in the responsibility of the whole organization and that there should be experts who can be borrowed in order to spread the knowledge about conducting user research. Furthermore, Expert 2 reminded that there is a correlation between self-organization and responsibility for success. She argued that experiences from user observations have a motivating effect on an agile team. Expert 3 posed the question of what qualifies an agile team to conduct user research. She emphasized that quality plays an important role in conducting user research and that individual team members need to be trained accordingly. Expert 4 had given an outlook into the future and emphasized the point that service provider and in-house will have a stronger partnership cooperation in the future. Moreover, the service provider would become the contact point for the provision of experts with deep knowledge in user research. All experts agreed that whether it makes sense to have a dedicated expert of user research as a member of the agile team depends on the context of the product development. d) TRANSPARENCY Expert 1 discussed the positive impact of Community of Practices. They allow one to share the knowledge across the organization. Moreover, Expert 3 said communicating is more important than documenting. In this respect, Expert 4 described an organization as having a learning curve in relation to the appropriate type of documentation. In agile environments, informal documentation methods like video, audio, or drawings on whiteboards are widely adopted [13]. e) VALUE FOR CUSTOMERS OR USERS Expert 1 reported that UX practitioners carry out preliminary context analyses prior to development, so that initial concepts are available at the beginning. He stated that prototyping is useful in terms of evaluating hypotheses. Prototyping is widely adopted in agile environments with the aim to create valuable products [39]. Summarizing the key takeaways of our panel discussion, we can highlight the following: •Agile way of working enables better planning of HCD. 1Confidence and Confidence Interval (p=0.05) 129770 VOLUME 8, 2020 E.-M. Schön et al.: Lean User Research for Agile Organizations •UX practitioners have the responsibility to enable agile teams to receive continuous feedback from users. •In doing so, the quality of the execution of user research should be ensured. V. DISCUSSION In an agile environment, the entire agile team or organization is responsible for the success of the product. Therefore, the tasks and responsibilities of an expert change. Experts are no longer the people who do special tasks alone because they have the knowledge. Their task is much more: they share their expert knowledge and enable the agile team to do the work together. The pattern Lean User Research (see Table 3) summarizes best practices in respect to the application of user research in an agile environment. In this context, patterns are a good way to share expert knowledge among an Agile Organization. A. MEANING OF FINDINGS General findings. The empirical studies helped us validate our pattern Lean User Research to a greater degree. The results reveal that the aspects of the solution (see solution a-e in Table 3) are important in terms of conducting user research in an agile environment. If we look at Anderson’s Economic Cost model for Lean software development [2], it becomes obvious that our pattern Lean User Research supports organizations to reduce wasteful activities. According to Anderson’s model, wasteful activities are allocated to transaction costs (e.g. project setup activities, delivery activities) or coordination costs (e.g. assign people to tasks, schedule events). For example, transaction costs are reduced by integrating user research methods into the agile development process so that teams can identify early on whether they deliver value for customers or users (see Table 3). On the other hand, coordination costs are reduced by ensuring the availability of participants for tests (see Table 3). Our pattern, therefore, enables the increase of value-added work. The results of the survey (see Section IV.B) show that the best practices of our pattern are rated as important by the UX community, but are not yet continuously applied. To this end, the application of our pattern can assist in making user research activities more acceptable in agile product development. This is still a problem in many organizations nowadays [29]. As our pattern Lean User Research becomes more widespread, the pattern can help organizations adapt Lean User Research best practices to the needs of their own organization. This is especially because many organizations today already use Agile UX or Lean UX [40]. Our pattern includes a collection of best practices for conducting user research in agile product development, and have been collected using empirical methods. Organizations can decide for themselves if they want to apply parts of the pattern or if the whole pattern is helpful for them. Moreover, our results are comparable with the ones reported by research literature [13], [14], [24], [38], [39]. For instance, problems and solutions regarding integrating user research into agile product development are similar (see Section IV.C). Findings related to RQ1: What impact does Lean User Research have in the industry? More and more organizations are starting to transform into Agile Organizations (see Figure 1). Concerning our first RQ, we can conclude that agile product development has an impact on how user research is conducted. UX practitioners are the ones responsible to enable the agile team to learn more about users and the use of the product. Moreover, they ensure that the agile team receives continuous feedback from users and stakeholders. This has an impact on the required skillset of UX practitioners. In addition to methodical skills, UX practitioners should also have personal skills, such as empathy or critical thinking [41]. In future, they will moderate and coach more frequently instead of conducting studies themselves. This change is also likely to have a positive impact on the working conditions of UX practitioners. The availability of UX practitioners has been a particular challenge in the development of enterprise software [42]. In future, UX practitioners will support the entire organization as well as individual agile product development teams as UX coaches instead of extinguishing small fires as lone fighters. For this reason, the culture of an organization must develop in such a way that everyone feels responsible for the success of the products. With regard to continuous feedback from users and stakeholders, it is essential to have a panel of test persons in order to ensure the availability of frequent feedback. This is why co-creation approaches are becoming more and more necessary [13], [26], [27]. Users and stakeholders must understand that they can influence product development through continuous feedback. Findings related to RQ2: How is Lean User Research applied in industry? The context-related application of the pattern (see Section IV.A) provides valuable insights into how user research is adapted in an Agile Organization (see Figure 1). KPIs are used to learn more about user behavior and to direct decision-making in terms of product development. For this reason, evaluations are carried out by the entire agile team instead of a result report being provided by a user research expert. In this context, the integration of user research methods into the agile development process is rated as very important by the participants in our questionnaire study (see Figure 12). Furthermore, the results show that knowledge in respect of user research and measurements are shared by means of Community of Practices among the whole organization so that teams can identify early on whether they deliver value for users. It is interesting to see that the aspects of Lean User Research (see solution a-e in Table 3) have received high value in terms of importance by the UX community of the DACH region (see Section IV.B). Nevertheless, the participants in the survey do not yet consistently apply them in their practical work, although best practices are well known (see Sections II and IV.A). This is an interesting finding, because agile process models are used more and more frequently and have an influence on the way UX practitioners VOLUME 8, 2020 129771