Work‐from‐home impacts on software project : A global study on software development practices and stakeholder perceptions

This is a self-archived version of an original article. This version may differ from the original in pagination and typographic details. Author(s): Title: Year: Version: Copyright: Rights: Rights url: Please cite the original version: CC BY-NC 4.0 https://creativecommons.org/licenses/by-nc/4.0/ Work‐from‐home impacts on software project : A global study on software development practices and stakeholder perceptions © 2023 The Authors. Software: Practice and Experience published by John Wiley & Sons Ltd. Published version Nguyen‐Duc, Anh; Khanna, Dron; Le Giang, Huong; Greer, Des; Wang, Xiaofeng; Martinez, Zaina Luciana; Matturro, Gerardo; Melegati, Jorge; Guerra, Eduardo; Kettunen, Petri; Hyrynsalmi, Sami; Edison, Henry; Sales, Afonso; Chanin, Rafael; Rutitis, Didzis; Kemell, Kai‐Kristian; Aldaeej, Abdullah; Mikkonen, Tommi; Garbajosa, Juan; Abrahamsson, Pekka Nguyen‐Duc, A., Khanna, D., Le Giang, H., Greer, D., Wang, X., Martinez, Z. L., Matturro, G., Melegati, J., Guerra, E., Kettunen, P., Hyrynsalmi, S., Edison, H., Sales, A., Chanin, R., Rutitis, D., Kemell, K., Aldaeej, A., Mikkonen, T., Garbajosa, J., & Abrahamsson, P. (2024). Work‐from‐home impacts on software project : A global study on software development practices and stakeholder perceptions. Software : Practice and Experience, 54(5), 896-926. https://doi.org/10.1002/spe.3306 2024 Received: 18 May 2023 Revised: 3 September 2023 Accepted: 2 December 2023 DOI: 10.1002/spe.3306 RESEARCH ARTICLE Work-from-home impacts on software project: A global study on software development practices and stakeholder perceptions Anh Nguyen-Duc1Dron Khanna2Giang Huong Le3Des Greer4 Xiaofeng Wang2Luciana Martinez Zaina5Gerardo Matturro6 Jorge Melegati2Eduardo Guerra2Petri Kettunen7Sami Hyrynsalmi8 Henry Edison9Afonso Sales10 Rafael Chanin10 Didzis Rutitis11 Kai-Kristian Kemell7Abdullah Aldaeej12 Tommi Mikkonen13 Juan Garbajosa14 Pekka Abrahamsson15 Correspondence Anh Nguyen-Duc, Department of Computer Science, University of South Eastern Norway, and the Norwegian University of Science and Technology, Trondheim, Norway. Email: [email protected] Funding information None Context: The COVID-19 pandemic has had a disruptive impact on how people work and collaborate across all global economic sectors, including software business. While remote workingisnotnewforsoftwareengineers,forcedWFHsituationscomewithbothlimitations and opportunities. As the ‘new normal’ for working might be based on the current state of Work-from-home (WFH), it is useful to understand what has happened and learn from that. Objective:ThisstudyaimstogaininsightsintohowtheirWFHarrangementimpactsproject management and software engineering. We are also interested in exploring these impacts in different contexts, such as startups and established companies. Method:We conducted a global-scale, cross-sectional survey during the spring and summer 2021. Our results are based on quantitative and qualitative analysis of 297 valid responses. Results: We characterize the profile of WFH in both spatial and temporal aspects, together with a set of common collaborative tools and coordination and control mechanisms. We revealedsomeareasofprojectmanagementthatarerelativelymorechallengingduringWFH situations, such as coordination, communication and project planning. We also revealed a mixed picture of the perceived impact of WFH on different software engineering activities. Conclusion: WFH is a situational phenomenon which can have both negative and positive impact on software teams. For practitioners, we suggest a unified approach to consider the context of WFH, collaborative tools, associated coordination and control approaches and a process that resolve those aspects that are sensitive to physical interaction. KEYWORDS COVID-19, empirical study, project management, software engineering, work-from-home Abbreviations: GSD, global software development; MVP, minimum viable product; PO, primary observations; RQ, research question; SE, software engineering; UX, user experience; WFH, Work-from-Home. For affiliations refer to page 26. This is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes. © 2023 The Authors. Software: Practice and Experience published by John Wiley & Sons Ltd. Softw: Pract Exper. 2023;1–31. wileyonlinelibrary.com/journal/spe 1 2NGUYEN-DUC et al. 1INTRODUCTION Work-from-home (WFH) is increasingly being recognized as a popular work arrangement due to its many potential benefits for both companies and employees (e.g., increasing job satisfaction and retention of employees).1–3 WFH has been known in many different terms, such as remote work,4,5 virtual teams,6,7 and teleworking.8–10 Remote work is not new,anditisnotuncommontohave softwaredevelopment undertaken in a multisite, multicultural,globally distributed setting.11,12 However, with the recent COVID-19 pandemic, many companies were forced to adopt the WFH model, regardless of suitability to their business model, product nature, the current setting of teams and organizations.13 As estimated recently, there might be close to 40% people working in the EU who began to telework full-time as a result of the pandemic.14 Many companies decided to switch to long-term remote work. In May 2020, Twitter’s CEO at that time informed their staff that they could work from home forever*. Coinbase has become a ‘remote-first’ company, allowing most staff who want to work remotely to do so indefinitely. Dropbox too will let all employees work from home permanently†and Amazon has stated that employees whose positions allow them to work from home can do so two days a week‡.While teleworking is not new in many organizational settings, recent studies in the context of COVID-19 agree that significant changes to the workplace or way of working will occur in post-pandemic times.15–17 Organizations might face various configurations of WFH, from a hybrid mode of office and online work to working from anywhere.17 Several research topics are useful for maintaining a safe and productive working environment in a WFH arrangement. Well-beingness, psychological safety and work-life balance have been quite well explored in a software industry context.18–23 Thesestudiesconceptualizeconstructs,suchaswell-beingness,fear,stress,distractions,communicationand happiness and relate them to individual performance on different software engineering activities. In contrast, there are relatively less studies about WFH at a team level. Since software development is a collaborative effort, it is essential to investigate the impact of WFH settings on the different activities of the software development processes. It may be difficult for project managers to communicate via e-communication tools since they cannot substitute face-to-face interaction, especially in a complex setting. Project managers might also need to adopt new practices or mechanisms to trace and coordinate software development activities in different WFH settings.24 The impact of new work arrangements on different software development activities, such as requirements engineering, coding, and testing, might be diverse due to the various need of physical communication and coordination. This effect might be heightened in scale for software development teams in startup and small companies due to their vulnerability to macro environmental changes. Bai et al. found that WFH practices are critical for startup companies to maintain their operations.25 To the best of our knowledge, no study about WFH has focused on software startups. Given the particular characteristics of this kind of company, such as their entrepreneurial characteristics, the relationship with the customers, small team size, lack of resources, and immature adoption of practices26,27 changes induced by new work arrangement could have unique implications. The first objective of this paper is to explore how professionals perceive the impact of WFH during COVID-19 on differentsoftware projectmanagementandsoftwareengineeringactivities.As part ofthisgoal,weaim tohighlight,from their perspective, which activities have been most impacted, both positively or negatively. The second objective of this paper is to compare the impactof the WFH situation on teams in software startups and established companies regarding differentsoftwaredevelopmentactivities.Basedontheseobjectives,thepresentstudyfocusesonaddressingthefollowing research questions: 1. RQ1–How does the way of working in software teams change when WFH is adopted? 2. RQ2–How is project management impacted when WFH is adopted? 3. RQ3–How are software engineering activities impacted when WFH is adopted? 4. RQ4–Is there any difference between startups and established companies regarding the above impact? Theoutcomesofthestudyarethree-fold.Firstly,itprovidesaworldwideanddiversesetofevidenceonWFHpractices in the software industry. Secondly, the study describes different practices to manage, communicate and coordinate in *https://www.forbes.com/sites/danabrownlee/2020/05/18/twitter-square-announce-WFH-forever-optionwhat-are-the-risks/. †https://www.businessinsider.com/what-spotify-twitter-goldman-sachs-said-about-long-term-remote-working-2021-3. ‡https://www.flexjobs.com/blog/post/companies-switching-remote-work-long-term/. 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License NGUYEN-DUC et al. 3 a WFH setting. Thirdly, the study is among the first to offer a worldwide survey on project management and software engineering with a comparative view between startups and established companies. The next section captures the relevant literature on WFH in software engineering and software startups. Section 3 presents our research approach. Sections 4and 5present and discuss our results while Section 6concludes the paper. 2RELATED WORK This section briefly reviews existing work on the three key concepts: WFH, WFH in software engineering, and software startups. In the end, we present our conceptual framework to guide our analysis and findings. 2.1 Work-from-home Work-from-home (WFH) has been referred to using many different terms, such as teleworking, remote working, and virtual team working. WFH is not a new phenomenon. Studies on the phenomenon has dated back to the 1970s, when the motivation for WFH was thought to be to manage resource shortages, reduce daily commuting or to balance work and family duties.6,10,28,29 WFH has been characterized as needing minimum physical requirements, individual control over work pace, defined deliverables, a need for concentration, and a relatively low need for communication.4 In general, WFH is often claimed to improve productivity30,31 and teleworkers do consistently report increased perceived productivity.32,33 WFH gives people more flexible working time providing better work-life balance, saves on the cost of maintaining a central working place and may give better job satisfaction. However, WFH is found to be associated with greater levels of both work pressure and work-life conflict34 because work intrudes into developers’ home lives through working unpaid overtime, thinking about working hours, exhaustion and sleeplessness.35 It may be that many organisations lack appropriate plans, supportive policies, resources or management practices for practicing WFH. 2.2 WFH and software industry The software industry, a dynamic and ever-evolving sector, serves as the digital backbone of the modern world, driving innovation across various domains. Given the software industry’s reputation for embracing innovative work methodologies, the introduction of remote work has had a substantial influence. However, it’s important to note that the extent and nature of this impact can vary widely, largely depending on the specific organizational context. Bao et al. conducted a quantitativeanalysis based onadatasetof developers’dailyactivitiesfromBaiduInc.23 The authors foundthatWFHhad both positive and negative impacts on developer productivity in terms of different metrics, for example, the number of builds/commits/code reviews. Forsgren et al. studied open source projects in Github and showed that developer activity in terms of number of pushes, pull requests, code reviews and commented issues remained similar or slightly increased compared to the pre-pandemic year.36 Ralph et al. performed an extensive study of the pandemic impact on programming, including productivity in the early months of WFH.20 They concluded that perceived productivity has declined (admitting a marginal effect size) as a result of negatively affected well-being and that organizations need to accept that expecting normal productivity under crisis circumstances is unrealistic. Ford et al. conducted a two-wave study on productivity in Microsoft.15 Both surveys indicated that productivity increased among some participants and stayed the same or decreased among the others. Russo et al. performed a two-wave longitudinal study with a diverse group of professionals, diving into the impact of over 50 psychological, social, situational, and physiological factors and their ability to predict the variance in well-being and productivity.21 The study concluded with a few associations between the studied factors and perceived productivity. Oliveira et al. gathered data from two online surveys of Brazilian professionals.37 The authors found that perceived productivity in WFH when comparing with the office times has increased. Another important finding made regarding the changesinperceivedproductivityduringthepandemicwasthatthenumberofpositivelyaffectedrespondentsgrewfrom 40% in the first wave to 60% in the second wave. Smite et al. studied 13 surveys in the literature finding that on average, perceived productivity had not changed significantly; there are developers who report being more productive and developers who are less productive when working from home.16 Also, positive trends are found in longitudinal surveys, that is, developers’ productivity in the later months of the pandemic show better results than those in the earlier months. Nolan et al. conducted a qualitative study on software engineering duringtheCOVID-19 pandemic.18 The authors showedthatsoftware companieswouldderivetangible 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 4NGUYEN-DUC et al. benefits from supporting their employees during this uncertain time through ergonomic home offices, listening to their concerns, as well as encouraging breaks and hard stops to boost long term well-being and productivity. Machado et al. explored genderedexperiencesofsoftwareengineersduring the COVID-19 crisis38 and foundthatwomen faceparticular challengesduringsocial isolation,astheyweremorelikelytolacksupportwithhousehold andchildcareresponsibilities. 2.3 WFH in software startups Software startups represent a unique and dynamic context within the software industry. Unlike established software companies, startups are typically newly formed ventures that are focused on developing innovative software products, services,or solutions.Bergetal.summarized acommondefinitiononsoftwarestartupsascompanieswithan innovation focus, lack of resources, working under uncertainty and time–pressure, highly reactive and rapidly evolving.26 Startups are found to be different from established companies in the strong presence of entrepreneurial personalities, behaviors, decision-making and leadership.27,39,40 Softwarestartups,inparticular,tendtoexcelatadaptabilityandflexibility27 andweshouldexpectthemtodemonstrate that in response to the COVID-19 crisis. Startups is known for their inherent agility and adaptability that allows them to pivotswiftlyinresponsetoevolvingmarketdynamics,whilelarger,establishedfirmsoftengrapplewithbureaucracyand legacy systems.27,41 Moreover, their tech-savvy nature could allow them to seamlessly transition to virtual collaboration, ensuring business continuity. Software startups have been instrumental in adopting the digital tools and environment needed to facilitate remote work for companies across industries. Consequently, we have a good reason to believe that we can observe the special impact of project management and work practices in software startups that can benefit the software industry. 2.4 Conceptual framework Collins et al. reviewed and discussed different roles of theory in empirical work, and one of them is to provide focus and organization to the study.42 Maxwell refers to a conceptual framework as “an idea context of the study”.43 The conceptual framework should assist the researcher in refining goals, developing research questions, discerning methodological choices, identifying potential threats to validity, and demonstrating the relevance of the research. The primary source of the conceptual framework, from his perspective,43 does not necessarily need to be an existing theory. Four primary sourcesareoptionsfromwhichtoderiveaconceptualframework:(1)knowledgebasedonexperience,(2)existingtheory, (3) exploratory research, and (4) ‘thought experiments’.42 Figure 1consists of three main conceptual blocks–distributed work arrangement due to the pandemic situation, perceived impact on project management, and perceived impact on software engineering. Distributed work arrangement depends on the policy and setting of individual companies. There are three important aspects of a distributed working condition that have been explored in global software development (GSD) research,11,44 namely the distribution profile,45 the usage of tools,46 and the coordination and control practices and mechanisms and approachesforcontrolandmanagement.45 Theconfigurationofandistributedteamisusuallyseenfromthecombination of the extent that a team spreads over geographical locations and timezones.45,47 Collaboration tools are essential in a virtual team. Various common tools that are reportedly common in global software development46 should be revisited in the context of forced virtual collaboration. We also take input from control and coordination practices that are previously reported in global teams.48–51 Project management: Communication, collaboration and coordination, the major managerial activities in a software project, are expected to be significantly affected by the shift to new working conditions. GSD research has highlighted important managerial aspects in software projects that are under high uncertainty and fast-changing environments, including team and task awareness, mid-term to long-term planning and team culture.52–54 Through one of our design meetings, we also considered team competence as a potential area of impact. Changing work arrangements might imply the needs of new roles or new competence in the team. Software engineering: We decided to focus on basic software engineering roles and activities to achieve broad coverage in the survey. The selected areas also reflect the research interests of the authors. The seven activities areas are Requirement Engineering, Software Archicture, UX Design, Implementation, Quality Assurance and Software Process. These areas also match the occupation of the majority of survey respondents, as shown in Figure 2. 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License NGUYEN-DUC et al. 5 FIGURE 1 A conceptual framework. FIGURE 2 Occupational distribution among participants. 3METHODOLOGY The original objective of the survey was to gain insights into how the way of working has changed into WFH and how it impacts software engineering, project management, innovation and resilience. This section presents our study design (Section 3.1), Instruments (Section 3.2), Sampling strategies (Section 3.3), Data collection (Section 3.4) and Data analysis (Section 3.5) 3.1 Survey design Thesurveyisdesignedasacross-sectionalstudywithamixofclose-endedandopen-endedquestion.55 Thesurveydesign process started in October 2020 and ended in March 2021. The initial foundation for the studies was a set of few studies about COVID-19 and startups or software engineering. It is noted that at the time the study was designed, there are not many papers found in this topic. The literature was updated gradually during the analysis and report writing. The study designwasinfluencedbyalargegroupofresearchers.Typically,abrainstormingsectionforstudydesigninvolvesbetween 15 to 25 researchers from Brazil, Norway, Italy, Finland, Sweden, UK, Portugal, Germany, Australia, Canada, China, and Vietnam.Manyofthesepeopleareco-authorsofthiswork.Weconductedbi-monthlyworksessionsviavideo-conference, 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 6NGUYEN-DUC et al. TABLE 1 Major versions of the survey. Ver. Major activities # meetings Sample size Tools 1 Testing and validating theoretical elements 5 meetings 5 internal participants Paper based 2Reducing the number of questions, adding opt-out options 3 meetings 25 pilot companies Google form 3 Adding more open-text questions, removing one-person companies 2 meetings 5 internal participants Lime survey 4Revising questions, correcting inviting text, languages 2 meetings 574 responses Lime survey either for brainstorming or focus group to design the survey, discuss and work on the project. The result of this design process is three major versions of the survey (details on piloting and validating these versions are shown in Table 1). The study’s target population is software development teams worldwide who switched from working in an office to working from home because of COVID-19. Stakeholders who had been working remotely before the pandemic are also important, but this study is about the switch, and the questions are designed for people who switched from working on-sitetoWFH.Theunitofanalysisinthisstudyisasoftwaredevelopmentteam.Weimplementedseveralapproachesto makingsureeachparticipantcouldvalidlyrepresenttheirteams.Inprinciple,thequestionnairewasopentoawiderange of software development stakeholders, from business analysts, designers, software developers, testers, Scrum masters to startup-specific roles, such as CEOs or CTOs. 3.2 Instruments The overall instrument used in this research constitutes in total 45 questions. The survey includes sections designed to (1) understand the current working conditions of the participants; (2) the contextual background of the participants, includingthecompanyandindividualcharacteristics;(3)theperceptionoftheparticipantsabouttheimpactofCOVID-19 onsoftware engineering activities in their teamand companies;(4)the perceptionof the participantsabouttheimpactof COVID-19 on their companies’ innovation and resilience; and (5) the perception of the participants about the impact of COVID-19 on perceived performance. We used both yes-no questions, multiple-choice questions (application domains, digital tools, team size etc.), and five-point Likert scale multiple-choice questions. Some questions, for example, regarding digital tools participants are using, employed a free text option, for example, ‘tool names’ or ‘other’ so that the respondents can specify their choice better.Wealsouseseveralopen-textquestionstogetmoredetailsfromtheparticipants.InaLikertordinal-scalequestion, we have five standard choices: (1) Strongly disagree, (2) Disagree, (3) Neither disagree nor agree, (4) Agree Strongly and (5) Agree. Additionally, we added the sixth option (6) Not applicable so the participants can opt-out of the question. Theinterviewquestionnairewasdevelopedin3months(asshowninTable1).Aninitialdraft(version1)ofthesurvey was first created based on the literature in software engineering and COVID-19. The first survey was created in Google forms. The second version of the survey was made after taking into account comments and adjustment from the whole author team. Some changes in later versions of thesurvey, for instance, re-coding scale labels of some questions, removal of few questions for better focus, adding team size/ company size value of 1 for filtering, adding open-text questions, and adding the option ‘Not applicable’ in questions. Apilotdatacollectionwas done in April 2021. We gatheredresponses from 25teamstovalidateourconstructs,scales and questions. We also asked for expert opinions from senior researchers in software engineering who conducted survey research before.20,56 The final survey was made ready on May 2021 and available via the Lime survey tool. The major changes made to versions of the survey is shown in Table 1. The details of the questions are available online§. We have several questions helping us to determine whether a company is a startup or an established one. We also implemented filtering questions to make sure only people who feel the impact of COVID-19 on their professional activities will continue filling in the survey: “Do you experience or observe an impact of COVID-19 to your work/ your company to any extent” The English version of the survey was translated into seven languages, which are Italian, Spanish, Portuguese, Arabic, Indonesian and Vietnamese. The translation was done by seven core members of the author team. The number of responses in each language is shown in Table 2. §https://COVIDnse.limesurvey.net/561361?lang=en. 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License NGUYEN-DUC et al. 7 TABLE 2 Number of responses by local languages. Survey language No. of responses Italian 5 Arabic 7 Indonesian 11 Chinese 12 Spanish 13 Vietnamese 42 Portuguese 90 English 394 N=574 3.3 Sampling strategies We have tried severalways topurposefullygathera sample that can representour targetpopulation.Atthe country level, wehavehadcontactfrom13+countries,andweexpectedtoget10–20responsespercountryviathesepersonalchannels. Conveniently, we invited participants through our professional and social networks. The invitation message was shared through co-authors’ social media, that is, LinkedIn, Twitter, Reddit, Quora, and Facebook. We also published the call for participation in several academic communities, that is, SEWorld, ICSOB, Software Startup Research Network, and so forth. We also explored our professional connections by asking co-authors to send invitation emails to those who they think are likely able to participate. We capitalized on each author’s local knowledge to reach more people in their jurisdiction. Rather than a single, global campaign, we used a collection of local campaigns. Each localization involved small changes inwording. In some cases, authorsprintedoutthesurvey and disseminatedthepaper-basedversioninstead. We also recruited participants from professional channels, such as Prolific¶. 3.4 Data collection The data collection process started in March 2021 and ended in August 2021. In an invitation-based approach, we invited one respondent as a representative of their teams. In a broad call for participation, we tried to control the sample representatives. Invitations led to a central survey tool, which can be configured to a suitable language version. The survey was spread over five online pages. As shown in Figure 3, a total number of 574 responses were collected. 324 respondents were able to complete the survey (the data inclusion rate is 43.55%). The completion time ranged from 4 to 155 min. A screening question was asked in the first place before other questions. If a respondent did not observe an impact of COVID-19 to their working environment in some ways, he or she is navigated to the end of the survey. In this way, the survey only gathers opinionsfrom people who actually experienced the change to their working setting. As shown in Figure 4, 290 people (50.5%) answered ‘Yes’, while 90 people (15.7%) answered ‘No’ to the questions. The rest had either no answer or answered ‘do not know’. As shown in Figure 5, the responses came from 35 countries, and were dominated by respondents from Brazil, UK, Vietnam, USA, and Poland. After filtering irrelevant and invalid responses in the data preprocessing step (section 4.5.1), the number of valid responses for analysis is 297 responses. 3.5 Data analysis We received 574 total responses, and after data pre-processing steps, 297 valid responses remained. ¶https://prolific.co/. 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 8NGUYEN-DUC et al. FIGURE 3 Research process. FIGURE 4 Responses of a screening question on the impact of COVID-19. 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License NGUYEN-DUC et al. 15 software engineering roles are asked, including Software Architect, Software Developer, Tester, UX Designer, Business Analyst,ProjectManagerandScrumMaster.TheresultisshowninFigure10.Forallsoftwareengineeringrolesthatwere named, there was a large portion of responses stating a neutral impact (from 37.4% to 51.2%). Overall, we also observed a relative balance between the negative and positive rates on the impact of WFH on software engineering roles. 4.3.1 Requirement engineering The distribution of respondents rates on requirement engineering activities is shown in Figure 11. We observed more responses with negative or little negative impact in activities like requirement gathering and customer involvement than that in activities like requirement prioritization and management. FIGURE 10 The perceived impact of WFH on software engineering roles. FIGURE 11 The perceived impact of WFH on requirement engineering activities. 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 16 NGUYEN-DUC et al. Negative impact can be found in the areas requiring customer involvement when shifting from a physical manner to a remote approach. Several respondents have commented in line with this observation, for example: “For customers, solicitingfeedbackhasbecomelaborious when done remotely, so mainly atimeissue” (Respondent60, UK), or “Alot of documenting that wasn’t noticed until COVID-19. More impact on gathering approaches–various meetings help organize work” (Respondent26, Poland) or “Hard to get hold of stakeholders/customers since they are not in their office” (Respondent140, Latvia),or“WehadcallsandmeetingsbeforeCOVID19butalotofinformalmeetupsandsyncsweredoneintheofficebutnow haveto beplanned,scheduled,andorganized.Itsloweddownsomestuffthatusuallywas veryeasytosolve”(Respondent17, Mexico) and “We think that doing an interview or understand our clients’ problems by using programs as Zoom instead of meeting in person may reduce the effectiveness of the reunions” (Respondent247, Uruguay). PO5: The impact of WFH on requirement engineering tends to be more negatively perceived in activities like requirement gathering and customer involvement than in activities like requirement prioritization and management. 4.3.2 Software architecture The distribution of respondents’ perception on the effect on software architecture activities is shown in Figure 12.Comparedtootherroles,scientistsandarchitectsinasoftwareprojecthavemorereportedneutralorpositiveexperiences.This includes (i) more focused time (given less spent on communication or travel) as typified by this response, “The development team has more research time and spends more time on design” (Respondent 101); (ii) greater flexibility as illustrated bythisresponse,“Ourbusinesswassettoahaltandsotheyprogrammersweremoreefficientandwereabletotestnewmethods.Thefactthattheyworkedmoreflexiblehoursreducedcostsgreatly.”(Respondent373,Germany)and(iii)betterrespect of formal processes as demonstrated in this response, “Inadaptationprocess,we gotbetterinlisten andinternalcommunication.Theteam is more cohesiveandworkingonmakingstandardprocessesbetter and nail it on that. All of themaremore creative and productive.”(Respondent 291, Brazil). An established technical setting for complex tasks like architecture is FIGURE 12 The perceived impact of WFH on software architecture activities. 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License NGUYEN-DUC et al. 17 essential to maintain productivity in a new working mode: "Since the target architecture and design was set up during the projectinitiationphase,thedevelopmentprocess was able toalignseamlesslytoit.Sincealldevelopersareexperienced in the standardarchitecture,thealignmenttoarchitecturewasneveranissue.Anyopenquestionswerealwaysflaggedintheonline daily standup meeting and closed quickly.” (Respondent 423, UK) PO6:ComparedtootherSE activities,software architecture showstheleastimpact by WFH.Many acknowledge the positive impact of WFH on architectural activities and quality. 4.3.3 UX design UX work also shares a common challenge with other roles in a WFH context as shown by several comments: “Interaction with users and customers during the pandemic has become quite complicated given social distancing” (Respondent 348, Vietnam), or “Testing has become harder because people can’t come to the office and user validation has to be done online only” (Respondent 364, Brazil). Some parts of UX work is not even possible in a WFH setting for example, “More difficult to have customers test on the devices we would like to use as they are only available in the office” (Respondent 382, Italy). An established process is reported as a factor to determine the quality of UX work online: “UXisbeingintegrated into our planning process earlier than it used to be; our overall organization is also implementing web compliance and accessibility guidelines more stringently” (Respondent 371, UK). Some UX designers enjoy the flexibility of WFH and see an alignment with creativity which is essential for their work: “Designers were able to perform better due to working from home and being flexible on their schedule. Zoom was pretty essential when they needed to cooperate.” (Respondent 373, UK) 4.3.4 Software implementation The distribution of respondents’ responses on implementation activities is shown in Figure 13. FIGURE 13 The perceived impact of WFH on implementation activities. 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 18 NGUYEN-DUC et al. Regarding Software developers’ responses, many have mentioned their wellbeing during the pandemic time, for instance, “Attitudes have suffered with declining mental health and isolation” (Respondent 11), or “We struggled a bit with energy due to not meeting up and generallyliving through a lockdown and torturous lack of things to do” (Respondent 216, Brazil). We also documented challenges regarding the maintenance of code standards, as stated in this comment:“In general, due to frequent changes, less care was taken with quality metrics or it was not possible to reach a satisfactory level of these metrics due to the short time, despite efforts to guide the team to maintain the standard.” (Respondent 202, UK), or “Problems that occur in remote work do not have the same urgency as they do in face-to-face work. As well as the charge for code quality has decreased.” (Respondent 315, Brazil) or "…releases slowed due to adaptation to change. Tech debt increased to do things quickly to deliver for customers facing exceptionally challenging conditions.” (Respondent 539, Greece) PO7:Itisamixed pictureof theimpactof WFHonsoftwareimplementation.Bothnegativeand positiveimpacts are found on the impact on frequency of releases, quality of code, developers’ attitudes, accumulated technical debts and technical risks. 4.3.5 Quality assurance Thedistributionofrespondents’responsesonqualityassuranceactivitiesisshowninFigure14.Itisquiteinterestingtosee many testerswithnegativeexperienceswithWFH.Onereasonmightbethat, thetestingprocesscanbetime-consuming, procedural and relateto a rangeof differentpeopleandfunctions,needinga lot of communication. Such communication canbenegativelyimpactedduetothechangeinworkingmode.Atestmanagerstated,“Theyusuallyweredonebyateamso again,itsnewmeetingscheduling,neworganizations,moretimegoingintothem,etc”(Respondent119,Turkey).Theimpact can also come to the quality of testing, for instance, “The testing team began to develop and change tests ’to pass’ instead of actuallytestingwhatwas supposed tobetested.” (Respondent198, Paraguay), or “Wedon’thave many accuratetests or even FIGURE 14 The perceived impact of WFH on quality assurance activities. 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License NGUYEN-DUC et al. 19 lack of tests on some products” (Respondent 246, Brazil). The issues with testing are also found with acceptance testing as stated in these comments: “Users had greater difficulty in validation and acceptance tests remotely when they did not fully knowtheproduct”(Respondent290,Brazil)or“Samepointaboutcommunicationbetweenteams,thisnewmodalityofwork has worsened the effectiveness of current code inspection (code review, walkthrough, etc) and the effectiveness of current user acceptance testing activities.” (Respondent 348, Vietnam). A senior tester stated “Automated testing ensured that there was nochangeintestingpracticesbeforeorafterCOVID.Codereviewandacceptancegottougherandmoretimehadtobeinvested in it, since communicating this feedback online was much more time intensive. The team had to be trained to perform this online.Earlier,itwasmucheasiertointeract,whereinwecouldjustwalktothedesktodiscussorgetfeedback.” (Respondent 423, Japan). PO8:ItisamixedpictureoftheimpactofWFHonsoftwaretesting.Bothnegativeandpositiveimpactsarefound on development testing, user testing and code inspection. 4.3.6 Software process Regardingprocessexperts,itseemsthat moremanagerialworkand responsibilityoccurduringtheshift inteamworking conditions, for example, “About management work, it is harder to co-ordinate activity and to communicate with everyone.” (Respondent70, UK). Many project managers shared difficult experiences in working with not only their teams, but also external stakeholders, for instance, “The pandemic has taken its toll on us so everything is a little more difficult than it used tobe,includingcommunication,thefeelofbeingateam,interviewsaboutaplaceinourcompany.”(Respondent82,Greece), or “Not being physically in the same room impinges upon the sharing of ideas and suggestions, it feels less natural and so it seemsslightly stifledasa creativeworking environment” (Respondent260,Pakistan).Coordinationalso suffers when WFH intervenes or disrupt traditional task-dependencies for example, “Despite the demands, there is still a lot of accumulated technical debt that is yet to be resolved but, for technical reasons, we still cannot start the improvements before some tasks” (Respondent 246, Brazil). Having a process in place is essential to overcome many of the above-mentioned challenges: “MVPs took longer than expected since UX designs were always getting delayed. Initially, it was tough to collaborate on everything online, and this caused some friction within the dev teams. However, since all tests are automated, we were able to align to the existing testing methodology and it didn’t make any difference to be performed online or on-premise. With Agile methodology being practiced, it was easy to flag potential technical risks early in the cycle and work out mitigation plans.” (Respondent 423, UK) PO9: Processes and practices that embrace the adoption of tools are important to reduce many of the negative impacts of WFH on Software Engineering activities. TherespondentsarealsoaskedabouttheimpactofWFHontheamountoftimespentondifferentSoftwareEngineeringactivities.Figure15showsaconsistentresponsepatternacrosssoftwareengineeringactivities.Themajorityperceived no change in time spent on these tasks. There are relatively more people who perceived a slightly increased amount of time needed on Requirement Engineering, Architecture, UX Design, Coding and Testing. P10: All Software Engineering tasks tend to take more time to complete than before. 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 20 NGUYEN-DUC et al. FIGURE 15 The perceived delay in software engineering task durations. 4.4 RQ4: Is there any difference between startups and established companies regarding the above impact? To investigate the possible relationship between types of software companies (startups vs. established companies), we conducted a chi-square test of Independence. This is a non-parametric hypothesis test that allows the conclusion on the relationship between two variables. We will test two set of hypotheses as follows: 1. H10: There is no difference between startups and established companies in the observed impact of WFH and project management 2. H1a: There is a difference between startups and established companies in the observed impact of WFH and project management as explored in Section 4.4.1 and 1. H20: There is no difference between startups and established companies in the observed impact of WFH and software engineering activities 2. H2a: There is a difference between startups and established companies in the observed impact of WFH and software engineering activities as explored in Section 4.4.2. We also investigate the possible difference among the observations between three other contextual variables: project size (small vs. large teams), company age (recent vs. long-time companies) and geographical locations (Europe and North-America, South-America, Asia and Africa). 4.4.1 Project management Thep-valuesofchi-squareresultsaredisplayedinTable6.Thesignificantvaluesaremarked*forp≤0.05,**forp≤0.01 and *** for p<0.001. Among four aspects of project management, none of the test for company types have a significant chi-square value, hence, we can not reject the null hypothesis H10. Aninterestingobservationis that geographicallocation can have relationstothe perceivedimpactofWFHonproject management.Table6showedsignificantp-valuesinthreeoutoffiveprojectmanagementaspects(organizationalculture, competence needs and project delay) The perceived impact of changed work arrangements and competence needs can be different between small and large teams, between company newly established and long-time companies and between companies in different continents. 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License NGUYEN-DUC et al. 21 TABLE 6 Chi-square test results for four contextual variables on project management. Questions Startups versus Established Small versus large team Recent versus long-time companies Among continents It is more difficult to have an overview of who does what 0.329 0.352 0.453 0.427 It is more difficult to plan for resources, time, risk, and milestones 0.353 0.102 0.370 0.182 It is more difficult to maintain organizational culture 0.194 0.384 0.580 0.015* WFH as an impact on competence needs in the project 0.033 0.002** 0.003** <0.001*** Theexperienceddelayinprojectsdue tothe shift to WFH 0.893 0.196 0.605 0.043* *p≤0.05; **p≤0.01; ***p<0.001. TABLE 7 Chi-square test results for four contextual variables on software engineering. SE activities Startups versus established Small versus large team Recent versus long-time companies Among continents Requirement engineering 0.295 0.294 0.436 0.001** UX Design 0.122 0.104 0.457 0.047* Architecture 0.574 0.300 0.662 0.017* Coding 0.384 0.440 0.384 <0.001*** Testing 0.748 0.660 0.971 0.158 Process (Scrum) 0.815 0.389 0.351 <0.001*** Project management 0.610 0.609 0.260 0.023* *p≤0.05; **p≤0.01; ***p<0.001. 4.4.2 Software engineering The p-values of chi-square results are displayed in Table 7. The significant value is marked * for p≤0.05, ** for p≤0.01 and *** for p<0.001. Among seven areas of software engineering, none of the tests for company types have a significant chi-square value; hence, we can not reject the null hypothesis H20. We also observed the effect of geographical location variables, as significant p-values are found in six out of seven software engineering activities (requirement engineering, UX Design, architecture, process and Project Management). The perceived impact of WFH and competence needs can be different between small and large teams, between company newly established and long-time companies and companies in different continents. P11: There is no significant difference found in the impact of WFH on Project Management and Software Engineering activities between startups and established companies. P12: There is a significant difference in the impact of WFH on Project Management and Software Engineering activities among software companies in different continents. 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 22 NGUYEN-DUC et al. 5DISCUSSION The summary of our findings is given in Table 8. In this section we discuss how our findings relate to existing studies, threats to validity and implications for research and practice. 5.1 Answering and discussing the RQs The findings for each RQs and discussion in line with existing studies are given in Sections 5.1.1 to 5.1.5 5.1.1 RQ1: How does the way of working in software teams change when WFH is adopted? Byfilteringparticipantswithoutawarenessofchangingworksituationsintheircompanies,weacquiredthedatasetof297 respondents who experienced the impact of COVID-19 and changing work arrangements on their software development projects. Previous work in GSD has described a distributed work setting in five dimensions: geographical, temporal, cultural, work and organizational dimensions. The shift in working patterns happened mainly in geographical dimensions, with project team members moving from offices to fully or mostly WFH situations. The extent of external collaboration seemstohaveremainedthesameduringthepandemictime.Theamountofinternalcollaborationincreasesovertemporal dimensions. The amount of external collaboration (i.e., interaction with customers) has not significantly changed (PO1). TABLE 8 Summary of the findings. POs Research findings RQs PO1 The WFH arrangement is characterized by geographical shift (mainly from offices to home), and increase internal collaboration within and across time zones. External collaboration does not significantly change in the new working mode RQ1 PO2 There are a wide range of tools being used to support collaboration and socializing during WFH. The top four collaborative types of tools are video conferencing, instant messenger, cloud storage and calendar sharing RQ1 PO3 Different mechanistic coordination approaches are introduced and implemented for WFH besides the adoption of online training and social media for work RQ1 PO4 It is a mixed picture of how software project management is affected by the new work arrangement. Team and task awareness do not seem to be a problem. It is perceived as more difficult for project planning than it was before RQ2 PO5 The impact of WFH on requirement engineering tends to be more negatively perceived in activities like requirement gathering and customer involvement than in activities like requirement prioritization and management RQ3 PO6 Among SE activities, software architecture shows the least impact by new work arrangement. Many acknowledge the positive impact of WFH on architectural activities and quality RQ3 PO7 Both negative and positive impacts are found on frequency of releases, quality of code, developers’ attitudes, accumulated technical debts and technical risks RQ3 PO8 Both negative and positive impacts are found on development testing, user testing and code inspection RQ3 PO9 Processes and practices that embrace the adoption of tools are important to reduce many of negative impacts of WFH on Software Engineering activities RQ3 P10 All Software Engineering tasks tend to take more time to complete than it was before RQ3 P11 There is no significant difference found on the impact of WFH on Project Management and Software Engineering activities between startups and established companies RQ4 P12 There is a significant difference in the impact of WFH on Project Management and Software Engineering activities among software companies in different continents RQ4 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License NGUYEN-DUC et al. 23 The increased amount of collaboration over the Internet is supported by the increased addition of a wide variety of collaborative toolsets (PO2). In addition, different mechanistic coordination approaches, social media and online training are introduced in many projects to compensate for the lack of physical appearance (PO3). We observed connections between PO2 and PO3. Coordination mechanisms (i) define how collaborative tools can be used, for example, setting frequency for working sessions or meeting online; (ii) support achieving effectiveness from collaborative tools, for instance, ‘Camera on while working/meeting’ linked to the adoption of video conferencing tools; and (iii) are enabled by the collaboration in place, that is, online coaching and training via video conferencing and collab tools.It seemsthattoolsandprocessesareintegral tothenew workingarrangements,andWFH isnota completelynovel situation. 5.1.2 RQ2: How is project management impacted when WFH is adopted? Project teams are equipped with new/ enhanced adoptions of collaboration tools and coordination and control mechanisms (from RQ1). In this context, respondents still report an additional project delay during the pandemic time. This aligns with the tendency for longer software engineering task durations observed in PO10. However, we are not able to distinguish between projects that started before COVID-19 (in other words, the work arrangement shift occurred in the middle of the projects), or projects that started during the COVID-19 time. A majority of respondents have agreed on the difficulty of planning resources, scope, time and risks in the WFH environment.Thismay not be a surprise since greateruncertaintiesandreducedvisibility of changes areexpected in this work arrangement. Project managers seem to not agree on the difficulty experienced in having an overview of who does what in their teams. As this was already documented as a challenge with tool adoption in GSD literature,59 we can argue that there is a certain level of preparation and maturity in tool adoption that increases the effectiveness of tracing and monitoring software projects. 5.1.3 RQ3: How are software engineering activities impacted when WFH is adopted Ford et al. conducted both qualitative and quantitative research on software engineers at Microsoft during Spring 2020.15 The authors reported a mixed experience for software engineers when working from home. They found that for the same factor, one engineer can perceive it as a benefit and another can perceive it as a challenge. Similarly, Smite and her colleagues conducted a study on software projects and did not observe a significant impact of COVID-19.17 Our work confirmsthepreviousobservationsinalargerandmorediversesetofglobalsoftwareprojects.Wedocumentedamajority numberofanswersof‘noimpact’or‘nochange’inWFHsituations.Thetendencyofanswerswitheithernegativeimpacts or positive impacts to software engineering activities is noted in the POs PO5 to PO10. The impact of WFH on requirement engineering tends to be significant and negative on activities involving stakeholder interactions (P5). This agrees with observations from a previously reported study that reports, “Understanding how the system under design will be used is challenging when site visits and observation are unavailable.”60 Our study characterizes these challenges on a large scale. Software architecture (both decisions and quality) is the least impacted by WFH situations, amongst those activities studied (PO6), followed by software implementation (PO8). Smite et al. revealed that during WFH, software engineers continue committing code and carrying out their daily duties without significant disruptions.16 The technical nature of the tasks might be less impacted by the social and organizational changes. In many situations, architects and developers enjoyedtheflexibilityand‘focused time’toimprovetheirwork.Onthenegativeside,developersalsoreportedchallenges with well-beingness, energy level and work quality. UX design shares a common challenge with certain requirement engineering activities (customer study, user validation) and general project tasks (lack of physical interaction). The impact of the workplace on creativity during UX design might be a relevant topic to explore, however, we did not acquire enough data to draw further conclusions on that. Quality assurance shows a mix of negative and positive perceived impacts on both the code inspection development testandusertest(PO8)Itisrelativelymorechallengingforremoteacceptancetestingduetonewwaysofcommunication betweenthedevelopmentteamand theusers. Weobservedreporteddifficultsituationswhenfewerautomatedtestswere in place, where there was lack of mature testing and on-boarding processes and explicit ways to ensure the work quality. 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 24 NGUYEN-DUC et al. FrompreviousPOs,wecanargue thatprocessesandpracticesareimportant toprepareforWFH,tomaintainproductivity and quality (PO9). Task dependency might be an important topic for preparing a process for WFH project. Smite et al. and Rodeghero et al. described socializing, communication and collaboration, and onboarding practices that have been adapted to a WFH context.16,61 While we cover some popular mechanisms in our survey, that is, turn-on cameras, social network for socializing, more meetings, and so forth, we lack insights on their effectiveness. Kettunen et al. conducted a survey on Agile practices during the pandemic in Sweden and Finland.62 When asked about the impact of the pandemic impact their companies, the authors reported that 53% of the respondents had perceived negative impacts and 33% of the respondents had perceived positive impacts. Our results tend to align with this observation, but we further provide specific details on which software engineering roles are affected and which aspects are perceived to be either negatively or positively impacted. Agile is also mentioned by some respondents as a useful approach in their new work arrangement. On one hand, the Agile mindset might help in reacting to uncertainties and changes introduced during the pandemic time. On the other hand, Agile does not encourage formal collaboration and plan-drivenwork,whichisshowntobeincreasinglyadoptedduringWFH.Thisisaninterestingresearchareathatneeds more insight. 5.1.4 RQ4: Is there any difference between startups and established companies regarding the above impact? It comes as a surprise that no difference was found regarding project management and software engineering (PO11) between startups and established companies. We can suggest three ways to explain this observation. Firstly, software startups might be agile in reacting to influences and that helps to make them as prepared as established companies, who have resources and competence for the process of work adaptation. Secondly, software startups might face the impact of thepandemicinotheraspects,suchasmarketandfinance.Theimpactofthepandemiconmanagementandengineering mightbeoverlooked.Thirdly,‘startups’mightnotbeaheterogeneousgroupandtheyareperceivedandevolvedifferently in different cultures and locations. Hence, our samples include various ‘types’ of startups that do not show a common work pattern. An interesting observation is a relationship between geographical locations and the impact of WFH on both project management and software engineering activities (PO12). This observation might be explained in two ways. Firstly, the impact of the pandemic, the macro environment, the level of reaction to the pandemic, and the condition of WFH might be different across the continent (e.g., Europe vs. Asia). Secondly, the characteristics (scope, maturity, types) of software engineering tasks might be differently defined and perceived across investigated locations. In any case, it seems that the geographical dimension introduces a boundary to any general strategy for communication, collaboration, coordination, management, and engineering activities during WFH situations. 5.2 Threats to validity We will discuss validity according to the four perspectives presented by Wohlin et al.,63 complemented by survey-specific validity aspects.64,65 Construct validity concerns the relationship between a theory behind an investigation and its observation.63 As the goal of the survey is to gain industrial insights on WFH and related practices, we do not aim at fully developing or validating hypotheses. However, the observations from our study can give hints for further research about the working environment (i.e., WFH) and different properties of software companies and software startups. To enhance the construct validity,weusedvalidatedscalesforsoftwareengineeringactivitiesandprojectmanagement.Wearealsoconfidentabout the confidentiality and anonymity of the respondents, hence reducing as much bias as possible. Internal validity deals with the relationship between a treatment and its results.63 We have a filtering question so that only respondents who experienced an impact on their work and their companies can answer questions. Our survey platform automatically collected log information, such as start date, completion time, and IP of the respondent. These data also helped us to filter poor-quality data. An inherent threat to survey research is that surveys can only reflect respondents’ perceptions rather than objective measurement. To some extent, we aim at revealing exactly those perceptions. To make the questions understood in the same ways by all respondents, we reviewed and revised them several times (Section 4.2). The survey versions were 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License NGUYEN-DUC et al. 31 TABLE A2 The classification rules for defining startups. Company type Last-known state of the main product Status (classified) Established Not applicable Established Established Other Established Established Product is rather stable, the focus is on gaining customer base. Established Established Product is stable, market size, share and growth rate are established. Focus is set on launching new variations of the product. Established Established Product was released to the market and is actively developed further with customer input. Established Established A product prototype is developed and has not yet been released to market. Startup Startup A product prototype is developed and has not yet been released to market. Startup Startup Not applicable Startup Startup Other Startup Startup Product is rather stable, the focus is on gaining customer base. Startup Startup Product is stable, market size, share and growth rate are established. Focus is set on launching new variations of the product. Established Startup Product was released to the market and is actively developed further with customer input. Startup Not sure A product prototype is developed and has not yet been released to market. Startup Not sure Not applicable Unknown Not sure Other Unknown Not sure Product is rather stable, the focus is on gaining customer base. Established Not sure Product is stable, market size, share and growth rate are established. Focus is set on launching new variations of the product. Established Not sure Product was released to the market and is actively developed further with customer input. Startup 1097024x, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/spe.3306 by University Of Jyväskylä Library, Wiley Online Library on [05/01/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License