Design for the analysis of variability management in the industry

Design for the analysis of variability management in the industry Ana Eva Chacón Luna Universidad Estatal de Milagro Milagro, Ecuador [email protected] Antonio Manuel Gutierrez Fernandez CDL VaSiCS, LIT CPS Lab, Johannes Kepler University Linz, Austria antonio.gutierr[email protected] David Benavides Universidad de Sevilla Sevilla, Spain [email protected] Lidia Fuentes ITIS Software, Universidad de Malaga Malaga, Spain [email protected] ABSTRACT Software Product Lines (SPL) enable systematic reuse within an organization, which reduces costs, effort, development time and the average number of defects per product. However, there is no empirical evidence on the adoption of SPLs in software development companies for product customization. To assess the need for SPL technologies in real software development, it is necessary to analyze the variability practices in companies to develop their products, pointing out the strengths and weaknesses of their approaches. To this end, we present the design of a study to evaluate how companies perform variability management. We start from the assumption that most companies manage variability, but not many of them are aware of the software product lines. CCS CONCEPTS •Software and its engineering → Software product lines;• General and reference →Empirical studies. KEYWORDS software product lines, variability management, empirical studies ACM Reference Format: Ana Eva Chacón Luna, Antonio Manuel Gutierrez Fernandez, David Benavides, and Lidia Fuentes. 2022. Design for the analysis of variability management in the industry. In 26th ACM International Systems and Software Product Line Conference - Volume B (SPLC ’22), September 12–16, 2022, Graz, Austria. ACM, New York, NY, USA, 6 pages. https://doi.org/10.1145/3503229.3547060 1 INTRODUCTION Software product lines (SPL) enable systematic reuse within an organization, leading to reduced cost, effort, development time, and average number of defects per product. Software product line engineering (SPLE) [ 4 , 24 – 26 ] has gained significant attention in recent years. Several studies have reported the benefits in using the strategies provided by this paradigm to develop a wide range of software products [ 6 , 30 , 32 ]. However, there are few case studies of the adoption of SPL lines in firms, and few experiments have been reported in the literature about their adoption in the industry [ 33 ]. We believe that this gap makes it difficult to strengthen or elaborate adjustments or improvements to SPL frameworks. For this empirical analysis, it is necessary to have a significant sample of companies that manage variability in their software products. Variability management is a practice that any company with a portfolio of similar software products has to face. Especially in recent years, many highly configurable systems are appearing with increasing demand for customization. However, we understand that there are a significant number of companies facing this challenge that are not familiar with SPLE techniques. Therefore, we believe that it is important to evaluate how companies perform variability management in these cases in order to understand the problems and challenges in the lack of adoption of these techniques. We hypothesize from the empirical study that most companies manage variability, but many of them are not aware of the SPL techniques. Previously, a preliminary study based on a case study has been presented in [ 5 ]. In this paper, we present the design of a survey to collect information on how companies manage variability in the products they develop, inquiring about their product customization/customization practices. The survey should consider a wide geographical area to improve the generalization of the results. The advantage of applying such analysis is to know the degree of adoption of SPL framework practices and to define steps that allow companies to manage variability with SPL techniques to improve the development process. In this work we set out to define a study protocol about variability management by companies, even if they do not know the domain of SPL. The framework of this work is based on the framework described by Pohl et al. [ 24 ], specifically considering the subprocesses of product management, domain requirements engineering and application requirements engineering. Specifically, in this paper we assume the hypothesis that in all companies that handle a portfolio of related products it is necessary to use techniques to manage variability and reuse of common components. Thus, this study aims to answer the question: Is there a relationship between the offer of similar or customizable products and the use of variability management mechanisms? SPLC ’22, September 12–16, 2022, Graz, Austria Chacon and Gutierrez, et al. To answer this question, we proposed the following research sub-questions. The objective of these sub-questions is to find out to what extent companies develop related products in their product portfolio and the knowledge and use of techniques for variability management. In this way we will be able to analyze whether there is a relationship between the product portfolio and these techniques. • RQ1.1. Do they manage a product portfolio in which similar or related products appear? The objective is to identify whether the product portfolio includes related products through customization or adaptation of existing products. • RQ1.2. Are product characteristics analyzed so that a family or portfolio of related products is identified? The objective of this sub-question is to identify whether the company systematically addresses the development of related products through feature analysis, derivation of a common platform and/or maintenance. • RQ1.3. Are the requirements of each project systematically analyzed so that common attributes and variables can be compared for potential reuse? The objective of this question is to identify whether the analysis and comparison of common and variable requirements in the product portfolio is systematically addressed to address requirements reuse. • RQ1.4. Do they have knowledge of SPL domain terms? The objective of this question is to identify the degree of knowledge of specific SPL vocabulary to analyze the relationship between this knowledge and the management of a portfolio of related products. For this purpose, we develop a study instrument in the form of a survey. The survey is descriptive in nature to collect information from a large group of people in a systematic way, allowing future work to look for patterns in the data and generalize to a broader population. This paper is structured as follows, in Section 2 the SPL definition is contextualized highlighting the processes of product management, domain requirements management and application of Pohl’s framework [ 24 ], in addition the works related to the realization of empirical studies in the area of SPLs are analyzed. In Section 3 the proposed study instrument is presented, following the guidelines of Kitchenham et al. [ 16 ]. Section 4 presents the next steps to apply the design and potential threats and the developed questionnaire is included in the AppendixA. 2 CONTEXT AND RELATED WORK 2.1 Software product lines In SPL engineering there are several frameworks related to the processes, concepts or activities used. In this case, we start from a widely accepted differentiation for all proposals between domain engineering and application engineering proposed by Weiss et al. [ 34 ]. According to Pohl et al. [ 24 ], SPL engineering consists of the following two processes. First, domain engineering process, which aims to produce the platform, including common and variable application functionalities. It consists of five key subprocesses: product management, domain requirements engineering, domain design, domain realisation, and domain testing. Second, application engineering process, which aims to achieve the highest possible reuse of domain assets when developing a product. It consists of four subprocesses: application requirements engineering, application design, application realisation and application testing. Product management deals with the management aspects of SPL and, in particular, market strategy. Its objective is the management of the organization’s product portfolio, using scoping techniques to specify what is in and out of scope for the product line. On the other hand, domain requirements engineering includes the activities to elicit and document the common and variable requirements of the product line. The objective is to determine reusable requirements, based on the product line variability model. Domain requirements engineering differs from requirements engineering for individual systems because requirements are analyzed to identify those that are common to all applications and those that are specific to particular applications. Finally, application requirements engineering performs the activities necessary to develop the application requirements specifications. The main responsibility of application requirements engineering is to detect the trade-offs between application requirements and available platform capabilities. Application requirements engineering differs from individual system requirements engineering primarily because it analyzes the convergence of application requirements and domain requirements to assess and document the required customization effort. 2.2 State of the art A study on how companies that are unfamiliar with SPL concepts manage variability was conducted in [ 5 ]. It was concluded that the observed company had some variability management practices, such as reuse of product assets, among other aspects. This study was conducted by implementing a case study. Although there are no previous documented studies on SPL practices in companies that are still unaware of SPL, there are several studies that can be considered as a basis for our work. In [ 1 , 2 ] the authors present studies on SPL adoption in small and medium sized companies in which they aim to justify the use of agile methods and the use of a multi-method approach respectively for SPL adoption. Also,Krüger and Berger [20] combine an interview to participants from large development organizations with a systematic literature review to compare the costs of using clone-and-own approaches and platform-oriented reuse. They conclude that "developing for platform-oriented reuse is more expensive, but simultaneously reduces reuse costs; and that platform-orientation results in higher code quality compared to clone & own". In [ 2 ], to achieve their objectives, they present a unique case study based on software scoping and requirements, since those disciplines are the ones that define the SPL lifecycle. Bastos et al. [ 1 ] conclude that the significant findings are an important step in establishing guidelines for SPL adoption. Conducting a case study aims to determine how companies unfamiliar with SPL concepts manage variability and to verify which SPL concepts are used in practice. As in our goal, Bastos also aims to define some transitional practices for implementing SPL techniques. In [ 27 ], a comparative study of the trend of SPL research in industry and academia is presented. This study revealed that more than one Design for the analysis of variability management in the industry third (34%) of the research in academia only provide artificial/toy examples and 27% of the papers do not present any evaluation. In the area of testing, several papers discuss software testing of SPLs in particular [ 7 , 10 , 21 , 22 ]. Recently, Berger et al. [ 3 ] reported a survey of twelve medium to large companies-from fields such as automotive, aerospace, and railway systems-on their adoption of variability management techniques. The results show that SPLE techniques in general are increasingly adopted, but as far as the use of such approaches for testing is concerned, there is little evidence. This feeds the assumption that companies deal with variability in their testing implicitly rather than systematically. Rincón et al . [28] proposes a framework for assessing how companies need and are prepared to adopt product line strategies that could be used in combination with an empirical study to serve as a starting point for the adoption of variability management practices. 3 METHODOLOGY According to Fink et al. [ 11 ], surveys are a method of data collection and should be used under appropriate conditions. In their work, it is also stated that a survey is a comprehensive research method and is not just a questionnaire or checklist to collect information. For the present study, a descriptive survey [ 16 ] has been proposed, which proposes to collect information and evidence the distribution of the respondents’ statements in relation to how they manage the variability of the products they develop. In addition, this empirical strategy allows describing, comparing and explaining the knowledge, attitudes and behaviors about the definition and application of SPL in products. For the planning of the study, we propose to follow the Kitchenham’s guidelines [ 16 ], which are summarized in Figure 1. Specifically, we divide the actions into 2 stages: Planning the study and Conducting the study. In this article we detail the actions corresponding to planning, leaving the conduct of the study as future work. •Identify research objectives •Plan the study instrument •Design the questionnaire •Define the data collection mechanism •Validation of the study instrument •Conducting the survey •Detailed case study 3.1 Identify research objectives According to Fink [ 11 ], different survey designs serve different objectives based on the classification can be experimental and descriptive. Experimental design is characterized by providing for the comparison of two or more groups, at least one of which is experimental [35]. Descriptive designs produce information about groups and phenomena that already exist. The present study applied a descriptive survey and the main objective of the study is to determine how variability is managed by companies offering a family of software products, regardless of whether they are aware of SPL concepts. We start from the hypothesis that, although they do not formally apply SPL techniques, they do apply solutions for variability management. To answer the proposed research questions, we will define the questionnaire (attached in the Appendix A). 3.2 Plan the study instrument The researchers developed the questionnaire over a period of several months within the context of the TASOVA (Network on new trends in software Architecture and variability) network 1 . For this study, the questionnaire will be sent by e-mail (since the researchers are not co-located) as of April 2021. In order to obtain the size of the study sample, the set of development companies was considered. For an initial study, the 10 nodes of the TASOVA network geographically spread throughout the country and extending the population through contacts and e-mail, thus also including international participants, have been considered. The researchers’ decision to send the survey in this way is based on a pilot survey that was sent to Ecuadorian companies, where it was sent to the entire population, with a response rate of only 15%, since we considered changing the sending strategy and abandoning the results obtained in the first attempt to collect information, considering that it would be more efficient to collect data through contacts with companies close to the nodes of the TASOVA network. 3.3 Design the questionnaire The survey consists of a questionnaire that was developed following the steps designed by [ 11 , 19 ]: search the relevant literature; construct an instrument; evaluate the instrument; and document the instrument. The survey instrument was motivated by previous work related to the adoption of the SPL [ 1 , 8 , 9 , 12 , 24 , 29 , 31 ], survey research [ 13 – 15 , 17 , 18 , 23 ], and from industrial evidence (case study [ 5 ]). A pilot project was conducted with the participation of a nearby development company. The main improvements were related to the sequence and organization of the questions. The introduction of the questionnaire included the motivation of the study (to know the variability management in companies that do not know SPLs), the importance of the participation of each individual and how the confidentiality of the collected answers would be implemented. Kitchemham[ 16 ] proposes that in self-administered questionnaires, the questions should be mainly closed. Thus, the study questionnaire is composed of 33 closed questions and was designed to be completed in twenty minutes. See Appendix A for the complete questionnaire. The questionnaire has 7 sections, including 14 questions with answers designed with ordinal frequency scales. The first 3 sections include questions on the profile of the company, the respondent and the products developed by the company, respectively. Specifically, section 1 (questions 1 to 7) asks about the scope, type and size of the company. Section 2 (questions 8 to 14) asks about the respondent’s professional situation, such as experience in the software production sector and the relationship with his position within the company. In section 3 of the form (questions 15 to 17) 3 questions were asked about the profile of the products they 1http://tasova.uma.es/ Chacon and Gutierrez, et al. Stage Planning the study Conducting the study Action Result Research Questions Identify the research objectives Design the questionnaire Define the data collection mechanism Questionnaire Study instrument planning Validation of the study instrument Participating companies Conduct the survey Revised questionnaire Detailed case study Results survey Results reviewed Plan the study instrument Data collection mechanism Figure 1: Methodology for survey design develop, size of the projects and number of products they currently have on the market. The research sub-questions are answered through sections 4, 5 and 6. Specifically, section 4 (questions 18 to 21) corresponds with research question RQ1.1 and includes questions on how companies handle customization, adaptation or extension of an already developed product. Section 5 (questions 22 to 25) on managing product variability, RQ1.2. Section 6 (questions 26 to 29) includes questions to learn about practices in requirements management and answer RQ1.3. Finally in section 7 (questions 30 to 33) inquire about the respondent’s level of knowledge on the topic of variability, RQ1.4. 3.4 Define the data collection mechanism According to Kitchenham [ 19 ] according to the way in which the survey is conducted, it can be classified into: self-administered questionnaires, telephone surveys and one-on-one interviews. In this study, considering the COVID context and the ubiquity of researchers we consider descriptive self-administered questionnaires. For this purpose, a self-administered questionnaire is created in April 2021 through Google Forms that can be sent by e-mail. 3.5 Validation of the study instrument During the development of the questionnaire, several review cycles were included, during which the formulation of the questions was discussed and agreed upon. The validation of the instrument was carried out, forming a discussion group with the objective of verifying that the questions proposed in the instrument satisfied the defined research questions, that is, the reliability of the application of the instrument was evaluated. In addition, it was verified that the method of data analysis was compatible with the answers that would be obtained from the application of the questionnaire. The understanding of these questions by participants outside the focus group was also validated by conducting a pilot test of the application of the questionnaire. 3.6 Conducting the survey A non-probabilistic sample of the ’snowball’ type is considered for the realization of the survey. The companies under study have registered activities in the software development sector. For the data analysis, it is considered the coding of the data in SPSS (which are initially collected in a calculation program). First it is necessary to check the correctness of the data and filter out incomplete surveys or surveys with a minimum number of responses. 3.7 Detailed case study Following the survey, we consider conducting interviews to obtain more information and a better understanding of the responses collected from the questionnaires in the previous step using an interview guide. The survey results can be used during the interview to identify areas where differences or ambiguities are detected. 4 CONCLUSIONS AND APPLICATION OF THE STUDY We believe that although SPLs have received significant attention from software engineers since the 1990s, there is insufficient empirical evidence of the degree of adoption of SPL contributions. This type of study can allow us to assess the degree of adoption of techniques for variability management and to identify problems in the application of these techniques. To carry out the survey, more than 180 participations have been requested so far. The data collected are still in the process of being analyzed, with a response rate of over 75% for the time being. It should be noted that due to the geographical distribution of the TASOVA network, the participants are mainly distributed in the several Spanish regions. Although there are international participants, their number is not significant, so it would be necessary to increase the application of the study to avoid possible bias due to the national nature of the study. The results of the empirical study can be extended with detailed case studies that will allow us to go deeper into the problems encountered in order to adopt variability management techniques. 4.1 Threats to the validity Construct Validity. A common threat to the validity of such an empirical study is a misunderstood question from a participant. To mitigate this, we translated the survey into Spanish, the native language of most of the participants. Internal Validity. An additional threat is the selection of participants. As we pointed out, the participants in the survey were selected by the authors contact network. Also, the use of contact groups can limit the sample size leading to reduce the validity of the results. We intend to extend this network to get a significant number of participants. External Validity. Most of the members in the author’s networks are Design for the analysis of variability management in the industry in Spain, which may introduce some cultural biases. We consider extend the network to include a higher number of international participants in the sample through research partners. ACKNOWLEDGMENTS This work was supported by the project OPHELIA (RTI2018-101204-B-C22) funded by FEDER/Ministry of Science and Innovation — State Research Agency; the COPERNICA (P20_01224), RHEA (P18-FR-1081) and METAMORFOSIS (FEDER_US-1381375) projects funded by Junta de Andalucía . The financial support by the Austrian Federal Ministry for Digital and Economic Affairs, the National Foundation for Research, Technology and Development, and the Christian Doppler Research Association is gratefully acknowledged. REFERENCES [1] Jonatas Ferreira Bastos, Paulo Anselmo da Mota Silveira Neto, Pádraig O’Leary, Eduardo Santana de Almeida, and Silvio Romero de Lemos Meira. 2017. Software product lines adoption in small organizations. Journal of Systems and Software 131 (2017), 112–128. [2] David Benavides and José A. Galindo. 2014. Variability Management in an Unaware Software Product Line Company: An Experience Report. In Proceedings of the Eighth International Workshop on Variability Modelling of Software-Intensive Systems (Sophia Antipolis, France) (VaMoS ’14). Association for Computing Machinery, New York, NY, USA, Article 5, 6 pages. https://doi.org/10.1145/2556624. 2556633 [3] Thorsten Berger, Jan-Philipp Steghöfer, Tewfik Ziadi, Jacques Robin, and Jabier Martinez. 2020. The State of Adoption and the Challenges of Systematic Variability Management in Industry. Empirical Software Engineering 25 (2020), 1755–1797. [4] Jan Bosch. 2000. Design and use of software architectures: adopting and evolving a product-line approach. Pearson Education, USA. [5] Ana E. Chacón-Luna, Elvira G. Ruiz, José A. Galindo, and David Benavides. 2019. Variability Management in a Software Product Line Unaware Company: Towards a Real Evaluation. In Proceedings of the 23rd International Systems and Software Product Line Conference - Volume B (Paris, France) (SPLC ’19). Association for Computing Machinery, New York, NY, USA, 82–89. https://doi.org/10.1145/ 3307630.3342421 [6] P. Clements and L. Northrop. 2002. Software Product Lines: Practices and Patterns. Addison-Wesley. https://books.google.at/books?id=tHGFQgAACAAJ [7] Ivan do Carmo Machado, John D. McGregor, Yguaratã Cerqueira Cavalcanti, and Eduardo Santana de Almeida. 2014. On strategies for testing software product lines: A systematic literature review. Inf. Softw. Technol. 56, 10 (2014), 1183–1199. [8] Christof Ebert. 2014. Software product management. IEEE Software 31, 3 (2014), 21–24. [9] Sascha El-Sharkawy, Nozomi Yamagishi-Eichler, and Klaus Schmid. 2019. Metrics for analyzing variability and its implementation in software product lines: A systematic literature review. Information and Software Technology 106 (2019), 1–30. https://doi.org/10.1016/j.infsof.2018.08.015 [10] Emelie Engström and Per Runeson. 2011. Software product line testing - A systematic mapping study. Inf. Softw. Technol. 53, 1 (2011), 2–13. [11] A. Fink. 2003. The Survey Handbook. SAGE Publications. [12] Andreas Helferich, Klaus Schmid, and Georg Herzwurm. 2006. Product management for software product lines: an unsolved problem? Commun. ACM 49, 12 (2006), 66–67. [13] Barbara Kitchenham and Shari Pfleeger. 2002. Principles of survey research: Part 5: Populations and samples. ACM Sigsoft Software Engineering Notes 27 (09 2002), 17–20. https://doi.org/10.1145/571681.571686 [14] Barbara Kitchenham and Shari Lawrence Pfleeger. 2002. Principles of Survey Research Part 4: Questionnaire Evaluation. SIGSOFT Softw. Eng. Notes 27, 3 (may 2002), 20–23. https://doi.org/10.1145/638574.638580 [15] Barbara Kitchenham and Shari Lawrence Pfleeger. 2003. Principles of Survey Research Part 6: Data Analysis. SIGSOFT Softw. Eng. Notes 28, 2 (mar 2003), 24–27. https://doi.org/10.1145/638750.638758 [16] Barbara Ann Kitchenham, David Budgen, and Pearl Brereton. 2015. Evidencebased software engineering and systematic reviews. Vol. 4. CRC press. 426 pages. [17] Barbara A Kitchenham and Shari Lawrence Pfleeger. 2002. Principles of survey research part 2: designing a survey. ACM SIGSOFT Software Engineering Notes 27, 1 (2002), 18–20. [18] Barbara A. Kitchenham and Shari Lawrence Pfleeger. 2002. Principles of Survey Research: Part 3: Constructing a Survey Instrument. SIGSOFT Softw. Eng. Notes 27, 2 (mar 2002), 20–24. https://doi.org/10.1145/511152.511155 [19] Barbara A Kitchenham and Shari L Pfleeger. 2008. Personal opinion surveys. In Guide to advanced empirical software engineering. Springer, 63–92. [20] Jacob Krüger and Thorsten Berger. 2020. An Empirical Analysis of the Costs of Clone- and Platform-Oriented Software Reuse. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Virtual Event, USA) (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA, 432–444. https: //doi.org/10.1145/3368089.3409684 [21] Beatriz Pérez Lamancha, Macario Polo, and Mario Piattini. 2010. Systematic Review on Software Product Line Testing. In Software and Data Technologies - 5th International Conference, ICSOFT 2010, Athens, Greece, July 22-24, 2010. Revised Selected Papers. Springer Berlin Heidelberg, Berlin, Heidelberg, 58–71. [22] Roberto E. Lopez-Herrejon, Stefan Fischer, Rudolf Ramler, and Alexander Egyed. 2015. A first systematic mapping study on combinatorial interaction testing for software product lines. In 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, Graz, Austria, 1–10. https://doi.org/10.1109/ICSTW.2015.7107435 [23] Shari Lawrence Pfleeger and Barbara A Kitchenham. 2001. Principles of survey research: part 1: turning lemons into lemonade. ACM SIGSOFT Software Engineering Notes 26, 6 (2001), 16–18. [24] Klaus Pohl, Günter Böckle, and Frank J van Der Linden. 2005. Software product line engineering: foundations, principles and techniques. Springer Science & Business Media. [25] Mikko Raatikainen, Juha Tiihonen, and Tomi Männistö. 2019. Software product lines and variability modeling: A tertiary study. Journal of Systems and Software 149 (2019), 485–510. [26] Rick Rabiser, Klaus Schmid, Martin Becker, Goetz Botterweck, Matthias Galster, Iris Groher, and Danny Weyns. 2018. A Study and Comparison of Industrial vs. Academic Software Product Line Research Published at SPLC. In 22nd International Systems and Software Product Line Conference (SPLC 2018). ACM, 14–24. [27] Rick Rabiser, Klaus Schmid, Martin Becker, Goetz Botterweck, Matthias Galster, Iris Groher, and Danny Weyns. 2018. A study and comparison of industrial vs. academic software product line research published at SPLC. In Proceedings of the 22nd International Systems and Software Product Line Conference-Volume 1. 14–24. [28] Luisa Rincón, Raúl Mazo, and Camille Salinesi. 2018. APPLIES: A framework for evaluAting organization’s motivation and preparation for adopting product lines. In 2018 12th International Conference on Research Challenges in Information Science (RCIS). 1–12. https://doi.org/10.1109/RCIS.2018.8406641 [29] Holger Schackmann and Horst Lichter. 2006. A cost-based approach to software product line management. In 2006 International Workshop on Software Product Management (IWSPM’06-RE’06 Workshop). IEEE, 13–18. [30] Klaus Schmid and Martin Verlage. 2002. The economic impact of product line adoption and evolution. IEEE software 19, 4 (2002), 50–57. [31] Thomas Thüm, Sven Apel, Christian Kästner, Ina Schaefer, and Gunter Saake. 2014. A classification and survey of analysis strategies for software product lines. ACM Computing Surveys (CSUR) 47, 1 (2014), 1–45. [32] Frank J Van der Linden, Klaus Schmid, and Eelco Rommes. 2007. Software product lines in action: the best industrial practice in product line engineering. Springer Science & Business Media. [33] Frank J Van der Linden, Klaus Schmid, and Eelco Rommes. 2007. Software product lines in action: the best industrial practice in product line engineering. Springer Science & Business Media. [34] David M Weiss and Chi Tau Robert Lai. 1999. Software product-line engineering: a family-based software development process. Vol. 12. Addison-Wesley Reading. [35] Claes Wohlin, Per Runeson, Martin Höst, Magnus C. Ohlsson, Björn Regnell, and Anders Wesslén. 2012. Experimentation in Software Engineering. Springer Berlin Heidelberg, Berlin, Heidelberg. A QUESTIONNAIRE SPLC ’22, September 12–16, 2022, Graz, Austria Chacon and Gutierrez, et al. Software variability management survey Section ID Question Answers Q1 Company name Open Q2 Company web Open Q3 Scope of the company’s market National/ International/ Other Q4 Type of company Public/ Private Q5 Company size Small (0-50)/ Medium (50-250)/ Large (+ than 250) Company profile Q6 How many years has the company been in business? 3 years or less/ Between 4 and 10 years/ More than 10 years Q7 Name and Family name Open Q8 Contact telephone Open Q9 Contact email Open Q10 Would you like to be contacted to send you the results or progress of the study? Yes/No Q11 City from where you work Open Q12 Years of work experience in the software production industry 3 years or less/ Between 4 and 10 years/ More than 10 years Participant profile Q13 Which of the following job descriptions most closely matches your job description? Manager/ Consultant/ Department head or project manager/ Analyst/ Programmer Q14 What type of products does the company develop (multiple choice)? Web Applications/ Mobile Applications/ Control Systems/ Critical Systems/ Internet of Things/ Data Analytics/ ERPs/ Security/ Information Systems/ Other. Q15 How big are the projects developed by the company? Small (less than 100,000 euros)/ Medium (between 100,000 and 500,000 euros)/ Large (more than 500,000 euros) Profile of the products developed by the company Q16 How many different products does the company currently have on the market and/or in development? Between 1 and 5/ Between 5 and 10/ Between 10 and 20/ More than 20/ Don’t know. Table 1: Company profile sections Software variability management survey Section ID Question Answers Q17 After the execution of a project, another product similar to the one developed in the project is produced. Always/ Frequently/ Sometimes/ Occasionally/ Never Q18 A new customer asks us to adapt a similar product from one already developed/offered. Always/ Frequently/ Sometimes/ Occasionally/ Never Q19 The company’s current methodology allows us to approach new products by customizing or adapting existing products in such a way that we save time and cost. Always/ Frequently/ Sometimes/ Occasionally/ Never Q20 Indicate the methodology you use to successfully approach the adaptation/customization/extension of an already developed product. Open Product customizationadaption Q21 ¿What percentage of projects developed by the company are made from scratch? Between 0 and 25%/ Between 25 and 50%/ 50%/ Between 50 and 75%/ Between 75 and 100% Q22 We prepare documentation detailing the specifications of the characteristics of each of your products. Always/ Frequently/ Sometimes/ Occasionally/ Never Q23 We have identified our own flagship products or platforms from which other products originate for other customers. Always/ Frequently/ Sometimes/ Occasionally/ Never Q24 When we perform evolution or maintenance tasks, we come up with ideas to improve the products already deployed in the market. Always/ Frequently/ Sometimes/ Occasionally/ Never Product variability Q25 If so, in what format do you store this information about the products and their characteristics? A spreadsheet/ A text document/ Web format/ Using specific software (e.g. DOORs))/ Open Q26 We prepare a matrix or similar where the common and variable requirements of each of the company’s similar products are detailed. Always/ Frequently/ Sometimes/ Occasionally/ Never Q27 When making a new product, we perform an analysis to determine which modules/components are common with respect to other existing products, and therefore reusable, and which are different or new. Always/ Frequently/ Sometimes/ Occasionally/ Never Q28 We develop a diagram or documentation that allows us to describe the interrelation between the characteristics of each product developed in the company. Always/ Frequently/ Sometimes/ Occasionally/ Never Analysis of requirements and variability of products Q29 The documentation where we collect the requirements of a new product also includes the analysis of the characteristics of other similar products. Always/ Frequently/ Sometimes/ Occasionally/ Never Q30 Software product lines, SPLs Very High/ High/ Medium/ Low/ Very Low Q31 Software Product Families Very High/ High/ Medium/ Low/ Very Low Q32 (Feature Models, FMs Very High/ High/ Medium/ Low/ Very Low Knowledge of variability terminology Q33 Variability intensive systems Very High/ High/ Medium/ Low/ Very Low Table 2: Sections on development management