On the selection and analysis of software product line implementation components using intelligent techniques

ON THE SELECTION AND ANALYSIS OF SOFTWARE PRODUCT LINE IMPLEMENTATION COMPONENTS USING INTELLIGENT TECHNIQUES JORGE LUIS RODAS SILVA ADVISED BY: PHD. DAVID BENAVIDES AND PHD. JOS ´ EA. GALINDO INTERNATIONAL DOCTORAL DISSERTATION First published in July 2019 by The Department of Computer Languages and Systems ETSI Inform´ atica Avda. de la Reina Mercedes s/n Sevilla, 41012. SPAIN Copyright c MMXIX Jorge Luis Rodas Silva [email protected] Classification (ACM 2012): Categories and subject descriptors: [100] - Software and its engineering: Software development process management. [100] - Software and its engineering: Software product lines. [500] - Information systems: Recommender systems. General Terms: Design, Theory, Algorithms, Performance Additional Key Words and Phrases: software product lines, feature models, automated analysis, recommender systems, implementation components. Support: This work has been partially supported by University of Milagro (UNEMI) with its scholarship program. It has also been partially funded by the EU FEDER program, the MINECO project OPHELIA (RTI2018-101204-B-C22); the Juan de la Cierva postdoctoral program; the TASOVA network (MCIU-AEI TIN2017-90644-REDT); and the Junta de Andalucia METAMORFOSIS project. 3 Don David Benavides, profesor titular del ´ Area de Lenguajes y Sistemas Inform´ aticos de la Universidad de Sevilla y Don Jos´ e A. Galindo, investigador Juan de la Cierva. HACEN CONSTAR que Don Jorge Luis Rodas Silva , Ingeniero en Sistemas Computacionales por la Universidad Estatal de Milagro, Ecuador; ha realizado bajo nuestra supervisi´ on el trabajo de investigaci´ on titulado On the Selection and Analysis of Software Product Line Implementation Components using Intelligent Techniques Una vez revisado, autorizamos el comienzo de los tr´ amites para su presentaci´ on como tesis doctoral al tribunal que ha de juzgarlo. Fdo. PhD. David Benavides y PhD. Jos´ e A. Galindo ´ Area de Lenguajes y Sistemas Inform´ aticos Universidad de Sevilla, Sevilla, julio de 2019 4 5 Yo, Jorge Luis Rodas Silva, con n´ umero de DNI 0921633988, DECLARO Ser el autor del trabajo que se presenta en la memoria de esta tesis doctoral que tiene por t´ ıtulo: On the Selection and Analysis of Software Product Line Implementation Components using Intelligent Techniques Lo cual firmo en Sevilla, julio de 2019. Fdo. Jorge Luis Rodas Silva 6 7 In addition to the committee in charge of evaluating this dissertation and the two supervisors of the thesis, it has been reviewed by the following researchers: •PhD. Edward Mauricio Alferez Salinas (Universit´ e du Luxembourg, Luxembourg) •PhD. Jaime Chavarriaga (Universidad de los Andes, Colombia) 8 A mis padres 16 List of Figures List of Tables 3.1 Metrics and methods for evaluating the performance of recommender systems(from[27]) ...........................................68 4.1 MAYAvs.otherproposals .....................................75 4.2 RESDECvs.othersproposals ..................................77 18 List of Tables Acknowledgements I remember my mother telling me since I was little “Everything you propose in life you will achieve: persevere and resist these words marked my life and became my encouragement voice to cope with the hard facets that appeared along the way and to be able to culminate this long process. When arriving this stage, many things cross my mind, it has been years of ups and downs, but at the same time of a lot of learning both academically and personally. Undoubtedly, perseverance and steadiness have been fundamental pillars to make it. First I want to thank God for having been my light and my guide; then, to the people who accompanied me during this journey and who were an important part of this puzzle. I would like to express immense gratitude to my parents and brothers for having always been there even when thousands of miles away separated us, at the beginning it was hard, but I can tell that it was worth it. To my friends of studies, of long days, with whom we have shared experiences (good and bad) but that in the end allowed us to enjoy the landscape while we were advancing in this adventure that ends today. Then, I want to thank the institution that has trusted me since always, my second family, the place where I was formed and which I am part of now, my beloved State University of Milagro. Thanks to its authorities for giving me the opportunity to fulfill one of my great wishes, to be able to study abroad. I have not disappointed them. Also, to INRIA and Professor Benoit Baudry, for opening the doors of this research center to me and having welcomed me in their DIVERSE working group for approximately sevent months. How can I leave aside Mike, Jorge, Juliana and Mathieu, people I met during this doctoral process and whose contributions supported the fulfillment of the objectives outlined in this thesis. To my friends from the development clan, Robert and Samuel, thank you for being there and holding out my hand. Finally, special thanks to my supervisors David and Jos´ e, to the first for the patience and for teaching me to overcome each challenge I assumed; to the 20 List of Tables second, for having been a counselor and support all the time. Abstract In recent years and with increasing technological advancement companies are no longer focused exclusively on designing a product for a customer (e.g. designing a website for Decameron Hotel), but on producing for a domain (e.g. designing websites for hotels.); that is, designing a product that can easily adapt to the different variations that may exist for the same product and that fits the individual tastes of customers. In software engineering, this can be achieved through the management of Software Product Lines (SPL). A software product line is defined as a set of systems that share a common set of features that satisfy the demand of a specific market. SPL intends to reduce the effort and cost to implement and maintain a set of similar software products over time. However, managing variability in these systems is a difficult task, the greater the number of products, the more complex it is to manage them. Feature models (FMs) are used to represent the common and variable parts of an SPL. Given the high number of products that can be derived from an feature model (FM), the management of feature models become a challenge task. The Automated Analysis of Feature Models (AAFM) is about the use of automated tools to extract information from feature models. The AAFM deals with feature model complexity. However, with current solutions, there are certain scenarios where the configuration of a product becomes a complex activity given the number of components that could exist to implement a certain feature and that feature models can be more complex in terms of layers, features or implementation components. In this thesis, we explore intelligent techniques to solve two problems that arise when managing an SPL: i. On the one hand, we have identified the problems that arise when a developer wants to keep up their applications with the latest advances in technology. The close relationship between application features and platform components that realize these features is hard to track. Developers need to be aware of the consequences on existing applications when the hardware where it will run changes, e.g., when an application shall be 22 List of Tables ported from a smartphone to a tablet computer or more generally, when a platform shall be upgraded to a new version. The different screen sizes and resolutions, the possible absence of a cellular radio or the increased amount of memory may all have positive or negative impacts on an application. In this context, since the features of the application and platform are conceptually separated, their features can be modeled in two separate models. Consequently the traceability between these two layers and the changes that certain features of one layer may affect the other is a problem to be addressed. ii. On the other hand, we have found how complicated it is for the application developer to configure a product when there are variety of implementation components for each feature. For example, a web developer in WordPress manually searches for those components (plugins) that are feasible and most optimal for each web site. This task takes time and does not always guarantee that the selected components are the most suitable (in terms of quality) for the required application. Two scenarios could arise during this configuration: first, the empirical selection of a component, in practice, may not provide the expected results. Furthermore, not having criteria based on other users’ experience regarding these components, could induce a bad selection and achieve a bad experience for the end user. In this context, managing the relationship between implementation components and their features is another problem to be solved. Specifically, the contributions of this thesis are detailed below; Multi-layer feature models: In this area we introduce a framework for the analysis of multi-layer feature models called MAYA. The objectives we pursue with this solution are: i) modeling the variability of software systems in two layers, i.e. a dependency mapping between two feature models to enable automated analysis application features of a system on a top layer and platform features on a bottom layer; ii) a definition of a set of operations that can be used on such models; iii) a reference implementation for multi-layer analysis based on an Android case study and, finally; iv) two empirical evaluations that demonstrate the feasibility of our proposal in practice. Implementation Components: A product configuration is one of the most error-prone activities, even more so when for each feature there is more than one component that implements it. To manage these configurations, we introduced a component-based recommender system called RESDEC List of Tables 23 that facilitates the selection of implementation components when creating products in an SPL. Specifically, the contributions presented with this proposal are: i) modeling of the implementation component selection problem as a recommendation task using collaborative-based and content-based filtering algorithms; ii) design of a prototype componentbased recommender system tool ready to be used and extended to other environments from the selection of implementation components and, finally; iii) an empirical evaluation based on e-commerce websites in WordPress. 24 List of Tables Resumen En los ´ ultimos a˜ nos y con el creciente avance tecnol´ ogico, las empresas ya no se centran exclusivamente en dise˜ nar un producto para un cliente (por ejemplo, el dise˜ no de un sitio web para el Hotel Decameron), sino en producir para un dominio (por ejemplo, el dise˜ no de sitios web para hoteles); es decir, el dise˜ no de un producto que pueda adaptarse f´ acilmente a las diferentes variaciones que puedan existir para un mismo producto y que se adapte a los gustos individuales de los clientes. En la ingenier´ ıa de software, esto puede lograrse a trav´ es de la gesti´ on de l´ ıneas de productos de software (SPL). Una SPL se define como un conjunto de sistemas que comparten un conjunto com´ un de caracter´ ısticas que satisfacen la demanda de un mercado espec´ ıfico. Una SPL intenta reducir el esfuerzo y el costo de implementar y mantener en el tiempo un conjunto de productos de software similares; sin embargo, manejar la variabilidad en estos sistemas es una tarea dif´ ıcil, a mayor n´ umero de productos m´ as complejo se hace manejarlos. Los modelos de caracter´ ısticas (FMs) se emplean para representar gr´ aficamente las partes comunes y variables de una SPL. Dada la gran cantidad de caracter´ ısticas que se pueden derivar de un modelo de caracter´ ıstica (FM), resulta dif´ ıcil de gestionarlos. Para hacer frente a estos problemas se ha propues- to el An´ alisis Autom´ atico de Modelos de Caracter´ ısticas (AAFM) que mediante el uso de herramientas asistidas por ordenador, se ocupa de la extracci´ on de informaci´ on de los modelos de caracter´ ısticas. No obstante, existen ciertos escenarios en los que la configuraci´ on de un producto se convierte en una actividad compleja dado el n´ umero de componentes que existen para implementar una determinada caracter´ ıstica. En esta tesis, exploramos t´ ecnicas inteligentes para resolver dos problemas que surgen al manejar una SPL: i. Por un lado, hemos identificado los problemas que surgen cuando un desarrollador desea mantener sus aplicaciones al d´ ıa con los ´ ultimos avances tecnol´ ogicos. La estrecha relaci´ on entre las caracter´ ısticas de aplicaci´ on y los componentes de plataforma es dif´ ıcil de rastrear. Los 32 Chapter 1. Introduction 1.1 Overview After the industrial revolution at the end of the 1980s, and with the appearance of the machines in industry, the production of goods and services completely changed the way of manufacturing a product; introducing a new production paradigm known as “mass production”. According to Zipkin [102] mass production had, as main objective, the production of a large quantity of the same product using standardized processes in a reduced commercialization time. This type of production mechanism, despite of guaranteeing production in large volumes of products to meet with high demand and immediate delivery, does not completely satisfy the customers since it does not allow the personalization of products according to their particular needs. In a highly competitive market with constant changes, mass production becomes insufficient, transforming its approach to a “mass customization”. According to McCarthy [57], mass customization consists in the production of (massive) products quantities and services in a personalized way to each client, maintaining quality and delivery time. The main advantage of this production system is the immediate response to the specific needs of each client without affecting the cost, achieving to maintain prices equivalent to the production in large quantities and complying with the efficiency of mass production. Figure §1.1 shows a motivating scenario of mass customization, where a website designed for an airline can be implemented in different airlines (e.g. KLM, Avianca and American airlines). Nowadays, with the great technological advances, the industry of the software engineering is not far from this mass customization. In software production, researchers and engineers worked hard to apply techniques of mass customization in software engineering to rise a new development paradigm known as “Software Product Lines (SPL)”. An SPL allows managing of creating reusable software artifacts, as well as to describe variability points and ensure they are reused properly. Clements and Northrop [28] define an SPL as a set of highly variable systems that share a set of common characteristics that meet the specific needs of a market segment. Managing variability in a product line is a difficult and complex task given the large number of possible software variants that could be derived. Generally, variability is represented through feature models (FMs). An FM [42] is a diagram that graphically describes the features of an SPL and 1.1. Overview 33 Massive website customization for airlines Airline website Let’s go flying Web programmer designing a website for airlines Figure 1.1: Websites mass-customization its relationships, as well as the possible restrictions that may arise. The features selection with their respective attribute values defines a configuration [9] (a.k.a. software product) that capture a personalization possibility that the system allows and that are used within the ecosystem where the SPL is developed. The number of configurations resulting from a feature model can grow when adding new features. To manage variability, the literature presents the automated analysis of feature models (AAFM) [15] that is about extracting information from feature models using computer-assisted mechanisms . In spite of the efforts made by researchers in the field of AAFM, there is still a need for solutions that reduce the inconvenience of managing configurations in order to prevent a time-consuming and costly process. One of the common difficulties is how to link the features that are defined at the application level with the platform level features components. In this context, we have identified two problems: i. What happens when we have two feature models and need to connect them to manage the dependencies of the application level with the platform level? ii. How to guarantee the optimal selection of components to implement features during the configuration of a product in an SPL? Figure §1.2 describes the motivation and the main research core presented 34 Chapter 1. Introduction in this document. Software Product Lines PlatformFeatures Radios Input Cellular NonCellular GSM GPRS Wifi BluetoothUMTS Communication Text Voice SMS Call VoIP Data InternetAccess AppFeatures Multi-layer feature model Implementation components Application Platform MAYA Framework towards the analysis of multi-layered feature models Communication Text Voice SMS Call VoIP Data InternetAccess AppFeatures C1 C3C2 C5 C4 Implementation components repository RESDEC Recommender systems for impementation components selection Traceability 1 Components selection 2 Figure 1.2: Overview of this thesis scope Specifically, in this thesis we have addressed the following areas: Multi-layers feature models There are well-established approaches in the literature to analyze variability intensive systems using feature models. However, there is a lack of approaches to analyze the application and platform features in multiple layers. In this first contribution, we present a framework for the analysis of multilayer feature models that is evaluated with an empirical implementation on Android that demonstrate the viability of the approach in real-world scenarios. Implementation components Given the variety of implementation components that can be used to configure product, identifying the best components set is a challenging task due to the high number of combinations and options that can be selected. In this contribution, we present a proposal that, through information associated with the components, introduces a recommender system that is later exploited in three usage scenarios. An empirical evaluation using WordPress information validates the capacity of our approach to guide users in the implementation components selection. 1.2. Research method 35 1.2 Research method In this work we have used the research method proposed by Ida and Ketil [39]. This method, in addition to solving a problem in a particular context, proposes the design and development of new software or the improvement of existing ones. The main focus is to meet all the requirements from the requirements engineering point of view. Note that this research method is focused on technological research. The main steps of the technological research are: iProblem statement: It is focused on identifying the necessary conditions for the new artifacts creation. In other words, the researcher is responsible for seeking the needs for the new technologies development. ii Contribution: The focus of this step is the solution development through the creation of the artifacts. iii Validation: In this last step, the artifacts designed to ensure they meet the necessary requirements to solve the problem initially identified are verified and validated. 1.3 Contributions In this section we summarize the main contributions of our work, which have been published in different conferences and journals. 1.3.1 Summary of contributions In this thesis we address two problems that originates when managing an SPL. First, we analyze the problem that arises when trying to link features that are defined at the platform level with application level features in multiple layers. To solve this, we introduce a framework called MAYA for the analysis of multi-layered feature models (i.e application layer and platform layer) using features attributes information. Then, we analyze the implementation components selection to configure a product of an SPL at the application level, using recommender systems. For this purpose, we designed a prototype of a tool called RESDEC (REcommender System that from selecteD fEatures suggest 36 Chapter 1. Introduction implementation Components) that allows implementation components selection using different recommendation techniques. Finally, we show real cases from the industry that allowed us to validate our contributions. The main objective of this thesis is to provide intelligent techniques that help the software engineer when selecting products that adjust to certain user requirements. That is, when working with multiple layers and when it is necessary to configure features from the implementation components selection. The main contributions of this thesis are summarized below along with the objective pursued in each one of them: i) Automated analysis of two-layered feature models with feature attributes Our first contribution proposes a mechanism to manage the dependencies between two feature models (application and platform) through the link of both models. –Problem statement: The proliferation of features and platforms in variability intensive systems, coupled with substantial technological progress, imposes several challenges for software developers and equipment manufacturers. Typically, the features of the application and platform are conceptually separated and can be modeled in two separated models organized in layers. The problem is the difficulty of tracking one layer features to the second features. This requires a means to model each set of features separately to reflect the possible independent evolution of each layer. –Contribution: We introduce MAYA, a two-layered feature model framework that includes application features of a system on a top layer and platform features on a bottom layer. In addition, a detailed dependency mapping between these two layers using features attributes to enable automated analysis is provided. –Validation: To validate our approach, two real-world empirical evaluations are presented regarding application handling and platform evolution in the application development domain for mobile phones. In this case, we demonstrate the applicability and need for working with twolayer models. See Chapter §5 for more details about this contribution. ii) Selection of software product line implementation components using recommender systems 1.3. Contributions 37 Once we were able to link two feature models, it was necessary to propose a technique to configure a product features from the selection of implementation components. –Problem statement: Selecting the best set of components to implement features in the product configuration of an SPL is a challenging task. In certain scenarios, given the high number of combinations and component options that can be selected, inappropriate selection could result in bad experience when configuring a product. –Contribution: We propose a component-based recommender system, called RESDEC. RESDEC aims to take advantage from a repository of information generated by users to efficiently search for appropriate components to implement features in a specific context. To do this, we adapt a set of algorithms commonly used by recommender systems. –Validation: We demonstrate the effectiveness of our approach through an empirical evaluation in an e-commerce website scenario using Word- Press. The evaluation is based on data from 116,000 users, 680 plugins, and 187,000 ratings. See Chapter §6 for more details about this contribution. 1.3.2 Publications in chronological order Results of our research work have been published in different congresses and journals. In Figure §1.3 we shown a complete list of these publications in chronological order. [2015] In this year, we made a stay at the INRIA research center (Rennes- France) with the DIVERSE research group, which allowed us to carry out formal investigations in a full time schedule. During this period, we began to work on the preliminary core of the dissertation. From the beginning, we were motivated by the advantages of the recommender systems techniques to personalize the user experience in different scenarios; which allowed us to analyze the applicability of these techniques in the configuration of software product lines through user reports. Preliminary results of this work were presented in the conference that is detailed below. •10CCC’15.Jorge Rodas-Silva, Jos´ e A. Galindo, David M´ endez, and David Benavides. Towards testing variability intensive systems using 38 Chapter 1. Introduction 2015 2017 2019 2016 2018 Conferences and Journals TIMELINE Conference 10CCC Bogotá - Colombia Conference JISBD’16 Salamanca - Spain Conference ICITS'18 Santa Elena - Ecuador Journal RISTI - Revista Ibérica de Sistemas e Tecnologias de Informaçã Conference JISBD'18 Sevilla - Spain Journal Journal of Visual Languages and Computing IEEE Access Conference SPLC'19 París - France Figure 1.3: Chronological order of conferences and publications user reviews. Proceedings of the 10CCC - Congreso Colombiano de Computaci´ on, Bogot´ a, Colombia. [2016] In this year, we continued with the stay in the INRIA research center. In this period, we went deeper into the different techniques and algorithms that a recommender system uses and its application in common problems faced by the software engineer when configuring or deploying a product of SPL. Specifically, we analyzed how these algorithms could help in the selection of configurations for the testing and deployment of applications. Results of this approach are published in the conference described below. •JISBD’16 Jorge Rodas-Silva, Javier Olivaro, Jos´ e A. Galindo, and David Benavides. Hacia el uso de sistemas de recomendaci´ on en sistemas de alta variabilidad. XXI Jornadas de Ingenier´ ıa del Software y Bases de Datos (JISBD 2016), Salamanca, Espa˜ na. 1.3. Contributions 39 [2017] In this year, our research was resumed at the University of Seville. In this period, the main research focus was the selection of deployment configurations to implement an SPL product using recommender systems. Parallel to this research, we began to collaborate on the proposal to manage the variability in a two-layer model and the traceability operations to connect these models. Some of the work done in this year, is published in the conference described below. •ICITS’17 Jorge Rodas-Silva, Jos´ e A. Galindo, David Benavides and Robert Soriano. Selecci´ on de configuraciones usando sistemas de recomendaci´ on en Android. The 2018 International Conference on Information Technology & Systems (ICITS), Santa Elena, Ecuador. [2018] In this year, the main contribution for the dissertation of this thesis was the publication of the results obtained when applying recommender systems in the selection of deployment configurations. Later, we designed a prototype tool called RESDEC based on collaborative filtering recommender systems that was implemented using a knowledge base built from Android data. Results of this work are published in the journal and conference described below. • RISTI’18 Jorge Rodas-Silva, Jos´ e A. Galindo, David Benavides, Robert Soriano. Selection of deployment configurations using recommender systems on Android. RISTI - Revista Ib´ erica de Sistemas e Tecnologias de Informac¸˜ a. •JISBD’18 Jorge Rodas-Silva, Jos´ e A. Galindo and David Benavides. RESDEC: Un prototipo de herramienta para la selecci´ on de configuraciones de despliegue basado en Sistemas de Recomendaci´ on. XXIII Jornadas de Ingenier´ ıa del Software y Bases de Datos (JISBD 2018), Sevilla, Espa˜ na. [2019] In this year, results were obtained which contributed to the development and culmination of the doctoral work. In this context, the proposal to connect layers of the application level with the level of platform that began to work in 2017 was published in a journal. Also, the proposals for recommender systems shown above were reinforced, this time focusing on the selection of implementation components to configure features. For this purpose, an improved version of RESDEC was designed in which new algorithms 40 Chapter 1. Introduction of recommender systems collaborative-based and content-based filtering were incorporated. In addition, the proposal was validated using real-world data from WordPress, which allowed us to validate the scope of our approach in the development of a product line based on websites built in this platform. Results of this work are published in the following report and conference. • JCL’19 Michael Letter, Jorge Rodas-Silva, Jos´ e A. Galindo, David Benavides. Automated analysis of two-layered feature models with feature attributes. Journal of Computer Languages; 0.971 Impact Factor (Q3). • IEEE’19 Jorge Rodas-Silva, Jos´ e A. Galindo, Jorge Garc´ ıa- Guti´ errez, David Benavides. Selection of software product line implementation components using recommender systems: An application to WordPress. IEEE Access; 3.557 Impact Factor (Q1) •SPLC’19 Jorge Rodas-Silva, Jos´ e A. Galindo, Jorge Garc´ ıa-Guti´ errez and David Benavides. RESDEC: Online Management Tool for implementation components selection in a Software Product Lines using Recommender Systems. Software Product Lines - 23rd International Conference, SPLC, Par´ ıs, France. 1.3.3 Tools During this PhD, two prototype tools were developed: •MAYA†1a framework towards the analysis of multi-layered feature models (see Chapter §7 for more details about this tool); and •RESDEC†2a component-based recommender system designed to select implementation components to configure features in an SPL (see Appendix §A for more details about this tool). †1https://github.com/FaMaFW/FaMA/tree/branches/fama-two-layers †2http://resdec.com 1.4. Research internships and collaborations 41 1.4 Research internships and collaborations The research results presented in this document and which are part of the contributions developed in the course of the doctorate, were carried out in three different countries under the supervision of different people. Figure §1.4 shows the universities and institutes in which they carried out research, along with each institution hosts and advisors in each country. Ecuador Francia Spain University of Seville David Benavides José A. Galindo PhD. student University of Milagro David Benavides PhD. student INRIA José A. Galindo PhD. student Research VISITS Figure 1.4: Stays to cope with the PhD objectives During the development of this thesis, we worked on stays in France (INRIA), complementing the work done in Sevilla (University of Seville) and Ecuador (University of Milagro). In France, the research stays was at the Institut National de Recherche en Informatique et en Automatique (INRIA). This research center is located in the city of Rennes and is considered one of the most important centers in the field of science and technology development in Brittany. In this center we stayed for a period of seven months between 2015 and 2016, the hosts were PhD. Jos´ e A. Galindo and PhD. Benoit Baudry, this last Senior Researcher of INRIA DIVERSE, department of which I was part. DIVERSE is a group of researchers focused on research in four areas: modeling and languages engineering, advanced testing, DevOps for distributed and heterogeneous system and variability engineering. During this period, the main research topic was the application of recommender systems in the context of software product lines. The results of this 48 Chapter 2. Software product lines 2.1 Introduction Software product lines (SPL) have become a new important paradigm in software development [31]. In software engineering, it represents a new and intensive area [85]. An SPL is a set of variability intensive systems that share a common, manageable set of features that meet the specific needs of a particular market segment [9, 31, 63, 85]. Its engineering is based on families that produce similar systems instead of the production of individual systems [15]. An SPL aim to reduce the time of commercialization of products and increase software quality through planned reuse of artifacts to meet customer needs and decrease customization effort [12, 34]. In addition, SPL seek to provide a common platform that allows the derivation of particular systems that are capable of adapting to the different needs required [76]. Specifically, an SPL aligns engineering resources with business objectives to ensure that efforts are focused on the most cost-effective features and functions of a product. The most characteristic element of an SPL is its variability, and its management is fundamental to define it; therefore, SPL paradigm is a solution for managing variability in a family of products [11]. Figure §2.1 shows an example of how the customization of a software product line is offered to mobile telephones consumers through web configurators in which users can select the functions that best adapt to their needs. Browse phones Compare phones >> Figure 2.1: Customization of an SPL in the mobile phone industry (from [94]) 2.2 Feature models One of the main artefacts of software product lines are features models (FMs) used to represent or model the common and variable features and the 2.2. Feature models 49 relationship between them [34, 42, 85]. In industry, FMs have been widely adopted to capture, organize and reuse the requirements of a set of similar applications in a software domain [100]. Since the first proposal of Kang et al. [42] FMs have become the de facto standard to represent variability and are used to model SPL in terms of features and relations among them [16, 81]. A feature represents a functionality operation in a system to meet a product requirement and provides a potential configuration element to enrichment an SPL [7]. Also, it is an increment in the product functionality [14]. An FM is a compact representation of the set of products in an SPL in terms of features and relations between them [87]. In general, FMs are tree-shaped hierarchical structures that describe the successive refinement of variability in a product line [40]; and where, restrictions between trees can be used to connect features [33]. FM can be seen as a “general landscape” of the functionalities of a software system, and their integration plays a fundamental role in software engineering tasks [22]. To use FMs, first is to build them, which generally involves a systematic analysis of common features and variability with the software domains in focus [100]. A feature can be abstract or concrete [97]. A feature is abstract, if it is not mapped to any implementation artifact, and is non-abstract or concrete, when at least one implementation artifact is assigned. The combinatorial features derived from the model are known as configurations and can be used to generate a product. An example used to illustrate a feature model is the one inspired by the web development industry, presented by Rodas et al. [80]. In the feature diagram, the features are represented by boxes and relationships by lines. As shown in Figure §2.2, the model illustrates how the features are related to create an e-commerce website. There are different proposals to represent FMs. For the most part, they have the common elements of Czarnecki et al. [30], the most used. Czarnecki’s notation for FM, represented in Figure §2.2 for the case of e-commerce website, proposes four types of relationships: mandatory, optional, alternative and orrelation. Relations.- Batory et al. [13] classifies the relationships into three groups: “and-group”, “or-group”, and “alternative-group”. And-group can be mandatory or optional. A relationship is mandatory when a child feature has a mandatory relationship with its father and is included in all the products in which its parent characteristic appears; and it is optional, when a child feature 50 Chapter 2. Software product lines e-Shop Payment SecuritySearchCatalogue High StandardPayPalCredit CardBank Transfer Legend: Mandatory Optional Or Alternative Abstract Concrete Requires Excludes Figure 2.2: Feature model inspired by the web development industry (from [80]) has an optional relationship with its parent and can optionally be included in all the products in which its parent characteristic appears. Following the model presented in Figure §2.2 an e-commerce website should implement a product catalog, a payment module, security policies and could optionally include a search tool. In addition to the parental relationships between features, a feature model may also contain constraints between unconnected features. These restrictions can be of two types: requires and excludes. A restriction is requires when an A feature requires a B feature, that is, the inclusion of A also implies the inclusion of B within the configuration of a product. For example, for the e-commerce website in Figure §2.2, the implementation of the credit card payment function requires high security policies. On the other hand, it is excludes when a feature A excludes a feature B, that is, both features cannot be part of the same product. In the case of the e-commerce website, high security policies and payments by bank transfer are incompatible, since the activation of high security policies requires the use of a payment via credit card or PayPal. Configurations.- Aconfiguration specifies a particular instantiation of the product line. It is characterized by specifying a set of selected and eliminated features. The configurations can be classified into different categories, for example, avalid configuration adheres to the defined relationships and constraints. A complete configuration contains all the features of the selected or deleted lists. When not all the features are contained in the selected/removed sets, it is called partial configuration. A complete configuration that only contains the set of selected features, while all other characteristics are implicitly eliminated, is known as product [15]. In the context of extended feature models, config- 2.2. Feature models 51 urations can also include the configuration of attributes (that is, determine a value for each attribute). Finally, for a product to be properly configured, a set of necessary components is required to implement each feature. This set of components is known as implementation components [61]. e-Shop Payment SecuritySearchCatalogue High StandardPayPalCredit CardBank Transfer Legend: Mandatory Optional Or Alternative Abstract Concrete PayPal Credit Car Payment, WooCommerce Stripe Payment Gateway, Authorize.Net Payment Gateway Wordfence Security, WordPress Security by CleanTalk, Lana Security WooCommerce Ajax Search, Advanced Woo Search, Search Manager Lite WooCommerce Catalog, Product Catalog, eCommerce Product Catalog 1 2 3 4 1 2 3 4 Implementation Components Feature Model Figure 2.3: Features model that describes a list of components for an ecommerce website As an example, the catalog, search, payment and security features on the e-commerce website presented in Figure §2.3 can be implemented using the list of components shown at the bottom of the figure to comply with the functionality required on the website. Keep in mind that different features can implement the same component. For instance, if we want to implement the catalog feature (identified with 1 in the graph), we could have more than one selection alternative to implement it. In Figure §2.3 we present a list of three components from which the user can choose one. In practice, we can find a wide variety of components available for selection and that could implement the catalog feature. Types of Extended Feature Models.- In addition to the basic feature models described above, a couple of extensions has been proposed. Schobbens [91] summarizes the general semantics of common feature diagrams in the literature. For example, an extended feature model is characterized by features that may contain attributes. This allows to model specific properties of a feature, such as costs or information in the version. At the same time, it allows complex constraints between trees that depend on the value of a given attribute, for 52 Chapter 2. Software product lines example, if the resolution of the camera is greater than a certain threshold X, then we need to have a high screen resolution. The alternative terms for extended feature models are “attributed” or “advanced” feature models [16]. 2.3 Automated analysis of feature models The Automated Analysis of Feature Models (AAFM) is a research topic that has attracted the researchers and professionals’ attention over the past two decades, time in which, the amount of tools and techniques that allow the analysis of feature models has increased and also its complexity [93]. The research carried out so far proposes the use of AAFM to cope with the variability management in software product lines [81]. The AAFM has recently been identified as one of the most important areas in the development of software product lines [34]. The AAFM consists of the computer-assisted extraction of information from models [15] and can be summarized in three steps. First, FMs are translated into a logical representation. Secondly, a specific algorithm or solution is used to perform a certain analysis operation (e.g. counting the number of products or verifying the consistency of an FM). Finally, the result is obtained and used in a specific domain such as the configuration or product derivation [34]. Recently, Galindo et al. [34], identify six different facets of variability where the AAFM is being applied: i. Product configuration and derivation. It is the most widely used automated analysis mechanism and is used to support the derivation and product configuration process. ii. Testing and evolution. It consists of using automation mechanisms that guide the process of selecting configurations for testing purposes. iii. Reverse engineering. The AAFM in this area from descriptive information of products and using logical formulas is in charge of the extraction of feature models. iv. Multi-model variability analysis. In certain scenarios it is necessary to analyze more than one model, in this case, AAFM has proposed analysis operations to manage the multilayer configuration process. v. Variability modeling. The analysis of models using basic constructions, in some cases are not enough. For this reason, it is necessary to use 2.4. Constraint satisfaction problems 53 additional information from the models for a improvements in analysis. For example, AAFM uses attributes to model different situations that help decision-making during the product configuration process. vi. Variability intensive systems. AAFM is not directly associated with SPL, its field of application has been extended and it is possible to use it in other areas of application. Among the activities that are carried out automatically using different approaches are: Discovering whether a product is valid, obtaining all the products, calculating the number of products, detecting errors, explaining errors, among other operations [34]. As specified by Benavides et al. [17], using operations, different aspects of the models are analyzed. More than thirty different operations can be found in the literature to date. For example, a Valid product operation will check if a specific product represents a valid combination of features belonging to a given feature model. Also, the Number of products operation determines the number of valid products. Other operations are aimed at comparing feature models, for example, in terms of similarity. Feature models analysis is a tedious and error-prone task. In order to help feature modeling professionals extract information, different computer-aided mechanisms have been proposed that allow different analysis of SPL in different contexts and domains. There are proposals based on specific algorithms, Binary Diagrams (BDD), SAT and CSP. Some available tools are FaMa [19], FaMiLiAr [3] and FeatureIDE [98], among others. Currently, there are efficient solutions for errors diagnosis in automated analysis of feature models, such as the proposal of Segura et al. [93], where they propose BeTTy, a framework that supports random feature models generation and its set of products allows, among other things, to generate expected inputs and outputs for a series of analysis operations in feature models that accelerate failure detection. On the other hand, Galindo et al. [36], propose to use automated analysis of feature models to automate the configuration selection and prioritization for testing in a products line; to this end, TESALIA (TESting vAriAbiLity Intensive Systems) is introduced, a method that seeks to analyze the set of coded products and the information associated with the features (costs) to prioritize the products that will be used for testing. 2.4 Constraint satisfaction problems As mentioned in the previous section, there is a great variety of tools that can be used in the AAFM as backends. One of them are CSPs (Constraint 54 Chapter 2. Software product lines Satisfaction Problems) A CSP [15] allows to create models formed by a set of variables in which it should indicated which possible values could be taken and the constraints that should be fulfilled between those variables. Given the model, the system is able to find those solutions that comply with the specifications established in the constraints [18]. Unlike propositional formulas that use binary values (true or false), a CSP solver also uses numeric values (e.g. integers or intervals). Figure 2.4: Mapping from feature model to CSP (from [15]) Figure §2.4 shows the process to map a feature model to a CSP using a CSP solver. According to Benavides et. al [15, 16] the steps to carry out this mapping process are the following: i. According to the type of variable the solver supports, a CSP is mapped to each feature model with values between 0...1 or TRUE - FALSE, as the case may be. ii. Depending on the type of relationship, restrictions are established for each relationship of the model giving rise to possible auxiliary variables. 2.5. Application engineering 55 iii. Finally, CSP is composed of the result of steps i and ii, that is, with values of the variables and the set of restrictions for each relation of the model. Also, an additional constraint is assigned for the root variable (i.e root ⇐⇒ true or root == 1) according to the variables domain. In the literature there are several proposals that propose the use of constraints programming for the AAFM. Some of these proposals can be found in [16, 18, 19]. 2.5 Application engineering The set of features and dependencies defined in an FM in the domain engineering phase specifies the complete set of valid features combinations of an SPL. Each resulting combination defines a different product that is part of the application engineering phase [8]. Application engineering is a systematic process for creating a product member from the main assets created at the domain engineering stage [41]. User needs Requirements Model (features, use cases) Analysis Model (Content, Navigation, Presentation) Design Model (Architecture) Feature & component selection Architecture selection Code Generation Domain Engineering 1 Application Engineering 2 Code Figure 2.5: SPL engineering process Thus, there are two distinct phases in the definition of SPL: domain engineering and application engineering as illustrated in Figure §2.5. The process begins by identifying the user’s needs. The domain engineering phase includes determining model requirements, model analysis and design; application engineering includes, features and components selection, architecture selection and codes generation. The domain engineering phase begins with domain analysis, where knowledge of the domain is used to identify common and variable features, and these are made during the design and implementation of the domain, while, applications engineering focuses on the products creation, first identifying the 56 Chapter 2. Software product lines needs of the client, which are then used to guide the derivation of the product [54]. The application engineering is responsible for selecting a set of features according to the requirements of stakeholders to obtain a software product that meets their needs [45]. To carry out this process, the application engineering phase meets the requirements for a product, sets out the features that meet those requirements and finally obtains a product configuration with the implemented features [75]. The selection of features of an FM and its possible combinations define a product configuration which must comply with the restrictions derived from the model and the requirements for the definition of the product [8]. Specifically, domain engineering ensures that the activities of analysis, design and implementation of a products family are carried out thoroughly for all members of the product, while application engineering ensures reuse of the main components of the product family for the product members creation [41]. 2.6 Summary In this chapter the theoretical aspects supporting the doctoral thesis were presented. First, we have introduced the paradigm of SPL, their definition, objective, benefits, and importance in software engineering. Given the importance of the concept of feature models, another section expanded on this key concept in the software product lines development; we also introduced automated feature model analysis, recently identified as one of the most important areas of software engineering. Domain engineering and application engineering represent the main phases of the development life cycle of SPL engineering; the first, focused on establishing a reuse platform, while the second is related to the effective reuse of assets in different products. In this review, a section outlining application engineering is included, given its application in this doctoral thesis. Chapter 3 Recommender systems We generate fears while we sit. We overcome them by action. Dr. Henry Link Arecommender system is defined as a system that provides users with a series of suggestions in a personalized way according to their tastes or preferences. In this chapter, the recommender systems are presented as a strategy for the information management by users in software engineering. 64 Chapter 3. Recommender systems To improve efficiency, the algorithm applies the Eckart-Young theorem [21] that obtains an approximation with only kfactors (k<n), so that the matrices would remain as Un×k,Sk×kand Vt k×m, as it is shown in Equation §3.7. SVD(M1m×n)≃Um×k×Sk×k×VT k×n(3.7) After this process, calculating the prediction of the user ukon the item p is reduced by multiplying the k-th row vector of the matrix U(i.e.,U(uk)) with the matrix Sand the p−th vector column of the matrix Vt,(Vt(p)) as shown in the Equation §3.8. ruk, p =¯ruk+U(uk)×S×VT(p)(3.8) Finally, although the collaborative filtering algorithms described use different equations to estimate the ratings (ruk,p) of an active user in a set of target items (see Equations §3.1, §3.4, §3.6, and §3.8), in all cases the best-rated k components are selected to build the list of recommendations. 3.2.2 Content-based recommender systems Content-based recommender systems [66] make recommendations based on the characteristics of the items. Without the need to use information from other users. They are generally used for information retrieval, such as search engines. In this thesis, we use TF-IDF (Term frequency – Inverse document frequency) algorithm to the SPL configuration domain. •TF-IDF: This algorithm [67] is commonly used to perform customized searches by internet search engines. It is characterized by being able to find the local weight and the global weight in a collection of documents that are being analyzed. The local weight is known as TF (Term Frequency) and specifies the number of times a word is repeated within a document; while the global weight, known as IDF (Inverse Document Frequency), indicates the number of documents in which that word appears at least once. The number of TF and IDF occurrences for each document determines the elements to recommend. This algorithm use a user rating matrix Mand a binary matrix Nthat relates the items with their features. The vector vudescribes the ratings of each user and a binary vector vpdetermines the profile of each item p. The vector vurepresents the frequency in which each feature appears in 3.2. Classification of recommender systems 65 the item rated by the user u, and the vector vpdetermines the presence or not of each characteristic in the items p(obtained by each row of the matrix N). Vectors vuand vphave the same dimension, determined by the number of features fin the matrix N. Once we attributed the values to the vectors, the algorithm uses the TFIDF strategy to obtain a weight for each characteristic, penalizing those that are not very similar and rewarding the most distinctive ones. Equation §3.9 presents the way to calculate the weighting of each feature fi. wfi=F(fi, uk).log n IF(fi)(3.9) Where F(fi, uk)represents the frequency which the characteristic fiappears in the items rated by the user ukand represents IF(fi)the inverse frequency or number of times the same characteristic appears but in the items that have not yet been rated by uk. Then, to recommend items to the user uk, the algorithm calculates the cosine similarity (see Equation§3.10) between the user profile (vuk) and the profile of the items not rated by the user (vp, p ∈P). In this computation, we use the weight vector calculated according to Equation §3.9. Scos(vuk, vp, w) = Pt i=1vuk(i)∗vp(i)∗w(i) qPt i=1(vuk(i))2∗qPt i=1(vp(i))2 (3.10) Finally, the algorithm recommends the kitems with higher similarity with the user. 3.2.3 Hybrid approaches This type of algorithm combines collaboration methods with those based on the content or with different variants of other collaboration methods. Hybrid filtering is commonly used in combination with demographic filtering or content-based filtering to exploit of each of these techniques; specifically, it is based on probabilistic methods such as genetic algorithms, diffuse genetics, neural networks, Bayesian networks, clustering and latent characteristics [25]. 66 Chapter 3. Recommender systems 3.2.3.1 Approach based on demographic data This algorithm is characterized by establishing recommendations based on the criterion that people with certain common personal attributes (e.g. sex, age, country, among others.) will also have common preferences [25]. 3.3 Evaluation metrics for recommender systems To assess the performance of the recommendations, there are many indicators and their choice depends on the system objectives, and it is determined by the judgment of their users [53]. Initially, most of the RS have been evaluated and classified according to their predictive power, their ability to accurately predict the user’s options; however, it is now widely accepted that, precise predictions are crucial but insufficient to implement a good recommender engine [95]. Figure §3.2, shows the importance of the evaluation process within the RS. Collect user's behaviors Recommender systems model/algorithm Sorting and recommendation of items Context Evaluating of Recommender Systems Users Figure 3.2: Scheme for the design and evaluation of recommender systems(from [27]) Although the core aspect of the RS is the algorithm, the evaluation of the performance of said algorithm is basic for its selection. In general, a new recommender system must complete the performance evaluation of three stages before its final release: off-line analysis, user study and online experiment [27]. To determine RS performance and compare results, commonly used indicators, metrics, or statistics used in other contexts are used that are useful in software engineering. These indicators include the following: Mean Absolute Error (MAE), and Root Mean Squared Error (RMSE), precision metrics that measure the closeness of predicted ratings to actual ratings; the Pearson product-moment correlation (PCC), the Spearman Correlation Coefficient 3.3. Evaluation metrics for recommender systems 67 (SCC), the Kendall’s Tau coefficient, the Normalized distance-based performance measure (NDPM), the Area Under ROC Curve (AUC), among others. Root Mean Squared Error (RMSE), is a very used indicator to qualify an algorithm (Rating accuracy metrics). RMSE is ideal for the prediction task because it measures inaccuracies in all grades whether positive or negative. Low values of RMSE correspond to better predictions in precision. Other metrics derived from this group are: Mean Square Error (MSE), Mean Average Error (MAE), and Normalized Mean Error (NMAE). Unlike NMAE that normalizes the MAE according to the range of ratings to facilitate comparison of errors between domains, RMSE penalizes errors much more than other evaluation metrics. Another way to assess precision of recommendations is to calculate the correlation between true ratings and system predictions (Rating and ranking correlations). For this purpose, Pearson’s product-moment correlation (PCC), Spearman’s correlation coefficient (SCC), and Kendall’s Tau coefficient are used, which measure the degree to which a linear relationship is present between the two sets of qualifications. Metrics have also been proposed for cases where only implicit ratings are available. In this case, the (Classification accuracy metrics) specially created for tasks such as “Finding good objects” appear. A well-known metric for evaluating this type of system is Area Under the Curve (AUC), which aims to measure the ability of an RS to distinguish relevant objects or objects of interest to users from irrelevant ones. For the RS performance evaluation, the metrics can also be classified according to different perspectives [27]: i. Perspective of machine learning, which includes MAE, MSE and RMSE. ii. Perspective of information retrieval, which includes the Precision measurements, the average precision (Mean Average Precision, MAP), the ROC curve (Receiver Operating Characteristic), the average Reciprocal Range (Mean Reciprocal Rank, MRR), the Spearman rank correlation coefficient (SRCC), the standardized discounted cumulative gain (nDCG), and, the Coverage. iii. Perspective of human-computer interaction and user experiences, which includes Diversity, Confidence, Novelty, and, Euphoria (Serendipity). iv. Perspective of software engineering, which includes Real Time, Robustness and Scalability. 68 Chapter 3. Recommender systems Perspectives Indicators Evaluation Content Methods Machine Learning MAE MSE RMSE Prediction Accuracy Offline analytics Precision Recall F-measure ROC Precision of Recommendations Offline analytics MAP MRR SRCC nDCG Ranking precision of recommended items Offline analytics Information Retrieval Coverage Coverage of users or items Offline analytics Diversity Diversity of recommended items Offline analytic or User study Trust User trust on the recommender systems Novelty Novelty of recommended items Human-computer interaction and user experience Serendipity Serendipity of recommended items User study Real-time Real-time performance of recommender systems User study or online experiment Robustness Robustness of recommender systems Software Engineering Scalability Scalability of recommender systems Online experiment Table 3.1: Metrics and methods for evaluating the performance of recommender systems (from [27]) Table §3.1 presents a summary of the perspectives, metrics and methods to evaluate the recommender systems performance. 3.4 Summary In this chapter we have presented the main concepts and classifications of the RS deepened in this thesis. Specifically, we explore the techniques employed by collaborative-based and content-based RS. In addition, the metrics used to evaluate the validity of the recommendations. Part III Contributions Chapter 4 Motivation Difficult and meaningful will always bring more satisfaction than easy and meaningless. Maxime Lagac´ e Recently, products configuration using automation mechanisms has been one of the most relevant research areas software products line field. However, current solutions are not practical enough to deal with the common problems the software engineer faces when models grow exponentially. In this chapter, we present some of the problems that motivate the search for new solutions and sustain our contributions. Section §4.1 describes the problems that develop around the configuration of products in an SPL; later in Section §4.2 we analyze the current proposals in the automated analysis of features models field based on two layers and the recommender systems applied to an SPL. Then, in Section §4.3, we summarize all the proposals and compare them with our contributions, emphasizing the advantages they provide and the need to implement them. Finally, we summarize the chapter in Section §4.4. 72 Chapter 4. Motivation 4.1 Introduction Software product lines has had a strong impact in recent years, which has led to different investigations and practical experiences from the industry. One of the main challenges the software product lines is the AAFM, where there is an important weakness that has not yet been completely resolved. For example, we find few solutions that allow linking two feature models and cover the problems that arise when working with models in multiple layers; that is, when we have features that interrelate with more than one model. Also, we find the lack of automation techniques for the selection of implementation components for the correct products configuration given the variability of components that may exist in an ecosystem to implement a certain feature. In this context, as we mentioned in Chapter §2, there are several proposals aimed at supporting automation processes with different approaches; however, none of them seems to be adequate enough to solve the automation problems described above, due to the following reasons: i. None of the proposals analyzes the features interaction through layers of different granularity (platform layer and application layer). ii. There are no tools that facilitate interaction in multilayer models to prevent the changes of certain features from affecting the related layer. iii. So far, none of the proposals have presented an approach to facilitate the implementation of features given the amount of components that may exist to implement them. iv. None of the proposals present configurators for the implementation components selection in an SPL. Therefore, we focused on developing new proposals and automation tools that cover the demand in this field of the SPL and which constitute the main motivation of this thesis. In the literature we review some of the proposals that motivate this work and support the contributions that presented in Chapters §5, §6. 4.2 Analysis of current solutions In this section we present different approaches and analyze the research gaps. 4.2. Analysis of current solutions 73 4.2.1 Product line engineering Back in 1990, the FODA (Feature-Oriented Domain Analysis) feasibility study by Kang et al. [42] introduced the feature modeling concept, which remained as one of the major research areas in product line engineering since then. Feature models were used in a variety of scenarios [15], including modeldriven development, feature-oriented programming, software factories or generative programming [29]. However, there are no proposals to explicitly model the interrelationships between the features in different layers, and to draw conclusions from top to bottom and from bottom to top (for example, what is the impact of a change of platform in an existing application). An extended work by Kang et al. was the Feature-Oriented Reuse Method (FORM) [43], which already recognized the importance of different granularity levels. Each feature therein pertains to one of four layers (capability layer, operating environment layer, domain technology layer, implementation technique layer), and features across layers could be connected using implemented by relations. The contribution we present in Chapter §5 is similar to this categorization given its different layers of FMs and focuses mainly on the relationships between the platform and implementation layers, to the point that, the useful consequences analysis of the features selection of the application layer or the platform (in both directions) is possible. Galindo et al. [35] propose a framework to configure multi product lines (i.e., multiple product lines from possibly different suppliers and potentially different notations). To connect these models, they define similar inter-model dependencies. While this highlights the importance of dependencies across models, their focus is quite different as the goal is to support the end-user product configuration (spanning multiple product lines), while our approach aims at analyzing the feature interaction across layers of different granularity (which for instance requires sophisticated relations for meaningful results). 4.2.2 Feature interaction In the early 1990s, feature interaction was acknowledged as a problem in the telecommunication domain, characterizing positive and, more importantly, negative side effects when introducing new features into an existing base system. It was then discovered that the basic problem of feature interac- 80 Chapter 4. Motivation Finally, in the work [72] a new proposal for the configuration of products is presented based on a recommender system that uses contextual information of the users. In this case, new analysis dimensions are introduced, which go beyond the two typical dimensions, users and items, which are usually used by a recommender system. For this purpose, the authors introduce the technique of collaborative filtering Tensor Factorization (TF) [44] to automatically prioritize the features and auto-configure the SPL at execution time according to the contextual information that it originates around users and improve performance in the process of configuring a product. All the proposals of Pereira et al. which have been mentioned above, employ a well-structured recommender system whose main elements are users, items and ratings. In addition, they make use of a historical knowledge base that is constantly updated. However, it can be perceived that the obtained ratings do not come from opinions that common users would give to a configuration of a product (such as errors during operation, poor design, among others). On the contrary, the information that is used is probably collected from the opinions of expert users, since common users could not accurately evaluate technical aspects related to the configuration of a product. On the other hand, the proposals only use collaborative-based recommender systems and focus exclusively on the configuration of products, leaving aside the components that implement the features during the configuration of an SPL. Comparing the works described above with RESDEC, based on the characteristics of columns [4-7] of Table §4.2, we can say that one of the main advantages of our proposal is the use of a knowledge base that feeds of information that comes from the experience that ordinary users have had when using products in an SPL domain. Specifically, in our proposal this information is represented by the ratings that users have made about the components used to implement features in the configuration of a product. We believe that it is more feasible and real, to obtain user evaluations about the implementation components, rather than the features of the product; since as mentioned before, a common user would not have the experience to evaluate technical aspects related to a product. The column “Tool” in Table §4.2 indicates whether the proposals include some prototype tool as part of contributions. All the proposals, except those presented by Pereira et al. [72, 73], propose a tool for configuring products in an SPL. However, RESDEC is the only tool that has been implemented using a well-defined scheme of recommender systems; and besides, it is the only one that incorporates a set of algorithms that are executed in three different scenarios that validate the platform’s capacity in terms of operationalization and scalability. 4.4. Summary 81 Finally, all proposals include an evaluation with information obtained from various business sectors. 4.4 Summary In this chapter we have reviewed some of the proposals that address the problems during the products configuration in an SPL. According to the results of analysis carried out in Section §4.3.1, we conclude that our first contribution is the only one that incorporates a set of options that validate analysis capacity in terms of operationalization, and considers aspects such as migration, support and the potential to solve problems when there are changes in the definition of models of two-layered features. In addition, from the analysis results we have presented in Section §4.3.2 about the automation of configuration processes in an SPL, we conclude that our second contribution is the first to introduce the concept of componentbased recommender systems, in the features configuration of a product based on the selection of optimal components, using collaborative-based and content-based filtering techniques. As seen, none of the proposals found in the literature manages to address all the problems at once. This justifies the originality of our contributions and contributes with the state of the art for future research. 82 Chapter 4. Motivation Chapter 5 Automated analysis of two-layered feature models If I don’t know one thing, I’ll look into it. Louis Pasteur The proliferation of features and platforms in variability intensive systems, coupled with substantial technological progress, imposes several challenges for software developers and equipment manufacturers—in some cases referred as technical sustainability. For instance, in the mobile application domain, developers often need to know the requirements and limitations of their applications to be supported on a specific platform. Conversely, an equipment manufacturer is interested in knowing what additional features become accessible on the application layer when the hardware or platform is being upgraded. To date, analyzing such interdependencies between specific feature and platform combinations is a tough problem, but important to solve. There are well-established approaches in the literature to analyze variability–intensive systems using feature models. However, there is a lack of approaches to analyze application and platform features in multiple layers.†1 †1Part of this chapter is published in the Journal of Visual Languages and Computing [50]. We have the agreement of all the authors to include the corresponding text as part of this manuscript. 84 Chapter 5. Automated analysis of two-layered feature models Chapter 6 Implementation components selection using recommender systems Everything seems impossible until it’s done. Nelson Mandela In a Software Products Line (SPL) there may be features that can be implemented by different components, which means, that we have different implementations for the same feature. In this context, the selection of the best set of components to implement a given configuration is a challenging task given the high number of combinations and options that could be selected. In certain scenarios, it is possible to find information associated with the components that could help in this selection task, such as user ratings and performance. We introduce a component-based recommender system, called RESDEC (REcommender System that from selecteD fEatures suggest implementation Components), that uses information associated with the implementation components to make recommendations in the domain of SPL configuration. †1 †1Part of this chapter is in press in the IEEE Access Journal [80]. We have the agreement of all the authors to include the corresponding text as part of this manuscript. 86 Chapter 6. Implementation components selection using recommender systems Part IV Validation Chapter 7 MAYA: Putting variability at the application and systems level. The greater the effort, the greater the glory. Pierre Corneille In a world where hardware frequently outpaces software in terms of innovation and speed up (which holds especially true for mobile phones and consumer electronics in general), a mechanism to understand the impact of these frequent technology changes on existing software is desirable. At the same time, new types of applications are enabled due to new or improved hardware components. It is equally interesting to see whether certain types of applications can be implemented on existing hardware, and to understand the limitations why some applications may not be realizable (e.g., the platform version in use doesn’t support a feature yet). In this chapter, we present an empirical evaluation of MAYA approach using case studies on two real world subjects. Finally, we show how MAYA could improve the state of the art in managing the evolution of applications and platforms in the field of mobile phone application development.†1 †1Part of this chapter is in press in the Journal of Visual Languages and Computing [50]. We have the agreement of all the authors to include the corresponding text as part of this manuscript. 96 Chapter 9. Conclusions and future work When it comes to platform and design decisions, both developers and handset manufacturers could profit from such a model, because potential problems and incompatibilities can be identified at a very early development stage. Furthermore, such an instrument can be used to discuss design decisions with marketing and other involved stakeholders who might not have detailed technical knowledge. The first objective of this dissertation was to provide two complex feature models for the respective layers, as a prerequisite for further research. We identified the required type of inter-tree relationships and how they are compared to well-known cross-tree relationships. Specifically the operations: (O1) platform capability analysis, that focus on identifies the minimally required platform features; (O2) platform compatibility analysis, that is focus on compare the required platform features for a specific application and determine their (in-)compatibility level; (O3) application functionality potential analysis, that focus on find the corresponding application features that are enabled by a certain platform, and; (O4) platform migration analysis potential, that focus on conflicts of a platform migration and how are existing application features affected by exchanging the underlying platform; set the stage for further analysis. A prototype implementation in Section §?? on Chapter §7 was used to demonstrate the validity of the proposed approach. The selection of optimal implementation components for the configuration of an SPL through the use of recommender systems will help the software engineer to make decisions in a more timely manner, saving time and available resources. Normally, a software developer searches manually those components that are feasible and most suitable to a certain application. This task takes time and does not always guarantee that the selected components are the most adequate (in terms of quality) for the required application. To the best of our knowledge in configurable development environments, there are no effective solutions to efficiently assist the developer in the task of finding suitable implementation components for its application. Consequently, the impact of implementation components selection in an SPL configuration is a costly and error-prone activity due to the large number possible components that could be selected to implement a feature. The second objective of this dissertation was the use of explicit usergenerated information about implementation components, specifically ratings made to the components according to their experience of use. For this pur- 9.2. Future work 97 pose, we have used collaborative-based and content-based recommender systems, which are executed in three scenarios where the software engineer usually faces problems when products configuration. The first scenario called Cold start that recommends components when there is no information associated with the user profile, that is, when the user has not had experience and for the first time is going to configure a product. The second scenario called Recommendations of implementation components based on ratings, which, based on the components linked to the user profile, recommends components that other users have used in past configurations. Finally, the scenario Recommendations of implementation components based on features, which recommends implementation components based on the features of the components associated with the user profile, that is, in the descriptive information of the components. The modeling of the problem for the implementation components selection using collaborative-based and content-based recommender systems algorithms and the design of a prototype tool for RESDEC are the new contributions of this dissertation. The results obtained in the evaluation (see Chapter §8) carried out using a WordPress dataset show that RESDEC is capable of making recommendations on implementation components with an error lower than 13%. This dissertation present results to support the configuration of implementation components in the domain of SPL. Moreover, this contribution could be also applied to other environments that face similar problems, such as, the selection of deployment environments for mobile applications. 9.2 Future work In this section we show the future work that arises from the contributions presented in this dissertation and that are necessary to cover all the research related to the current trends in automated analysis applied to the configuration of an SPL. It is divided into two areas: first shows the future work derived from the use of multiple layers in feature models; and the second, in the implementation components selection to configure an SPL. 98 Chapter 9. Conclusions and future work 9.2.1 Two-layered feature models In this dissertation we have addressed the use of two-layered feature models through the use of computer-assisted mechanisms to manage them. However, there are some unresolved problems that we intend to address in the future. Here are some of the proposed challenges: Extending the operation catalogue.- While we propose four operations in the contributions shown in Chapter §5 of this dissertation, there are others open for extension. For instance, a possible new operation would be to ask for the minimally required platform version to support a particular application. However, this would require to take into account model versioning that is a feature not supported in our current proposal. Also, we can imagine operations that instead of retrieving a configuration work over the inter-tree relationships and analyze them looking for inconstancies or errors. Graphical frontend for analysis input/output.- As mentioned in Chapter §7 of this dissertation, for our prototypical implementation we do not employ tool support for product configuration of the input, nor is there a graphical representation of the analysis output. There is ongoing work†1to integrate FaMa toolsuite with various modeling tools (e.g., MOSKitt, pure::variants). While these tools help to model each layer individually, we plan an extension that supports configuration on both layers concurrently, e.g. by combining them in a graphical frontend. Providing support for the complex task of defining mappings is very important [38]. At the same time, such assistance also ensures the integrity of the inter-tree relations, as only valid features and relation types can be selected. Presenting the output in a visual format, similar to the figures in Section §??, will be another beneficial extension. The different colors to reflect the meanings of the different output categories (e.g., a conflicting feature), will help the user to easily understand the impact of his operation. It is envisioned to be implemented in a way that by just hovering over single features, the respective feature(s) on the connected layer can be highlighted, giving the user a real-time feedback upon (de-)selection of a feature. Analysis with a unified feature model.- As mentioned in Section §??, proposal shown in Chapter §5 of this dissertation was designed to work exclusively with two-layer models since in a real environment there are different practitioners that make hardware and software, where each layer evolves in a different way[35]. Having the platform and application layers separate enables us to control the changes that may occur in the applications when they †1http://www.isa.us.es/fama/?FaMa Current Projects 9.2. Future work 99 update or when they migrate to different platforms without affecting their performance. However, in the future we could evaluate the integration of the two layers in a single feature model where the hardware and software layers would be represented by two mandatory sub-features. In this way, we would show how a change in one feature in the software can affect the hardware or vice versa and how these variations could be managed in both scenarios to determine which responds better to the transitions that feature models can experience. More-than-two layers.- In this dissertation we have proposed a framework relying on two levels abstraction. However, we envision that there might be scenarios where having more than two levels is interesting. In future work, we plan to extend this framework to support as many levels of feature models required in the same spirit as in other domains such as in DSLs [84]. Operation formalization.- In the past, we have already formalized several FMs operations [32] for single model scenarios. We plan to perform the same formalization for the operations shown in Chapter §5 considering more than one model in future work. 9.2.2 Implementation components selection In this dissertation we have approached computer-assisted automation techniques based on recommender algorithms for implementation components selection that help the customization and optimization of SPL products. However, there are still some challenges to be solved: Recommendations using implicit information.- In the contribution presented in Chapter §6 shown in this dissertation, we have only considered recommender techniques that take into account explicit information of the users (i.e., ratings). As future work, we aim to include implicit information from events defined indirectly by the user, such as, number of clicks, number of views, etc. For example, in addition to including user ratings on the plugins, we could also include implicit information as the number of downloads, number of views and versions of the plugins. Information that would allow the recommender system to work in a more personalized way. In the literature, there are several recommender algorithms that use implicit information to make the recommendations [47, 79]. Our future aim is to adapt this algorithms to enrich the recommendations of implementation components in the domain of SPL configuration. 100 Chapter 9. Conclusions and future work Recommendations using of contextual information.- We aim at recommending components based on contextual information derived from the users, features and configurations by adapting the algorithms proposed in Pereira et al. [71] and which can be processed according to the RESDEC components presented in Figure §?? of Chapter §6. The objective that we seek with the implementation of this scenario is to guide the user during the process of configuring a product. This way, as the user progresses in the configuration of a product, RESDEC is able to automatically suggest which feature can be selected to complement the partial configuration based on associated descriptive information to the features, users and configurations. To face the handling of contextual information and offer a personalized product to the user of better quality, we intend to extend the benefits offered by the techniques of Factoring Matrix by introducing the technique of collaborative filtering Tensor Factorization (TF) [72] that allows an integration of contextual data that does not focus only on information from matrices of user and items. In our case, this technique will allow us to explore beyond the contextual information of the components of implementation of the features, and will facilitate us to involve other dimensions of study such as, for example, contextual information of the features and configurations not considered in this dissertation. An approximation to this type of recommendation could be the following, suppose that the user who set up a website for tourism promotion also set up a website for travel; when new users set up a tourism promotion website, it is likely for the system to recommend the configuration of the website for travel. Note that in this case we make use of contextual information that is developed around a valid configuration of a product. Recommendation of configurations for testing.- The RESDEC objective, shown in Chapter §6, is to be able to recommend configurations more susceptible to errors and therefore could be candidates for testing. The aim is to provide to the person in charge of supervising the quality of the SPL an automated mechanism that allows him to select the configurations more error-prone. For this, we are based the hypothesis that, the configurations with the lowest rating by the users are those which tend to contain more errors. For example, if we have designed a mobile application for tourist promotion, the system should be able to recommend which configurations of mobile devices testing should be performed. For this purpose, mobile devices in which similar applications have had unfavorable ratings will be recommended for testing. To make this type of recommendations, we will use the RESDEC recommendation elements and algorithms shown in Figure §?? of Chapter §6. 9.2. Future work 101 As mentioned above, results obtained provide a solid basis for our doctoral thesis and leaves the way open for future research contributing to the progress of software engineering in software product lines area. 102 Chapter 9. Conclusions and future work Part VI Appendix Appendix A RESDEC: Online Management Tool for Implementation Components Selection In this appendix we describe a prototype component-based recommender system called RESDEC (REcommender System that suggest implementation Components from selecteD fEatures) designed to generate implementation component recommendations in the configuration of a product line based on WordPress e-commerce websites. In addition, we present demo video publish on YouTube, that introduce RESDEC tool. 112 Appendix A. RESDEC: Online Management Tool for Implementation Components Selection Figure A.6: Recommendations of implementation components based on ratings pose that we are going to configure a tourism website that has already been implemented and we need to replace or complement the feature Social Media, in this case, first the user selects the plugin implemented, social-media-widget, then the system will display the list of tags associated with the plugin through which it will establish recommendations. Then, we specify the number of desired recommendations and select the execution algorithm (TF-IDF). In this case, RESDEC establishes the recommendations based on the features, that is, on the similarity of the tags associated to the selected plugin with other plugins that use one or more of these tags. In this scenario, RESDEC also uses the information associated to the user’s profile (see Figure §A.7). iv. The option Case Study applied to e-commerce website, shows a Feature Model that was built from information about websites designed in WordPress that are available on the Internet. This Feature Model has been implemented in an interactive way and describes the relations be- A.2. RESDEC Tool Suite 113 Figure A.7: Recommendations of implementation components based on features tween features, the same ones that can appear when configuring an ecommerce website on this platform. In the Feature Model, for example, by clicking on the Shopping Cart feature, the lower part of the model is configured for each scenario, showing only information associated with the selected feature. Thus, for scenario 1 it will show only the list of tags associated to that feature, in the same way in scenarios 2 and 3, it will display only the plugins that implement that feature. The recommendations in each scenario are executed in a similar way as described above (see Figure §A.8). All recommendations presented in the different scenarios, show for each plugin a brief description and a button called “View More” that links to the WordPress website to learn more about it. Additionally, there is an additional list called “You may also like” that shows other plugin options similar to those recommended and that may be of interest to the user (see Figure §A.9). 114 Appendix A. RESDEC: Online Management Tool for Implementation Components Selection Figure A.8: Case Study applied to e-commerce Website Finally, there is an option called “About this case study”, when you click on this option, it shows in detail the process that was carried out to build the feature model. In addition, We present a video demonstration of the RESDEC web tool that is available on YouTube at https://youtu.be/iz bpj0rJLE (see Fig- A.2. RESDEC Tool Suite 115 Figure A.9: Screen “You may also like” that show recommendation alternatives ure §A.10). Figure A.10: Screen of RESDEC tool demo Bibliography [1] A. Abele, Y. Papadopoulos, D. Servat, M. T¨ orngren, and M. Weber. The cvm framework-a prototype tool for compositional variability management. VaMoS, 10:101–105, 2010 [2] M. Acher, P. Collet, A. Gaignard, P. Lahire, J. Montagnat, and R. B. France. Composing multiple variability artifacts to assemble coherent workflows. Software Quality Journal, 20(3-4):689–734, 2012 [3] M. Acher, P. Collet, P. Lahire, and R. B. France. Familiar: A domainspecific language for large scale management of feature models. Science of Computer Programming, 78(6):657–681, 2013 [4] G. Adomavicius and A. Tuzhilin. Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6):734–749, June 2005 [5] M. Al-Hajjaji, T. Th¨ um, J. Meinicke, M. Lochau, and G. Saake. Similaritybased prioritization in software product-line testing. In Proceedings of the 18th International Software Product Line Conference-Volume 1, pages 197–206. ACM, 2014 [6] A. Anandhan, L. Shuib, M. A. Ismail, and G. Mujtaba. Social media recommender systems: Review and open research issues. IEEE Access, 6:15608–15628, 2018 [7] S. Apel and C. K¨ astner. An overview of feature-oriented software development. Journal of Object Technology, 8(5):49–84, 2009 [8] J. Arriel. Personalized recommender systems for software product line configurations. PhD thesis, Otto von Guericke Universit¨ at Magdeburg, 2018 [9] M. A. Babar, L. Chen, and F. Shull. Managing variability in software product lines. Software, IEEE, 27(3):89–91, 2010 118 Bibliography [10] K. Bak, Z. Diskin, M. Antkiewicz, K. Czarnecki, and A. Wkasowski. Clafer: unifying class and feature modeling. Software & Systems Modeling, 15(3):811–845, 2016 [11] M. Bashari, E. Bagheri, and W. Du. Dynamic software product line engineering: a reference framework. International Journal of Software Engineering and Knowledge Engineering, 27(02):191–234, 2017 [12] R. Bashroush, M. Garba, R. Rabiser, I. Groher, and G. Botterweck. Case tool support for variability management in software product lines. ACM Comput. Surv., 50(1):14:1–14:45, March 2017 [13] D. Batory. Feature models, grammars, and propositional formulas. In International Conference on Software Product Lines, pages 7–20. Springer, 2005 [14] D. Batory, D. Benavides, and A. Ruiz-Cortes. Automated analysis of feature models: challenges ahead. Communications of the ACM, 49(12): 45–47, 2006 [15] D. Benavides, S. Segura, and A. Ruiz-Cort´ es. Automated analysis of feature models 20 years later: A literature review. Information Systems, 35(6):615–636, 2010 [16] D. Benavides, P. Trinidad, and A. Ruiz-Cort´ es. Automated reasoning on feature models. In Advanced Information Systems Engineering, pages 381–390. Springer, 2005 [17] D. Benavides and J. A. Galindo. Automated analysis of feature models: Current state and practices. In Proceedings of the 22Nd International Systems and Software Product Line Conference - Volume 1, SPLC ’18, pages 298–298, New York, NY, USA, 2018. ACM [18] D. Benavides, S. Segura, P. Trinidad, and A. Ruiz-Cort´ es. A first step towards a framework for the automated analysis of feature models. Proc. Managing Variability for Software Product Lines: Working With Variability Mechanisms, pages 39–47, 2006 [19] D. Benavides, P. Trinidad, A. Ruiz-Cort´ es, and S. Segura. Fama. In Systems and Software Variability Management, pages 163–171. Springer, 2013 [20] J. Bentahar, M. Taghavi, K. Bakhtiyari, and C. Hanachi. New Insights Towards Developing Recommender Systems. The Computer Journal, 61(3):319–348, 06 2017 Bibliography 119 [21] M. W. Berry, S. T. Dumais, and G. W. O’Brien. Using linear algebra for intelligent information retrieval. SIAM review, 37(4):573–595, 1995 [22] V. Bischoff, K. Farias, and L. Gonc¸ales. Evaluating the effort of integrating feature models: A controlled experiment. 07 2018 [23] L. Blair, G. S. Blair, J. Pang, and C. Efstratiou. Feature interactions outside a telecom domain. In FICS, pages 15–20, 2001 [24] J. Bobadilla, A. Hernando, F. Ortega, and J. Bernal. A framework for collaborative filtering recommender systems. Expert Systems with Applications, 38(12):14609 – 14623, 2011 [25] J. Bobadilla, F. Ortega, A. Hernando, and A. Guti´ errez. Recommender systems survey. Knowledge-Based Systems, 46:109–132, 2013 [26] R. Burke. Hybrid recommender systems: Survey and experiments. User modeling and user-adapted interaction, 12(4):331–370, 2002 [27] M. Chen and P. Liu. Performance evaluation of recommender systems. International Journal of Performability Engineering, 13(8), 2017 [28] P. Clements and L. Northrop. Software Product Lines: Practices and Patterns. The SEI series in software engineering. Addison-Wesley, Boston and Mass. and London, 2001 [29] K. Czarnecki and U. Eisenecker. Generative programming: Methods, tools, and applications. Addison Wesley, Boston, 2000 [30] K. Czarnecki and S. Helsen. Feature modeling. Generative Programming, pages 82–130, 1998 [31] P. Donohoe. Introduction to software product lines. In 2011 15th International Software Product Line Conference, pages 350–350, Aug 2011 [32] A. Dur´ an, D. Benavides, S. Segura, P. Trinidad, and A. R. Cort´ es. FLAME: a formal framework for the automated analysis of software product lines validated by automated specification testing. Software and System Modeling, 16(4):1049–1082, 2017 [33] J. A. Galindo, D. Benavides, and S. Segura. Debian packages repositories as software product line models. towards automated analysis. In International Workshop on automated configuration and tailoring of applications (ACoTA), pages 29–34, 2010 120 Bibliography [34] J. A. Galindo, D. Benavides, P. Trinidad, A.-M. Guti´ errez-Fern´ andez, and A. Ruiz-Cort´ es. Automated analysis of feature models: Quo vadis? Computing, pages 1–47, 2018 [35] J. A. Galindo, D. Dhungana, R. Rabiser, D. Benavides, G. Botterweck, and P. Gr¨ unbacher. Supporting distributed product configuration by integrating heterogeneous variability modeling approaches. Information and Software Technology, 62:78–100, 2015 [36] J. A. Galindo, H. Turner, D. Benavides, and J. White. Testing variabilityintensive systems using automated analysis: an application to android. Software Quality Journal, 24(2):365–405, 2016 [37] Y. Gonzalez-Fernandez, S. Hamidi, S. Chen, and S. Liaskos. Efficient elicitation of software configurations using crowd preferences and domain knowledge. Automated Software Engineering, pages 1–37, 2018 [38] F. Heidenreich, J. Kopcsek, and C. Wende. Featuremapper: Mapping features to models. In Companion of the 30th International Conference on Software Engineering, ICSE Companion ’08, pages 943–944. ACM, New York, NY, USA, 2008 [39] S. Ida and S. Ketil. Technology research explained. Technical report, 2007 [40] M. Javed, M. Naeem, and H. A. Wahab. Towards the maturity model for feature oriented domain analysis. Computational Ecology and Software, 4(3):170–183, 2014 [41] W. Jirapanthong. Experience on re-engineering applying with software product line. arXiv preprint arXiv:1206.4120, 2012 [42] K. C. Kang, S. G. Cohen, J. A. Hess, W. E. Novak, and A. S. Peterson. Feature-oriented domain analysis (FODA) feasibility study. Technical report, DTIC Document, 1990 [43] K. C. Kang, S. Kim, J. Lee, K. Kim, G. J. Kim, and E. Shin. Form: A feature-oriented reuse method with domain-specific reference architectures. Annals of Software Engineering, 5:143–168, 1998 [44] A. Karatzoglou, X. Amatriain, L. Baltrunas, and N. Oliver. Multiverse recommendation: n-dimensional tensor factorization for context-aware collaborative filtering. In Proceedings of the fourth ACM conference on Recommender systems, pages 79–86. ACM, 2010 Bibliography 121 [45] A. Kn¨ uppel, T. Th¨ um, S. Mennicke, J. Meinicke, and I. Schaefer. Is there a mismatch between real-world feature models and product-line research? In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, pages 291–302. ACM, 2017 [46] J. A. Konstan, B. N. Miller, D. Maltz, J. L. Herlocker, L. R. Gordon, and J. Riedl. Grouplens: applying collaborative filtering to usenet news. Communications of the ACM, 40(3):77–87, 1997 [47] Y. Koren. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 426–434. ACM, 2008 [48] Y. Koren, R. Bell, and C. Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8), 2009 [49] S. K. Lam and J. Riedl. Shilling recommender systems for fun and profit. In Proceedings of the 13th international conference on World Wide Web, pages 393–402. ACM, 2004 [50] M. Lettner, J. Rodas-Silva, J. A. Galindo, and D. Benavides. Automated analysis of two-layered feature models with feature attributes. Journal of Computer Languages, 2019 [51] G. Linden, B. Smith, and J. York. Amazon. com recommendations: Item- to-item collaborative filtering. Internet Computing, IEEE, 7(1):76–80, 2003 [52] L. Lorentsen, A.-P. Tuovinen, and J. Xu. Modelling feature interactions in mobile phones. In FICS, pages 7–13, 2001 [53] L. L¨ u, M. Medo, C. H. Yeung, Y.-C. Zhang, Z.-K. Zhang, and T. Zhou. Recommender systems. Physics reports, 519(1):1–49, 2012 [54] M. Marques, J. Simmonds, P. O. Rossel, and M. C. Bastarrica. Software product line evolution: A systematic literature review. Information and Software Technology, 105:190 – 208, 2019 [55] J. Martinez, G. Rossi, T. Ziadi, T. F. D. A. Bissyand´ e, J. Klein, and Y. Le Traon. Estimating and predicting average likability on computergenerated artwork variants. In Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, pages 1431–1432. ACM, 2015 This document was typeset on // using R C–BOO Kα. for L A T EX2. Should you want to use this document class, please send mail to [email protected].