A Method Based on AHP to Define the Quality Model of QuEF

A Method Based on AHP to Define the Quality Model of QuEF M. Espinilla1, F.J. Domínguez-Mayo2, M.J. Escalona2, M. Mejías2, M. Ross3,4, and G. Staples4 1 University of Jaén, Campus Las Lagunillas s/n 23071, Jaén, Spain [email protected] 2 University of Seville, Avda. Reina Mercedes s/n. 41012 Seville [email protected], [email protected], [email protected] 3 Southampton Solent University, East Park Terrace, Southampton, Hampshire, UK [email protected] 4 British Computer Society (BCS) Abstract. QuEF is a framework to analyze and evaluate the quality of approaches based on Model-Driven Web Engineering (MDWE). In this framework, the evaluation of an approach is calculated in terms of a set of information needs and a set of quality characteristics. The information needs are requirements demanded by users of approaches. On the other hand, the quality characteristics are specific aspects that the approaches provide to their users. In these lines, there is a gap in the importance of each quality characteristic in the QuEF and the degree of coverage of each information need regarding the quality characteristics. In this contribution, we propose a method to define the Quality Model within QuEF. This method is based on the Analytic Hierarchy Process in order to establish the importance of the quality characteristics and the degree of coverage of each requirement of the information needs regarding the set of quality characteristics. Furthermore, a software application that develops the proposed method is presented. Keywords: Model-Driven Web Engineering, Quality Assurance, Analytic Hierarchy Process, Information Needs, Quality Characteristics. 1 Introduction A good strategy to manage quality is essential to obtain good results on the improvement of the quality. In these lines, it is important to define what are the goals and the set of steps to achieve the goals. The web development is currently being an important task to take into account in the sense that more and more web applications are developed every day. So, it is important to define the right steps to manage the development of this kind of products. In this context, Model-Driven Engineering (MDE) [1] paradigm plays a key role because it pertains to software development; MDE refers to a range of development approaches that are based on the use of software modeling as a primary form of expression. Model-Driven Web Engineering (MDWE) is a specific domain of the Model-Driven Engineering (MDE) paradigm [1] which focuses on Web environment. The growing interest on the Internet has led to generate several MDWE approaches, which offer a frame of reference for the Web environment. Therefore, it is necessary to analyze and evaluate the quality of different approaches in order to choose the most appropriate taking into account the user requirements, i.e., users’ information need. However, the process of quality evaluation of an approach is a complex task because there are lot of MDWE approaches without standard consensus [2] [3] [4] and an important gaps between the users’ information needs and quality characteristics [5] [6]. Recently, in our previous research, we have defined QuEF (Quality Evaluation Framework) [7] [8][9], an approach to establish an environment in order to analyze and evaluate MDWE (Model-Driven Web Engineering) approaches under quality characteristics. The framework has been structured and organized with four different components: Thesaurus & Glossary component, Quality Model component, Approach Features Template component and Quality Evaluation Process component. The most important component is the Quality Model, which defines and describes the set of quality criteria, its weight and its relations with the users’ information needs. Each approach will be assessed in this evaluation framework. Therefore, a process of decision among users of approaches, decision-makers, on the definition of this evaluation framework is crucial in the success of QuEF. A common approach in decision making is the Analytic Hierarchy Process (AHP) introduced by Saaty [10] [11], which is a widely accepted as a multi-criteria decisionmaking methodology. In this process, the criteria are structured in several levels and then, different decision alternatives are evaluated and prioritized, taking into account the preferences of a set of decision makers. In this contribution, the essence of the AHP is used to propose a method to define the Quality Model in QuEF in order to establish the importance of the set of quality characteristics and the degree of coverage of each information need regarding the set of quality characteristics. Furthermore, the proposed method is developed in a software application in order to support the process of definition of the Quality Model. In this way, this definition process can lay the bases of a framework for quality assessment of approaches on Model- Driven Web Engineering (MDWE) approaches. The outline of this contribution is as follows. Section 2 describes Quality Evaluation Framework. Section 3 reviews the Analytic Hierarchy Process. Then, Section 4 proposes an AHP-based method to build a quality model. Section 5 presents a software application that develops the proposed method. Finally, the conclusion and future works are described in the Section 6. 2 QuEF and the Quality Management QuEF is a framework to analyze and evaluate the quality of approaches based on Model-Driven Web Engineering (MDWE), although it could be extended to other areas or domains. The main objective of QuEF is to provide a set of guidelines, techniques, processes and tools for the structuring of specifications, which are expressed as models. The framework is a basic conceptual structure composed of a set of four elements used to evaluate, in this case, MDWE approaches (see figure 1). These elements are based on existing literature for the evaluation of the quality of MDWE methodologies and they are described as follow: Fig. 1. Basic Conceptual Structure of QuEF 1. Quality Model Component. This component includes a set of information needs and a hierarchy of quality criteria composed by quality subcharacteristics and characteristics which provide the basis for specifying quality requirements and evaluating quality in a specific domain (in this case, MDWE). Furthermore, the model contains association links between the information needs and the quality subcharacteristics. These links represented the degree of coverage of each requirement of the information needs regarding the set of quality subcharacteristics. In this contribution, we focus in this component in order to establish the weight of the quality criteria in the hierarchy and the degree of coverage of each requirement of the information needs regarding the set of quality subcharacteristics. 2. Thesaurus & Glossary component. An important element for QuEF is the thesaurus component. A thesaurus is a list containing the "terms" used to represent concepts, themes or contents of documents in order to standardize the terminology which improves the access channel and communication between users of different MDWE methodologies. This component is necessary to carry out a standardization of terminology to improve the access channel for communication on MDWE. A set of concepts for MDWE methodologies is currently being described and related. 3. The Approach Characteristic Template Component. Templates with users’ information needs based on the Quality Model are used to describe an input methodology. These templates are used as input to QuEF. They are analyzed in the evaluation process and compared with the model quality of the Quality Model component. Templates for MDE, Web Modelling, Tool Support and Maturity have already been developed. 4. The Quality Evaluation Process Component. The Quality Evaluation Process component contrasts the information from each input approach template with information from the Quality Model. The main purpose of this evaluation is to identify tradeoffs and sensitivity points of the methodology under study. The idea is to determine which aspect needs to be improved on MDWE methodology. 3 Analytic Hierarchy Process In this contribution, we focus on the Quality Model component of QuEF in order to establish the weight of the quality criteria in the hierarchy and the degree of coverage of each requirement of the information needs regarding the set of quality subcharacteristics. Different approaches will be assessed in QuEF. So, it seems appropriate that several users, decision makers, are involved in the definition of the Quality Model component. To do so, we will use the essence of the Analytic Hierarchy Process that is reviewed in this section. The Analytic Hierarchy Process (AHP), proposed by Saaty [10] [11], is systematic analysis technique developed for multicriteria decision by means of creating a ratio scale corresponding to the main alternatives. The output of AHP is a ranking indicating the overall preference for alternative. This process is based on three axioms: (1) breaking down the problem; (2) pairwise comparison of the various alternatives and (3) synthesis of the preferences. Conventional AHP includes four steps: modelling, valuation, priorization and synthesis, which are detailed below. 1. Modelling: The first step builds a hierarchy where the goal is at the top, criteria and sub-criteria are respectively placed at levels and sub-levels, and decision alternatives appear at the bottom of the hierarchy. 2. Valuation: The second step analyzes the elements of the problem by means of reciprocal pairwise comparison matrices, which are provided for each decision maker. This step involves two states: a) Weight of the criteria: group pairwise comparisons are performed to determine the relative scores for the weights of the criteria and sub-criteria in the hierarchy. b) Judgments. The assessment is conducted by means of reciprocal pairwise comparison matrices against a third element. In this way, the process obtains the preferences of the individuals regarding the different components of the model (criteria, sub-criteria, alternatives). Each individual provides a preference in terms of importance, preference or probability, assigning a numerical value, which measures the intensity of their preference. So, Saaty suggested a nominal scale with 9 points, the socalled “Saaty’s Fundamental Scale”, shown in Table 1, in order to provide judgments of each individual. Table 1. The rate of importance of criterion Y over X Numerical rating Linguistic judgments 1 X is equally preferred to Y 2 X is equally to moderately preferred over Y 3 X is moderately preferred over Y 4 X is moderately to strongly preferred over Y 5 X is strongly preferred over Y 6 X is strongly to very strongly preferred over Y 7 X is very strongly preferred over Y 8 X is very strongly to extremely preferred over Y 9 X is extremely preferred over Y 3. Prioritation: In this step, the local and global priorities are obtained by using, respectively, any of the existing prioritation procedures. The eigenvector method and the row geometric mean method are the most widely used. 4. Synthesis. In this step, the total priorities are derived by applying any aggregation procedure. It can be additive or multiplicative. One of the main characteristics of AHP is the ability to assess the degree of inconsistency present in the judgments expressed by the decision makers in the reciprocal pairwise comparison. The consistency ratio is obtained by comparing the consistency index with the random index [9][10] which is an average random consistency index derived from a sample of randomly generated reciprocal matrices using the scale in Table 1. Saaty defined the consistency ratio (CR) as: Consistency ratio= Consistence index / Random index (1) Where the consistency index (CI) is in the form: CI= (λmax – n) / (n-1) (2) λmax is a principal eigenvalue of a pairwise matrix such that λmax ≥ n. This method is good for measuring consistency by using the eigen system method. A value of the consistency ratio CR ≤ 0.1 is considered acceptable. Larger values of CR require that the decision-maker revise his judgments. Table 2. Random consistency index (RI) N 12345678910 RI 0 0 .52 .89 1.11 1.25 1.35 1.40 1.45 1.49 4 AHP-Based Method to Build a Quality Model QuEF represents a valid framework for the quality management of MDWE approaches. In this framework is necessary to fix in the Quality Model the importance of the criteria in the hierarchy and the degree of coverage of each requirement of the information need regarding the set of quality criteria. Furthermore, in QuEF is important that the definition of the Quality Model is performed by a group of users, decision makers, in order to fix a consensus quality model. In this section, we present a method to define the Quality Model of QuEF based on the essence of the AHP, which takes into account the opinions of different decision makers. Our method contains the following steps that are described in detail in the following sections. 4.1 Hierarchy Design In this step, all the elements in the Quality Model are identified: 1) the objetive, 2) criteria and sub-criteria and 3) alternatives. In our case, the aim is the establishment of the degree of coverage of each information need in a quality framework. The criteria and sub-criteria composed the quality hierarchy; they are the factors that affect to the objective. In the Quality Model, they are the set of quality characteristics and quality sub-characteristics that are shown in Table 3. Table 3. Hierarchy of quality criteria Quality Characteristics Quality Sub-Characteristics Q1=Usability q 11 = Learnability q 12 = Understandability q 13 = Simplicity q 14 = Interpretability q 15 = Operability q 16 = Attractiveness Q2=Functionality q 21 = Suitability q 22 = Accuracy q 23 = Interoperability q 24 = Compliance q 25 = Interactivity q 26 = Applicability q 27 = Accessibility q 28 = Flexibility Q3=Maintainability q 31 = Stability q 32 = Analyzability q 33 = Changeability Q4= Reliability q41 = Maturity q42 = Recoverability case of failure q43 = Fault Tolerance q44 = Availability q45 = Currently q46 = Compactness q 47 = Relevancy Finally, the alternatives are proposes that can achieve the goal. In our case, they are the set of information needs that are represented as features or requirements. These information needs can group as shown in Table 4. Table 4. Alternatives represent as information needs Information needs F1 = MDE f 11 = Standard Definition f 12 = Model–Based Testing f 13 = Traces f 14 = Level of Abstraction f 15 = Transformations F2=Web Modelling f21 = Web Conceptual Levels f 22 = Interfaces f 23 = Content Modelling f 24 = Presentation Modelling f 25 = Navigation Modelling f 26 = Business Modelling f 27 = Development Process F3=Tool Support f31 = Analysis Tool Support f 32 = Code Generation and Specific Tool Support f 33 = Team Work Tool Support f 34 = Creation, Edition and Composition Tool Support f 35 = Transformation Tool Support f 36 = Trace Tool Support F4= Maturity f41 = Modelling Examples f42 = Publications f43 = Topicality f44 = Application in Real-World Projects f45 = External Web References f 11 = Standard Definition 4.2 Development of Judgment Matrices In this step, the information is obtained from decision makers in order to establish the Quality Model. At the beginning, each individual provides his preferences about the weight of the set of criteria in the quality hierarchy, i.e., the importance that each quality characteristics and quality sub-characteristics has in the Quality Model. These weights are provided by means of reciprocal pairwise comparison matrices, using the scale shown in Table 1. When the priority of the quality hierarchy has been provided, the evaluation of information need takes place. Each decision maker provides his preferences about the information needs, i.e, the degree of coverage of each information need regarding the quality sub-characteristics. This involves yet another set of pairwise comparisons, this time between each alternative against each quality sub-characteristics. Finally, the consistency ratio for each judgment matrix is checked. If CR ≥ 0.1, the pairwise matrix is not consistent, then the comparisons should be revised by the decision maker. 4.3 Compute the Weights of the Quality Criteria Once judgment matrices have been obtained by decision-makers, the proposed method computes a collective matrix for the weights of the set of quality characteristic and quality sub-characteristic which summarizes preferences of the group decision. This aggregation can be carried out applying some types of OWA operator [13]. To use an OWA operator is recommended because it reorders arguments according to the magnitude of their respective values. Furthermore, they satisfy some interesting properties such as compensativeness, idempotency, symmetry and monotonicity. 4.4 Compute the Degree of Coverage of Each Information Need In this step, the proposed method computes the collective preferences to obtain a final vector of degree of coverage for each information need. This is done by following a path from the top of the hierarchy down to each alternative at the lowest. The outcome of this aggregation is a normalized eigenvector of the overall weights of the options [10][11] level, and multiplying the weights along each segment of the path. 5 A Software Application of the AHP-Based Method In this section, we present an application that develops the proposed method. The objectives of the application are two. The first objective is to obtain the preferences of decision makers in a quick and simple way. The second objective is to automate the computations to obtain the importance of the criteria in the quality hierarchy and the degree of coverage of each information need. This application generates a Web application with the set of surveys that consider all the elements in the Quality Model in order to carry out the proposed method. The Quality Model has been defined using the Enterprise Architect tool support as is shown in Fig. 2. This tool support can generate an XML file of the model defined in the tool. On the other hand a Windows form application has been implemented using the Visual Studio .NET environment to generate automatically all the code of a Web Application that include all the Surveys for the elements which have already defined in the Quality Model. If the Quality Model or the set of questions to carry out the AHP method is changed the Web application can be generated automatically. This program can be used to carry out the AHP method for any other domain. Fig. 2. Application of the method Fig. 3. Web application Finally, the Web Application is generated. The Visual Studio .NET has been used to implement this environment as is shown in Fig. 3. 6 Conclusions The Quality Model of QuEF defines and describes an hierarchy of quality aspects and the relationships between them and the information needs. In this contribution, we have presented a method to define the Quality Model based on the Analytic Hierarchy