An MDA Proposal To Integrate the Measurement Lifecycle Into the Process Lifecycle

An MDA proposal to integrate the measurement lifecycle Into the process lifecycle A. Meidana, J. A. García-Garcíaa, I. Ramosa, David Lizcanob, M.J. Escalonaa a University of Seville, Avenida Reina Mercedes s/n, Seville, 41012, Spain. b School of Computer Science, Madrid Open University (UDIMA), Collado Villalba, 28400, Madrid, Spain. Context: Measuring the Software Development Process (SDP) supports organizations in their endeavor to understand, manage, and improve their development processes and projects. In the last decades, the SDP has evolved to meet the market needs and to keep abreast of modern technologies and infrastructures. These changes in the development processes have increased the importance of the measurement and caused changes in the measurement process and the used measures. Objective: This work aims to develop a solution to support the measurement activities throughout the process lifecycle. Method: Study the current state of the art to identify existing gaps. Then, propose a solution to support the process measurement throughout the SDP lifecycle. Results: The proposed solution consists of two main components: (i) Measurement lifecycle; which defines the measurement activities throughout the SDP lifecycle, (ii) Measurement definition metamodel (MDMM); which support the measurement lifecycle and its integration into the process lifecycle. Conclusion: This proposal allows organizations to define, manage, and improve their processes; the proposed information model supports the unification of the measurement concepts and vocabulary. The defined measurement lifecycle provides a comprehensive guide for the organizations to establish the measurement objectives and carry out the necessary activities to achieve them. The proposed MDMM supports and guides the engineers in the complete and operational definition of the measurement concepts. Keywords: Software development process, process measurement, process Metrics and indicators, measurement lifecycle. 1. Introduction Defining and improving the development process is one of the most important strategies used by organizations to enhance productivity and improve the quality of the developed software. The development process is the primary guide for the management of the work teams and the production process. It is also used as a basis for project planning and monitoring. Defining, monitoring, and improving the software development process (SDP) aims to produce high-quality software products and more predictive and productive projects. Software development is considered to be comprised of three essential components: products, processes, and resources (Fenton, 1991). Developing software is a long, costly, and complex process. The outcome of this process is not only the final product but the production of many intermediate and supplementary artifacts during the development endeavor. The quality of this development process significantly impacts the quality of the resulting product (Cugola & Ghezzi, 1998; Fuggetta, 2000; B. Kitchenham & Pfleeger, 1996). Measuring the SDP and its outcomes is the only way to gain knowledge about them. Besides, the obtained measurements could be used in models for prediction purposes (Lennselius et al., 1987). Moreover, software process measurement provides support for better understanding, evaluation, and control of the development process, project, and the resulting product (Ebert et al., 2007). Measurement also enables organizations to have insight into its processes, predict, and improve its quality and performance, which give organizations a better position to make appropriate and informed decisions as early as possible during the development process (Abreu Fernando Brito & Carapuça, 1994; García et al., 2006). In the last decades, the SDP has evolved to meet the market needs and to keep abreast of modern technologies and infrastructures that have influenced the product development and its use. These changes in the development processes have increased the importance of the measurement (Bourgault et al., 2002) and caused changes in the measurement process and the used measures (Tihinen et al., 2012). For instance, cloud computing allowed to merge software development, deployment, and operation in what is known as DevOps. Measurement is one of the four DevOps perspectives (Collaboration culture, automation, measurement, and sharing) (Bang et al., 2013). In this context, measurement promotes communication and the common understanding between development and operations. On the other side, today's software is increasingly developed by teams working in different geographic locations, time zones, and cultures. Management of these kinds of projects is more challenging and complicated than traditional on-site development. The measurement is an essential element for the success of these development projects (Tihinen et al., 2012). These evolutions in the development process, technologies, and infrastructures create new challenges and obstacles for the measurement, regarding data collection, storage, analysis, interpretation, and decision-making based on the measurement results. These challenges and difficulties emphasize the importance of the measurement in the context of the SDP. This work aims to use the Model-Driven Engineering (MDE) paradigm (Schmidt, 2006) to integrate the measurement process into the process lifecycle in a way that allows the definition and modeling of the process measures explicitly and operationally during the process modeling phase. It also aims to use the MDE transformations to derive the measurable process execution model from the process definition model. The result of this work is a theoretical solution guided by models to improve the measurement of the software processes, as well as a software tool to support the application of this theoretical solution in practical environments. The remainder of this paper is organized as follows: The next section discusses the related works. Section three describes the identified gaps and the established objectives. Section four presents the components of the proposed solution. And section five describes the development of the tool which allows the practical use of the proposed solution. Section six, describe the validation project and the results obtained from this experience. And finally, section seven states the conclusions. 2. Related Work This section is divided into two parts: the first part presents the previous research carried out by the authors to comprehend the current state of the art and to discover the existing gaps in the domain. The second part discusses the existing proposals, modeling languages, and tools related to this work. Also, describes the process lifecycles and measurement lifecycles found in the literature. 2.1 Understand the current state of the art The first study performed by the authors to understand the current state of the domain is a survey on the existing open-source Business Process Management Suites (BPMS) (A. Meidan et al., 2016), this study aims to investigate to what extent the existing BPMSs support the process lifecycle. Also, provide a guide for the organizations to plan and perform a comparative on the existing BPMSs. Which allows them to discover which BPMS best meets their process management needs. One of the findings of this study indicates a lack of the definition and integration of the Process Performance Indicators (PPI) into the process model, also, in linking the PPIs with the service level agreements. This finding prompted the authors to perform the second study, a Systematic Mapping Study (Ayman Meidan et al., 2018) that focuses on the measurement of the software development process and its execution projects, mainly to give insight on the measurement related to the “Project” and “Process” entities. These previous two studies reveal the lack of support for the measurement in the existing process modeling and management tools and proposals. The survey demonstrates the existing weakness in the definition of the measurements and its integration into the process lifecycle; the majority of the investigated proposals do not support the definition and integration of the process measurement, this integration promotes the process monitoring and improvement. Furthermore, the mapping study demonstrates the scarcity of research on defining the measurements in the form that allows its integration into the process lifecycle. This study also reveals that: the definition of measurements in a complete and operational form (Deming, 1986), as well as considering the measurement issue in all the process stages is essential for strengthening process improvement and project management. 2.2 Related Proposals This section describes the existing research attempts to define and integrate the measurement into the SDP, also reveal how the main process modeling languages and tools support and integrate the measurement issues. Moreover, this section outlines the main existing process and measurement lifecycles. 2.2.1 Relevant Research, Modeling languages, and Tools Measurement is essential for the quantitative management and improvement of the SDP, for that it has gained significant interest from both researchers and practitioners. There are many proposals in the literature related to the measurement definition, modeling, and execution. This section focuses on the model-based proposals. In (Bendraou et al., 2006) authors present a metamodel based proposal for software process modeling. This proposal does not define the measurement as a process element, but the authors mention the necessity to measure the different process elements during the process execution for monitoring purposes. They also discuss the need to apply changes to the process elements to support its measurement (e.g., add some attributes to the process elements). In (Mora et al., 2009; Mora, Garcia, et al., 2008; Mora, Piattini, et al., 2008) the authors propose a measurement framework based on Model Driven Architecture (MDA) (Singh & Sood, 2009) to measure any software entity (e.g., database structure, process model, and requirement document) based on the metamodel that represents them. They also present a graphical notation language which allows the users to define software measurement models based on software measurement ontology. This work focuses mainly on measuring model elements based on its metamodel (e.g., count the number of tables in a relational database scheme). Thus, this proposal does not focus on measuring the process execution perspective such as the elapsed time to perform an activity. In (Larrucea & Iturbe, 2010) authors present an approach to combine different metamodels (e.g., SMM (GROUP, 2009) and SPEM 2.0 (OMG, 2002)) to model the process and the measures to provide control over the execution of processes. This approach allows the definition of measures only for processes and task elements but does not allow the process modeler to model the measures in an explicit and operational form within the process model. In (Freire et al., 2011) the authors present a model-driven approach for the definition, execution, and monitoring of SDPs, it supports the automatic collection of quantitative measures during project execution. The authors define a metamodel to define the measures. This approach does not define the measures explicitly in the process model, does not consider the manual measures, and also does not measure the process artifacts. Furthermore, the measure definition does not address how the values of the measures will be analyzed and used. The authors in (Del-Río-Ortega et al., 2013) provides a metamodel and tool for the definition and the designtime analysis of PPIs independently of the language used to model the process. This proposal does not reflect the relation between the information needs, the indicators, and the data collected to satisfy this information needs. Moreover, this proposal focuses only on the measures that will be collected automatically; does not support the definition of the manual measurements (e.g., specifies the necessary methods and tools to perform the measurement activities). Furthermore, the proposal does not allow the definition of context data to be collected with the measurement value. In the proposal (Garcia-Garcia, 2015; García García et al., 2015) the authors present a metamodel to define the development process. This metamodel defines the measure as a process element, the proposal derives the process execution model from the definition model, but the resulting model does not include the measure element defined in the definition model. This proposal could be developed by extending the metamodel to define the measurement concepts (e.g., information needs), and by adding more attributes (e.g., performer role, unit, context, collection method, etc.) to the measure element, and also by representing the measure element in the process execution model. On the other side, the industrial standard BPMN 2.0 (Allweyer, 2015) does not define the measures as a process element. The commercial implementations of this standard (e.g., Bonita BPM (Bonitasoft, 2016)) allows the modeler to define an attribute to be measured but does not define the measures as an element to allow the modeler to include it in an explicit and operational way. SPEM 2.0 (OMG, 2002) defines the measure as a process element but in basic and abstract form, wherefore, the process modeling tools which use SPEM 2.0 metamodel (e.g., EPF(Eclipse, 2017) and RMC (IBM, 2017)) does not support modeling the measures operationally and explicitly within the process model. These academic and industrial works fail to integrate the measurement into the process lifecycle in such form that allows: (i) the process engineer to define and model the measurement concepts (e.g. information needs, performer, procedure, and context) in operational form during the process definition and modeling phase. (ii) using this definition in the process deployment phase to perform the necessary configurations to collect and store the measurement values. (iii) collecting the measures data during the process execution phase according to the measure's definition. (iv) analyzing the measured data during the process monitoring and analysis phase according to the method indicated in the measure's definition. Furthermore, (v) reporting the measures and its analyses to the indicated role to determine the necessary actions to control, optimize, and improve the process. 2.2.2 Process Improvement and Lifecycles Lifecycle can be defined as a series of activities grouped in a set of phaseseach with a specific focus - performed to achieve specific and integrated objective. Given the wide range of application areas, different views of the process lifecycle have been proposed over the past decades. The most recent process lifecycles are summarized below. Authors in (Hill et al., 2006) propose a global process revision cycle to create value for organizations. To do this, they contemplate modeling processes as the first step to achieve this goal. In this way, before initiating any design or process review, the organization must decide the scope of its initial activities. The process lifecycle proposed by these authors is based on the following nine phases: discovery, modeling, simulation, deployment, execution, monitoring, analytics, optimization, and refine. On the other hand, in (W.M.P. van der Aalst, 2004; Wil M. P. van der Aalst, 2004) authors establish a process lifecycle that is much more compact than that presented in the previous proposal. In this case, the lifecycle is based on four phases: process design, system configuration, process enactment, and diagnosis. 2.2.3 Measurement Process Lifecycles The term measurement lifecycle refers to the entire phases of the measurement process (e.g., measurement definition, application, and the exploitation of the measurement result)(Habra et al., 2008). This process aims to collect, analyze, and report objective data and information to support effective management and demonstrates the quality of the products, services, and processes. (ISO/IEC/IEEE 12207-2017-International Standard - Systems and software engineering -- Software lifecycle processes 2017, ISO/IEC/IEEE 15288-Systems and software engineering System lifecycle processes 2015) Over the last decades, several authors have identified and described phases of the measurement process. The main and most recent proposals are summarized below. Jacquet et al. (Jacquet & Abran, 1997) have decomposed the measurement lifecycle into four successive steps: Design of the measurement method. This step includes: defining the measurement objectives, define the measurement object, characterize the measurement concept, and defining the assignment rules. Measurement method application. In this step, the measurement data are collected, and the measurement methods -defined in the previous stepare performed to produce the measurement results. Measurement result analysis. The results obtained in the previous step are documented, evaluated, audited, and analyzed in this step. The exploitation of the result. In this step, the measurement results are used in many several forms (e.g., characterizing and predicting purposes). In a similar form, authors in (Y. Zhang & Sheth, 2006) have divided the measurement process into four steps: definition, collection, analysis, and in the last phase, the analysis results are used to control and improve the process. On the other hand, in (Del-Río-Ortega et al., 2009) the authors propose a measurement lifecycle comprise of four phases: Definition. During this phase, the measures are identified, defined, and linked with the process objectives. Measuring. Where the data is gathered. Analysis. In this phase, the measured values are compared with the target values, and the causes of any unexpected value are identified and report. In this phase, the analysis results are summarized and reported to the users. Furthermore, the recent version of the standard ISO 15939-2017 (ISO/IEC/IEEE 15939-2017 International Standard - Systems and software engineering--Measurement process, 2017) defines a measurement process of four phases: Establish and sustain measurement commitment. In this phase, the measurement requirements and scope are defined, the management committee is established, and resources are assigned for the measurement activities. Prepare for measurement. This phase includes several activities, such as: Define the measurement strategy and identify & prioritize the information needs. Perform measurement: which includes collecting, storing, and verifying data. Evaluate measurement: this phase emphasis the quality of the measurement process and the information needs. 3. Problem definition, Objectives, and Influences In the past years, the software engineering community has proposed many methods, standards, and techniques (e.g., GQM, PSM (McGarry, 2002), and ISO 15939) to guide the selection and definition of the measurement concepts to optimize the measurement process. Unfortunately, most of these methods and processes stop at the point of selecting and identifying the measures and the measurement concepts that satisfy different needs (e.g., monitoring, controlling, estimating, and improving). However, they do not focus on defining the measurement concepts (e.g., indicators, measurement method, and context) in the form that support the measurement process throughout its lifecycle (B. A. Kitchenham et al., 2001). Previous studies conducted by the authors and the relevant proposals discussed in the previous section show that this situation remains to date. Defining the measurement concepts in an unambiguous and rigorous (operational) form is essential to support the collection, storing, and analysis of the measurement values. Moreover, it promotes the interpretation and reporting of the measurement results in the form that support engineers and managers to adopt quantitative management, make informed decisions, and develop the improvement plan. Furthermore, the operational definition of the measurement concepts motivates and supports the integration of the measurement into the software process (Barcellos et al., 2013). After introducing the importance of the measurement process and its impact on the SDP, the following section summarizes the problems addressed in this work: The first problem is related to the definition of the measurement concepts in the form that support the measurement throughout its lifecycle (Del-Río-Ortega et al., 2009; Habra et al., 2008; Jacquet & Abran, 1997). Defining the measurement concepts in such form supports the integration of the measurement into the SDP. The research tries to answer several questions to address this problem, among them: (i) What are the essential measurement concepts? And what are the necessary aspects (e.g., unit, scale type, performer, and context data) to define these concepts operationally? (ii) How to enrich the definition of the measurement concepts in the form that support its integration into the process lifecycle? (iii) How to consolidate the existing measurement selection and definition methods to support the operational definition of measurement concepts. The second problem is related to the integration of measurement issues (e.g., concepts, artifacts, and activities) into the process lifecycle. To address this problem, the research focus on (i) Identify the software process and the measurement process lifecycles. (ii) Define the main activities of these lifecycles. (iii) Integrate the measurement lifecycle into the software process lifecycle. The third problem is related to the necessity to provide a tool to support the management of both lifecycles (i.e., software process and measurement process) (Bandara et al., 2005) in the form that enhances their integration throughout their lifecycles. Resolving this problem requires the following: (i) Study the existing process management tools. (ii) Develop a solution to integrate the management of the measurement process into these tools. 3.1 Main Objectives After defining the scope of the problem, the main objectives are described below. The first objective is defining the main measurement concepts and identifying the characteristics that should be satisfied to define them operationally (in the form that supports the measurement throughout its lifecycle). The second objective is defining the measurement lifecycle, and integrating it into the process lifecycle, for this purpose we propose three metamodels. The first is the Measurement Definition Metamodel (MDMM) which allows the definition and modeling of the measurement concepts through the process modeling phase in the form that integrates these concepts into the process lifecycle. The second is the Measurement Execution Metamodel which supports the measurement during the process execution phase (e.g., collecting and validating the measures data). The third metamodel is the monitoring metamodel which supports the monitoring and reporting of the measurement data. And finally, merging the measurement metamodels with the process metamodels to complete the integration. The third objective is defining the required transformation rules to derive the necessary measurement artifacts (e.g., execution and monitoring models, and measurement documentation) from the measurement definition model. The fourth objective is developing a tool to support the practical use of the proposed solution. This tool allows the process engineers to (i) Define and model the process and its related measurement concepts. (ii) Execute the process considering the measurement issues. (ii) Use the measurement data to support process management and improvement. And the last objective is validating the proposal by applying it in a real environment and evaluating the results of this experience. To increase the readability of the paper we have excluded the measurement execution metamodel, monitoring metamodel, and the MDE transformation rules. 3.2 Influences This section outlines the previous works and the main technological and conceptual aspects that influenced the development of this work. 3.2.1 The conclusions obtained from the previous research. The results obtained from the survey and the mapping studies (presented in section 2.1) have demonstrated the lack of supporting the process measurement in the existing process management tools. This issue could be divided into two aspects: On the one hand, the conceptual aspect, this aspect is related to the definition of the measurement concepts in the form that supports the measurement objectives and the alignment with the business needs. Existing process definition languages (e.g., SPEM and BPMN) only allow defining the measures in a simple and generic form (e.g., measure name, description, and value); this definition lack of defining important measurement aspects such as the information needs to be satisfied by this measure, the validation rules, and the context data. For that, it is essential to enrich the measurement definition to comprise all the necessary data to support the measurement configuration, collection, validation, analysis. On the other hand, the integration aspect, this aspect is related to integrating the measurement into the process lifecycle in the form that uses the measurement definition to (i) Configure the measurements in the process deployment phase, (ii) Collect the process execution data, and (iii) Analyzing and reporting the measurements data in the process monitoring and improvement phases. Integrating the measurement in such a form allows achieving the measurement goals and satisfying the organization's needs. 3.2.2 Model-Driven Engineering Using model-based approaches is a growing trend when developing software processes modeling languages (García-Borgoñón et al., 2014). Due to the high-level abstraction and code reuse (or regeneration) that provide, it seems appropriate to consider this approach when developing a solution. Adopting the model-driven approach reduces the development time, enhance the quality of the final code, and also facilitate and improve the process of applying changes or maintenance (Czarnecki & Helsen, 2003; Kleppe et al., 2003). Model-Driven Engineering paradigm has emerged to address the complexity of the software systems to express the concepts of the problem’s domain adequately. In this line, the basic principle of MDE is «Everything is a model» (Bézivin, 2005). MDE seems appropriate to achieve the objectives of this work because it uses models to represent the information of a given domain. In the context of this work, using the models allows the formalization of the measurement information. Also because this paradigm applies transformations which are a possible tool to describe, perform, and automate the transformations between the models. Furthermore, and as mentioned earlier, using model-based approaches is the current trend in the development of the software processes modeling languages. 3.2.3 Product Lifecycle Management for Business-Software (PLM4BS) framework The PLM4BS is a model-driven based framework (García-Borgoñon et al., 2013; Garcia-Garcia et al., 2017) that aims to model and manage software processes. It defines metamodels or domain-specific languages to define and executes processes. Furthermore, it establishes systematic protocols to support the necessary transformations between the process models. PLM4BS is based on a continuous improvement lifecycle. This lifecycle comprises four phases: modeling phase, execution and orchestration phase, monitoring phase, and the continuous improvement phase. PLM4BS framework has been developed in the same research group in which this work was developed; the Web Engineering and Testing Early (IWT2) research group has started the development of PLM4BS to support the evolution of the NDT (Navigational Development Techniques) (Escalona & Aragon, 2008; Escalona, 2004) methodology which was developed and used by the group to support the software development lifecycle. For this reason, this work is influenced and motivated by this framework. As mentioned early, PLM4BS defines the process lifecycle in four phases. Currently, the framework provides support for the first two phases (modeling and execution & orchestration) only. Therefore, the main goal of this work is to support the investigations that aim to cover the rest of the process lifecycle defined by the PLM4BS (Monitoring and improvement). Precisely, this work is part of the research that seeks to promote the integration of the measurement issues into the process lifecycle (e.g., design, modeling, execution, and monitoring) in the way that supports the process monitoring and improvement. After presenting the problem addressed in this work, the main objectives, and the key influences which guided and motivated this work, the next section introduce the proposed solution to achieve the established objectives. 4. The proposed solution In previous sections, we have defined the problem addressed by this work and the main objectives established to resolve it. This section presents the proposed solution to support the measurement throughout the process lifecycle. 4.1 Define and integrate the lifecycles As described in previous sections, several proposals have been presented to determine and describe the main stages of the process lifecycle. Each of these proposals focuses on a specific perspective according to its context of use. By studying these proposals, we find that the main activities of the process lifecycle can be categorized into the following stages: Process discovery and Design, Modeling and simulation, Deployment, Execution, Monitoring and analysis, and Process continuous improvement. 4.1.1 Measurement lifecycle In recent years, several proposals have been presented to define the measurement lifecycle, main proposals are described in previous sections. Below, we propose a more comprehensive measurement lifecycle. As shown in figure 4.1, the proposed lifecycle defines all the activities related to the measurement process: Figure 4.1. Measurement process lifecycle. The first phase of the measurement lifecycle is measurement Selection and Definition: in this phase, the measurement objectives and concepts are defined. The measurement selection and definition methods (e.g., GQM, and PSM) are used to choose the appropriate set the measurement concepts that satisfy the measurement objectives. Modeling: In this phase, the measurement definition resulted from the previous phase is represented in a formal and operational form, and also integrated into the process model. The defined measurement concepts and their relationships could be analyzed and optimized (e.g., for consistency, correlation, and causality issues) (Del-RíoOrtega et al., 2013; Popova & Sharpanskykh, 2010) in this phase. Configuration: In this phase, the measurement definition established in the previous stage is used to perform the necessary configurations to achieve the measurement activities; the process execution environment is prepared to allow the collection, validation, storing, and also reporting the measurement data. Collection: During the process execution, the defined measures are gathered, validated, prepared, calculated, and stored according to the definition established in the first phase. Analysis and Reporting: In this phase, measurement data is analyzed and reported according to the measurement definition; the resulting information is generated, formed, and communicated to the pre-defined roles as indicated in the measurement definition. Evaluation and improvement: In this phase, the measurement process is evaluated, lesson learned, and feedback about the process is gathered and assessed to discover improvement opportunities. There are many validation and evaluation frameworks (e.g., (Fenton & Pfleeger, 1996; Habra et al., 2008; B. Kitchenham et al., 1995)). Also, the industry standard ISO/IEC/IEEE 15288:2015 (Martin, 2015) could be used to validate the measurement from two main perspectives: Relative verification which evaluates the design objectives of the measurement, the necessary precision, the maturity of the available knowledge about the attribute, etc. And the Absolute verification that focuses on evaluating the measurement principle in itself; to ensure that the process is characterizing what it intended to measure (Habra et al., 2008). 4.1.2 Integrating the process and measurement lifecycles The measurement process is closely related to the SDP since the measurement process provides support for the software process in various phases throughout its lifecycle such as design and simulation, monitoring and improvement(Mora et al., 2009). Therefore, the integrated management of both lifecycles is essential to transform the organization toward quantitative management (management by facts). This integration defines the measurement activities which should be carried out at each stage of the development process. This integration also encourages people to adopt the measurements as part of their work. (Daskalantonakis, 1992; Dekkers & McQuaid, 2002). The potential benefits of this integration include: (i) The integration of the measurement process into the development process establishes a connection (Figure 4.2) between the two parts of the development process (that is, the management process and the production process (McLeod, Raymond and Schell, 1990; Ruiz-gonzález & Canfora, 2004)). This connection allows the data flow from the production process to the management process, which is fundamental for management and decision making. (ii) Minimize the redundancy of the measurements and improve their consistency in the organization. (iii) Provide a clear and comprehensive measurement plan at the early stage of the development process. This plan identifies and defines the necessary measurement concepts, activities, and artifacts throughout the process life cycle. (iv) Promotes objective communication between the stakeholders by using common concepts and terminology. (v) Defining how the development process (for example, activities, stakeholders, and results) will be measured at an early stage of the development process promotes the achievement of the process objectives in terms of performance, productivity, and quality. (“Tell me how you will evaluate me to tell you how I will behave. (Eliyahu & Goldratt, 1990)”). Figure 4.2. The measurement process connects the two parts of the SDP. This integration could be done by introducing the measurement activities into its corresponding phases in the software process lifecycle and allowing the transition of the artifacts between the two lifecycles. Figure 4.3 shows the relationship between the software process lifecycle and the proposed measurement process lifecycle. The integration details are described below: Process discovery and design: Throughout this phase, process engineers define the process's main objectives, activities, roles, and outputs. The measurement team - in collaboration with the process engineerscan use these details to (i) Define main measurement goals and concepts. (ii) Derive the indicators and measures from these goals using measurement selection methods (e.g., GQM, GQIM, and PSM). Taking the measurements into consideration at this stage have several benefits: (i) allow the management team to communicate their information needs, prioritize their objectives, design the format of the reports, defines the expected values, and analysis models, etc. (ii) Support the measurement team to a better understanding of the measurement requirements and objectives. (iii) Demonstrate the management's commitment to the measurement processes which is an essential success factor for the measurement process (Tahir et al., 2016). As mentioned before, this paper only covers the measurement definition metamodel, the rest of the metamodels and the transformation rules will be introduced in future papers to improve the readability of this paper. The next section introduces the proposed Measurement definition metamodel. The measurement definition metamodel (MDMM) aims to support the measurement modeling phase. Its allows the engineers to define the established measurement goals and the necessary measurement concepts to achieve these goals, these goals and concepts were identified in the first phase of the measurement process (Selection and Definition). As shown in figure 4.6 the proposed measurement language is described in form of a MOF metamodel and presented by the UML class diagram notation. Moreover, we have defined the necessary semantic constraints -as recommended by the Object Management Group (OMG)- using the OCL language ISO/IEC 19507 (ISO/IEC 19507:2012.Information technology - Object Constraint Language (OCL), 2012). We describe below the main elements of the proposed metamodel. The AbstractMeasure is the main metaclass in the proposed language. It represents a generalization of the three types of measures (base measure, derived measure, and indicator). This metaclass defines the common attributes (detailed in section 4.2.2) and relations of these metaclasses. The associations of this metaclass allow the definition of the stakeholder role, the measurement context, the measured attribute, the process to which the measure belongs, and the annotation which allows adding custom attributes and notes to the measure. Also has an association with the MeasurementMethod metaclass to define how the measurement value is obtained. The BaseMeasure metaclass represents the measure that quantifies a specific attribute of an entity (e.g., process, process element, or work product). The associations of this metaclass allow the definition of the human role responsible for performing the measurement, and a measurement method to obtain the value of the measure, this measurement method should define at least one procedure. The OCL expression which implements this restriction is: context BaseMeasure inv measureHasProcedure : self.mMethod.mProcedures->size()>=1 The DerivedMeasure metaclass represents a relationship or algorithm/function between multiple measures (i.e., base measures or derived measures). It has an association with the MeasurementMethod metaclass, this association allows the definition of an algorithm to obtain the value of this measure. The following OCL expression implements this restriction: context DerivedMeasure inv measureHasAlgorithm : self.mMethod.mAlgorithm->size()>=1 The Indicator metaclass allows the evaluation of the measurement objective based on defined analysis rules, also suggests actions based on defined decision criteria. The attribute Analysis guide define the necessary details to support and guide the analysis and interpretation of the indicator results, where the attribute Decision criteria specify actions to be taken based on specific indicator results, and the attribute Presentation guide provide a guide about how to visualize and report the indicator results. Moreover, this metaclass defines associations to specify the analyzer role and the InformationNeeds evaluated by this indicator. InformationNeeds metaclass represents the required information to track a goal or constraint, this metaclass defines associations to specify the author's role and the indicators that evaluate the information needs element. The metaclass MeasurementMethod defines how the measurement value is obtained, its associations allow the specification of the measures, algorithms, and procedures related to this element. It is not allowed to associate procedure and algorithm with the same MeasurementMethod, this restriction is defined in the metamodel using the UML logical operator «XOR» associated with the mProcedures and mAlgorithm. This metaclass should be associated with at least one «mProcedures» or one «mAlgorithm», this restriction is implemented using the following OCL expression: context MeasurementMethod inv hasProcedureOrAlgorithm : (self.mProcedures->size()>=1) or (self.mAlgorithm->size()>=1) Procedure metaclass defines how the attribute of the entity is characterized. The associations related to this element specify the MeasurementMethod associated with the procedure and the instruments which implement the procedure (if it exists). The Algorithm metaclass defines a relation between measures to calculate the derived measures or indicators. The associations of this element allow specifying the measurement method and the measures related to the algorithm element. The Stakeholder metaclass represents the human roles (e.g., performer, responsible, and author) involved in the measurement activities. And, the Instrument metaclass represents the necessary instruments to obtain the measurement value. Moreover, the Context metaclass represents the necessary information to verify, interpret, or evaluate the measurement value, where the Annotation metaclass allows the user to add more notes or attributes to define the measure element. Finally, the enumeration Scale-type classifies the measurement scale type, and the CollectionM enumeration defines the possible values of the collection methods The process engineers will use this metamodel (MDMM) –in the measurement modeling phaseto describe the measurement concepts, the output of this phase is a measurement definition model, this model contain the formal description of the measurement concepts and relations. This data is needed to support the rest of the measurement lifecycle phases. As shown in figure 4.7, the measurement definition model will be used to automatically generate –using MDE transformationsthe necessary artifacts (execution and monitoring models and the measurement documentation) to support the measurement process and its integration into the process lifecycle. After defining the MDMM and describing its role in supporting the measurement process lifecycle, next section describes the developed tool which allows the practical use of this proposal. Figure 4.6. The measurement definition metamodel (MDMM). class Measurement Definition MetaModel «metaclass» AbstractMeasure Name :String Description :String Scale-type :Scale-type Scale :String Unit :String Scheduling :String Validation guide :String Accessibility :String Priority :String CollectionMethod :CollectionM «metaclass» BaseMeasure Objective :String «metaclass» Indicator Objective :String Analysis guide :String Decision criteria :String Presentation guide :String «metaclass» DerivedMeasure Objective :String «metaclass» InformationNeeds Title :String Description :String Accessibility :String Version :String Priority :String «metaclass» MeasurementMethod Name :String Description :String «metaclass» Procedure Name :String Description :String Instructions :String «metaclass» Instrument Name :String Description :String Guide :String Locator :String «metaclass» Algorithm Name :String Description :String Formula :String = «metaclass» Process Name :String Description :String «metaclass» ProcessElement Name :String Description :String «metaclass» Stakeholder Name :String Description :String Role :String «enumeration» Scale-type Nominal Ordinal Interval Ratio Custom «metaclass» Annotation Name :String Content :String {XOR} «metaclass» Context Name :String Description :String «metaclass» Entity Name :String Description :String «metaclass» WorkProduct Name :String Description :String «Enumeration» CollectionM manually script service «metaclass» Attribute Name :String Description :String absM 1hasAn annts 0..* responsible1 hasResp aMeasures 0..* procedures 0..* hasIns instrument 0..* quantifies 0..* Measurand 1 measures 1..* hasMM mMethod 1 analyzer 1 analyzedBy indicators 0..* author 1 autheredBy inf_needs_Items 0..* algorithms 0..* involves aMeasures 1..* aMeasure 1hasC contexts 0..* mMethods 1..* hasP mProcedures 0..* mMethods 1 hasAl mAlgorithm 0..1 0..* Composed_By 1 belongTo 0..* 1 performer 0..1 isPerformer bMeasures 0..* indicators 1..* isEvaluatedBy inf_needs_Items 1 Figure 4.7. The role of the measurement definition metamodel in the proposed solution. 5. Measurement Modeling tool The previous section has presented the proposed metamodel to model the measurement concepts of the software processes. However, to support the practical use of this proposal, it is necessary to develop a tool that allows the engineers to model the measurement concepts, this section presents our Measurement Modeling Tool (MMT). As mentioned before, this work is influenced by our previous works; This proposal aims to support the modeling and integration of the measurements in our PLM4BS process modeling tool (see section 3). PLM4BS is based on Enterprise Architect (EA) (Sparx Systems, 2018)) as a modeling CASE tool. Therefore, the development of our MMT consists of integrating the proposed measurement definition language into the PLM4BS process modeling tool (i.e., integrating our measurement definition language into the EA). This integration consists of two steps: (i) Develop a domain-specific modeling notation (i.e., UML profile) which allows the practical use of our MDMM, and (ii) Integrate this notation into the PLM4BS process modeling language. 5.1 Develop the Specific language (Measurement UML profiles) UML profile provides a usable, expressive, and flexible mechanism to adapt a theoretically defined metamodel with specific constructs for a particular domain (Object Management Group, 2015). This profile allows the instantiation of the MDMM using a visual notation that can be used by CASE tools (e.g., NDT-Tool (Escalona et al., 2003), IBM Rational Software Architect Designer (IBM Corporation, 2018), and Enterprise Architect. The UML extension protocol is based on three basic mechanisms: «Stereotype», «Tagged value», and «Constraint». The UML profile that implements our MDMM defines a stereotype for each metaclass of the MDMM and includes the required tagged values in each stereotype to represent the attributes of each metaclass in the metamodel. It also adapts the semantic constraints of the metamodel to restrict the behavior of the UML metaclass used. 5.2 Integrating the measurement definition profile into PLM4BS process modeling tool To allow the practical use of the solution proposed in this work, we need to integrate the proposed metamodel into our process modeling tool (PLM4BS). As mentioned earlier, this tool is based on Enterprise Architect (EA) CASE tool; EA supports the creation of visual instances of the metamodels that describe the process (e.g., software process, clinical guides). To perform this integration, we need to add the UML profile developed in the previous step to the EA. One of the advantages that EA provides is its extension capacity, which allows the development of Add-Ins. For this purpose, EA provides a Model Driven Generation (MDG) Technology, which allows the development of custom packages and deploys them in the EA project, providing a solution tailored to specific domains or environments. We have used the MDG technology to develop an Add-In to allow the instantiation of the MDMM proposed in this work. Figures 4.9 and 4.10 show the measurement toolbox defined for our MMT. 6. The application of the proposed solution The previous section has described the development of our Measurement Modeling Tool and its integration into our process modeling tool (PLM4BS). This section describes the application and evaluation of our MMT in a real project. 6.1 The Application of the proposal We have applied and evaluated our proposal in a real project related to the health industry; the IDE4ICDS project (Integrated Development Environment for Improving Clinical Decision Support based on Clinical Guides). This project aims to develop the IDE4ICDS platform, which establishes a real working philosophy oriented to clinical guides (Audet et al., 1990), together with effective, systematic, and automatic mechanisms within the health sector organizations. This approach allows the representation, maintenance, execution of the clinical guides, also, capturing feedback about its use to improve the quality of the health care received by the patient; all using software tools that enable these tasks as well as the interoperability between systems to transfer and share clinical knowledge. The main objective of the IDE4ICDS platform is implementing the clinical guides lifecycle which is similar to the software process lifecycle (i.e., Design, modeling, deploying, executing, monitoring, etc.) The previous description of the project and its objectives highlights the role that the proposal presented in this work can play in achieving these objectives. Our proposal has contributed to monitoring the status of the clinical guides by defining the measurement concepts which evaluate and monitor the execution of the clinical guides. We describe below how the proposed solution was applied during the clinical guide lifecycle: In the clinical guide design phase, the stakeholders (e.g., health professionals and process engineers) define the clinical guide objectives and requirements (e.g., identifying the biomedical best practices and references, the technical requirements to execute the process, etc.). Integrating the activities of our measurement selection and definition phase (see section 4.1) has allowed the stakeholder to define the measurement objectives (based on the clinical guide goals). Furthermore, supported the team in identifying the main measurement concepts that satisfy these objectives. Furthermore, these activities supported them in applying measurement methods to select and define these concepts in an operational manner. And in the modeling phase, stakeholders describe the different perspectives of the clinical guide in a formal language; the objective is ensuring a common understanding of the clinical guide perspectives between the various stakeholders. The MDMM and the UML-profile proposed in this work allow stakeholders to describe formally the measurement objectives and concepts defined in the previous step. Figure 6.1 shows part of a clinical guide modeled using the PLM4BS framework, also shows the defined measures and indicators integrated into the clinical guide model. And figures (6.2 and figure 6.3) demonstrate parts of the formal description of the measurement concepts defined for this clinical guide using our MMT. Figure 4.9. The elements and relations of the MMT toolbox. Figure 4.10. Measurement toolbox. 6.2 The results of applying the proposed solution. The proposed solution has provided the support needed by the project team to carry out the measurement activities during the project; the proposed lifecycle has been used to plan and identify the required measurement activities, and the proposed measurement modeling language (MDMM) has been used to describe the measurement objectives and concepts formally and operationally. The feedback of the project team has highlighted the following benefits as the main contributions of applying the solution. The proposed measurement concepts and information model have contributed to the unification of the measurement vocabulary used in the project and connected them coherently, also ensured the traceability between these concepts. Using the proposed measurement concepts and information model has promoted a clear and common understanding of the measurement goals and concepts and its relationships, which has supported the communication between the project stakeholders. Moreover, the proposed measurement language has supported the operational definition of the measurement concepts. The proposed measurement lifecycle has provided a clear and comprehensive guide to the project team; it has defined and consolidated the measurement activities which should be performed during the clinical guide lifecycle. Furthermore, this lifecycle has supported the project team in planning and performing the measurement activities by defining why? When? And how? These activities should be conducted. As well as, the formal definition of the measurement concepts using the proposed MDMM has supported the communication between the different roles in the project and reduced the errors, time, and costs. Besides this, since the proposed solution addresses the international standards (e.g., ISO 1593-2017) and the best practices available in the literature, it has supported the project team to follow and comply with the measurement standards. Figure 6.1. Part of a clinical guide model. class Class Model KO (> 160/90 mm Hg) OK (< 160/90 mm Hg) «HumanActiv ity» pressure reading «HumanActiv ity» pressure reading «Stakeholder» Health professional «Stakeholder» Patient Control blood pressure «HumanActiv ity» Appointment for the next test within 3 months «HumanActiv ity» Appointment for the next test within 3 months «HumanActiv ity» Re-measure blood pressure «HumanActiv ity» Re-measure blood pressure «Indicator» Patients with a blood pressure > 160/90 mm had re-measured «Metric» M1_Number of patients «Metric» M2_ Patients have to re-measured «Metric» M3_Patients having had re-measured «Indicator» Percentage of patients with blood pressure > 160/90 mm Hg «MeasuredBy» «MeasuredBy» «MeasuredBy» «MeasuredBy» «MeasuredBy» Figure 6.2. Part of the definition model of the measurement concepts (1) class Percentage of re-measured Patients «InformationNeeds» Percentage of re-measured patients tags Accessibility = Health professionals, Health management Description = Percentage of re-measured patients Priority = 3 Title = Percentage of re-measured patients Version = 2 «Indicator» Indicator2 tags Accessibility = Health professionals Analysis guide = Target :90% of the patients should have their blood pressure re-measured. CollectionMethod = Script Decision criteria = Green (>= 90%), Yellow (89%-70%), Red (<70%) Description = Percentage of re-measured patients Name = Percentage of re-measured patients Objective = Presentation guide = Priority = 3 Scale = Real number Scale-type = Ratio Scheduling = Each 6 months Unit = Percentage Validation = valid value between 0-100 «MeasurementMethod» MeasurementMethod3 tags Description = Calculate % of re-measured patients Name = MM1 «Algorithm» Algorithm2 tags Description = (Re-measured patients / Patients with high blood pressure) * 100 Formula = (M2/M3)*100 Name = Percentage of re-measured patients «isEvaluatedBy» «hasMM» «hasAl» Figure 6.3. Part of the definition model of the measurement concepts (2). 7. Conclusions and future work In this paper we have outlined several issues related to the process measurement domain, the main problems discussed in this research are (i) the operational definition of the measurement concepts, and (ii) the integration of the measurement issues (e.g., concepts, artifacts, and activities) into the process lifecycle. To address these problems, we have proposed a measurement process lifecycle, this lifecycle defines all the necessary concepts and artifacts to define and achieve the measurement objectives. The proposed lifecycle also describes the necessary activities to achieve the measurement goals in each phase of the process lifecycle (e.g., design, modeling, execution). After defining the measurement lifecycle, we have described how this lifecycle can be integrated into the process lifecycle. As we have explained, this integration describes (i) how the measurement activities will be performed throughout the different phases of the process lifecycle, and (ii) how the artifacts will be exchanged between the activities of both lifecycles. Furthermore, we have proposed a measurement information model to define the measurement concepts and also to describe the relationships between the measurement goals and the necessary objective data (measures) to satisfy these goals. Besides, we have proposed our measurement definition metamodel, which supports the modeling phase of the measurement process lifecycle. This metamodel allows the engineers to define operationally the established measurement goals and the necessary measurement concepts to achieve these goals. To allow the practical use of this metamodel, we have developed a measurement modeling tool and integrated this tool into our process modeling tool (PLM4BS). Moreover, we have validated our proposal in several ways, on the one hand, we have applied our proposal (the MIM, measurement concepts and its operational definition) to many scenarios from the literature and from real class Base Measures_1 «BaseMeasure» BM1_Number of patients tags Accessibility = CollectionMethod = Manually Description = Total number of patients Name = BM1_Number of patients Objective = Priority = Scale = integer Scale-type = Ratio Scheduling = each 6 months Unit = Patient Validation = «MeasurementMethod» MeasurementMethod2 tags Description = Count the number of patients manually Name = MM2 «Stakeholder» Stakeholder1 tags Description = Name = Health Prof_1 Role = Health Professional «Procedure» Procedure1 tags Description = Describe how to count the patients Instructions = The patient is added to the number of patients upon arrival at the clinic Name = Proc_1_count the Patients procdef = <memo> «hasMM» «hasResp» «isPerformer» «hasP» cases. As an example -and to increase the readability of this paperwe have included only one scenario (see section 4.2.3), this has served as a proof-of-concept that shows the applicability of our proposal. On the other hand, the proposal has been applied to several industry experiences. As an example, we have included the experience of applying it to a project related to the health sector, the results obtained from this experience has shown that the proposal is useful and provide an important support to the project team in defining and integrating the measurement issues into the clinical guide lifecycle. In this work, we have presented our proposal to support the measurement definition and modeling phases of the measurement process lifecycle. As future work, we plan to provide support for the rest of the measurement process lifecycle. Currently, we are developing and testing the measurement execution metamodel to support the measurement collection phase, and the measurement monitoring metamodel to support the analysis and reporting phase. The measurement execution and monitoring models will be derived automatically from the measurement definition model presented in this work, moreover, we are developing the necessary transformations to derive the artifacts (models and documentations) which support the measurement process lifecycle. Acknowledgements This research has been supported by the NICO project (PID2019-105455GB-C31) of the Spanish Ministry of Science, Economy and University and by NDT4.0 (US-1251532) of the Andalusian Regional Ministry of Economy and Knowledge References Abreu Fernando Brito, & Carapuça, R. (1994). Object-oriented software engineering: Measuring and controlling the development process. Proceedings of the 4th International Conference on Software Quality, 186, 1–8. Allweyer, T. (2015). Business Process Model and Notation (BPMN) Version 2.0. https://doi.org/10.1007/s11576-0080096-z Audet, A.-M., Greenfield, S., & Field, M. (1990). Medical practice guidelines: current activities and future directions. Annals of Internal Medicine, 113(9), 709–714. Bandara, W., Gable, G. G., & Rosemann, M. (2005). Factors and measures of business process modelling: model building through a multiple case study. European Journal of Information Systems, 14(4), 347–360. https://doi.org/10.1057/palgrave.ejis.3000546 Bang, S. K., Chung, S., Choh, Y., & Dupuis, M. (2013). A grounded theory analysis of modern web applications: knowledge, skills, and abilities for DevOps. Proceedings of the 2nd Annual Conference on Research in Information Technology, 61–62. Barcellos, M. P., de Almeida Falbo, R., & Rocha, A. R. (2013). A strategy for preparing software organizations for statistical process control. Journal of the Brazilian Computer Society, 19(4), 445–473. https://doi.org/10.1007/s13173-013-0106-x Bendraou, R., Gervais, M.-P., & Blanc, X. (2006). UML4SPM: An executable software process modeling language providing high-level abstractions. Enterprise Distributed Object Computing Conference, 2006. EDOC’06. 10th IEEE International, 297–306. Berander, P., & Jönsson, P. (2006). A goal question metric based approach for efficient measurement framework definition. Proceedings of the 2006 ACM/IEEE International Symposium on International Symposium on Empirical Software Engineering - ISESE ’06, 316–325. https://doi.org/10.1145/1159733.1159781 Bézivin, J. (2005). On the unification power of models. Software & Systems Modeling, 4(2), 171–188. https://doi.org/10.1007/s10270-005-0079-0 Bonitasoft. (2016). Bonitasoft. http://www.bonitasoft.com/