NDT-Agile: An Agile, CMMI-Compatible Framework for Web Engineering

NDT-Agile: An Agile, CMMI-Compatible Framework for Web Engineering Carlos J. Torrecilla-Salinas 1 , Tatiana Guardia 1 , Olga De Troyer 2 , Manuel Mejías 1 , and Jorge Sedeño 1 1 IWT2 Group, Universidad de Sevilla, Seville, Spain {carlos.torrecilla,tatiana.guardia, jorge.sedeno}@iwt2.org, [email protected] 2 Department of Computer Science, Vrije Universiteit Brussel (VUB), Brussels, Belgium [email protected] Abstract. Agile and Web Engineering show important synergies, making Agile a common approach for Web development. Besides, several initiatives emerged to support CMMI-DEV within Agile, where CMMI-DEV aims to improve organizations’ software development process. An approach integrating Agile, Web and CMMI-DEV might be of great value, since they might allow Web development teams to use Agile, as well as progress through CMMI-DEV maturity levels. For this purpose, we developed NDT-Agile, an NDT-based Agile framework to achieve the goals of CMMI-DEV in the context of Web Engineering. It was developed by mapping Agile practices to the goals of CMMI-DEV so as to identify existing gaps. Next, we searched for suitable Agile practices to cover the gaps and integrated them into a framework called NDTAgile, which was validated using an expert-judgment technique: the Delphi method. This paper describes how we integrated Agile and CMMI-DEV into a Web Engineering framework. Besides, it also analyzes its initial evaluation, together with a first tool developed to support it. Keywords: Agile  Scrum  CMMI  Web engineering  Expert judgment 1 Introduction Agile methodologies, i.e. those that can be grouped under the principles and values described in the Agile manifesto [1], emerged as an alternative to classic software development approaches, which were frequently based on heavy up-front planning and on freezing requirements before the development started. Agile brings a completely different view on how to handle and approach requirements [2]. It is based on improved communication, close collaboration with business representatives and reduced delivery cycles, among other elements [1]. Several approaches can be found within the label Agile, like Scrum [3], eXtreme Programming (XP) [4] or Kanban [5], being Scrum and XP the most popular ones [6, 7]. Web Engineering has established itself as the field of Software Engineering in charge of developing Web Systems, those conceived, developed, deployed and used on the Web [8]. Several methodological approaches have been proposed for Web Engineering, such as Navigational Development Techniques (NDT) [9]. In turn, NDT proposes a Model-Driven approach to Web Engineering. NDT is a methodology that focuses on the first phases of the Web development lifecycle and utilizes a bottom-up process. It uses a highly-detailed requirement gathering phase guided by objectives with three sub-phases: requirements capturing, definition and validation. It is important to note that NDT was not developed bearing the Agile approach in mind. Both Agile and Web Engineering emerged simultaneously and independently. However, they show great synergies [10]. As such, several Web development teams have already applied an Agile approach. CMMI (Capability Maturity Model –Integration) [11] is a well-known approach designed to improve organizations’processes. Out of the different maturity models proposed by CMMI, CMMI-DEV (Capability Maturity Model –Integration for Development) is the one focusing on software development. Currently, CMMI is used in more than 5,000 companies [12]. The progress through the different CMMI maturity levels is normally associated with increases in quality and customer satisfaction [13]. During several years, the approaches proposed by Agile and by CMMI were seen as opposite and even contradictory [14], but after that initial reluctance period, initiatives emerged from both sides trying to find common grounds [15]. Recently, we can find several proposals trying to combine these approaches, both for generic software development projects [16] as well as for Web specific projects [17]. Summarizing all the aforementioned elements, we can conclude that an approach based on Agile principles that allows progressing through the different CMMI-DEV maturity levels, simultaneously supporting Web specificities, will be of great value as, on the one hand, it will enable organizations to keep using an Agile approach for their Web development projects but, on the other hand, it will ensure repeatability and institutionalization by means of the process improvement approach carried out by CMMI. Based on the foregoing arguments, we have developed NDT-Agile, a framework conceived to help organizations achieve the specific and generic goals of CMMI-DEV in the context of Web Engineering, while keeping agility and ability to respond to changes. This paper has the following objectives: (i) present how agility, CMM-DEV and Web Engineering have been integrated into a coherent framework, called NDT-Agile; (ii) describe the assessment of NDT-Agile by means of an expert-judgment process based on the Delphi method [18]; (iii) introduce a first version of a tool to support NDTAgile; and (iv) draw relevant conclusions and present further lines of research. To achieve the listed objectives, the paper is organized as follows: after this introduction, Sect. 2asks the research question and describes the utilized research approach. Section 3discusses related work. Then, Sect. 4introduces NDT-Agile by describing its main elements. Afterwards, Sect. 5presents the expert-judgment validation process, as well as the developed supporting tool and, finally, Sect. 6states the main conclusions and further lines of research. 2 Research Question and Research Method The main research question that we addressed in our research was: “Can we develop an Agile approach compatible with CMMI-DEV and usable for organizations developing Web systems?”To answer this question, we asked a few concrete research sub-questions: •RQ1: What are the existing gaps between the current most used Agile approaches and the specific and generic goals of CMMI-DEV for Web systems? •RQ2: Are there any existing Agile techniques to cover those gaps, in case they exist? •RQ3: How can we combine the characteristics of the most used Agile approaches with the Agile techniques identified in RQ2 in a single coherent framework suitable for Web systems development? •RQ4: How can we validate this framework, in case it can be developed? Once the research questions were asked, the next step consisted in defining a suitable research approach, which finally comprised the following steps: •Perform a gap analysis: To distinguish if the existing and most popular Agile approaches can cover the different specific and generic goals of CMMI-DEV maturity level. As previously stated, Scrum and XP are the most used Agile approaches [6] whose practices are mapped to CMMI-DEV. The results of our gap analysis will be presented in Sect. 3. •Identify suitable Agile techniques to cover the gap: To identify goals not covered by Scrum or XP practices by means of the gap analysis. The next step consisted in searching (in existing Agile literature) for other suitable Agile practices to cover the gaps. Results of this exercise will be also presented in Sect. 3. •Combine the identified techniques in a single coherent framework: To define a coherent framework, named NDT-Agile, where the identified Agile practices should be combined in a suitable way, avoiding duplicities, gaps or contradictions. NDTAgile will be presented in detail in Sect. 4. •Validate the proposed framework: To validate the proposed framework by means of an expert judgment process based on the Delphi method [18], before performing real-life experiments. The goal of this expert evaluation was to obtain an initial validation of the framework. This process will be presented in Sect. 5. •Develop an initial version of a tool to support the framework: To develop a first version of a supporting tool to backup future deployments of the framework. Such tool will be introduced in Sect. 5. 3 Related Work This section summarizes related work that was collected by means of a Systematic Literature Review (SLR) based on Kitchenham’s approach [19]. This process is described in more detail in [17]. The main goal was to identify previous works tackling the relations among Agile, Web and CMMI. From the results of the SLR, different types of papers were identified such as: other existing SLRs in the context [16]; papers tackling different angles of the research problem; and theoretical studies or case studies coming from both Agile [20–22] and CMMI side [23], some of them including the Web perspective [24–26], but some others not [20,22]. In this section, we focus on those works that performed a gap analysis or a mapping between Agile and CMMI, regardless of whether or not they considered the Web perspective. In [21], a Scrum-based model named Model C-S is presented. It maps the specific practices of CMMI levels 2 and 3 to Scrum ones. This model includes 123 practices, but excludes some CMMI-DEV process areas linked to organizational issues. The work comprises a mapping describing which practices are fully or partially covered, or they are not covered at all, and some ad-hoc modifications to Scrum. Besides, the proposed model incorporates supporting elements to deploy and assess the model together with two case studies. Further on, [22] assesses, from a theoretical point of view, whether the standard Scrum practices can cover the goals of a set of CMMI-DEV process areas from maturity levels 2, 3 and 4 (those linked to project management). The work presents an analysis of the coverage provided by Scrum to 22 of CMMI-DEV practices, establishing whether they are fully or partially covered, or they are not covered at all. In [20], a theoretical study on whether Scrum standard practices can cover the goals of a set of CMMI-DEV level 2 process areas is presented. It particularly analyzes Project Planning (PP), Project Monitoring and Control (PMC) and Requirements Management (REQM). It also includes a case study based on an internal project assessment. In [27], the coverage provided by XP practices to CMMI levels 2 and 3 process areas is studied from a theoretical point of view. From its conclusions, we took out that XP supports most level 2 practices and some level 3 ones. The paper also highlights some limitations to CMMI coverage depending on the project size. Additionally, [28] maps specific goals of CMMI maturity level 3 process areas to three different Agile methods (Scrum, XP and Kanban). Then, it evaluates which of the practices proposed by the analyzed Agile methods can cover the different goals of CMMI and provide a percentage of coverage to each of the analyzed techniques. The main conclusion of this work is that there is compatibility between Agile approaches and CMMI level 3, as many of the goals of maturity level can be covered. Finally, the paper includes a case study to validate the proposal. After this review process, we concluded that all the above-described works focus on generic development and do not consider Web specificities. Moreover, we confirmed that all of them are partial, not presenting a full gap analysis or a complete mapping to all CMMI-DEV maturity levels’process areas. As mentioned in Sect. 2, the first step in our research was conducting a gap analysis to identify whether the existing and most popular Agile approaches can cover the different specific and generic goals of CMMI-DEV maturity level. This analysis specifically included Web specificities. In [26], we compared Scrum practices with the goals of CMMI-DEV maturity level 2. This work analyzed theoretically the gap between Scrum practices and those of CMMI-DEV level 2, concluding that, even though there is no full coverage between both, they are highly compatible. In this paper, we also included a proposal to extend Scrum with the aim to cover the identified gap. Later, in [25], we proposed a mapping between Agile practices (including Scrum and XP standard practices, but going beyond them) and goals of CMMI-DEV maturity level 3. The paper settles that Agile techniques and CMMI-DEV are still highly compatible. Finally, in [24] we performed a gap analysis between Scrum and XP, and the proposed goals of CMMI-DEV maturity levels 4 and 5. The main conclusion was that Scrum or XP standard practices do not cover CMMI practices. Based on that assumption, we identified a set of Agile practices that could be suitable to cover the gap. Table 1summarizes the identified coverage (from [24–26]) to the different CMMI-DEV maturity levels. To obtain the percentage of coverage, the number of CMMI-DEV specific practices fully covered by Scrum/XP standard practices of a particular maturity level was calculated and then divided by the total number practices defined in the maturity level. Table 2describes the identified and proposed extensions spotted in [24–26]to complement Scrum and XP with the aim to cover all the goals of CMMI-DEV. It also indicates whether the proposal is either an existing Agile practice (and then points to a reference describing it), or it is just an ad-hoc modification: 4NDT-Agile: An Agile CMMI-Compatible Framework for Web Engineering In the previous section, we identified works related to RQ1 and RQ2, including our own gap analysis and mapping exercise [24–26]. We also identified a suitable list of Agile practices or ad-hoc modifications that simultaneously support Web specificities and all the specific and generic goals of CMMI-DEV. Nevertheless, a list of practices is not useful for organizations that focus on looking for a coherent framework to implement and customize. In this section, we present NDT-Agile, an Agile framework built upon the conclusions of the gap-analysis and on top of Scrum and XP practices, including all identified proposed extensions listed in Table 2.NDT-Agile also supports Web specificities by integrating NDT (i.e. a Web development methodology) and incorporating it into an Agile lifecycle. The description looks at the way in which CMMI-DEV is supported. NDT-Agile is composed of 3 main components (Fig. 1): Table 1. Identified coverage of Scrum/XP per CMMI-DEV maturity level. CMMI-DEV level 2 CMMI-DEV level 4 Approach Coverage Approach Coverage Scrum 72.2% Scrum 0.0% XP 66.7% XP 0.0% Combined Scrum/XP 92.6% Combined Scrum/XP 0.0% CMMI-DEV level 3 CMMI-DEV level 5 Approach Coverage Approach Coverage Scrum 34.8% Scrum 0.0% XP 54.7% XP 0.0% Combined Scrum/XP 60.5% Combined Scrum/XP 0.0% Table 2. Identified proposed extensions to Scrum/XP standard practices. CMMI-DEV level 2 –Proposed extensions Extension Type Sprint 0 Agile practice [26] Establish measurement objectives, how to measure them and how to store measures and collect data during the project Ad-hoc modification Establish how, when and where to store the project data and use the selected sources during the project Ad-hoc modification Establish how to communicate and manage the project data and follow the agreed approach during the project Ad-hoc modification Establish quality objectives, briefly documenting the agreements Ad-hoc modification Agile contracts techniques Agile practice [29] CMMI-DEV level 3 –Proposed extensions Extension Type Agile Project Management Agile practice [30] Scrum at Enterprise Level Agile practice [31] Lean Software Development Agile practice [32] Agile Risk Management Agile practice [33] CMMI-DEV level 4 –Proposed extensions Extension Type Performance and KPI baselines Ad-hoc modification Adapt the process to achieve desired quality and performance objectives Ad-hoc modification Select measures and techniques to be used for quantitative management Ad-hoc modification Use Agile Performance Indicators Agile practice [34] Agile EVM Agile practice [35] CMMI-DEV level 5 –Proposed extensions Extension Type Lean Software Development Agile practice [32] •NDT-Agile lifecycle, an iterative and incremental lifecycle that describes the way projects are identified, planned, approved and developed, and which encapsulates NDT techniques. It focuses on covering the lifecycle related to the goals of CMMI-DEV levels 2 and 3. •Agile complementary techniques, based on techniques identified in the gap analysis and complementing the framework beyond the scope of a project lifecycle. They cover the remaining not organizational-related goals of CMMI-DEV. •NDT-Agile governance, which wraps the previous two elements and ensures a proper framework rollout, customization and improvement. It covers those goals of CMMI-DEV levels 4 and 5 that have an organizational dimension. 4.1 NDT-Agile Lifecycle NDT-Agile lifecycle is used to manage projects (identify, plan and execute them) [35]. It comprises two main phases and Fig. 2depicts the lifecycle: •Project launching, which is the only non-iterative phase of the framework, where an initial plan is developed by means of Agile estimation techniques [36] combined with Agile Project Management inception techniques [30]. It is presented in the form of an Agile project charter for the organization’s management to approve. Fig. 1. NDT-Agile Components Fig. 2. NDT-Agile lifecycle [35] •Project development, is an iterative phase that, based on the Scrum lifecycle, includes all identified modifications to achieve most project managements related to the goals of CMMI-DEV levels 2 and 3. After a ground-setting Sprint 0, a succession of Sprints is run with the aim to develop the project. The initial plan is adjusted Sprint-by-Sprint in order to ensure that business priorities are always identified and implemented. During this phase the requirements engineering, definition and validation proposed by NDT take place. 4.2 Agile Complementary Techniques Agile complementary techniques were incorporated into our framework aiming to cover all the goals of CMMI-DEV that go beyond the scope of a project lifecycle and mainly come from the identified list of Agile practices resulting from the gap analysis. We identified and included a total of seven complementary techniques: •Agile EVM [35]: It provides an Agile way to control project constraints like budget and schedule without including extra overhead. They are included to cover the remaining goals of Quantitative Project Management (QPM) process area not covered by the standard Scrum/XP practices. •Agile productivity metrics [34]: They are proposed to cover the goals of OPP (Organizational Project Performance), helping to measure consistently teams’ productivity and ensure continuous improvement. •Agile reporting: It is established to cover the generic goals of CMMI-DEV, as described in CMMI-DEV standard, and propose an Agile approach so as to enhance communication with the stakeholder. It includes classic Agile elements such as burn-down or burn-up charts. •Agile Learning: Coming from the Lean approaches [32], it is proposed to fully cover the goals of Organizational Training (OT), Causal Analysis and Resolution (CAR) and Organizational Project Management (OPM). It contains techniques to ensure both team and organization improvements, by means of elements like retrospectives or communities of practices. •Agile risk management [33]: It is included to meet the specific goals of RSKM (Risk Management) and provide Agile projects with explicit risk management capabilities without extra overhead. •Agile contracting [29]: It is proposed to cover the goals of Supplier Agreement Management (SAM) and as a way to ensure an Agile relation with providers. That guarantees that risk is well balanced, thus all parties gain with such a relationship. •Agile engineering practices [4]: They mainly come from XP and are proposed to cover all CMMI-DEV engineering process areas, including Agile design, test and validation elements. 4.3 NDT-Agile Governance NDT-Agile governance is the third component of the framework. It is proposed to cover the goals of level 4 and 5 process areas. It is based on Schwaber’s proposal to scale Scrum to organizational levels [31] and prescribe the establishment of a governance body (named Enterprise Transition Team), which is set at organization level. It comprises the following main objectives: •Tailor the framework according to the organization’s specific needs. •Define the different organizational assets (tool or lessons learnt, among others). •Establish project baselines and define organizational KPIs. The proposed governance body also ensures that Agile practices like Scrum of Scrums (in order to coordinate the different existing Agile teams) or the maintenance of an organization wide product backlog (to have a view of the progress at organizational level) are established. 5 Validation of the Approach and Supporting Tool In this section, we explain how we carried out an initial validation of our proposal. For this purpose, an expert-judgment exercise based on the Delphi method was performed (Sect. 5.1). Furthermore, we also describe the first version of the supporting tool that enabled the framework deployment in practice (Sect. 5.2). 5.1 Expert-Judgment Process As the implementation of new methods or frameworks within organizations always implies economical risks and presents organizational challenges, companies may be reluctant to incorporate them. If an initial validation conducted by a set of well-known experts is presented beforehand, some of these reluctances can be overcome and organizations might be more willing to experiment with the new working methods or frameworks. One of these expert-judgment techniques is the Delphi method [18], which consists in a panel of experts who, by means of structured and anonymous questionnaires and a series of rounds, reach consensus on a specific topic. In order to validate NDT-Agile proposal using the Delphi method, a panel of 20 experts in one or more of the analyzed fields, coming from 8 different countries, was created and three consecutive rounds took place. The questionnaire used was composed of 21 statements, distributed among 4 different domains (i.e. dimensions) as follows: •Agile dimension: 6 statements were used to assess the agility of the framework. •CMMI dimension: 5 statements were used to evaluate the compliance with the different goals of CMMI maturity levels. •Web dimension: 7 statements were provided to supervise the support given to Web specific characteristics. •Framework dimension: 3 statements were used to test the internal coherence of completeness of NDT-Agile. The questionnaire was made available to the experts in three rounds that were organized between February and June 2016. In each round, experts were asked to express their agreement with each of the statements by means of a Likert scale [37] ranging from “Complete disagreement”(value 1) to “Complete agreement”(value 5).