Chapter 5. The task of quantitative assessment of changes in the long-term IT project management system

© The Author(s) of individual chapters, 2025 DOI: 10.21303/978-9908-9706-8-4.ch5 Abstract The object of the study is the process of change management in an IT project. The study addressed the problem of quantitative assessment of changes that arise during the implementation of a long-term IT project. Existing common methods of change management do not allow for quantitative assessment of the main parameters of changes. Modern research is mainly aimed at solving the problem of quantitative assessment and change management within the entire IT project. The issue of quantitative assessment of changes that arise in the process of work of individual teams of IT project performers remains practically unexplored. In the course of the study, it was proposed to improve the existing method for quantitative assessment of changes, based on the Beckhard and Harris's model. To improve this method, it was decided to use a descriptor approach. An improved method was developed, which, unlike the existing one, allows for quantitative assessment of changes based on the values of descriptors. These descriptors are formed in the process of performing sprints of a long-term IT project by its performers. Based on the improved method, elements of information technology for automated solution of the problem of quantitative assessment of changes in the management system of a long-term IT project have been developed. A description of the architecture of information technology has been proposed, its technological stack has been defined, and the results of the development of software elements have been presented. Experimental verification of the obtained research results was carried out within the framework of the long-term IT project "Web Constructor". The verification was carried out on the results of the work of one team of project developers. Maksym Yevlanov Nataliia Vasyltsova Iryna Panforova Anastasiia Popova CHAPTER 5 The task of quantitative assessment of changes in the long-term IT project management system 171 The task of quantitative assessment of changes in the long-term IT project management system Chapter 5 The calculation was based on data obtained during the team's execution of seven project sprints. The verification results showed that the predicted values of the indicators quite accurately coincide with the values of the time actually spent on implementing changes (0.97 and 0.81 for the sixth and seventh sprints of the IT project, respectively). Overall, the application of the improved method improved the change implementation rates during the IT project implementation and improved the team's attitude to the change management process by 14.5%. This allows to consider the improved method and implemented IT elements as a usable tool for change management of individual teams of IT project performers. Keywords IT project, change, Beckhard and Harris's model, least squares method, Huber method, descriptor, sprint. 5.1 Introduction The last decades are characterized by the widespread use of long-term projects, the implementation period of which is three or more years. Such projects are typical for various branches of the IT sphere: development and implementation of information technologies (IT), information systems (IS), IT infrastructure, scientific research in this field, etc.[1,2]. Analysis of the characteristics of modern long-term IT projects allows to highlight their main features[2]: –complexity of description, caused by a large number of functions, processes, elements, data and connections between them; –the presence of a set of closely interconnected subsystems, each of which has its own local goals and objectives; –the need to integrate existing and re-developed applications; –heterogeneity of individual groups of developers in terms of the level of classification and established traditions of using tools; –a constant flow of changes in modern IS. A significant number of changes in the implementation of such projects occur at the operational level of tasks, which makes high-quality change management at this level necessary. For example, poor quality of time-to-market time-variance assessment of individual project operations can lead to project overruns, resource overstrain, and unforeseen delays in product launch. On the contrary, systematic timeto-market time-variance assessment allows project management to make informed 172 Management of a modern IT company: theoretical and technological aspects decisions regarding resource planning, task allocation, and priorities in the event of changes in project requirements or conditions. Implementing such an assessment as a separate functional task of the long-term IT project management system helps reduce the risks of project delays and ensures its successful completion within the established deadlines. The modern point of view considers change management as one of the tasks of change management. The term "change management" should be understood as a comprehensive, cyclical, and structured approach to changing individuals, groups, and organizations from their current state to a future state with expected business benefits[2,3]. This approach is now described in the form of a framework– conventions, principles and methods of change management in the field of program management, project portfolios and individual projects. But, unfortunately, the existing change management framework[3] and standards and body of knowledge on IT project management[2,4] do not provide specific recommendations for the selection of change management methods and their implementation in IT project management systems. Therefore, conducting research on solving change management problems as elements of the IT project management system is relevant from a theoretical and applied point of view. 5.2 Analysis of the approach and modern methods of project change management The general features of the implementation of processes, subprocesses and individual works of the existing change management framework are determined by the change management life cycle adopted in[3]. According to this life cycle, each of the change management subprocesses is proposed to be considered as a set of individual works, each of which is performed in the order established by the framework (but not necessarily strictly sequentially one after the other). The entire change management process and its subprocesses in[4] are proposed to be described using an iterative life cycle model, taking into account the possibility of constant occurrence of adaptive changes in response to changing circumstances. Such a representation of the change management life cycle allows minimizing the connections between individual subprocesses and works of the existing change management framework and considering them as separate independent objects of scientific research. At the technological level, this means the possibility of creating and using for automation of the corresponding subprocesses and works of individual IT, which interact with each other according to the service-oriented paradigm. 173 The task of quantitative assessment of changes in the long-term IT project management system Chapter 5 Since the existing regulatory documents do not provide recommendations for choosing a specific change management method in an IT project, it was decided to consider the features of the change management methods that are considered the most common. These methods include[5]: –Prosci ADKAR model[6,7]; –the Accelerating Implementation Methodology (AIM) method[8,9]; –Beckhard and Harris's model[10]; –Bridges Transition model[11]; –Kotter's 8-Step Change model[12]; –Kübler-Ross model[13]; –a method based on the Kurt Lewin model[14]. ADKAR is an abbreviation that represents five key stages necessary for the successful implementation of changes: "Awareness", "Desire", "Knowledge", "Ability", "Reinforcement"[7]. The Prosci ADKAR model was created by J.Hyatt in the late 1990s. This method offers a deep approach to change management, focused on personnel. It is assumed that in the process of implementing changes, itis necessary to clearly explain to personnel the reasons for the changes and their importance, which contributes to the involvement of personnel in the process. Next, it is important to train each employee in the methods of implementing changes, which allows to show the level of their professional knowledge and skills in the process of introducing changes. The final stage involves consolidating the changes, ensuring the sustainability of innovations in the organization's activities[6]. The Prosci ADKAR method is part of a broader approach to change management developed by PROSCI. It is based on the method using the ADKAR model, but includes additional tools, methods and practices aimed at effectively implementing changes in the organization[7]. AIM is a powerful and disciplined method for managing organizational change, including transformational change, until the full return on investment. AIM can be applied to any type of initiative or project, but most organizations direct the main resources and energy to the technical and business process components[8, 9]. One of the advantages of this method is the ability to systematically analyze and anticipate possible difficulties in the change process, which allows organizations to prepare and respond to them more effectively. However, AIM can require significant re sources and time spent on using the method, and can also become difficult in the event of adverse circumstances or unforeseen events during the change implementation process[9]. The method, based on the Beckhard and Harris's model, provides five stages of change management aimed at identifying the need for change, developing a strategy for its implementation, forming an action plan and identifying responsible 174 Management of a modern IT company: theoretical and technological aspects executors. The main advantage of the method, based on the Beckhard and Harris's model, is a systematic approach to the change process, which allows to structure it and manage it effectively. Praxie has developed software applications for change management using a method based on the Beckhard and Harris's model and provides training[10,11]. The method, based on the Bridges Transition model, enables organizations and individuals to better understand the human and organizational aspects of change and manage them effectively. The main advantage of this method is its focus on internal transition, which can provide a deeper and more sustainable change in the organizational environment. However, it may be less effective when it is necessary to respond quickly to external and unpredictable changes, as it focuses on the internal aspect of the transition[11]. The method, based on the Kotter's 8-Step Change model, is designed to increase staff involvement in change management and ensure its acceptance by all employees. The method includes the following eight stages[12]: "Creating the emotional need for change", "Building a coalition", "Building a vision", "Communicating the vision", "Implementing actions", "Formation of short-term achievements", "Consolidation of achievements", "Implementation of changes into the culture". Skipping one of these stages can lead to problems in making changes and complicate the change process. The main advantage of the method is its systematic approach to change management and emphasis on involving personnel in the process. The following can be noted as disadvantages of the method based on the Kotter's 8-Step Change model: –implementation of all eight stages of the method may require significant effort and time, especially in large organizations or with a large scale of changes; –the method focuses mainly on the organizational aspects of changes, leaving aside individual emotions and needs of employees; –the method does not always take into account external influences, such as economic or political factors, which may affect the success of changes. The method, based on the Kübler-Ross model, describes the stages of personnel behavior change, including: personnel resistance to change, lack of awareness of the consequences of change, personnel adaptation to new working conditions, positive attitude of employees to change and their acceptance. This method is important for understanding and predicting personnel reactions to changes in the organization. However, it is worth considering that the reaction to change can be individual and does not always correspond to this sequence of stages. From a practical point of view, the method can be used to identify any obstacles in the early stages of change projects and develop appropriate strategies. The main advantages of the method 175 The task of quantitative assessment of changes in the long-term IT project management system Chapter 5 include the fact that it is simple but effective for managing organizational changes. The disadvantage of the method is that since all employees react at different speeds, they are at different stages marked on the change curve. This often leads to problems when planning changes[13]. The method, based on the model of Kurt Lewin and developed in the middle of the twentieth century, remains one of the most popular at the current stage of change management development. This method is notable for its systematicity and practicality, which makes it effective for implementing changes in various types of organizations. However, it can require significant resources and time to implement, and may be less effective in the case of complex organizational structures or indiscriminate application of change management methods[14]. To compare the considered methods of change management in projects, the study proposed basic criteria (indicators). The results of the comparison of change management methods by criteria are given in Table5.1[5]. Table 5.1 Comparison of existing methods of change management in projects Indicator Method No.1 Method No.2 Method No.3 Method No.4 Method No.5 Method No.6 Method No.7 Ease of use – – +– – + + Complexity of implementation – – +–+ + + Time required for use + + –+ + + – Availability of quantitative assessment of changes – – – – – – – Flexibility of the method – – + + –+– Use in rapid changes –+ + – – – – Possibility of certification + + –+ + – – Possibility of training + + + + + + + Availability of an informative website + + + –+– – Availability of free training – – +– – + + Availability of use in complex projects + + + –+– – Cost of training, $ 2000 1790 – – 6000 (850) – – Source:[5] 176 Management of a modern IT company: theoretical and technological aspects Table5.1 uses the following notations[5]: –method No.1– Prosci ADKAR model; –method No.2– AIM method; –method No.3– Beckhard and Harris's model; –method No.4– Bridges Transition model; –method No.5– Kotter's 8-Step Change model; –method No.6– Kübler-Ross model; –method No.7– method based on the Kurt Lewin model. The main drawback of the considered change management methods is their lack of opportunities for quantitative assessment of changes at the level of individual project tasks. Therefore, an analysis of modern research aimed at eliminating this drawback was conducted. A significant amount of modern work in the field of change management in projects and, in particular, in IT projects is based on the idea of change management to the tasks of general (organizational) project management as a whole. For example, in[15] a new hybrid model of the IT project life cycle is proposed, which combines predictive planning with iterative implementation. This model emphasizes defined requirements, short sprints and early feedback, focusing on the interaction of people and customers, while limiting changes during the project life cycle[15]. But this model does not take into account the features of long-term IT projects. In addition, this model, like the change management methods discussed above, is focused on solving the problems of project personnel management, and not on quantitative assessment and change management. In[16] mathematical models and methods of change management in megaprojects caused by integrative actions of stakeholders under complex external conditions are investigated. The term "megaprojects" in[16] defines large-scale investment programs with complex organizational structures that unite many stakeholders, the interaction of which leads to the redistribution of power and the creation of temporary control centers. To describe the management of megaprojects, vector-matrix models of a dynamic system with feedback on the results of changes were used. To identify recurring patterns of negative events, the method of event-based analysis was used. As a result of the study, in[16] a prototype of IS was proposed, which is based on: –a mathematical model of change management in megaprojects; –a methodology of neural network analysis based on the use of the large language model Qwen 2.5-Plus for processing text information and forming quantitative estimates; –a software interface for uploading documents, automated data processing and visualization of results. 177 The task of quantitative assessment of changes in the long-term IT project management system Chapter 5 This prototype provides users with the ability to analyze stakeholder interactions, assess the intensity of change, and predict potential risks based on historical data[16]. The disadvantage of this IS prototype is its focus on processing textual information, through which stakeholders can express their own attitude to the megaproject. This means that quantitative assessments of changes are formed in this prototype on the basis of individual statements, which can be influenced by the economic or political environment of stakeholders. In addition, this prototype is also focused on managing changes and risks of the entire megaproject as a whole. Solving the problem of project evaluation by using neural networks is one of the current research directions. Thus, in[17] it was proposed to use a neural network of radial basis functions to predict project efficiency, which illustrates changes in efficiency levels during the phases of failures and project recovery. To train this network in[17], data from 64 completed construction projects were used. The results show that the discrepancy between the predicted and actual values of the stability of the assessed project is less than 10%. But this approach has quite significant disadvantages. Among these shortcomings, it is necessary to highlight, in particular: the orientation of the proposed IT to the evaluation of the project as a whole; the need to train the neural network on a large volume of historical data; the impossibility of quantitatively assessing changes in individual project tasks. A separate issue is the analysis of resistance to changes by project employees. Existing methods for assessing resistance are based on closed questionnaires and binary classifications. However, such methods, as indicated in[18], limit the expression of opinions and do not provide a nuanced segmentation of employees' positions on changes. Therefore, in[18] it is proposed to use an innovative auto mated methodology for analyzing resistance to changes by project employees, which combines specialized Large Language Models (LLM) with a zero result (in particular, DeBERTa-v3-large-zeroshot) and rapid engineering methods. However, the disadvantage of this methodology is that it, like existing methods, is based on employees' responses to pre-prepared questionnaires. Therefore, this issue requires additional scientific research that goes beyond the scope of this study. In general, the limitations and shortcomings inherent in modern research in the field of change management in projects and, in particular, in IT projects, can be formulated as follows: –the absence in the overwhelming majority of widespread applied change management methods of a tool for quantitatively assessing these changes; –the orientation of the overwhelming majority of methods and IT change management precisely on managing project personnel and stakeholders, as well as their relationships with each other within the project life cycle; 178 Management of a modern IT company: theoretical and technological aspects –the use of neural networks for various purposes in the proposed modern IS and IT change management and assessment studies, which significantly complicates the design and implementation of these IS and IT; –the use of large language models in IS and IT change management and assessment, which leads to the formation of change assessments based not on specific actions of project stakeholders, but on their statements on this issue. Therefore, the main direction of work on automation of change management is the expansion of existing project management systems and, in particular, IT projects by developing separate analytical services. These services should be aimed at solving the problems of assessing changes in those phases, processes and activities of an IT project that are relevant for stakeholders in specific periods of time. A feature of these services should be their readiness for operation in the conditions of the socalled "cold start" (i.e., in the absence of an array of historical data or a small amount of such data for a specific project). An example of research in this direction is the work[19], devoted to the analysis of the impact of changes together with differences in the code during the verification of the software code of an IT project. To carry out this analysis, in[19] it was proposed to combine methods of dependency analysis based on call graphs and methods of intelligent history analysis. Using this combination of methods made it possible to calculate a set of file metrics and an overall risk score for each change request. The obtained estimates were not very accurate, but their accuracy generally satisfied stakeholders and IT project personnel. In addition, the time of analytical calculations during the experimental verification of the obtained combination of methods ranged from 7.4 to 22.43 seconds[19]. This makes it possible to apply the corresponding IT to change management in the management systems of any IT projects based on almost any Agile or hybrid methodologies and frameworks. Therefore, the purpose of this study is to develop a service for automated solution of the problem of quantitative assessment of changes for the long-term IT project management system. The operation of this service will reduce the costs of implementing long-term IT projects by reducing unplanned time costs for performing individual tasks of these projects. To achieve this goal, the following tasks were solved in the study: –to improve the method based on the Beckhard and Harris's model for quantitative assessment of changes in a long-term IT project; –to develop IT elements of quantitative assessment of changes based on the improved method; –to carry out experimental verification of the obtained research results. 185 The task of quantitative assessment of changes in the long-term IT project management system Chapter 5 After that, the task of finding the values of the regression parameters is solved. There are many methods for building regression models. However, the most commonly used method is the Ordinary Least Squares (OLS). Before using the OLS method, it is necessary to check the fulfillment of the prerequisites for using regression analysis. As a result of such analysis, outliers may be detected that will need to be processed. In the case of outliers in the data collected during the implementation of the project tasks, the developed combined method proposes to use Huber regression, because it has properties related to robustness[24]. After building statistical models, it is proposed to check their performance using the following indicators[24]: –coefficient of determination (R2); –forecasting coefficient (Q2); –forecasting coefficient calculated using the Leave-one-out cross-validation procedure ( QLOOCV 2 ); –standard deviation ( σ ); –assessment of the significance of the statistical model (F-criterion). If, during the verification of statistical models, these indicators acquire unsatisfactory values, another descriptor is selected and the statistical models are recalculated. If the values of these indicators are satisfactory, the changes are assessed using the constructed model. When using the obtained assessment, an assumption is made about the possibility of implementing changes at the task level in the current iteration of the IT project. If the assessment of the required task completion time corresponds to this possibility, the changes are approved. If the assessment does not correspond to the possibility of implementing changes, the results of Stage3 are analyzed, after which, if necessary, the possibilities of either attracting additional human resources from other teams or overtime work of the current team members are analyzed. If it is determined that the changes cannot be implemented in the current project iteration, a new task is created. This task is included in the list of tasks when planning subsequent project iterations. The stage "Action Planning" (Stage6) is necessary for carrying out operations to re-plan the IT project taking into account the introduced and evaluated changes and implementing this plan. It is proposed to consider this stage as a set of the following activities: –identification of key participants in the change process and responsibilities of each person making the changes; 186 Management of a modern IT company: theoretical and technological aspects –development of a specific change implementation plan, including resources and deadlines; –determination of the sequence of steps for implementing the changes; –preparation of a communications plan to inform customers about the changes and their impact on the project. The stage "Transition Management" (Stage7) is necessary for operational management of work on the direct implementation of the IT project iteration. It is proposed to consider this stage as a set of the following activities: –implementation of the change plan for the task; –monitoring the impact of changes on the project and timely identification of problems; –providing support for users during the transition to a new state of the project. The stage "Evaluation of the success of the change management process"(Stage8) is necessary to analyze the progress of the IT project iterations and assess the success of the planned changes to the IT project. This stage is recommended to be carried out once every several (two or more) iterations of the project or at the request of the project stakeholders. In order to assess the success of the change management process, it was proposed to use the following indicators at Stage 8[5]: –the share (in percent) ChCp of time changes canceled due to the impossibility of their implementation, excluding changes that became unnecessary for a certain period; –the share (in percent) of time changes that were accepted by the customer and approved as successfully implemented for a certain period of time, ChCp ; –the difference between the time spent on work and the estimated time for a certain period of time, δ ChT ; –the share (in percent) of changes that were completed on time for a certain period of time, ChiTp . The share (in percent) of changes canceled over a certain period of time due to the impossibility of their implementation ChCp , excluding changes that have become irrelevant, can be calculated by the formula[5] Ch Ch Ch Cp C  100% , (5.2) where ChC – the number of canceled changes for a certain period of time due to the impossibility of their implementation, excluding changes that have become irrelevant; Ch– the total number of changes for a certain period of time. 187 The task of quantitative assessment of changes in the long-term IT project management system Chapter 5 Large values ChCp indicate poorly planned changes. The share (in percent) of changes that were accepted by the customer and approved as successfully implemented for a certain period of time ChSp can be calculated by the formula[5] Ch Ch Ch Sp S  100% , (5.3) where ChS – the number of changes that were accepted by the customer and approved as successfully implemented for a certain period of time. A large value ChSp indicates a better change management process. The difference between the time spent on performing tasks of a long-term ITproject and the estimated time for a certain period of time δ ChT can be calculated by the formula[5]  Ch tt ti T i p i a i     () () () 0, (5.4) where ti p() – the time planned for implementing the change, days; ti a() – the time spent on implementing the change, days; t– the time period for evaluation, days; i– the number of changes for the time period t. The indicator δ ChT indicates whether changes are performed on time and in accordance with the change plan. The lower the indicator, the better organized the change management. The share (in percent) of changes that were completed on time for a certain period of time ChiTp can be calculated by the formula[5] Ch Ch Ch iTp iT  100% , (5.5) where ChiT – the number of changes that were completed on time for a certain period of time. A high value ChiTp indicates a better change management process and adherence to the planned schedule. The use of the proposed improved method for solving the problem of quantitative assessment of changes made it possible to formulate the main requirements for the service that should provide an automated solution to this problem. To ensure the possibility of multiple use of the obtained solutions, this service during design and implementation (design&development) is proposed to be considered as a separate IT for automated solution of the problem of quantitative assessment of changes. 188 Management of a modern IT company: theoretical and technological aspects 5.5 Elements of information technology for automated solution of the problem of quantitative assessment of changes IT for automated solution of the problem of quantitative assessment of changes (hereinafter referred to as IT for quantitative assessment of changes) is proposed to be developed for automated implementation of the proposed improved method. Therefore, it was decided to use this IT to automate only those stages of the improved method that are directly related to the collection, processing and storage of data and information on changes that arise during the implementation of a longterm IT project. These stages include[5]: –Stage1 "Internal project analysis"; –Stage4 "Analysis of differences between the current state and the desired one"; –Stage7 "Transition management". Based on this decision, it was proposed to present the developed IT as a sequence of the following stages and steps[5]. Stage1.Survey of the team of performers on the current perception of the change management process. Step1.1. Forming a questionnaire and conducting a survey of all employees who form the IT project teams regarding their current attitude to the change management process. Step1.2. Processing the survey results and forming current assessments of the level of employee satisfaction with the change management process. Stage2. Forming and storing descriptors of individual IT project work. Step2.1. Determining the set of descriptors of individual IT project work. Step2.2. Forming sets of values of the defined descriptors. Step2.3. Storing the formed sets of values of the defined descriptors. Stage3. Statistical analysis of IT project descriptors. Step3.1. Determining indicators of IT project changes. Step3.2. Selecting the defined descriptors to create a change model. Step3.3. Analysis of the sets of values of the selected descriptors for the presence of outliers. Step3.4. If the analysis results obtained as a result of Step 3.3 indicate the absence of outliers, then build a change model using OLS. Otherwise, build a change model using the Huber regression method. Step3.5. Calculate the performance indicators of the built change model. Stage4. Survey of the team of performers on the final perception of the change management process. 189 The task of quantitative assessment of changes in the long-term IT project management system Chapter 5 Step4.1. Form a questionnaire and conduct a survey of all employees who form the teams of performers of the IT project on the final attitude to the change management process. Step4.2. Process the survey results and form final assessments of the level of employee satisfaction with the change management process. A description of the IT architecture of quantitative change assessment in the form of a data flow diagram is given in Fig.5.2[5]. When creating this diagram, the Yordon-DeMarco notation was used. Fig.5.2 does not indicate the names of the flows that are directly related to the data warehouses, because these names coincide with the names of the data warehouses themselves. For the further implementation of IT, the features of its technology stack were determined. The technology stack will be understood here and now as a set of technologies that are used together to develop and support software[25]. Manager and the IT project management team Formation of the survey Results of the survey Results of the survey Definition of descriptors and their values Sets of values of IT project work descriptors Results of the survey Results of the survey Responses to the survey questions Results of the current survey Results of the final survey Responses to the survey questions Definition of descriptors Definition of indicators of change and their relationship with descriptors Modeling results Modeling results Formation of the survey Team of performers of the IT project Survey of the team of performers on the current perception of the change management process Formation and storage of descriptors of individual IT project works Statistical analysis of project descriptors Survey of the team of performers on the final perception of the change management process Fig. 5.2 Description of the architecture of the information technology of quantitative change assessment Source: [5] 190 Management of a modern IT company: theoretical and technological aspects The technological stack of IT quantitative change assessment has the following components: –for the development of the elements "Survey of the team of performers on the current perception of the change management process" and "Survey of the team of performers on the final perception of the change management process", it is proposed to use the Google Forms service, which is included in the free Google Docs editor package from Google; –for the development of the element "Formation and storage of descriptors of individual IT project works", it is proposed to use existing IT project management systems (for example, Jira[26]) and tools for storing descriptor descriptions and their numerical values (for example, Microsoft Excel, or its analogues, or database management systems Firebase, MongoDB, PostgressSQL, etc.); –for the development of the element "Statistical analysis of IT project descriptors", it was proposed to develop a specialized service using the Python programming language. Microsoft Excel was used to implement data warehouses where it was planned to store descriptor descriptions and their numerical values. A fragment of the program code for calculating the values of the Gaussian density function is shown in Fig.5.3. A fragment of the program code for plotting the calculated Gaussian density function and checking the results obtained using the three-sigma method is shown in Fig.5.4[5]. A fragment of the program code for creating a model using the OLS method is shown in Fig.5.5. A fragment of the program code for creating a model using the Huber regression method is shown in Fig.5.6[5]. A fragment of the program code for calculating the performance indicator of the model "Coefficient of determination" is shown in Fig.5.7. A fragment of the program code for calculating the performance indicator of the model "Forecasting coefficient" is shown in Fig.5.8. A fragment of the program code for calculating the performance indicator of the model "Forecasting coefficient QLOOCV 2 " is shown in Fig.5.9[5]. A fragment of the program code for calculating the performance indicator of the model "Standard deviation ( σ )" is shown in Fig.5.10. A fragment of the program code for calculating the performance indicator of the model "F-criterion" is shown in Fig.5.11[5]. Fig. 5.3 A fragment of the program code for calculating the values of the Gaussian density function Source: [5] 191 The task of quantitative assessment of changes in the long-term IT project management system Chapter 5 Fig. 5.4 A fragment of the program code for plotting the calculated Gaussian density function and checking the results obtained using the three-sigma method Source: [5] 192 Management of a modern IT company: theoretical and technological aspects Fig. 5.5 A fragment of the program code for creating a model using the least squares method Source: [5] Fig. 5.6 A fragment of the program code for creating a model using the Huber regression method Source: [5] Fig. 5.7 A fragment of the program code for calculating the performance indicator of the model "Determination coefficient R2" Source: [5] Fig. 5.8 A fragment of the program code for calculating the performance indicator of the model "Forecasting coefficient Q2" Source: [5] 193 The task of quantitative assessment of changes in the long-term IT project management system Chapter 5 Fig. 5.9 A fragment of the program code for calculating the performance indicator of the model "Forecasting coefficient QLOOCV 2" Source: [5] Fig. 5.10 A fragment of the program code for calculating the performance indicator of the model "Standard deviation ( σ )" Source: [5] Fig. 5.11 A fragment of the program code for calculating the performance indicator of the model "F-criterion" Source: [5] 194 Management of a modern IT company: theoretical and technological aspects 5.6 Description of an example of solving the problem of quantitative assessment of changes during the implementation of a long-term ITproject 5.6.1 Description of the features of the long-term IT project "Web Constructor" Experimental verification of the obtained research results was proposed to be carried out during the implementation of the long-term "Web Constructor" IT project. The duration of the implementation of this IT project is 6years. The main result of this IT project is the "Construct" system. This system implements an administrative portal, where customers can use the library of components to create sites. The "Construct" system supports more than 20 languages and allows to create web pages for most countries of the world. The results obtained from the implementation of the long-term "Web Constructor" IT project are already in operation and continue to expand. The main goal of the long-term "Web Constructor" IT project is the development and support of the created "Construct" system. According to the classification of the main groups of projects by interests of the Project Management Institute (PMI), this project belongs to IS development projects[2,3]. Table5.2 provides an additional classification of the "Web Constructor" project. This classification was carried out according to classification criteria according to the materials of sources[27,28]. Table 5.2 Additional classification of the "Web Constructor" project Classification feature Project type By scale Large By complexity Technically complex By implementation time Megaproject By resource constraints of a set of projects Program By the nature of the project and the level of participants International By the nature of the project's target task Marketing By the main reason for the project Need for structural and functional transformations By the location of the customer External Degree of customer participation in the project Average 201 The task of quantitative assessment of changes in the long-term IT project management system Chapter 5 Table 5.3 Assessment of employees' attitude to the method based on the Beckhard and Harris's model Questions for evaluation on a scale from 0 to 10 Employee Average score per question 12345678 Speed of change implementation (10– fast) 65467453 5 Comfort of change management processes (10– comfortable) 54567687 6 Presence of stressful situations during change implementation (10– absent) 78677686 6.875 Need for overtime work (10– absent) 8 9 8 10 9 10 9 9 9 Difficulty of change assessment (10– not difficult) 67567854 6 Average employee rating 6.4 6.6 5.6 7 7.4 6.8 7 5.8 6.575 According to the survey results, the average score of the current change management method was 6.575. Then, with the participation of all project team members, an analysis of the use of descriptors was conducted. A conditional name was chosen for each descriptor for use in the descriptor databases. The main descriptors at the time of discussion were the following: –number of task characteristics (C_Amount); –number of task characteristics to be implemented on the client side (C_FE_ Amount); –number of task characteristics to be implemented on the server side (C_BE_ Amount); –planned number of days to complete the task (W_Days_Planned); –number of days spent to complete the task (W_Days_General); –number of task characteristics at the time of completion of work on it (C_Amount_ Final); –difference between the number of task characteristics at the time of taking the task into work and the number of characteristics that occurs at the time of completion of work on the task (C_Delta); –the difference between the number of days planned for the task at the time of its initiation and the number of days spent on its implementation at the time of its completion (W_Days_Delta). During Stage2, information was collected in the form of descriptors by entering them into the descriptor database. 202 Management of a modern IT company: theoretical and technological aspects The tasks that were performed during the last five sprints were analyzed. The description of the descriptors for these tasks is given in Table5.4. Table 5.4 Descriptors for tasks performed during the last five sprints Task ID C_Amount, pieces C_Amount_ Final, pieces C_FE_ Amount, pieces C_BE_ Amount, pieces W_DaysPlanned, days W_DaysGeneral, days WEBCO_2342 80 92 30 50 15 20 WEBCO_2343 80 94 65 15 23 31 WEBCO_2344 20 25 10 10 8 10 WEBCO_2345 35 38 15 20 7 8 WEBCO_2346 42 47 12 30 12 15 WEBCO_2347 76 85 26 50 9 14 WEBCO_2348 12 12 12 0 5 5 WEBCO_2349 56 61 26 30 16 18 WEBCO_2350 15 20 10 5 5 7 WEBCO_2351 71 75 25 46 14 16 WEBCO_2352 12 12 0 12 3 3 WEBCO_2353 7 10 7 0 3 4 WEBCO_2354 2 2 2 0 1 1 WEBCO_2355 1 8 1 0 2 6 WEBCO_2356 35 40 32 3 18 20 WEBCO_2357 8 10 1 7 4 5 WEBCO_2358 54 65 53 1 50 55 WEBCO_2359 34 53 25 9 74 84 WEBCO_2375 50 43 30 20 15 12 WEBCO_2376 40 33 25 15 12 9 WEBCO_2377 35 28 20 15 10 8 WEBCO_2380 20 15 12 8 7 5 WEBCO_2384 70 62 40 30 20 16 WEBCO_2385 50 42 28 22 15 12 WEBCO_2386 45 32 25 20 13 8 WEBCO_2387 40 30 22 18 12 8 WEBCO_2388 35 32 20 15 11 10 WEBCO_2342 80 92 30 50 15 20 During the next (sixth) sprint, descriptors were also entered into the descriptor database. The values of these descriptors are shown in Table5.5. 203 The task of quantitative assessment of changes in the long-term IT project management system Chapter 5 Table 5.5 Descriptors for tasks performed during the sixth sprint Task ID C_Amount, pieces C_Amount_ Final, pieces C_FE_ Amount, pieces C_BE_ Amount, pieces W_DaysPlanned, days W_DaysGeneral, days WEBCO_2389 30 26 18 12 10 9 WEBCO_2395 20 20 12 8 8 8 WEBCO_2396 25 21 15 10 9 7 WEBCO_2397 30 35 18 12 10 13 WEBCO_2398 22 24 14 8 8 9 WEBCO_2399 35 35 20 15 11 11 WEBCO_2360 65 75 40 25 20 25 WEBCO_2361 25 59 15 10 8 23 During Stage3 of the sixth sprint, when performing task WEBCO_2360, it became necessary to make changes at the task level. The task included updating the designs of the bottom menu component of the page (footer). During the task, it was discovered that changes to the graphical interface designs were necessary. The designer updated the graphical interface, and the business analyst updated the component characteristics in accordance with the design updates. During Stage4, the difference between the task states without changes and with changes was studied in detail. The team was explained the changes necessary for implementation. The development team requested Stage5 of the improved method. To build statistical models of changes, a descriptor was selected that determines the difference between the number of task characteristics at the time the task was started and the number of characteristics that occur at the time the task was completed (C_Delta). The change indicator was chosen as the difference between the number of days planned for the task at the time of its start and the number of days spent on its implementation at the time of completion of work on it (W_Days_Delta). First, the prerequisite for using regression analysis was checked. It was carried out for data collected on the basis of the first five sprints. Using the developed software, a Gaussian distribution was constructed, presented in Fig.5.12[5]. The mean, variance, and standard deviation of the descriptor values collected for the first five sprints were calculated. The mean was 1.52. The variance was 62.72. The standard deviation was 7.92. There were no outliers outside 3 σ . The check showed that there were no outliers in the data, so it was decided to build a model using the OLS method. The model calculations were performed for the descriptors collected from the first five sprints. 204 Management of a modern IT company: theoretical and technological aspects Gaussian distribution C_Delta Probability density function –15 –5 0510 15 20 –10 0.06 0.05 0.04 0.03 0.02 0.01 0 0 Fig. 5.12 Gaussian distribution for the data on the number of changed characteristics collected for the first five sprints Source: [5] The model obtained as a result of the OLS calculations is represented by the formula y x ii 046026.. , (5.6) where yi– the value of the change indicator W_Days_Delta (system property); xi – the value of the descriptor C_Delta. The change model, built using the OLS method for the data of the first five sprints, is shown in the graph (Fig.5.13)[5]. C_Delta W_Days_Delta Observed values OLS –15 –10 –5 0 5 10 15 20 10 8 6 4 2 0 –2 –2 –4 –6 Fig. 5.13 Change model built using the OLS method for data from the first five sprints Source: [5] 205 The task of quantitative assessment of changes in the long-term IT project management system Chapter 5 After building the model, its performance was checked using the following indicators: R2, Q2, QLOOCV 2 , F-criterion and standard deviation σ . The calculated performance indicators for the change model, built on the basis of descriptors for the tasks of the first five sprints, are given in Table5.6. Table 5.6 Performance indicators for the change model Method Regression equation R2Q2Q2 LOOCV F σ OLS yi=0.46⋅xi+0.26 0.98 0.98 0.97 481.61 0.59 The values of the calculated indicators showed that the model is workable and can be used to estimate the change indicator. During the sixth sprint, issues were resolved regarding updating the list of characteristics of the WEBCO_2360 task after it was put into operation. The initial number of characteristics was 65units. The team was tasked with choosing one of the proposed options: –add 20 new characteristics to the current task; –add 10 priority new characteristics to the current task and create a task with 10 other characteristics to be performed during the next sprint. The calculation of the change implementation time estimates under the conditions of adding 10 and 20 new characteristics is given in Table5.7. Table 5.7 Calculation of the change implementation time required for the WEBCO_2360 task due to changes in the number of characteristics Method Regression equation Time of change (20characteristics) Time of change (10characteristics) OLS yi=0.46⋅xi+0.26 9.46 4.86 Using the resulting change estimate, it was decided to add 10 new features during the current sprint. The project manager assigned people responsible for making changes and checking them. The changes took 5 days, which was slightly longer than planned. This fact showed that using the improved method, an estimate was obtained that was close enough to the actual value. The calculation of the ratio of the predicted value to the actual value is given in Table5.8. During the seventh sprint, issues were addressed regarding the reduction of the list of characteristics of the WEBCO_2361 task after it was taken into operation. The initial number of characteristics was 55 units. After taking the task into operation, 206 Management of a modern IT company: theoretical and technological aspects it was found that 7 characteristics were no longer relevant. The development team requested Stage 5 of the improved method. Table 5.8 Calculation of the ratio of the predicted value of time to the actual value of time Method Forecast change, days Actual change, days Ratio of forecasted and actual results OLS 4.86 5 0.97 To build statistical models of changes, it was decided to use the descriptor C_Delta and the change indicator W_Days_Delta, as for previous calculations. The data collected on the basis of the first six sprints were checked for outliers. The Gaussian distribution for the values of these descriptors, which were collected during the previous six sprints, is shown in Fig.5.14[5]. The calculated mean was 2.4. The variance was 82.6. The standard deviation was 9.09. The three-sigma test showed the presence of an outlier in the data with a value of 34. Since the test showed the presence of outliers, models were built using Huber regression. The models were calculated on the descriptor values of the first six sprints using the following values of the parameter δ : 1.345, 0.8, 0.1, 0.02, and 1.5. A fitness check was performed for the found change models. The results of the calculations are given in Table5.9. Fig. 5.14 Gaussian distribution for descriptor values collected during the previous six sprints Source: [5] Gaussian distribution C_Delta Probability density function –15 –10 –5 0510 15 20 25 30 35 0.045 0.040 0.035 0.030 0.025 0.020 0.015 0.015 0.010 0.005 0 207 The task of quantitative assessment of changes in the long-term IT project management system Chapter 5 Table 5.9 Models calculated using Huber regression and their fitness indicators Parameter δ Regression equation R2Q2Q2 LOOCV F σ 1.345 yi=0.25⋅xi+0.01 0.76 0.76 0.78 51 2.09 0.8 yi=0.24⋅xi+0.01 0.74 0.74 0.76 46 2.16 0.1 yi=0.37⋅xi+0.01 0.94 0.94 0.93 259 1.02 0.02 yi=0.41⋅xi+0.02 0.97 0.97 0.97 517 0.74 1.5 yi=0.25⋅xi+0.01 0.76 0.76 0.78 51 2.07 According to the calculated performance indicators, the value δ =0.02 was chosen for further calculations. The model built using Huber regression for the data of the first six sprints is shown in the graph in Fig.5.15[5]. Observed value Huber regression C_Delta W_Days_Delta –15 –10 –5 0510 15 20 25 30 35 15 10 5 0 –5 –10 Fig. 5.15 Model built using Huber regression, for data from the first six sprints Source: [5] The calculated performance indicators of the model showed that the model is workable and can be used to estimate the indicator of changes in task execution time. The calculation of the estimate of released days under the condition of changing the number of characteristics of the task WEBCO_2361 is given in Table5.10. For task WEBCO_2361, it was decided to remove 7 characteristics from the task description in the current sprint. 4 days less were spent on solving the changed task. Calculations of the ratio of the predicted value to the actual value are given in Table5.11. 208 Management of a modern IT company: theoretical and technological aspects Table 5.10 Calculation of the estimate of released days under the condition of changing the number of characteristics of the task WEBCO_2361 Method Regression equation Change indicator Huber regression yi=0.41⋅xi+0.02 2.85 Table 5.11 Calculation of the ratio of the predicted value to the actual value for task WEBCO_2361 Method Forecasted value, days Actual value, days Ratio of forecasted and actual results Huber regression 2.85 3.5 0.81 Thus, the calculated change estimates turned out to be close to real changes. Therefore, it was decided that the results of the experimental verification of the improved method and the elements of the corresponding IT are adequate to the real processes of implementing the long-term "Web Constructor" IT project. 5.7 Discussion of the research results Improving the method based on the Beckhard and Harris's model using the descriptor approach made it possible to solve the following tasks: –conduct a quantitative assessment of changes based on the values of descriptors that are formed in the process of implementing sprints of a long-term IT project by its performers; –automate the solution of the problem of quantitative assessment of changes by forming and analyzing statistical models of the dependence of change indicators on descriptors. The developed IT elements of quantitative assessment of changes allow to improve the quality of change management in a long-term IT project. This possibility was achieved through the use of the descriptor approach and statistical models of quantitative assessment of changes. The use of software implementation developed by IT greatly simplifies the work on assessing changes in the time parameters of individual tasks of a long-term IT project[5]. Unlike the derived method based on the Beckhard and Harris's model[10], the use of the improved method allowed to improve the performance indicators of the change assessment process. The results of comparing the values of these indicators 209 The task of quantitative assessment of changes in the long-term IT project management system Chapter 5 calculated for the derived method based on the Beckhard and Harris's model and after the implementation of the improved method are given in Table5.12. Table 5.12 Results of comparing the values of indicators of the change assessment process Indicator Application of the derivative method based on the Beckhard and Harris's model Application of the developed information technology for quantitative assessment of changes Change in the indicator Proportion (in percent) of time changes canceled due to impossibility of their implementation ChCp 7 5 Decreased by 2 Proportion (in percent) of time changes that were accepted by the customer and approved as successfully implemented ChSp 92 95 Increased by 3 Difference (in hours) between the time spent on work and the estimated time, δ ChT 7 2 Decreased by 5 Proportion (in percent) of changes that were completed on time ChiTp 79 86 Increased by 7 Employee attitude assessment towards change management processes (from 1 to 10) 6.575 8.025 Improved by 14.5% The developed improved method has the following limitations: –the duration of the project must be large enough to allow collecting the necessary amount of information for analysis; –there is a need for additional storage of information for analysis; –models built using this approach are relevant only for specific projects on the basis of which data was collected for building models (however, exceptions are possible). The main limitation of the use of the developed IT elements is the need to operate IS and IT management of the work of project teams, which could provide the formation of derived data arrays for further calculation of the values of the selected descriptors of the tasks of a long-term IT project. In addition, a significant drawback of the developed IT is a significant increase in the duration of model calculations with an increase in the number of descriptors taken for analysis. To overcome this drawback, it is necessary to conduct additional research to reduce the time and computational complexity of the algorithms for implementing the developed IT[5]. 210 Management of a modern IT company: theoretical and technological aspects Research on the development of an improved method for quantitative assessment of changes is proposed to focus on determining the possibility of its application for assessing and predicting other (except time) parameters of changes that arise during the implementation of a long-term IT project. To conduct such research, itwill also be necessary to conduct additional research to determine the best indicators that characterize changes and the success of the change assessment process in IT projects of various types. 5.8 Conclusions In the process of research, a method based on the Beckhard and Harris's model was improved for quantitative assessment of changes in a long-term IT project. To improve the method, it was proposed to use a descriptor approach. The improved method, unlike the existing one, allows for quantitative assessment of changes based on the values of descriptors that are formed during the implementation of sprints of a long-term IT project by its performers. The results of improving the change assessment method were implemented in the form of IT elements of an automated solution to the problem of quantitative assessment of changes in the long-term IT project management system. A service architecture was selected for implementation, and the developed IT was considered as a means of implementing the corresponding service. The IT architecture was defined, a technological stack was proposed, and elements of the software implementation of the service were developed. To experimentally verify the obtained research results, it was decided to test these results during the implementation of the long-term "Web Constructor" ITproject. The test was performed on the results of the work of one project development team. The calculation used data obtained during the team's execution of seven project sprints. The calculations were performed for two cases of changes that occurred during the sixth and seventh sprints. In the first case, the ratio of the predicted time estimate to the actual time spent on implementing the change was 0.97, in the second case– 0.81. These results showed that the predicted values of the indicators quite accurately coincide with the values of the time actually spent on implementing the changes. In general, the use of the improved method improved the indicators of change implementation during the implementation of the IT project (Table5.12) and improved the team's attitude to the change management process by 14.5%.