Applying a Requirements-Focused Agile Management Approach for Machine Learning-Enabled Systems

Evaluation of requirements-focused Agile Management in R&D&I Projects Focused on Building Machine Learning-Enabled Systems Dear participant, Thank you for taking part of your valuable time to complete this questionnaire. Objective: Evaluate the application of requirements-focused agile management, supported by Lean R&D principles, in the context of MVP development within R&D&I projects aimed at building machine-learning-enabled systems (ML). This research follows a high academic standard and is conducted anonymously. We will not associate your email address with your responses. For more information/questions, contact: Lucas Romão: [email protected] Marcos Kalinowski: [email protected] Júlia Condé Araújo: [email protected] Marina Condé Araújo: [email protected] Pontifical Catholic University of Rio de Janeiro (PUC-Rio) Opening 1.1 – What is your academic background? 1.2 – What is the field of your undergraduate degree? 1.3 – What is your approximate experience in software projects (in years)? 1.5 – What is/was your role(s) during these projects? Main Block During the Initial Specification, the team specifies the Machine Learning components based on PerSpecML, promoting a shared understanding among business, software, and machine learning teams. At this stage, requirements are elicited from the perspectives of system goals, user experience, infrastructure, model, and data. 2.1 – How do you evaluate the adequacy of the Initial Specification to promote a shared vision among business, software, and machine learning teams? During the Conception phase, requirements are collaboratively refined. The Product Owner structures the Software and ML Backlogs with support from the software and ML teams, also defining the infrastructure and architectural Enablers that support the integration between software and machine learning components. 2.2 – How do you evaluate the adequacy of the Conception stage in building the product backlog from the Initial Specification requirements? 2.3 – How do you evaluate the separation between the Software backlog and the ML backlog regarding requirement organization and the integration between system and model responsibilities? 2.4 – How do you evaluate the role of infrastructure and architectural Enablers in integrating the software system and machine learning pipelines? During Technical Feasibility, the software and ML teams analyze the technical feasibility of ML components and their integration with the system, through Data Feasibility, Model Feasibility, and the creation of a Demo API (LoD 0). 2.5 – How do you evaluate the adequacy of the Technical Feasibility stage in anticipating and dealing with technical uncertainties in agile management of ML-enabled systems? 2.6 – How do you evaluate the effectiveness of the Data Feasibility stage in ensuring availability, quality, and representativeness of the data required for ML model development? 2.7 – How do you evaluate the contribution of the Model Feasibility stage in analyzing algorithm adequacy and complexity of the proposed models? 2.8 – How do you evaluate the usefulness of the Demo API (LoD 0) for anticipating the integration between the ML model and the software system? During the Development phase, system development is conducted iteratively, following Agile4MLS principles and maintaining synchronization between software and ML deliveries using Layers of Done (LoD) and Minimal Viable Model (MVM). 2.9 – How do you evaluate the impact of joint iterations and sprints between the software and machine learning teams on continuous value delivery? 2.10 – How do you evaluate the principle of the machine learning team working “two sprints ahead” of the software team regarding development flow and integration? 2.11 – How do you evaluate the use of Layer of Done in communicating the maturity and reliability level of machine learning models? 2.12 – How do you evaluate the adequacy of the Minimal Viable Model (MVM) definition to validate business hypotheses before full model optimization? Acceptance of the Approach in R&D&I Projects The following are general questions about the approach. Their purpose is to assess perceived usability, perceived ease of use, and intention of use. 2.13 – How do you evaluate the usefulness of the requirements-focused agile management approach in R&D&I projects focused on ML-enabled systems? 2.14 – How do you evaluate the ease of use of the requirements-focused agile management approach in R&D&I projects focused on ML-enabled systems? 2.15 – Would you recommend the use of the requirements-focused agile management approach to other teams or projects? Why? Closing 2.16 – What limitations do you see in the approach for managing requirements and the agile development process of projects with machine learning components? 2.17 – Are there any other statements or lessons learned you would like to comment on?