Edge-AI Assurance in the REBECCA Project

Edge-AI Assurance in the REBECCA Project Clara Ayora Universidad de Castilla-La Mancha Albacete, Spain [email protected] Arturo S. García Universidad de Castilla-La Mancha Albacete, Spain arturosimon.garc[email protected] Jose Luis de la Vara Universidad de Castilla-La Mancha Albacete, Spain joseluis[email protected] ABSTRACT In critical domains, assurance corresponds to the set of activities to provide confidence that a system can be deemed dependable, e.g., safe and secure. This essential systems and software engineering process are usually conducted according to standards. For novel applications running at the edge and containing artificial intelligence, and for their corresponding platforms, how to conduct assurance in a systematic way is still undefined. This paper introduces the work in the large-scale REBECCA EU project to contribute to filling this gap. A new assurance framework will be defined, addressing the management of compliance, assurance cases, and assurance evidence. The assurance framework will be based on the results of a systematic study to characterize Edge-AI assurance needs, and four systems will be used to validate the framework via case study research. The assurance framework will provide guidance about what needs to be considered for safety and security assurance of Edge-AI applications, and how. CCS CONCEPTS • Artificial Intelligence • Social and professional topics → Professional topics → Management of computing and information systems → System management → Quality assurance KEYWORDS Edge, Artificial Intelligence, Assurance, Compliance ACM Reference format: Clara Ayora, Arturo S. García and Jose Luis de la Vara. 2024. Edge-AI Assurance in the REBECCA Project. In Proceedings of The ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM’24). ACM, New York, NY, USA, 4 pages. 1 INTRODUCTION Among the current and future key ICT technologies [1], edge computing has emerged as an architectural paradigm that brings computation and data storage closer to data sources and that is expected to save time and bandwidth. Another key technology is artificial intelligence (AI), which is facilitating and will continue to facilitate the automation of processes that are largely reliant on human cognitive abilities and the execution of complex cognitive tasks that humans are unable to perform. The use of AI on the edge can be referred to as Edge AI [1]. Edge-AI applications are gaining attention nowadays and are starting to be developed and deployed for a wide range of contexts and application domains, including critical ones, such as automotive, aerospace, healthcare, and Industry 4.0. When Edge-AI applications, as well as others, perform critical functions whose failure can have a negative impact on, e.g., safety or security, they are subject to system assurance [2]. Assurance can be defined as the set of activities to provide adequate justified confidence that a system satisfies given requirements, e.g., for system safety and security, thus for system dependability [3]. For instance, system assurance is needed to have confidence that an autonomous system will not harm someone. This confidence is often developed by satisfying certain objectives that mitigate the potential risks that a system can pose during its lifecycle. This is usually performed in compliance with assurance and engineering standards, e.g., IEC 61508 [4]. However, for systems including AI at the edge, and although several initiatives aim to tackle Edge-AI complexity and to ensure its quality, a unified standard remains elusive [5]. For cost-effective assurance of novel critical Edge-AI applications, several issues need to be tackled [6]: multi-concern assurance, as it is not enough to address only, e.g., safety, but security must also be considered due to the connectivity of edge devices; specific Edge-AI characteristics regarding, e.g., where and how data is generated, stored, and used, where and how decisions are made, and the dedicated software and hardware developed, such as accelerators and hypervisors, and; assurance needs of novel applications, considering the distinguishing characteristics of the Edge-AI applications, e.g., a drone monitoring infrastructures versus a system that interacts with people. In this context, REBECCA (Reconfigurable Heterogeneous Highly Parallel Processing Platform for safe and secure AI; https://rebecca-chip.eu/) is a large-scale EU project that will provide a novel Edge-AI assurance framework. This is a part of the overall work in the project to enable efficient Edge-AI solutions that can overcome physical limitations, are dependable, and enhance European strategic autonomy. As a whole, REBECCA considers hardware and software development for Edge AI, their integration and validation, their safety and security, their use in different systems, and compliance. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). WOODSTOCK’18, June, 2018, El Paso, Texas USA © 2018 Copyright held by the owner/author(s). 978-1-4503-0000-0/18/06...$15.00 https://doi.org/10.1145/1234567890 ESEM’24, October, 2024, Barcelona, Spain C. Ayora et al. In this paper, we focus on the work in REBECCA that (1) aims to provide a new method for systems and software engineering, specifically, for system assurance, and (2) is being developed and will be validated in an empirical way. System assurance is transversal to the rest of system lifecycle processes. It must be considered from system inception to decommission, imposing needs and constraints, e.g., to comply with standards. For framework development and validation, we are following design science [7] and evidence-based software engineering [8], with engineering activities and those aimed at generating knowledge, and to rigorously search and provide scientific evidence. The assurance framework will be applied in four real-world use cases. The rest of the paper presents the REBECCA project, the development of the assurance framework, the plan for its validation, and our main conclusions. 2 PROJECT OVERVIEW REBECCA will develop efficient Edge-AI solutions that can overcome the physical limitations of edge devices, e.g., AI processing resources. This will involve designing and developing hardware, software, and middleware to accelerate computationintensive parts of AI and conventional applications, ensure deterministic response times, and satisfy quality constraints. The complete technology stack will be centered around a RISC-V CPU [9] and will aim to offer higher performance, energy efficiency, and confidence in safety and security than existing solutions. REBECCA will develop a novel platform comprising two tightly-coupled chiplets and reconfigurable devices containing the following components (Figure 1): (1) RISC-V multicore, (2) Neuromorphic AI Accelerator, (3) Programmable array AI Accelerator, (4) AI Accelerator with a hierarchical processing architecture, (5) Deep Neural Network Accelerator, (6) Reconfigurable hardware, and (7) Near-Memory-Processing. The project will demonstrate its approach on four real-world use cases: underwater robot, unmanned aerial vehicles, AI-powered fridges, and industrial equipment inspection. REBECCA started in February 2023 and will finish in July 2026 (i.e., 42 months). It is funded by the Chips Joint Undertaking of the EU [10] (within the program Horizon Europe) and is powered by a consortium of 24 partners from 9 European countries. Currently, REBECCA is in month 18 (July 2024) finalizing the component specification (e.g., requirements analysis, architecture design). The development of Edge-AI-specific hardware and software has also already started (e.g., hypervisor and library development for distributed deep learning), as well as the definition of the assurance framework. These results will be integrated and applied in the first version of the use cases that will be demonstrated in October 2024 (via emulation). During the project, safety and security will be addressed; mostly at design time and through several fronts, including hardware and software development (e.g., redundancy and formal verification) and the Edge-AI assurance framework. The framework will provide guidance about what needs to be addressed for safety and security assurance of Edge-AI applications, such as functional and architectural features (product aspects) and techniques to use in their development, verification and validation (process aspects). The determination of these aspects is part of the research work to conduct in the work package focused on development tools and on security and safety verification. The work on the Edge-AI assurance framework is relevant to the empirical software engineering community. As detailed in this paper, it is a new method for a systems and software engineering process that is being developed in a scientifically rigorous way. The framework will also be validated via case study research. Figure 1: REBECCA hardware and software stack 3 EDGE-AI ASSURANCE FRAMEWORK The assurance framework will address the arguably main and more specific assurance areas [2]: • Compliance management: Assurance and engineering standards define best practices and criteria to ensure system safety and security. This includes requirements to meet, activities to execute, techniques to use, and artefacts to manage during a system’s lifecycle [11]. Edge-AI projects need to analyze how compliance progresses and possible gaps. The relevant standards and parts to comply with need to be selected. • Assurance case development: Assurance cases provide structured justifications of why a system can be deemed dependable, explaining how system dependability has been addressed and showing how claims about it (e.g., about hazard elimination) are supported by evidence [12]. EdgeAI-specific aspects to justify and justification structures to do it (e.g., according to usage context) need to be provided. • Assurance evidence management: Assurance evidence corresponds to artefacts that contribute to developing confidence in the dependable operation of a system and to showing the fulfilment of the requirements of one or several assurance standards [13], e.g., system specifications and testing results. The types of evidence to collect need to be determined, and assurance evidence needs to be evaluated. Assurance evidence traceability and change impact analysis need to be considered as well. Although prior publications have worked on these areas (e.g., [14, 15, 16]), they do not consider Edge-AI applications, they only ESEM’24, October, 2024, Barcelona, Spain C. Ayora et al. cover some assurance area for Edge AI partially, or do not compile all this information in a systematic way. As the first step to develop REBECCA’s Edge-AI assurance framework, we are currently collecting the existing knowledge in the literature about Edge-AI assurance and characterizing its needs. This is being performed by means of a systematic mapping study – a documented and repeatable process through which the literature on a given subject is examined and the current state of knowledge is recorded [8]. In this mapping, we are investigating aspects that affect compliance management, assurance case development, and assurance evidence management for Edge AI: • Dependability attributes: Dependability is a concept that encompasses different attributes, such as safety, security, trust, integrity, availability, reliability, integrity, maintainability, and confidentiality. There exist relationships between them, so safety and security impact and are impacted by others. There also exist AI-specific attributes, such as transparency. Investigating this aspect will allow us to determine what attributes need to be considered for REBECCA’s outputs. • Standards used: Demonstrating compliance with standards involves collecting evidence that shows that the relevant criteria in the standards are met. Standards refer to one or more dependability attributes, e.g., safety. Compliance often proves to be a very challenging task, especially for new and innovative technologies such as Edge-AI. Thus, we investigate what standards (e.g., IEC 61508) might need to be considered for REBECCA’s results assurance. This will in turn allows us later to analyze the extent to which the results are compliant with them. • Dependability justification techniques. We aim to identify means that are used to explain why Edge AI can be deemed dependable, e.g., structured graphical argumentation and checklists. Identifying these techniques will help us to determine what and how an adequate, convincing, and valid justification of Edge-AI dependability can be, which can in turn help in its assessment. • Assurance evidence. Some examples of assurance evidence are requirements specifications, architecture specifications, testing results, and lifecycle plans. There can be specific evidence artefact types or with specific characteristics for Edge AI, e.g., about learning aspects. Determining what relevant assurance evidence is for Edge AI will aid in substantiating compliance and confidence in dependability for a given application in given context. • Artificial Intelligence techniques. The goal is to identify what aspects specific to AI techniques need to be considered for Edge-AI assurance. For example, assurance needs might vary according to the use of supervised vs. unsupervised learning and local vs. distributed learning. • Edge-specific aspects. These refer to the edge computing technologies and components used for Edge AI, e.g., accelerators and hypervisors. They might have their own assurance needs, from usual ones such as hardware vs. software assurance to more specific ones such as virtualization assurance. The aggregation of the knowledge about these aspects will result in the Edge-AI assurance framework: a specification of needs that might have to be considered to provide adequate justified confidence that Edge AI is dependable. The framework is intended to be generic so that it can be used in different scenarios. However, since certain dependability attributes (e.g., safety) depend on the Edge-AI application and its usage context (e.g., drone versus plane), the framework will need to be adapted to the specific characteristics of, e.g., the REBECCA’s results for the four project use cases. They are concrete examples of an Edge-AI system and of Edge-AI applications. The outcome from applying the assurance framework will contribute to provide evidence of the degree of confidence in the safety and the security of the REBECCA’s results in a specific context. This includes compliance assessments against applicable standards, e.g., IEC 61508. 4 VALIDATION THROUGH USE CASES In line with prior related empirical validations (e.g., [3, 13]), we will validate the assurance framework via case study research [17] in the context of REBECCA. This type of research aims to investigate contemporary phenomena within their real-life context, especially when the boundary between the phenomena and the context cannot be clearly specified. Case study research is typically exploratory and flexible, and uses qualitative data as primary source. Several case studies will be conducted. The goal of the case studies will be to evaluate the effectiveness of the assurance framework to provide adequate justified confidence that Edge AI is dependable. Two research questions will be investigated: • RQ1. Is the assurance framework a feasible means for compliance management, assurance case development, and assurance evidence management for Edge-AI safety and security? • RQ2. How does the assurance framework contribute to providing information about the degree of confidence in Edge-AI safety and the security? The design to answer these RQs consists in: (1) selection of relevant and representative case studies of Edge-AI applications and of concrete usage scenarios; (2) use of the assurance framework in the case studies and usage scenarios, and (3) study of the use of the framework in the case studies and usage scenarios. The case studies and the usage scenarios are extracted from the real-world use cases of the REBECCA project. For each use case, safety and security requirements have been identified. The use cases will use the REBECCA platform. The first use case is an underwater robot that inspects underwater infrastructures, such as quay walls, sheet pile walls, and pillars. There can be damage on those objects, e.g., cracks in walls, and an AI recognition algorithm will help in detecting it. The REBECCA’s results will gather video and other sensor data in a cost-effective way and with precise positioning directly in the robot. Protection of the content and integrity of the inspection images, as well as of the resulting maps, need to be assured. The second use case corresponds to unmanned aerial vehicles for real-time defect detection in photovoltaic power panels. The usual approach is based on the collection of images or videos on-site and a post-processing on remote servers. Within REBECCA, a novel multi-stage architecture will be implemented for anomaly detection and processing on-site. Data needs to be ESEM’24, October, 2024, Barcelona, Spain C. Ayora et al. securely transmitted to avoid a wrong defect detection or a wrong defect location. Compliance with IEC 61508 [4] is required. The third use case is related to AI-powered fridges that distinguish food with image recognition. Current IoT architectures depend on centralized servers/clouds to process all the data collected. This adds security and privacy risks, latency in time-critical actions, and additional costs. By using REBECCA’s results, decisions can be made faster without relying on network connectivity, also reducing data storage in the cloud and power consumption. Compliance with ETSI EN 303 645 [18] is required. Finally, the fourth use case deals with industrial equipment inspection. Current industrial vision applications suffer from limited scalability. As more cameras are used, the processing capabilities of a centralized CPU solution is impacted. The use of REBECCA’s results will provide higher processing speeds for more intelligent and numerous visual inspections. There are safety-critical axes that may cause physical harm, with redundant communication channels in case they need to be shut down. In addition to the use cases, the Edge-AI assurance framework will be used for the REBECCA platform in isolation, out of the context of any specific application. The corresponding analysis will be the basis of what is usually referred to as component qualification, i.e., the assurance of the REBECCA platform and of its dependable use in a system under certain conditions. All applications of the assurance framework will allow to answer RQ1, considering the assurance areas described in Section 3. For RQ2, this is the initial set of metrics defined: • Compliance criteria coverage (regarding compliance management): Number of fulfilled relevant compliance criteria / total number of relevant compliance criteria Example: The unmanned aerial vehicles must fulfil relevant IEC 61508 requirements, e.g., “The software safety requirements specification shall specify and document any safety-related or relevant constraints between the hardware and the software”. • Claims coverage (regarding assurance case development): Number of fulfilled relevant compliance criteria / total number of relevant compliance criteria Example: For industrial equipment inspection, it must be justified how physical harm hazards have been addressed. • Assurance artefact evidence coverage (regarding assurance evidence management): Number of valid evidence artefacts / total number of evidence artefacts Example: Security analysis reports of the AI-powered fridges must be evaluated and deemed valid. • Assurance gaps coverage (regarding lack of coverage in the different assurance areas): Number of justified assurance gaps / total number of assurance gaps Example: Assurance gaps might exist for the underwater robot because REBECCA will not pay detailed attention to, e.g., deployment (which is out of scope), and this must be justified. The metrics correspond to fundamental information about the degree of confidence in system assurance. They might be extended or refined depending on the final work in the real-world use cases, e.g., the dependability justification techniques finally used. In addition, the evaluation criteria and metrics can be tailored to the possibility of assigning weights to the relevant compliance criteria, the claims, the evidence artefacts, and the assurance gaps. This can be based on the judgement of their importance or of their contribution to safety or security. 5 CONCLUSION Novel Edge-AI applications pose new challenges for system developers, e.g., regarding physical limitations and response time. Assurance of these applications is not an easy task either. Several quality concerns may need to be addressed at the same time, the specific characteristic of Edge-AI need to be considered, and the context where these applications must be carefully considered. The REBECCA project will develop efficient Edge-AI systems and will provide a novel Edge-AI assurance framework that addresses the management of compliance, assurance cases, and assurance evidence. The framework will be based on the results of a systematic mapping study. To validate the framework, four different real-world use cases will be used for case study research, and the REBECCA platform. The assurance framework will contribute to providing information about what needs to be addressed for assuring safety and security in Edge-AI applications. ACKNOWLEDGMENTS The work leading to this paper has received funding from the REBECCA (HORIZON-KDT ref. 101097224; MCIN/AEI ref. PCI2022-135043-2; NextGen.EU/PRTR), ETHEREAL (MCIN/AEI ref. PID2020-115220RB-C21; ERDF), AETERNAL (MCIN/AEI ref. PID2023-149753OB-C21; ERDF), and “Paradigmas de interacción para la nueva era de resiliencia digital” (UCLM ref. 2022-GRIN34436; ERDF) projects. REFERENCES [1] ECS. 2023. Strategic Research and Innovation Agenda 2023. [2] N. Mansourov et al. 2010. Syst. Assur. beyond detecting vulnerabilities. Elsevier. [3] J.L. de la Vara et al. 2021. Assurance and Certification of Cyber-Physical Systems: The AMASS Open Source Ecosystem. J. Syst. Softw. 171: 110812. [4] IEC. 2011. IEC 61508 - Functional safety of electrical/electronic/programmable electronic safety-related systems, 2nd ed. [5] Y. Himeur et al. 2024. Edge AI for Internet of Energy: Challenges and perspectives. Internet Things 25: 101035. [6] B. Gallina et al. 2022. Multiconcern, Dependability-Centered Assurance Via a Qualitative and Quantitative Coanalysis. IEEE Softw. 39(4): 39-4. [7] Roel J. Wieringa. 2014. Design Science Methodology for Information Systems and Software Engineering. Springer. [8] B.A. Kitchenham et al. 2015. Evidence-Based Software Engineering and Systematic Reviews. Chapman & Hall/CRC Press. [9] A. Waterman et al. 2019. The RISC-V Instruction Set Manual, Volume I: UserLevel ISA, Document Version 20191213. RISC-V Foundation. [10] Chips JU: https://www.chips-ju.europa.eu/ [11] J.L. de la Vara et al. 2016. Model-Based Specification of Safety Compliance Needs: A Holistic Generic Metamodel. Inf. Softw. Technol. 72: 16-30. [12] OMG. 2023: Structured Assurance Case Metamodel, version 2.3. [13] J.L. de la Vara et al. 2022. Model-Based Assurance Evidence Management for Safety-Critical Systems. Software and Systems Modeling 21(6): 2329-2365. [14] S. Nair et al. 2014. An extended systematic literature review on provision of evidence for safety certification. Inf. Softw. Technol. 56(7), 689-717. [15] R. Wei et al. 2024. ACCESS: Assurance Case Centric Engineering of Safety– critical Systems. J. of Syst. Softw., 213, 112034. [16] D. Bacciu et al. 2021. Towards Functional Safety Compliance of Recurrent Neural Networks. CAIP 2021 (p. 86). [17] P. Runeson et al. 2012. Case Study Research in Software Engineering - Guidelines and Examples. Wiley. [18] ETSI. 2020. ETSI EN 303 645 - Cyber Security for Consumer Internet of Things.