Towards a Holistic Human-centered Semantics for Collaborative Planning and Execution

Towards a Holistic Human-centered Semantics for Collaborative Planning and Execution 1st Elisa Foderaro National Research Council of Italy and University of Genova, Italy [email protected] 2nd Alessandro Umbrico National Research Council of Italy Rome, Italy [email protected].it 3rd Andrea Orlandini National Research Council of Italy Rome, Italy [email protected].it Abstract—This work addresses the integration of Human Factors (HF) into Human-Robot Collaboration (HRC) within the transition from Industry 4.0 to Industry 5.0. While collaborative robots and AI enable flexible, adaptive, and safe production environments, industrial deployment remains challenging due to the need for accurate and semantically rich knowledge representations. We propose an ontological model that formalizes the production environment, robot capabilities, human skills, and psycho-physiological states of operators. This approach aims to enhance task planning and execution by supporting both efficiency and human well-being in collaborative industrial settings. Index Terms—Knowledge Representation, Ontology, HumanRobot Collaboration I. CONTEXT Industry 4.0 has changed the manufacturing sector, integrating advanced technologies to increase flexibility and efficiency, while achieving a customer-oriented and personalized production [1]. Human-Robot Collaboration (HRC) paradigm enables such flexibility and reconfigurability. Collaborative robots (cobots) are specifically designed to work next to human operators [2], thus enabling more dynamic production environments merging human flexibility and robotic precision. In this context, AI enables robots to perceive, understand, and adapt to the working environment, creating an effective and safe shared workspace. However, its integration in an industrial setting is challenging since it requires an accurate representation of the knowledge and input specifications that are usually difficult to design. The emerging Industry 5.0 promotes a more human-centered perspective, which also requires the analysis and consideration of human factors (HF). These can be defined as the physical, cognitive, social, emotional, and environmental factors that influence human operators in collaborative environments [3]. Taking into account these elements not only ensures human safety and well-being, but can also enhance collaboration [4]. In this context, our investigation aims to provide robot control systems with a formal and structured knowledge of the collaborative environment, explicitly including HFs. Here we present an initial result regarding the design of an ontological model capable of representing not only the production environment, the robot’s capabilities, and the This work was partially supported by the EU Commission under the Horizon Europe framework within the TRIFFID project (G.A. 101168042). human operator’s skills, but also their psycho-physiological status. This knowledge would constitute a solid and semantically rich formalism to enhance the planning and execution of robot tasks, while considering a comprehensive state of the operator [5]. To achieve this goal, we first identified relevant HFs in collaborative manufacturing and then explored how they can be effectively represented within the state-of-the-art ontology SOHO [6]. SOHO is a domain ontology designed for HRC that characterizes collaborative dynamics by capturing the capabilities of different types of resources, actors with different interacting features, and production requirements. Despite their importance, there is little research on how HFs affect AI-based manufacturing. Several works focus on safety [7] by mainly considering physical and/or cognitive ergonomics. However, human behavior and performance are also affected by factors such as gender, age, or level of expertise, together with previous experience with cobots, and attitude toward robots [8]. It is important to point out that our current scope is the individual dimension of human workers within a single collaborative cell, disregarding organizational or logistical aspects of the entire shop-floor and multi-agent dynamics. Therefore, the target of our analysis is the dyadic human-robot relationship, and the identified HFs that are relevant to our purpose are listed in Table I. These HFs can be assessed through different methods, ranging from subjective evaluation tools, e.g. questionnaires, to physiological signals. II. HUMAN FACTORS FOR HRC Building on this analysis, we refined SOHO [6] to represent HFs in a structured way. We specialized the concept AgentProperty to formally characterize qualities of agents. In particular, AgentSocialProperty expresses humans’ ability to relate themselves to the technology, thus subsumes complex relational qualities, e.g. Trust, Usability,AttitudeToTechnology, that can be updated using profiling questionnaires. Other individual properties, such as Age,Gender, and Experience are subsumed by AgentProfileProperty. Within AgentEmbodimentProperty, the concept WorkerEmbodimentProperty characterizes physical and cognitive dimensions relevant to the state of human workers. Furthermore, we introduced the concept of PhysiologicalProperty to capture additional metrics 2025 I-RIM Conference October 17-19, Rome, Italy ISBN: 9788894580570 10.5281/zenodo.17629792 161 TABLE I OVERVIEW OF HUMAN FACTORS CONSIDERED IN THIS WORK Human Factor Subfactor Description Physical Posture Body position of the user Ergonomic Physical Strain Physical strain or exertion associated with the execution of motions Physical Fatigue Physical fatigue associated with the execution of motions Cognitive Ergonomic Cognitive/Mental Workload Cognitive, mental, or intellectual demand Usability Effectiveness, efficiency, satisfaction in achieving specific goals Trust Confidence of the user that the system will act to reduce risk Stress/Anxiety Emotional pressure and strain arising when interacting with the system Frustration Negative emotional response to unsatisfied needs Perceived Enjoyment Positive emotional response associated with the use of the system User Gender Gender of the user Characteristics Age Age of the user Level of Expertise Expected level of expertise Previous Experience with Cobots Degree of prior experience with cobots Attitude Toward Technology Subjective attitude toward technology Task Related Factors User Preferences Individual preferences regarding system behavior Risk Level Inherent risk level of a task (e.g., blood pressure) that can be monitored to adapt robot behaviors. Pellet and Hermit were leveraged to perform a semantic check of the revised ontology. The resulting ontology1represents a step toward building a system fostering effective and efficient collaboration between cobots and human workers. The knowledge captured in SOHO can be exploited in many ways. For instance, task planning can take into account the state of the operator to avoid their overloading and benefit the collaboration. To illustrate the concrete applicability of the ontology, we considered the same collaborative cell described in [9], where a robotic arm is mounted on a linear track. Different working positions are available, and the robotic arm and the human operator perform assembly/disassembly tasks either in the same position (entailing strict parallelism or synchronization of the robot and human tasks) or on two different ones. We thus implemented the ontology-based modeling of HFs and started investigating its integration with task planning. As a first step, we modeled two distinct operators, with the same age and gender, but with different expertise levels: a novice operator (Wn), requiring more assistance, and an expert one (We), who can work autonomously. Moreover, operator Wedemonstrates a positive attitude toward robots, while operator Wnexhibits a negative 1Available at: https://github.com/pstlab/SOHO one. Even at this early stage, SOHO can be used to infer the operator’s profile and support personalized task allocation. For instance, operator Wewould be associated with a highly reliable profile, which entails low uncertainty in task execution times. On the other hand, Wnis inferred as an unreliable profile, for which no reliable estimates of execution times can be made. These profiles are then exploited by a task planner to generate plans which take into account the operator’s characteristics. As an example, thanks to their positive attitude toward robots, Weprofile would enable the planner to increase parallelism by allowing the robot and the worker to operate nearby when needed. Conversely, plans generated for Wnlimit parallelism and prevent close physical proximity between the human and the robot during task execution. Moreover, in the latter case the planner would also define the operative behavior of the worker without making any assumption on the expected duration of the tasks (i.e., [1, +∞]), which results in a less effective task allocation. III. CONCLUSIONS In summary, we propose an ontological model that integrates human factors into Human-Robot Collaboration, capturing both technical and psycho-physiological aspects. This formalism supports more effective task planning while promoting safety, well-being, and efficiency in Industry 5.0 settings. Future work will investigate a thorough evaluation of the proposed ontology in a real-world industrial scenario, and how this enriched knowledge can help cobots adjust their behavior in real-time by leveraging a planning framework as in [10]. REFERENCES [1] H. Lasi, P. Fettke, H.-G. Kemper, T. Feld, and M. Hoffmann, “Industry 4.0,” Business & information systems engineering, vol. 6, 2014. [2] R. Gervasi, L. Mastrogiacomo, and F. Franceschini, “An experimental focus on learning effect and interaction quality in hrc,” Production Engineering, vol. 17, no. 3, 2023. [3] A. de Nobile, D. Bibbo, M. Russo, and S. Conforto, “A focus on quantitative methods to assess human factors in collaborative robotics,” International Journal of Industrial Ergonomics, vol. 104, 2024. [4] R. Gervasi, M. Capponi, L. Mastrogiacomo, and F. Franceschini, “Analyzing psychophysical state and cognitive performance in hrc for repetitive assembly processes,” Production Engineering, vol. 18, 2024. [5] E. Foderaro, A. Umbrico, and A. Orlandini, “A Human-Centered Ontology for AI-Driven Human-Robot Collaboration,” in Proceedings of 3rd European Symposium on Artificial Intelligence in Manufacturing (ESAIM), 2025. To appear. [6] A. Umbrico, A. Cesta, and A. Orlandini, “Enhancing awareness of industrial robots in collaborative manufacturing,” Semantic Web, 2023. [7] J. Vermeulen, G. Caldwell, M. Teixeira, A. Burden, and M. Guertler, “To Safety and Beyond! A Scoping Review of Human Factors Enriching the Design of Human-Robot Collaboration,” Interaction Design and Architecture(s), 2024. [8] S. Hopko, J. Wang, and R. Mehta, “Human Factors Considerations and Metrics in Shared Space hrc: A Systematic Review,” Frontiers in Robotics and AI, vol. 9, 2022. [9] A. Umbrico, A. Orlandini, A. Cesta, M. Faroni, M. Beschi, N. Pedrocchi, A. Scala, P. Tavormina, S. Koukas, A. Zalonis, N. Fourtakas, P. S. Kotsaris, D. Andronas, and S. Makris, “Design of Advanced Human–Robot Collaborative Cells for Personalized Human–Robot Collaborations,” Applied Sciences, vol. 12, p. 6839, Jan. 2022. [10] E. Foderaro, A. Cesta, A. Umbrico, and A. Orlandini, “Simplifying the A.I. Planning modeling for Human-Robot Collaboration,” in 2021 30th IEEE RO-MAN, pp. 1011–1016, 2021. 162