EmoACT: Integrating Identity, Impression, and Emotion for Synthetic Affective Agent

EmoACT: Integrating Identity, Impression, and Emotion for Synthetic Affective Agent Francesca Corrao1, Alice Nardelli1, Antonio Sgorbissa1and Carmine Tommaso Recchiuto1 Abstract—Equipping robots with emotional capabilities enhances human–robot interaction by increasing trust, social presence, and predictability. Existing approaches largely rely on Cognitive Appraisal Theory. We propose EmoACT, a platformindependent framework for synthetic emotion generation based on Affect Control Theory (ACT). ACT models emotions as emerging from social interaction in a three-dimensional Evaluation–Potency–Activity (EPA) space. EmoACT integrates three modules: an Impression Estimator that infers real-time user impressions, an Identity Generator that combines personality, role, and comfortability, and an Emotion Generator that produces affective states in EPA space and maps them to basic emotions. Results show that EmoACT generates personalitysensitive emotions with low computational cost, enabling realtime use in social robots. Index Terms—Synthetic Emotions, Affective Computing, Affect Control Theory I. INTRODUCTION Affective computing [1] enables robots to understand and express emotions, improving human–robot interaction by strengthening the human–robot bond [2], enhancing social presence [3], and increasing acceptability [4]. Emotions also help humans to interpret robot behavior using cues from human-human interactions [5]. Most approaches for synthetic emotion generation rely on multiple interconnected modules that process external stimuli based on psychology-inspired models [6]. Cognitive Appraisal Theory [7]is the most widely used and has been implemented in various systems [8], [9]. We recently proposed EmoACT [10], a platformand taskindependent framework for synthetic emotion generation based on Affect Control Theory (ACT). Unlike appraisal-based models, ACT focuses on how emotions emerge through social interaction [11], offering a promising approach for emotionally capable artificial agents. In this work, we present a new version of the EmoACT framework featuring an improved emotion equation, a novel impression estimation method, and the integration of a new module for generating robot identities that accounts for synthetic personality traits. Results suggest that EmoACT produces diverse emotional responses according to different personality trait and enhances the perception of personality traits in robotic agents. This work was supported by the RAISE project and by the Alzheimer’s Association (24AARG-NTF-1200708). 1All authors are with the RICE Lab, DIBRIS Department, University of Genoa, Via All’Opera Pia 13, 16145 Genoa, Italy. Corresponding author: [email protected] II. METHODOLOGY EmoACT, is a platform-agnostic framework for synthetic emotion generation grounded in ACT. ACT describes emotions as arising during interaction from the discrepancy between one’s identity and the impression others have of them [11]. These three concepts are represented in a 3D affective space measuring Evaluation, Potency and Activity (EPA). The EmoACT architecture comprises three main components: Impression Estimator, Identity Generator, and Emotion Generation (Figure 1). Impression ([E,P,A]) Impression Estimator Emotion Generation Identity ([E,P,A]) Identity Generator User's Sentence User's Emotion User's Gaze User's Proximity Comfortability Personality Fig. 1: EmoACT architecture. A. Impression Estimator The Impression Estimator continuously updates the user’s estimated impression of the robot using the Actor-BehaviorObject (ABO) paradigm [12], where the Actor is the human, the Behavior is inferred from perception cues, and the Object is the robot. It relies on multiple cues such as the user’s emotion, sentence, gaze, and proximity, provided by external perception modules. The user’s emotion is used as a modifier for the default Actor value, which corresponds to EPA value of the label ”human”. Other perception cues are used to estimate the Behavior, by computing the mean of three values: •Emotional behavior: determined by performing sentiment analysis of the user sentence; •Attention behavior: derived from gaze changes to determine whether the user is paying attention to the robot; •Distance behavior: derived from proximity to determine whether the user stays close to the robot. B. Identity Generator The Identity Generator uses personality traits and comfortability level to generate the robot’s identity. The identity is obtained as a weighted mean between the role of the robot 2025 I-RIM Conference October 17-19, Rome, Italy ISBN: 9788894580570 10.5281/zenodo.17629714 91 (Ir), the current personality trait(Ip), and the comfortability level (Ic), as described in the following Equation (1). identity =wr∗Ir+wc∗Ic+wp∗Ip(1) The robot’s role (Ir) is defined by default as that of a social robots [13]. The comfortability parameter (Ic) is assigned to either the ”comfortable” or ”uncomfortable” label depending on the input value. Personality (Ip) is obtained by computing a weighted mean (Equation (2)) between three of the Big Five personality traits: Conscientiousness, Extroversion and Agreeableness (CEA). We considered labels for each of the six CEA extremes (Distracted-Conscientious, Introvert-Extrovert, Disagreeable-Agreeable), which are then weighted according to input values. Ip=wc∗C+we∗E+wa∗A(2) C. Emotion Generation The Emotion Generation node receives the robot’s identity and impression of the robot and, based on ACT equations [11], produces the emotion the robot should portray. The structure of ACT equations is the one presented in Equation (3). emotion =k1∗identity −k2∗impression +k3(3) Emotions are generated in EPA space, but the closest basic emotion label among Ekman’s set [14] is also determined. Computing Ekman’s basic emotions facilitates emotional expression, since few studies have attempted to convey emotions based on affective level [15]. The emotional server provides both EPA values and basic emotion labels as output. III. RESULTS We evaluated EmoACT’s capabilities to generate personality-driven emotional responses using a dataset of 40 English user sentences for each of the seven basic emotions. The dataset was generated by an LLM and validated by an expert. Results show that EmoACT’s emotion generation is influenced by personality traits, particularly Agreeableness. We also integrated EmoACT into a robotic agent and tested it in a dyadic, general-purpose conversation. Findings suggest that, compared to an LLM, EmoACT enhances the perception of the Agreeableness traits. Due to the lack of standardized benchmarks and validated questionnaires for assessing the accuracy of synthetic emotional display, direct comparison with other emotion-generation system was not possible. IV. CONCLUSION We extended prior work on implementing ACT, resulting in a platform-agnostic framework that incorporates personality into emotion generation. Additionally The system requires minimal processing power; the only computationally demanding component is perception, which runs at low frequency and does not impact interaction latency. This makes EmoACT a promising approach for robots operating in real-world environments. In future work, we aim to use this architecture as the empathetic component of a companion agent designed to assist people with neurodegenerative diseases and reduce caregiver workload. REFERENCES [1] J. Tao and T. Tan, “Affective Computing: A Review,” in Affective Computing and Intelligent Interaction, J. Tao, T. Tan, and R. W. Picard, Eds. Berlin, Heidelberg: Springer, 2005, pp. 981–995. [2] Deukey Lee, Ho Seok Ahn, and Jin Young Choi, “A general behavior generation module for emotional robots using unit behavior combination method,” in RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication. Toyama: IEEE, Sep. 2009, pp. 375–380. [Online]. Available: http://ieeexplore.ieee.org/ document/5326239/ [3] C. L. Bethel and R. R. Murphy, “Survey of Non-facial/Nonverbal Affective Expressions for Appearance-Constrained Robots,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 38, no. 1, pp. 83–92, Jan. 2008, conference Name: IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews). [Online]. Available: https://ieeexplore.ieee.org/abstract/document/4378439 [4] R. Stock-Homburg, “Survey of Emotions in Human–Robot Interactions: Perspectives from Robotic Psychology on 20 Years of Research,” International Journal of Social Robotics, vol. 14, no. 2, pp. 389– 411, Mar. 2022. [Online]. Available: https://link.springer.com/10.1007/ s12369-021-00778-6 [5] E. Osuna, L.-F. Rodr´ ıguez, J. O. Gutierrez-Garcia, and L. A. Castro, “Development of computational models of emotions: A software engineering perspective,” Cognitive Systems Research, vol. 60, pp. 1–19, May 2020. [Online]. Available: https://linkinghub.elsevier.com/ retrieve/pii/S1389041719305108 [6] Z. Kowalczuk and M. Czubenko, “Computational Approaches to Modeling Artificial Emotion – An Overview of the Proposed Solutions,” Frontiers in Robotics and AI, vol. 3, Apr. 2016. [Online]. Available: http://journal.frontiersin.org/Article/10.3389/frobt.2016.00021/abstract [7] A. Moors, P. C. Ellsworth, K. R. Scherer, and N. H. Frijda, “Appraisal Theories of Emotion: State of the Art and Future Development,” Emotion Review, vol. 5, no. 2, pp. 119–124, Apr. 2013, publisher: SAGE Publications. [Online]. Available: https: //doi.org/10.1177/1754073912468165 [8] C. Breazeal, “Emotion and sociable humanoid robots,” International Journal of Human-Computer Studies, vol. 59, no. 1-2, pp. 119–155, Jul. 2003. [Online]. Available: https://linkinghub.elsevier.com/retrieve/ pii/S1071581903000181 [9] M. Ehtesham-Ul-Haque, J. D’Rozario, R. Adnin, F. T. Utshaw, F. Tasneem, I. J. Shefa, and A. A. Al Islam, “EmoBot: Artificial emotion generation through an emotional chatbot during generalpurpose conversations,” Cognitive Systems Research, vol. 83, p. 101168, Jan. 2024. [Online]. Available: https://linkinghub.elsevier.com/ retrieve/pii/S138904172300102X [10] F. Corrao, A. Nardelli, J. Renoux, and C. T. Recchiuto, “EmoACT: a Framework to Embed Emotions into Artificial Agents Based on Affect Control Theory,” Apr. 2025, arXiv:2504.12125 [cs]. [Online]. Available: http://arxiv.org/abs/2504.12125 [11] K. J. Lively and D. R. Heise, “Emotions in Affect Control Theory,” in Handbook of the Sociology of Emotions: Volume II, J. E. Stets and J. H. Turner, Eds. Dordrecht: Springer Netherlands, 2014, pp. 51–75, series Title: Handbooks of Sociology and Social Research. [Online]. Available: http://link.springer.com/10.1007/978-94-017-9130-4 4 [12] D. R. Heise and L. Smith-Lovin, “Impressions of Goodness, Powerfulness, and Liveliness from Discerned Social Events,” Social Psychology Quarterly, vol. 44, no. 2, pp. 93–106, 1981, publisher: [Sage Publications, Inc., American Sociological Association]. [Online]. Available: https://www.jstor.org/stable/3033705 [13] D. B. Shank, A. Burns, S. Rodriguez, and M. Bowen, “Software Program, Bot, or Artificial Intelligence? Affective Sentiments across General Technology Labels.” [14] P. Ekman and D. Cordaro, “What is Meant by Calling Emotions Basic,” Emotion Review, vol. 3, no. 4, pp. 364–370, Oct. 2011. [Online]. Available: http://journals.sagepub.com/doi/10.1177/1754073911410740 [15] A. Beck, L. Canamero, and K. A. Bard, “Towards an Affect Space for robots to display emotional body language,” in 19th International Symposium in Robot and Human Interactive Communication. Viareggio, Italy: IEEE, Sep. 2010, pp. 464–469. [Online]. Available: http://ieeexplore.ieee.org/document/5598649/ 92