Computational affective knowledge representation for agents located in a multicultural environment

May 2024 l Volume 14 RESEARCH'(Research'Manuscript)''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''Open'Access! Human-centric Computing and Information Sciences (2024) 14:30 DOI: https://doi.org/10.22967/HCIS.2024.14.030 Received: April 14, 2022; Accepted: July 25, 2022; Published: May 30, 2024 Computational Affective Knowledge Representation for Agents Located in a Multicultural Environment Joaquin Taverner1,*, Andreas Brännström2, Dalila Durães3, Emilio Vivancos1, Paulo Novais3, Juan Carlos Nieves2, and Vicente Botti1 Abstract In this paper, we propose a new computational model for affective knowledge representation that will be used for affective agents located in a multicultural environment. To this end, we present the results of two experiments, the first of which determines the most appropriate labels to define the pleasure-arousal dimensions in the culture and language of the agent’s location. As an example, we use the Portuguese and Swedish languages. The second experiment identifies the most suitable values of pleasure-arousal dimensions for each emotion expressed in these example languages. The results obtained are compared with a previous model developed for agents interacting with European Spanish-speaking people. Results show significant differences in the values of pleasure and arousal associated with emotions across languages and cultures. The results also show no significant differences in gender or age when associating levels of pleasure-arousal to emotions. We propose two applications of these representation models, such as a model of an agent capable of adapting its affective behavior to different cultural environments and a human-aware planning scenario in which the agent uses this dimensional representation to recognize the user’s affective state and select the best strategy to redirect that affective state to the target state. Keywords Affective Computing, Human Emotion Modeling, Human-Machine Interaction, Affective Agents, Emotion Representation, Cross-Cultural Emotion Representation 1. Introduction Human affective behavior encompasses a set of constructs and cognitive processes that make it complex to simulate using software agents. One of the main constraints of simulations corresponds to how humans interpret affective knowledge. Humans typically use different abstractions and metaphors to express, represent, and interpret the affective knowledge. For example, through the use of words or labels such as “sadness” or “joy.” This way of representation is commonly used in the area of affective computing [1–4]. However, the use of words or labels is a simplification that does not lead to representing all the information inherent to affective states, such as the intensity with which an affective state is ※ This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. *Corresponding Author: Joaquin Taverner ([email protected]) 1Valencian Research Institute for Artificial Intelligence (VRAIN), Universitat Politècnica de València, Valencia, Spain 2Department of Computing Science, Umeå University, Umeå, Sweden 3ALGORITMI Centre, University of Minho, Braga, Portugal Page 2 / 19 Computational Affective Knowledge Representation for Agents Located in a Multicultural Environment perceived. In this paper, we use an alternative to this representation by the use of models based on different cognitive dimensions. One of the most common dimensional models in literature is the Core Affect model [5]. That model defines affective states through two dimensions of pleasure and arousal. Dimensional models have been widely used in affective computing [6–8]. Still, existing proposals also simplify affective states, losing some of the information inherent to affective states. Moreover, as we show in this paper, the cognitive conceptualization of an affective state may vary across cultures and languages, thus influencing the levels of pleasure and arousal associated with that affective state [9]. The use of representation models that are not adapted to the culture and language may produce misunderstandings and errors that compromise the simulations of affective behavior and users’ experience [10]. The method for representing affective knowledge for intelligent agents proposed in this article solves some of the most important limitations of previous proposals, namely (i) facilitating the use of the affective state in the agent’s reasoning process; (ii) representing the universal meaning of the emotion independently of language and culture; and (iii) allowing the agent to adapt the emotion to the language and culture at the moment of expressing the emotion. This paper proposes a new computational model for affective knowledge representation. We describe the results of two experiments conducted with Portuguese and Swedish participants to find the most suitable translations for the affective categories, and adapt the pleasure-arousal space to both cultural environments. The results of the experiment show that there are significant differences in the pleasure and arousal values assigned to each emotion, based on the language and culture of the participant. However, we did not detect any sexor age-dependent differences. We finally present a computational model for affective knowledge representation in affective agents and a potential application of this crosscultural model in the area of human-aware planning. 2. Related Work Affective states are constructs that define different experiences, including emotions and moods. From the area of affective computing [1] different approaches focused on the recognition, interpretation, processing, or simulation of different emotional behaviors and affective states have been conducted [3, 11–13]. However, not much attention has been paid to developing models to represent the knowledge related to affective states designed explicitly for its use in computational systems. The two common approaches in the literature to represent affective states are referred to as categorical and dimensional [14]. Categorical approaches are based on using a limited set of words or labels to define the different affective states [15]. These labels provide a means to verbalize affective states, which constitute a critical factor in human affective interactions. One of the best-known models is the model of basic emotions proposed by Ekman [16] in which six labels are defined as “sadness,” “happiness,” “anger,” “fear,” “disgust,” and “surprise.” This model is one of the most widely used in the area of affective computing [4, 17]. However, by a label-based representation, we assume a high level of abstraction since we have condensed the entire spectrum of affective states into a limited set of labels. Therefore, we are losing part of the affective information, such as the intensity of the affective state, or the probability of moving from one affective state to another depending on a certain stimulus [18, 19]. For example, if a person is “sad” and receives good news, the probability of transitioning to a positive affective state such as “joy” may be lower than if the person is “happy.” In addition, this representation assumes that labels/words representing emotions have the same meaning for all cultures and languages, and one word can be translated by another one representing the same affective state. Nonetheless, depending on the model of emotional representation being used, translating emotions from one language to another can lead to cultural misunderstandings since emotions are not always understood in the same way across different cultures and languages [9]. In contrast to categorical approaches, theories based on dimensional approaches allow reducing the level of abstraction by representing affective states with different dimensions. These theories have their origin in psychological constructivism, according to which a finite set of labels cannot establish affective Human-centric Computing and Information Sciences Page 3 / 19 states since they depend on an individual’s subjective experience that occurs in a particular context (e.g., a specific culture or language) [20]. One of the most representative dimensional models is the model of core affect proposed by Russell [5]. In that model, affective experiences are defined as emotional cues and represented by two dimensions of pleasure and arousal. Russell [21] also evidenced through experimentation that the labels used to define the different affective states follow a circular pattern along these two dimensions. From these results, he devised the circumplex model of affect. In that model, all the labels referring to the different affective states are represented as points composed of pleasure and arousal components, and are organized in a circular way around these two dimensions. Being a representation based on continuous dimensions, it is possible to establish relationships between the distribution of the different affective states. For example, it can be assumed that, in general, affective states related to the word “happiness” are related to positive values of the pleasure and arousal dimensions, while those related to the word “sadness” are related to negative values in these two dimensions. Some authors have discussed the number of dimensions that define affect. For example, in [22], it is proposed to introduce the dominance dimension to the pleasure-arousal space resulting in a model known as the PAD model. Nevertheless, some studies show that this dimension does not substantially impact the representation of affect across cultures [21] and that people exhibit difficulties and confusion when trying to assign a dominance value to the emotions they are feeling [23]. Over the years, different authors have shown through experimentation the existence of variations in the levels of pleasure and arousal associated with each affective state across different cultures and languages [9, 10, 24]. For example, the study presented in [25] analyzed the discrepancy of 24 emotional labels in about 2,500 languages. That study concludes that there are significant variations between the values of pleasure and arousal across cultures. Therefore, the direct translations of emotion labels can be biased because the meaning of the words can differ between languages. The circumplex model of affect has also been used in affective computing [6, 7]. For example, in [26], a model for estimating the facial expressions of a humanoid robot is presented. Authors use the circumplex model of affect to represent emotions as points composed of the pleasure and arousal dimensions. However, the circumplex model of affect in affective computing also presents some disadvantages. Let’s suppose that we define affective states labels by a single point in the dimensional representation space, and then we generate a random point in that space. If the point does not match with any of the points predefined for each affective state label, it is not possible to determine the label associated with that point. A possible approach used in some proposals [27, 28] warrants estimating the distance of a random point from each predefined point for the affective labels in the model and then assigning the nearest affective label. By this approach, it is necessary to assume that the distribution of affective labels in the pleasure-arousal space is equal and uniform. However, this way of representing affective labels is artificial and far from how affective states are distributed throughout the pleasurearousal space as demonstrated in [29]. In addition, in a point-based approach, it is complex to assign a level of certainty for whether a random point belongs to a specific label or represents the transitions between two affective states. It is also difficult to simulate other affective behaviors such as the influence on the affective state of other affective factors like personality [30]. Still, in a point-based representation, the intensity of affective states is not considered [31]. Despite this, there is empirical evidence that establishes a direct relationship between the levels of pleasure and arousal and the intensity with which different affective states are experienced; for example, values close to zero in both dimensions would indicate very low or no intensity while values close to one would indicate high intensity [32]. A possible alternative to the point-based representation is a vector-based representation. In this representation, the direction of the vector is used to determine the label that represents a point in the pleasure-arousal space. For example, in [33], a vector-based representation using the pleasure-arousal space to express emotions in a robot is proposed. Unfortunately, that representation presents the same problem as the point-based representation since the way in which the variations in the dispersion of Page 4 / 19 Computational Affective Knowledge Representation for Agents Located in a Multicultural Environment affective states are represented is symmetrical and thus, far from the mental representation that people have of affective states. Finally, there seems to be a trend towards developing affective models based on fuzzy logic [34, 35]. The fuzzy logic adds uncertainty [36] to the representation model by allowing a point in the pleasurearousal space to belong to more than one label. Moreover, fuzzy logic allows us to express affective states in the same terms that people use, such as “very sad” or a “little happy.” For example, in [37], a fuzzy logic-based model for translating emotional labels from Spanish to English is proposed. A methodology based on an experiment in which eight Spanish speakers assigned the values of pleasure, arousal, and dominance to 30 Spanish emotional labels/words was proposed to establish the discriminant values of the fuzzy membership functions for each emotion. 3. Cross-Cultural Model of Emotions for Affective Computing We present a new cross-cultural model of emotion representation for affective computing which differs from the representations discussed in Section 2. This new system uses probability areas to define each affective state in the pleasure-arousal space. By using probability areas uncertainty is added since each point in the space composed of the pleasure and arousal components has a certain probability of belonging to each affective label. Our model follows a vector-based representation in a polar coordinate system in which the direction indicates the probability of experiencing a particular affective state and the modulus indicates its intensity. However, in a polar coordinate system, traditional linear statistical models may not be suitable since they do not satisfy the linearity constraint, because they are constrained between 0 and 2p. The main problem underlying the application of linear statistics arises when studying the probability regions near 0 and 2p, since in a linear model, these two regions would be independent; however, in a polar coordinate system, these two regions are dependent. A possible method to avoid these problems entails replacing the usual linear statistics with circular statistics [38]. The most suitable approach to estimate the dispersion in a circular representation space composed of the pleasure and arousal dimensions is through the circular standard deviation s, which is defined as follows [39, 40]. σ=#−2 ln)*!+,!-" !, (1) where * and , represent the arithmetic mean for the pleasure and arousal dimensions, respectively. On the other hand, to estimate the probability distribution in a circular representation space, the von Mises distribution [41] can be considered, which allows to estimate the Gaussian (or normal) distribution in circular data. Given an angle α, the von Mises probability density function is defined as follows: 0(α|µ,κ)=6#$%&(()*) 2π8,(κ), (2) where µ represents the mean of the distribution, κ represents the concentration parameter of the distribution which is calculated as the inverse of the circular variance (1/;!), while 8,(κ) is the modified Bessel function of order zero used to normalize the function [42]. Fig. 1 shows an example of the model of emotion representation adapted to European Spanish speakers. In this example, the outer rings show the probability areas of each of the ten emotions using one circular standard deviation, the vector <- = = = > represents an affective state, and the inner rings represent four regions of intensity, namely strong, medium, weak, and neutral. The grey shaded area shows an example of the probability region related to the emotion “boredom” using one circular standard deviation. Note that the different grey tones represent the different intensity values for the “boredom” emotion. Through this model, the different affective states can be represented by abstracting from the labels that represent each affective state, which will only be necessary when a software agent needs to express or Human-centric Computing and Information Sciences Page 5 / 19 interpret an affective state. Vectors can be modified over time by a sequence of events that an agent perceives. This makes it possible to evaluate the relationship between previous and current affective states, allowing for the simulation of different human affective behaviors. For example, if the agent has an affective state at instant ?. and receives a new positive event at instant ?./", the agent’s emotional response can be adapted to the affective state it had in ?. by modifying the intensity of the emotion generated by the positive event in ?./". However, since this model results from an experiment with European Spanish speakers, it may not be suitable for use in other cultural environments. Therefore, an experiment-based methodology has been proposed to develop a new affect representation model establishing the levels of pleasure and arousal for each emotion in the language of the model the agent will use. An agent can have more than one probability model for different languages and adapt its expression of affective states to the language and culture of its human interlocutor Fig. 1. Distribution of probability functions in the pleasure-arousal space using 1 standard deviation for Spanish resulting from the experiment conducted in [31]. 4. Experiment Design To create the affect representation model, we need to find the areas associated with each affective state word or label in the language of the agent’s location (the target language). We propose a methodology based on two experiments as follows. In the first experiment, the words used to represent the dimensions of pleasure and arousal are translated into the target language, and then the results of the first experiment are used in a second experiment to obtain the levels of pleasure and arousal associated with each affective state in the target language. In this paper, as an example of the application of this methodology, we use an agent designed for representing and expressing affective states in Portuguese and Swedish. 4.1 Methodology of the First Experiment This first experiment is designed to identify the labels that best represent the meaning of the pleasure and arousal dimensions in a target language. In the literature, there are different references to the pleasurearousal dimensions. Generally, these dimensions are defined into four affective categories of pleasure or its antithesis misery and arousal or its antithesis sleepiness [21]. However, a literal translation of these terms to other languages may result in cultural misunderstandings as the term associated with the translation may not fully reflect the original meaning of an affective category label. Neutral Arousal Arousal  ve 0.75 0.5 Weak Medium Strong 0.25 Disgust Excitement Calm -1 Boredom Anger Sadness Fear Surprise Arousal Happiness Pleasure Sleepiness Neutral -1 +1 +1 Page 6 / 19 Computational Affective Knowledge Representation for Agents Located in a Multicultural Environment Hypothesis: The main hypothesis to be validated by this first experiment is that the literal translation of the labels associated with the dimensions of pleasure and arousal is not adequate for agents located across different cultures and languages. Participants: Participants were recruited through Prolific [43]. In our experiments, we required participants’ nationality, residence, and birth place to be either Portuguese or Swedish in each case. In addition, we also specified that their first language should be Portuguese or Swedish in each case. For this experiment, 50 participants were recruited for each target language. For Portuguese, the sample was composed of 15 women and 35 men, ranging in age between 18 and 67 years old (@=26.28,;=9.98). For Swedish, the sample was composed of 13 women, 35 men, and two participants who preferred not to indicate their gender, with their ages ranging between 18 and 65 years old (@=29.96,;=9.99). Materials: For the first experiment (concerning the identification of suitable pleasure-arousal end points), a set of labels was chosen for each language. These labels were selected from the common Portuguese, proposed in [44, 45], and Swedish, proposed in [46]. In order to use the same words that regular people use and minimize the bias, we complemented both sets of labels with different synonyms collected from the dictionaries of Priberam [47] and Infopedia [48] for Portuguese and Bab.la [49] and Synonymer [50] for Swedish. The words and synonyms were chosen by native speakers of the target languages, considering the words found in dictionaries and their knowledge of commonly used words. (a) (b) (c) (d) Fig. 2. Degree-of-acceptance of candidate words in Portuguese for each affective category: (a) pleasure, (b) misery, (c) arousal, and (d) sleepiness. (a) (b) (c) (d) Fig. 3. Degree-of-acceptance of candidate words in Swedish for each affective category: (a) pleasure, (b) misery, (c) arousal, and (d) sleepiness. From this research, we selected 28 Portuguese words and 30 Swedish words. Then we divided these words into four sets of words, containing between five and nine words in each set for Portuguese (Fig. 2) and between seven and nine words in each set for Swedish (Fig. 3). Then to explore the words that participants related most closely to each of the four affective categories in each target language, we designed an online questionnaire composed of four sections, one for each affective category. Two checkbox questions were designed to identify the best words to translate each affective category in each section. The first question asked each participant to read a set of words and select the word(s) that clearly do not -20 -10 0 10 20 30 Prazer Satisfação Felicidade Contente Júbilo Encantado Contentamento Alegria (a) Pleasure. -40 -30 -20 -10 0 10 20 30 Aborrecimento Miséria Aflição Insatisfação Desprazer Infelicidade Desespero Sofrimento Angústia (b) Misery. -30 -20 -10 0 10 20 30 Ativação Alerta Entusiasmo Excitação Agitação (c) Arousal. -20 -10 0 10 20 30 Entorpecimento Sonolência Apatia Desativação Cansaço Dormência (d) Sleepiness. 1 -50 -40 -30 -20 -10 0 10 20 Lycka Glädje Tillfredsställelse Belåtenhet Behag Gottfinnande Lust Åtrå (a) Pleasure. -30 -20 -10 0 10 20 30 Irritation Otillfredsställelse Motvilja Missnöje Obehag Ogillande Olust Lidande Ångest (b) Misery. -18 -12 -6 0 6 12 18 Aktivering Affekt Spänning Upphetsning Agitation Känslosvall (c) Arousal. -40 -24 -8 8 24 40 Dåsighet Sömnighet Slöhet Deaktivering Trötthet Kraftlöshet Utmattning (d) Sleepiness. 1 Human-centric Computing and Information Sciences Page 7 / 19 express the same meaning as the rest of the words in the set. In the second question, the same set of words was shown, and each participant was asked to read the set of words and select the word(s) that best represent the concept of that set. Finally, we added an extra section containing an attention check question. Procedure: At the beginning of the experiment, participants were provided with instructions describing the task to be performed along with two examples of questions for illustrative purposes. Then, participants were asked to complete the four sections, one for each affective category, of the questionnaire without a time limit. Those who failed the attention check question were dismissed and replaced by new participants until a total of 50 participants was reached in each language. 4.2 Methodology of the Second Experiment This second experiment aims to adapt the emotional representation space of the agent to the language of the agent’s location. The experiment defines the regions of the pleasure-arousal space associated to each emotion label. Initially, for this experiment, we selected the ten emotions proposed in [9] since they represent a “broad range of feeling as possible,” namely “fear,” “surprise,” “disgust,” “anger,” “boredom,” “sleepiness,” “sadness,” “excitement,” “calm,” and “happiness.” In the same vein, to the first experiment, suitable labels for representing emotions were chosen that are common with the Swedish labels used in [46] and with the Portuguese labels presented in [44] and [45], coupled with the resulting end-point labels from the first experiment. Hypothesis: The main hypothesis for this second experiment is that the levels of pleasure and arousal associated with an emotional label depends on the culture and language of the agent’s location. Participants: In this second experiment, the Prolific platform was also used to recruit participants by applying the same demographic and language filters as in the first experiment. For Portuguese, 50 women and 50 men ranging in age between 18 and 52 years (@=24.10,;=5.91) were recruited, while for Swedish, 50 women and 50 men between the ages of 18 and 67 years (@=30.79,;=10.37) were recruited. Materials: We designed an online questionnaire composed of ten prompts, one for each emotion label. Each prompt contained only an emotion label. Two 7-item Likert scales were used to assign the levels of pleasure and arousal associated with the emotion label. The arousal scale ranged from very sleepy to very aroused and the pleasure scale ranged from very miserable to very pleased. In addition, an attention check question was shuffled along with the ten prompts of the experiment. Procedure: At the beginning of the experiment, an explanation of the objective of the experiment along with a brief introduction to the dimensions of pleasure and arousal was given to the participants. In addition, instructions describing the task to be performed were also provided along with an illustrative example. Participants were then asked to complete the ten questions of the experiment and the attention check question with no time limit. Those who failed any of the attention check questions were dismissed and replaced by new participants until a total of 100 participants was reached in each language. 5. Results 5.1 First Experiment To measure the suitability of a word J to represent the translation of an affective category K into a target language, we define the degree-of-acceptance. The degree-of-acceptance Laccept$(J)R is defined as the difference between the number of times the word J has been selected as representative for the affective category K and the number of times the word has been selected as non-representative of that affective category. 5.1.1 Portuguese model Page 8 / 19 Computational Affective Knowledge Representation for Agents Located in a Multicultural Environment Fig. 2 shows the degree-of-acceptance of the candidate words for the four affective categories. For example, in the case of the affective category misery, the degree-of-acceptance for “infelicidade” was accept01&234(Infelicidade)=22, indicating that the participants considered this word as very representative for that affective category. In contrast, the degree-of-acceptance for “aborrecimento” was accept01&234(Aborrecimento)=−34, indicating that the majority of the participants rejected this word to be representative for the affective category of the group. Fig. 2 shows the degree-of-acceptance of the candidate words in Portuguese for each affective category. In order to perform the second experiment, we decided to use the three words with the highest level of acceptance. 5.1.2 Swedish model The results obtained for the first experiment conducted with Swedish participants are shown in Fig. 3. Analyzing the degree-of-acceptance of these results, we can appreciate that there is a larger set of words with a positive degree-of-acceptance than in the case of the experiment conducted with Portuguese participants. Nevertheless, it can be seen that in all cases, there are two words that outperform the rest. Therefore, from these results, we decided to use the two words with the highest level of acceptance. 5.2 Second Experiment To analyze the results of this second experiment, we will refer only to the dimensions of pleasure and arousal, considering that misery is negative pleasure and sleepiness is negative arousal. Those responses in which the participants selected both neutral values (i.e., the 4th item in both Likert scales) in the pleasure and arousal dimensions were removed. This decision was made according to previous studies in which it was evidenced that at values close to zero in both dimensions, the intensity of emotions is so low that it can be considered that there is no emotion [32]. Then we calculated the angle of each response from the pleasure and arousal values, with this angle representing an approximation to the conceptualization that each participant associates with an emotion in the pleasure and arousal space. Finally, we estimated the dispersion of the samples for each emotion in each of the two languages, and removed the possible outliers. Table 1 summarizes the results obtained in the second experiment for the example languages Portuguese and Swedish, as well as the results of the previous experiment conducted with European Spanish-speaking participants [29]. This table presents the mean angle for each emotion and its circular standard deviation obtained using Formula 1. To facilitate the visualization and analysis of the distribution of the ten emotions in the circular space composed of the pleasure and arousal dimensions,!we proposed the use of circular boxplots [51] which can be seen in Figs. 4 and 5. Table 1. Circular mean and circular standard deviation in degrees for each emotion Emotions in English Emotions in Portuguese Emotions in Swedish Emotions in Spanish Label/word Mean SD n Label/word Mean SD n Label/word Mean SD n Fear Medo 160.12 39.25 94 Rädsla 136.93 21.62 82 Miedo 125.51 15.61 93 Surprise Surpresa 62.79 21.02 91 Överraskad 75.06 18.86 91 Sorpresa 71.63 26.38 92 Disgust Aversão 195.37 41.88 86 Avsky 167.88 34.00 96 Asco 182.58 43.65 97 Anger Raiva 151.60 37.72 98 Ilska 140.95 8.20 75 Enfado 138.55 16.90 83 Boredom Aborrecimento 245.96 21.11 93 Uttråkad 230.33 9.46 84 Aburrimiento 245.34 21.41 98 Sleepiness Sonolência 259.19 18.04 87 Trött 254.39 16.79 93 Somnolencia 263.59 15.56 95 Sadness Tristeza 207.78 15.51 92 Ledsen 205.69 20.31 96 Tristeza 196.02 22.48 98 Excitement Excitação 53.18 8.64 85 Spänning 76.78 20.96 96 Emoción 39.97 10.32 87 Calm Calmo 312.68 46.76 70 Lugn 325.44 37.77 88 Calma 318.12 35.89 74 Happiness Felicidade 40.22 7.66 73 Lycka 32.66 15.64 92 Felicidad 25.09 19.02 96 “n” represents the number of samples without outliers and neutral value (i.e., the 4th item in both Likert scales). Human-centric Computing and Information Sciences Page 15 / 19 However, there may be more or less suitable ways for achieving stress relief. Whether to decrease arousal or increase pleasure in a moment depends on the current emotional state of the individual, the current context, and a variety of personal and cognitive attributes. Desirable plans for stress relief are generated by the system’s decision-model. Decision-model: A method for defining a decision-model for picking suitable actions to influence emotions is to conduct knowledge elicitation processes [60] with relevant experts, such as psychologists. By consulting experts (as well as relevant psychological theories), we can understand how to appropriately deal with emotional change and capture this knowledge in explicit decision rules, which help the agent anticipate how the mental model of the individual will change and pick suitable actions in each state of interaction. A decision-model can be modeled in the structure of a transition system [61] (consisting of states h & transitions between states), in which each state corresponds to a set of emotions and transitions correspond to suitable change, triggered by an action or set of actions, between emotional states. A sequence of actions that trigger transitions between states constitute a plan to reach emotions that represent stress relief (e.g., from a state representing “stress,” a set of actions and transitions lead to a state representing “calm”). Through the constraints of the transition system, the agent can decide which emotions to aim for next, depending on the current emotional state. The decision-model can, for instance, be implemented in action reasoning [61]. In action reasoning, a set of (emotion influencing) actions , are defined. An action _∈, has preconditions stating from which emotional states an action can be executed, as well as post-conditions stating the estimated emotional change (proceeding state) after the action is executed. The solutions of the planning problem (the generated plans) are returned as trajectories of length j, ⟨`,,,",`",,!,…,,.,`.⟩, which are sequences of sets of actions ⊆, and states `. of h satisfying the conditions for suitable emotional change specified by the decision-model. Interaction: The actions of , are by extension connected to specific interactive actions by the agent, depending on how it is embodied/ visualized and on the particular application, e.g. changing the behavior, facial expressions, voice feedback, or by other means influencing the individual’s beliefs, thereby changing pleasure and arousal indirectly. For instance, interactions could result in the following trajectory, consisting of emotional states and sets of actions: ⟨ s,:{Fear,Anger},,":{Pause,Speak slowly,z^_b{0| 6}f6K?_?{cj`}, g ggg`":{~6b<ca`j6``},,!:{h•{^6,hℎcJ {j`?baK?{cj`}, g ggggggggggggggggggggggg`!:{zc^^6K?6Ä,,5: {*_a`6,h•{^6,Å{jÄ <c{K6},`5:{z_^•}⟩ . (4) The interaction is dependent on reliable ways to elicit pleasure and arousal levels of the human user. 8. Conclusion In this paper, we have proposed a new computational model for affective knowledge representation that will be used for affective agents located in a multicultural environment. We’ve shown how our multidimensional model based on the pleasure-arousal dimensions can be adapted to the language and cultures where it will be used, using two experiments. The first experiment finds the most feasible translations for the affective categories which define the dimensions of pleasure and arousal. The second experiment associates regions of the pleasure-arousal space to the different emotions. The analysis of the proposed models shows the existence of significant differences in the values of pleasure and arousal associated with emotions across languages and cultures. These differences increase when the languages come from different language families. The results also show that there do not appear to be significant differences in gender or age when associating levels of pleasure and arousal with emotion labels. In addition, we have evidenced how the distribution of emotions does not follow a uniform pattern in the pleasure-arousal space. The use of models based on points or distances that are grounded on a uniformity criterion can produce misinterpretations or biases that will affect the perception, interpretation, or simulation of affective states in intelligent agents. These results have been used to propose a new cross- Page 16 / 19 Computational Affective Knowledge Representation for Agents Located in a Multicultural Environment cultural multidimensional model for affective knowledge representation in affective computing based on probability regions. This multidimensional model provides affective agents with enhanced capability of emotional differentiation, which is increasingly important in phase with the advances of NLP-based conversational interfaces. Also, we have proposed a cross-cultural human-aware planning scenario. The agent represents the affective state of the user as a function of its cultural environment. Then, the agent uses the vector representation to guide its actions in order to influence the user’s affective state and redirect it towards a target affective state. This work presents some limitations derived from the multicultural nature of emotions, namely the models are valid only for the target languages. The models are also limited by the number of emotions considered as well as the dimensions used. That said, we are currently working on the design of an affective multicultural affective agent using this model. Author’s Contributions Conceptualization, JT, AB, DD, EV, PN, JCN, VB; Investigation & methodology, JT, AB, DD, EV, PN, JCN, VB; Software, JT, AB, DD, EV, PN, JCN, VB; Validation, JT, AB, DD, EV, PN, JCN, VB; Visualization, JT, AB, DD, EV, PN, JCN, VB; Writing of the original draft, JT, AB, DD, EV, PN, JCN, VB; Supervision, EV, PN, JCN, VB. Funding This work is partially supported by the Spanish Government project PID2020-113416RB-I00 and TAILOR, a project funded by EU Horizon 2020 under GA No 952215. Competing Interests The authors declare that they have no competing interests. References [1] R. W. Picard, Affective Computing. Cambridge, MA: The MIT Press, 1997. [2] G. Gaudi, B. Kapralos, K. C. Collins, and A. Quevedo, “Affective computing: an introduction to the detection, measurement, and current applications,” Advances in Artificial Intelligence-based Technologies. Cham, Switzerland: Springer, 2022, pp. 25-43. https://doi.org/10.1007/978-3-030-80571-5_3 [3] L. Schoneveld, A. Othmani, and H. Abdelkawy, “Leveraging recent advances in deep learning for audio-visual emotion recognition,” Pattern Recognition Letters, vol. 146, pp. 1-7, 2021. https://doi.org/10.1016/j.patrec.2021.03.007 [4] R. Zall and M. R. Kangavari, “Comparative analytical survey on cognitive agents with emotional intelligence,” Cognitive Computation, vol. 14, pp. 1223-1246, 2022. https://doi.org/10.1007/s12559-022-10007-5 [5] J. A. Russell, “Core affect and the psychological construction of emotion,” Psychological Review, vol. 110, no. 1, pp. 145-172, 2003. https://doi.org/10.1037//0033-295x.110.1.145 [6] T. P. Jung and T. J. Sejnowski, “Utilizing deep learning towards multi-modal bio-sensing and vision-based affective computing,” IEEE Transactions on Affective Computing, vol. 13, no. 1, pp. 96-107, 2022. https://doi.org/10.1109/TAFFC.2019.2916015 [7] F. Schweitzer, T. Krivachy, and D. Garcia, “An agent-based model of opinion polarization driven by emotions,” Complexity, vol. 2020, article no. 5282035, 2020. https://doi.org/10.1155/2020/5282035 [8] M. S. Zitouni, P. Lee, U. Lee, L. J. Hadjileontiadis, and A. Khandoker, “Privacy aware affective state recognition from visual data,” IEEE Access, vol. 10, pp. 40620-40628, 2022. https://doi.org/10.1109/ACCESS.2022.3165622 [9] J. A. Russell, M. Lewicka, and T. Niit, “A cross-cultural study of a circumplex model of affect,” Journal of Personality and Social Psychology, vol. 57, no. 5, pp. 848-856, 1989. https://doi.org/10.1037/0022-3514.57.5.848 [10] D. Sunar, S. Cesur, Z. E. Piyale, B. Tepe, A. F. Biten, C. T. Hill, and Y. Koc, “People respond with different moral emotions to violations in different relational models: a cross-cultural comparison,” Emotion, vol. 21, no. 4, pp. 693-706, 2021. https://doi.org/10.1037/emo0000736 Human-centric Computing and Information Sciences Page 17 / 19 [11] S. Bursic, G. Boccignone, A. Ferrara, A. D’Amelio, and R. Lanzarotti, “Improving the accuracy of automatic facial expression recognition in speaking subjects with deep learning,” Applied Sciences, vol. 10, no. 11, article no. 4002, 2020. https://doi.org/10.3390/app10114002 [12] B. Alfonso, E. Vivancos, and V. Botti, “Toward formal modeling of affective agents in a BDI architecture,” ACM Transactions on Internet Technology, vol. 17, no. 1, article no. 5, 2017. https://doi.org/10.1145/3001584 [13] H. Zhang and M. Xu, “Multiscale emotion representation learning for affective image recognition,” IEEE Transactions on Multimedia, vol. 25, pp. 2203-2212, 2022. https://doi.org/10.1109/TMM.2022.3144804 [14] J. A. Russell, “Psychological construction of episodes called emotions,” History of Psychology, vol. 24, no. 2, pp. 116-120, 2021. https://doi.org/10.1037/hop0000169 [15] P. E. Ekman and R. J. Davidson, The Nature of Emotion: Fundamental Questions. New York, NY: Oxford University Press, 1994. [16] P. Ekman, “An argument for basic emotions,” Cognition & Emotion, vol. 6, no. 3-4, pp. 169-200, 1992. https://doi.org/10.1080/02699939208411068 [17] G. M. Smith and J. Carette, “What lies beneath: a survey of affective theory use in computational models of emotion,” IEEE Transactions on Affective Computing, vol. 13, no. 4, pp. 1793-1812, 2022. https://doi.org/10.1109/TAFFC.2022.3197456 [18] W. J. Villano, A. R. Otto, C. E. Ezie, R. Gillis, and A. S. Heller, “Temporal dynamics of real-world emotion are more strongly linked to prediction error than outcome,” Journal of Experimental Psychology: General, vol. 149, no. 9, pp. 1755-1766, 2020. https://doi.org/10.1037/xge0000740 [19] M. Wrbouschek and T. Slunecko, “Moods in transition: theorizing the affective-dynamic constitution of situatedness,” New Ideas in Psychology, vol. 62, article no. 100857, 2021. https://doi.org/10.1016/j.newideapsych.2021.100857 [20] J. De Leersnyder, B. Mesquita, and M. Boiger, “What has culture got to do with emotions,” in Handbook of Advances in Culture and Psychology. Oxford, UK: Oxford University Press, 2021, pp. 62-119. [21] J. A. Russell, “A circumplex model of affect,” Journal of Personality and Social Psychology, vol. 39, no. 6, pp. 1161-1178, 1980. https://doi.org/10.1037/h0077714 [22] A. Mehrabian, “Pleasure-arousal-dominance: a general framework for describing and measuring individual differences in temperament,” Current Psychology, vol. 14, pp. 261-292, 1996. https://doi.org/10.1007/BF02686918 [23] A. Kazemzadeh, S. Lee, and S. Narayanan, “Fuzzy logic models for the meaning of emotion words,” IEEE Computational Intelligence Magazine, vol. 8, no. 2, pp. 34-49, 2013. https://doi.org/10.1109/MCI.2013.2247824 [24] D. T. Cordaro, R. Sun, D. Keltner, S. Kamble, N. Huddar, and G. McNeil, “Universals and cultural variations in 22 emotional expressions across five cultures,” Emotion, vol. 18, no. 1, pp. 75-93, 2018. https://doi.org/10.1037/emo0000302 [25] J. C. Jackson, J. Watts, T. R. Henry, J. M. List, R. Forkel, P. J. Mucha, S. J. Greenhill, R. D. Gray, and K. A. Lindquist, “Emotion semantics show both cultural variation and universal structure,” Science, vol. 366, no. 6472, pp. 1517-1522, 2019. https://doi.org/10.1126/science.aaw8160 [26] N. Lazzeri, D. Mazzei, L. Cominelli, A. Cisternino, and D. E. De Rossi, “Designing the mind of a social robot,” Applied Sciences, vol. 8, no. 2, article no. 302, 2018. https://doi.org/10.3390/app8020302 [27] C. Katsimerou, J. A. Redi, and I. Heynderickx, “A computational model for mood recognition,” in User Modeling, Adaptation, and Personalization. Cham, Switzerland: Springer, 2014, pp. 122-133. https://doi.org/10.1007/978-3-319-08786-3_11 [28] A. Landowska, “Towards new mappings between emotion representation models,” Applied Sciences, vol. 8, no. 2, article no. 274, 2018. https://doi.org/10.3390/app8020274 [29] J. Taverner, E. Vivancos, and V. Botti, “A multidimensional culturally adapted representation of emotions for affective computational simulation and recognition,” IEEE Transactions on Affective Computing, vol. 14, no. 1, pp. 761-772, 2023. https://doi.org/10.1109/TAFFC.2020.3030586 [30] M. X. Zhou, G. Mark, J. Li, and H. Yang, “Trusting virtual agents: the effect of personality,” ACM Transactions on Interactive Intelligent Systems, vol. 9, no. 2-3, article no. 10, 2019. https://doi.org/10.1145/3232077 [31] P. H. Bucci, X. L. Cang, H. Mah, L. Rodgers, and K. E. MacLean, “Real emotions don't stand still: toward ecologically viable representation of affective interaction,” in Proceedings of 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII), Cambridge, UK, 2019, pp. 1-7. https://doi.org/10.1109/ACII.2019.8925534 Page 18 / 19 Computational Affective Knowledge Representation for Agents Located in a Multicultural Environment [32] R. Reisenzein, “Pleasure-arousal theory and the intensity of emotions,” Journal of Personality and Social Psychology, vol. 67, no. 3, pp. 525-539, 1994. https://doi.org/10.1037/0022-3514.67.3.525 [33] F. Jimenez, T. Yoshikawa, T. Furuhashi, and M. Kanoh, “An emotional expression model for educationalsupport robots,” Journal of Artificial Intelligence and Soft Computing Research, vol. 5, no. 1, pp. 51-57, 2015. https://doi.org/10.1515/jaiscr-2015-0018 [34] L. Chen, W. Su, Y. Feng, M. Wu, J. She, and K. Hirota, “Two-layer fuzzy multiple random forest for speech emotion recognition in human-robot interaction,” Information Sciences, vol. 509, pp. 150-163, 2020. https://doi.org/10.1016/j.ins.2019.09.005 [35] S. Jain and K. Asawa, “EMIA: emotion model for intelligent agent,” Journal of Intelligent Systems, vol. 24, no. 4, pp. 449-465, 2015. https://doi.org/10.1515/jisys-2014-0071 [36] O. Abu Arqub, “Adaptation of reproducing kernel algorithm for solving fuzzy Fredholm–Volterra integrodifferential equations,” Neural Computing and Applications, vol. 28, pp. 1591-1610, 2017. https://doi.org/10.1007/s00521-015-2110-x [37] A. Kazemzadeh, “Using interval type-2 fuzzy logic to translate emotion words from Spanish to English,” in Proceedings of the International Conference on Fuzzy Systems, Barcelona, Spain, 2010, pp. 1-8. https://doi.org/10.1109/FUZZY.2010.5584884 [38] E. Batschelet, Circular Statistics in Biology. New York, NY: Academic Press, New York, 1981. [39] A. Pewsey, M. Neuhauser, and G. D. Ruxton, Circular Statistics in R. Oxford, UK: Oxford University Press, 2013. [40] S. Rao Jammalamadaka and A. SenGupta, Topics in Circular Statistics. Singapore: World Scientific, 2001. [41] J. T. Horwood and A. B. Poore, “Gauss von Mises distribution for improved uncertainty realism in space situational awareness,” SIAM/ASA Journal on Uncertainty Quantification, vol. 2, no. 1, pp. 276-304, 2014. https://doi.org/10.1137/130917296 [42] A. Hakim, S. Marsland, and H. W. Guesgen, “Statistical modelling of complex emotions using mixture of von Mises distributions,” in 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, Geneva, Switzerland, 2013, pp. 517-522. https://doi.org/10.1109/ACII.2013.91 [43] S. Palan and C. Schitter, “Prolific.ac: a subject pool for online experiments,” Journal of Behavioral and Experimental Finance, vol. 17, pp. 22-27, 2018. https://doi.org/10.1016/j.jbef.2017.12.004 [44] A. C. Crispim, R. M. Cruz, C. M. Oliveira, and A. B. Archer, “Core affect and the circumplex approach: evidence for construct validity,” Avaliação Psicológica, vol. 16, no. 2, pp. 145-152, 2017. https://doi.org/10.15689/AP.2017.1602.04 [45] A. A. Lopez, P. R. Lourenco, I. Dimas, and C. Figueiredo, “PJAWSN: escala portuguesa do bem-estar afectivo no trabalho: contributos para a sua validação,” in A emoção nas organizações. Viseu, Portugal: PsicoSoma, 2012, pp. 155-178. [46] I. Knez and S. Hygge, “The circumplex structure of affect: a Swedish version,” Scandinavian Journal of Psychology, vol. 42, no. 5, pp. 389-398, 2001. https://doi.org/10.1111/1467-9450.00251 [47] Priberam Dictionary, “Priberam English–Swedish dictionary,” 2023 [Online]. Available: https://dicionario.priberam.org. [48] Infopedia Dictionary, “Infopedia English–Portuguese dictionary,” 2023 [Online]. Available: https://www.infopedia.pt. [49] Bab Dictionary, “Bab English–Swedish dictionary,” 2023 [Online]. Available: https://sv.bab.la. [50] Synonymer Dictionary, “Synonymer Swedish dictionary of synonyms,” 2021 [Online]. Available: https://www.synonymer.se. [51] D. Buttarazzi, G. Pandolfo, and G. C. Porzio, “A boxplot for circular data,” Biometrics, vol. 74, no. 4, pp. 1492-1501, 2018. [52] L. F. Barrett, J. Gross, T. C. Christensen, and M. Benvenuto, “Knowing what you're feeling and knowing what to do about it: mapping the relation between emotion differentiation and emotion regulation,” Cognition & Emotion, vol. 15, no. 6, pp. 713-724, 2001. https://doi.org/10.1080/02699930143000239 [53] R. J. Larsen and E. Diener, “Promises and problems with the circumplex model of emotion,” in Emotion. Thousand Oaks, CA: Sage Publications, 1992, pp. 25-59. [54] L. Sjoberg and E. Engelberg, “Emotional intelligence: theory and empirical research in a psychological perspective,” SSE/EFI Working Paper Series in Business Administration, Stockholm School of Economics, Stockholm, Sweden, 2003. Human-centric Computing and Information Sciences Page 19 / 19 [55] A. Kaver, KBT i utveckling: en introduktion till kognitiv beteendeterapi. Stockholm, Sweden: Natur & Kultur, 2006. [56] B. Peeters, Semantic Primes and Universal Grammar: Empirical Evidence from the Romance Languages. Amsterdam, Netherlands: John Benjamins Publishing Company, 2006. [57] G. B. Gil, A. B. de Jesus, and J. M. M. Lopez, “Combining machine learning techniques and natural language processing to infer emotions using Spanish Twitter corpus,” in Highlights on Practical Applications of Agents and Multi-Agent Systems. Heidelberg, Germany: Springer, 2013, pp. 149-157. https://doi.org/10.1007/978-3-642-38061-7_15 [58] M. Cirillo, L. Karlsson, and A. Saffiotti, “Human-aware task planning: an application to mobile robots,” ACM Transactions on Intelligent Systems and Technology, vol. 1, no. 2, article no. 15, 2010. https://doi.org/10.1145/1869397.1869404 [59] J. Du, J. Huang, Y. An, and W. Xu, “The relationship between stress and negative emotion: the mediating role of rumination,” Clinical Research and Trials, vol. 4, no. 1, pp. 1-5, 2018. https://doi.org/10.15761/CRT.1000208 [60] N. R. Shadbolt and P. R. Smart, “Knowledge elicitation: methods, tools and techniques,” in Evaluation of Human Work. Boca Raton, FL: CRC Press, 2015, pp. 163-200. [61] M. Gelfond and V. Lifschitz, “Action languages,” Computer and Information Science, vol. 3, no. 16, pp. 1-16, 1998.