An emotion-based agent architecture

Luís Morais Sarmento An Emotion-Based Agent Architecture Tese submetida à Faculdade de Ciências da Universidade do Porto Para obtenção do grau de Mestre em Inteligência Artificial e Computação sob orientação do Professor Eugénio de Oliveira Departamento de Ciência de Computadores Faculdade de Ciências da Universidade do Porto Março / 2004 3 aos meus pais Maria Helena e Luís Carlos 4 5 AGRADECIMENTOS Este trabalho nunca teria sido possível sem a ajuda e contribuição a vários níveis de pessoas que certamente não conseguirei nomear na totalidade. Em primeiro lugar, gostaria de agradecer ao Professor Eugénio de Oliveira pelo confiança depositada em mim, pelo constante desafio que me colocou, pela forma esclarecida e objectiva como ia levantando os problemas e apontando os caminhos a seguir, e por me ter permitido expandir os meus horizontes em variadas direcções. Gostaria de agradecer a toda a equipa do Núcleo de Inteligência Artificial Distribuída e Robótica (NIAD&R) da Faculdade de Engenharia pelo constante encorajamento, pelos comentários e ideias e pelo agradável ambiente de trabalho que sempre proporcionaram. Gostaria de agradecer em particular ao Daniel Moura, com o qual tive o prazer de colaborar, pelas inúmeras discussões e trocas de ideias sobre Agentes Emocionais. Relativamente à execução do relatório gostaria também de agradecer à Cristina Sá, pela ajuda na paginação e execução gráfica e pelo longo encorajamento que me proporcionou, e também à Ana Sofia Pinto, pela ajuda na edição em inglês e pela inesgotável fonte de boa disposição. Gostaria de deixar também uma palavra de agradecimento à Professora Belinda Maia e à Professora Diana Santos pela paciência e pelo encorajamento e suporte que me deram durante este tempo que tenho pertencido à equipa da Linguateca. Agradeço a todos aqueles que durante execução deste trabalho tiveram a paciência de me apoiarem incondicionalmente e aos quais eu fui roubando o tempo e a energia que me mereciam: Cristina, Ricardo, Sahra e tantos outros. Agradeço em especial à Teresa toda a força que me dá. Por fim, agradeço aos meus pais, ao meu irmão, à minha avó e à minha restante família por todas as coisas que estas páginas não chegariam para descrever. 6 7 The question is not whether intelligent machines can have any emotions, but whether machines can be intelligent without any emotions. The Society of Mind - Marvin Minsky 8 9 RESUMO A Emoção Humana estabelece um conjunto de interacções funcionais com a Cognição. Ao contrário do que foi durante muito tempo defendido, estas interacções foram identificadas como essenciais para a emergência de comportamento inteligente em ambientes complexos e com requisitos de tempo-real. Este trabalho procura introduzir em Arquitecturas de Agentes Autónomos mecanismos com funcionalidades semelhantes aquelas que as Emoções possuem na Cognição Humana, de forma a permitir melhorar o comportamento de Agentes Autónomos em ambientes complexos, próximos da realidade que nos rodeia. Tal arquitectura poderá ser denominada de Arquitectura de Agentes baseada em Emoções. Começaremos por discutir a motivação para este trabalho assim como colocar o nosso trabalho numa plano mais vasto que cruza a Inteligência Artificial, a Psicologia e a Neurologia. Em seguida será feito um resumo das propriedades funcionais da Emoção Humana que tomamos como base para a construção de um modelo de Mecanismos Emocionais. Iremos também apresentar algum do trabalho que tem sido desenvolvido por investigadores da Inteligência Artificial com objectivos semelhantes ao nosso, e que nós consideramos mais relevante e inspirador. Em seguida iremos apresentar um modelo genérico de Mecanismos Emocionais que permite a implementação de 4 propriedades funcionais da Emoção, nomeadamente: (i) a utilização da Emoção como Informação, (ii) a utilização da Emoção como mecanismo de Controlo de Processos Cognitivos, (iii) a Emoção como mecanismo para a Distribuição de Recursos Computacionais e (iv) como método para definição de uma Estratégia Global de Processamento de Informação. Seguidamente, apresentaremos o simulador de fogos florestais Pyrosim, que serviu de base para o desenvolvimento da Arquitectura de Agentes baseada em Emoções. Esta plataforma foi desenvolvida de raíz e permite simular um ambiente em que os Agentes cooperam para combater um fogo florestal. A complexidade e os requisitos de tempo-real do Pyrosim tornam evidente a necessidade da utilização de Mecanismos Emocionais, proporcionando desta forma uma base conveniente para o desenvolvimento de uma Arquitectura de Agentes baseada em Emoções. Prosseguiremos com a apresentação da Arquitectura, que se encontra adaptada ao ambiente proporcionado pelo simulador Pyrosim. Serão descritos os módulos que a compõem e a forma como estes são divididos em duas camadas sobrepostas. Será explicada a forma como os Mecanismos Emocionais estão integrados nesta Arquitectura e o modo como permitem suportar as propriedades funcionais das Emoções Humanas. Explicaremos como 3 Mecanismos Emocionais concretos, “Medo”, “Ansiedade” e “Auto-Confiança”, foram instanciados na Arquitectura e proporcionam uma vantagem efectivam em termos de adaptabilidade e gestão de recursos computacionais. Serão explicadas interacções funcionais (a alto e a baixo nível) entre os Mecanismos Emocionais e os módulos funcionais específicos da Arquitectura e a forma como elas se repercutem nos comportamentos de alto nível. 16 7.4.1. The Elicitation 127 7.4.2. The Interactions 128 7.4.3. Additional remarks 130 7.5. Self-Confidence 130 7.5.1. The Elicitation 130 7.5.2. The Interactions 132 7.5.3. Additional remarks 133 7.6. Summary and Conclusions 134 Chapter 8. Conclusions 135 8.1. Overview 135 8.2. Current State of Implementation and Limitations 136 8.3. Future Research Direction 137 References 139 Appendix A. The Mathematical Model of Pyrosim 145 A.1. Introduction 145 A.2. Some facts about forest fires 145 A.2.1. The Process of Ignition 145 A.2.2. The Spreading of the Fire-Front 146 A.2.3. The Point of Extinction 149 A.3. Pyrosim Simulator 150 A.3.1. Topography 150 A.3.2. Vegetation 151 A.3.3. Fire Propagation 153 A.3.4. Revision of Heat/Temperature Exchange Equations 159 A.3.5. The Extinction Process in Pyrosim 161 A.4. Modeling Agent Mobility In Pyrosim 162 Chapter 1General Introduction 17 CHAPTER 1. GENERAL INTRODUCTION 1.1. Introduction In this thesis report we introduce an Agent Architecture that includes a set of Emotional-like Mechanisms1 that deeply interact with the operating modes of the Agent similarly to the way Emotion interacts with Human cognition. These Emotional-like Mechanisms try to functionally adapt the Agent’s information processing capabilities in order to improve its global performance in the environment. We follow the view of those arguing that these Emotional-like mechanisms become a requirement for intelligent behavior for resource-bounded Agents operating in complex, dynamical and real-time environments. According to several theories developed in the fields of neuroscience and psychology, Emotions are crucial for the feasibility of the Human intelligence apparatus in the real world [Dam94][Fri86]. Humans are undoubtedly resource bounded-agents with multiple simultaneous goals. Nevertheless, they are able to live with some degree of success in very complex environments, performing actions in real-time. Emotions indeed participate in the intelligence processes required for this success. Following this line of reasoning, we have developed an Agent Architecture that includes mechanisms functionally analogous to Human Emotion but applied to Synthetic Agents. As we will try to show in the following Chapters, Emotion is a complex subject that crosses several fields of knowledge including Psychology, Neuroscience, Artificial Intelligence and Philosophy. In each of these fields, one will most certainly find many different perspectives on the study of Emotion and different schools of thought, whose ideas are sometimes opposing or difficult to conciliate. Therefore, whoever starts a study in this field faces not only a large amount of almost contradictory literature, but also an ideological and terminological nightmare. The ideas supporting this thesis are rather dispersed over such literature. In this thesis, much of the effort has been devoted to mining the field in order to build a coherent view of the subject. However, we must emphasize that along this process there has always been an effort to keep an engineering view about Emotion. This means that each idea or model used or developed would always have to make sense from a functional 1 Throughout this thesis the term “Emotion” will be applied only when referring to Human Emotion. We will use the term “Emotion-like Mechanism” or simply “Emotional Mechanism” when mentioning any computerized model or implementation based on Human Emotion. 18 point of view. This way, we obviously come closer to the perspective of the Functionalists, with all the inherent advantages and disadvantages [Mof00]. A detailed discussion about the Functionalities of Emotions will be developed in Chapter 2. In the following sections of this chapter we will explain the motivation behind this work. We will also try to place our work in the wider panorama of the study of Emotion within Artificial Intelligence or, more generally, Computer Science. We will also make a very brief description of our testing scenario, the Pyrosim environment, through which we have tested our Emotion-based Architecture. Finally, we will present the outline of the main body of this thesis. 1.2. Motivation The beginning of this work has been greatly inspired by Damásio’s ideas about the role of Emotion and Feelings in the rational processes of intelligence, described in the book “Descartes’ Error” [Dam94]. Although Damásio has not been the first author to address these interactions, his book has had the merit of attracting both the specialized audience and the general public to the subject of Emotion and, therefore, the merit of starting a wide debate about this subject. Our interest in Emotion came in a rather natural way after reading Damásio’s book. The ideas described in it suggested the existence of some very efficient engineering processes performed by the Emotional mechanisms. The connection between Emotion and decision-making and Emotion and planning, described by Damásio, immediately drew our attention, as both these capabilities are central to the field of Autonomous Agents. The implication that Emotion would not only help Humans on deciding and planning their actions rationally but also that its absence would render most of the decision-making and planning processes unfeasible in the real world, seemed very intriguing. At the same time, these ideas became very appealing if we considered that Humans, despite of the limited intellectual and physical resources they have available, are reasonably successful Autonomous Agents: they are capable of operating in real-time in a very complex, dynamic and uncertain environment, the real world. Since the “real world” is the ultimate application scenario for Autonomous Agents, it was certainly worth looking at Emotion, whatever this could be, as a possible interesting and useful contribution to Agent Architectures. The relationship between Emotion and decision-making became even more interesting if we considered that the Human brain is a highly distributed and massively parallel system. Each intelligence-based process involves several brain (and also body) components that work both in parallel and cooperatively in order to produce a final result. For example, a decision-making process would require at one stage a method for assessing the current situation as well as detecting which features are most relevant for the decision. This would require both the analysis of the state of the world and of one’s previous experiences. Thus, a mechanism optimized to access memory for “relevant” information according to the current situation would also be needed. At the same time, one Chapter 1General Introduction 19 would draw a projection in the future of several possible courses of action, based on past experiences and also according to the notion of how the world may change after our actions. The evaluated options would take into account one’s personal interests and preferences, social rules and norms, the notion of one’s own capabilities and several other criteria. If we now consider that each of these sub-processes is itself complex, depending on multiple parameters, and possibly making use of other complex capabilities running in parallel, we may conclude that trying to obtain a coherent response from all elements involved is a major coordination challenge. Achieving it, is a huge engineering accomplishment. Damásio pointed out that Emotion plays a very significant role in the success of these type of interactions, for example by speeding up memory searches using Emotional cues, by reducing the space of analysis to a set of reasonable hypothesis, or by providing a way to compensate the uncertainty of data gathered by perception. As engineers working in the field of Autonomous Agents, it was obvious for us that there was a strong engineering potential in several of the concepts described by Damásio for the development of better Agent Architectures. Most of the present proposals for Agent Architectures have been inspired, at least remotely, on the Human brain and explore the knowledge gathered about its structural and functional properties. However, it is rather surprising to observe that until very recently few were the cases where Emotionbased concepts were clearly included as key concepts of the Agent Architecture. Some notable exceptions include the work of Herbert Simon [Sim67] and Aaron Sloman [Slo81] who have pioneered this field and whose work and ideas have been great sources of inspiration for more recent researchers. Several reasons may explain this apparent lack of interest in Emotion. The first one is related to the common and long debated misconception that Emotion is the opposite notion of Reason. Acting Emotionally has always implied that one’s action would be impulsive and unthought, and therefore always less appropriate or efficient than action taken after a long Reasoning process. Emotion and Reasoning have historically been placed on opposite fields. The only interaction Emotion was considered to have with Reasoning was clearly of a destructive nature. The other reason for the absence of Emotion-based concepts in Agent Architectures is quite basic: we simply did not know enough about Emotion to build a comprehensive model. It was not until the mid 80’s that researchers had the chance to observe the Brain performing in real-time using very powerful image acquisition and visualization techniques such as CAT, MR and PET [Mau93]. These techniques allowed researchers to thoroughly analyze the structure of the brain and to visualize its activity patterns and internal interactions, providing deeper insights into Emotion machinery. In spite of these undeniable advances, Emotion is still a big mystery. Many basic questions remain to be answered, including, for example, the concise definition of Emotion itself. The nature of Emotion is still under intense debate [Ekm94] and there is no assurance that a definite answer will 20 ever be found. There are still many questions to be answered. We will try to address some of them in the next chapter, which is completely dedicated to Emotion. One issue, however, may now be considered to be completely settled in this debate: it is no longer possible to draw a complete picture of Intelligence excluding the processes related to Emotion. It is precisely at this point that artificial intelligence researchers reenter the scene, nearly 30 years after the work developed by Herbert Simon. When Agent Architectures are applied in the development of intelligent Robots, researchers find similar implementation problems to those that would be found in the hypothetical design of the Human brain. In fact, Autonomous Robots share many design requirements with Humans (although simplified). Robots should be able to operate in complex environments, subjected to some time constraints, mechanical limits, energy storage limits and, of course, limited computational power. At the same time, the world outside the Robot is complex, noisy, constantly changing and demands attention to address multiple simultaneous needs or concerns. During the last decade, researchers have produced a considerable number of interesting works in which Emotional-based concepts were applied in Agent Architectures. We will make a revision of some of these works in Chapter 3. Still, it is worth mentioning some authors whose work has greatly influenced our ideas, either by their functional and practical perspective or, on the other hand, by their high-level and integrating approach. For example, Velásquez [Vel98] has applied Emotional-based mechanisms to control behavior selection of a physical robot named Yuppi. Cañamero [Cañ97][Cañ00] has also developed some very interesting work in modeling and applying Emotionbased concepts to action-selection in software Agents. Gadanho & Hallam[Gad98] and Gadanho[Gad02] built a simulation where they have applied Emotional Mechanisms to perform reinforcement learning. From a more theoretical point of view, Aaron Sloman and his students in the Cognition and Affect Group, University of Birmingham [Wri97][Bea94][Slo00] followed a designbased approach to develop Agent Architectures. These Architectures were based on the study of the requirements needed for building systems with Human-like intelligence. In such Architectures, Emotion-like manifestations would arise in the system as a natural result of interactions established between architectural components. Besides their strong scientific contribution, these works, among others, had the merit of bringing the study of Emotion back to AI, after the initial works developed in the 60’s. However, the complexity surrounding Emotion has, most of the times, led AI researchers to focus on very specific Emotion-Cognition interactions such as, for instance, interactions between Emotion and Learning or between Emotion and Action-Selection. In our opinion, this specificity, although almost unavoidable at that stage, has had three undesirable effects regarding the development of Autonomous Agent Architectures: 1. the over-fitting of the models of Emotion-Cognition interaction, which prevented their easy transposition to other application scenarios or even to other possible Emotion-Cognition interactions within the same scenario; Chapter 1General Introduction 21 2. the inclusion of very human-specific elements in the model, such as hormones and other “organic” elements, whose direct application in the context of Synthetic Autonomous Agents, despite of its metaphorical nature, may lead to rather artificial concepts. 3. a great fragmentation of models of Emotion-Cognition interaction within the realm of Autonomous Agent Architectures, and the lack of a general, large scope model; Another thing that seemed apparent in the early steps of our work is that AI researchers tend to choose one particular “school of thought” about Emotion and do not usually consider complementing the models they develop with ideas from others “schools”. This is not to be blamed because, obviously, one should not expect AI researchers to develop a complete model of Emotion when researchers from Psychology and Neurophysiology are still debating very fundamental issues. Nevertheless, in this thesis we have tried to follow some of the latest trends in psychology [Sch00][Bow00]: that try to integrate previous contributions under a more global model. Thus, we have taken the liberty of borrowing concepts from different “ schools of thought” whenever we felt that they had a strong functional use. Having said all this, we may now present more clearly the motivation for our work, which includes essentially 3 points: 1. The study of Emotional-phenomena from a very functional point of view, trying always to explore possible practical applications in Autonomous Agent Architectures. 2. The development of a model of Emotional-based mechanisms compatible with various functional models of Emotion dispersed among artificial intelligence, psychology and neurosciences. 3. The development of an Agent Architecture where such Emotional-based mechanisms provide a performance advantage, especially for resource-bounded agents that operate in complex and dynamic environments. 1.3. Some Previous Remarks It is important to make a few considerations about the motivations presented before. Firstly, during this work we will make a great effort to develop a clear understanding about Emotion-Cognition interactions but our focus will be exclusively on possible functional aspects. This implies that a large number of Emotion-related phenomena (cognitive or physiological) will simply not be addressed. However, this also motivates us to try to filter out those aspects of Emotion that could be ignored in a first approach when thinking of Synthetic Agents. For example, when dealing with the physiological aspects of Emotion, we have tried not to transpose them directly to Synthetic Agents. Instead, we have tried to adapt them or generalize them. This adaptation follows the idea of 22 Embodied Intelligence [Dam94] where the Human body is itself a part of the global intelligence machine. Therefore, the “body” of a synthetic agent may have functionally analogous interactions with Emotion but will certainly have different implementations. For example, and just for the sake of (naïve) illustration, if the Human heartbeat is accelerated in response to a fearful situation in order to prepare the body for a possible powerful response, in the case of a Synthetic Agent this could correspond to speeding up the processor clock to cope with a “dangerous” flow of data, or boost the global performance of the Agent. On the other hand, if that heartbeat acceleration is triggered by a hormonal discharge, which functions as a global communication mechanism between the brain and the body, then, from a functional point of view, it might not make much sense to try to introduce the idea of a “synthetic hormone discharge” because a similar communication mechanism can be simply performed by means of a hardware or software interrupt. Note, however, that in both cases it would probably not be possible to sustain such a powerful state for a long time either due to certain physical limits, as in the case of Humans, or to possible energy and temperature constraints in the case of Synthetic Agents. Even by reducing the scope of our work to the study of functional interactions, we do not claim that we will try to develop a global model of functional Emotion-Cognition interactions, applicable in all cases to Synthetic Agents. That would be out of the scope of this kind of work. In spite of that, by trying to look at Emotions as abstract functions of intelligence, whose “implementation” in humans and animals is tightly related to specific physical and mental limits, one may find principles and rules of design that can be applied in the construction of Synthetic Agent Architectures. That has been the aim of our effort. 1.4. More on Emotion and Computer Science Before proceeding with the discussion regarding how Emotions may interact functionally with Agent’s internal processes, it might also be interesting to look at other perspectives on Emotion within the field of AI and Computer Science. In fact, the complexity of Emotion has motivated a great amount of work where Emotion and Emotion-based concepts play different roles in a computer system. Besides the perspective we are following, there are at least three different ones that should be considered: 1. Recognition of Human Emotion 2. Expression of Emotional-like behavior 3. Modeling and Simulation of Human Behavior The categories listed here should be seen only as a reference for the major concern of Chapter 1General Introduction 23 researchers. In practice, actual projects may fit into more than one category as these subjects tend to blend in. However, they represent perspectives on Emotion significantly different from the one we have been pursuing. The purpose of the following section is just to emphasize that difference. 1.4.1. Recognition of Human Emotion Emotion recognition is a basic feature of human intelligence. By recognizing Emotions, people extract additional information that may not be conveyed directly by speech, gesture, text or other forms of explicit communication. Emotion recognition plays a central role in decoding signals during a conversation and helps to reduce the possible ambiguity of a message. People usually communicate large amounts of Emotional information, either intentionally or unconsciously. The message becomes not only "what" is transmitted but also "how" it is transmitted. One might consider two parallel communication channels: along with a "plain" information channel, people use what we may call an "Emotional information" channel. Emotions influence our body movements and expressions. This effect is known as Sentic Modulation [Pic94]. Our motor system may be seen as the carrier of Emotional responses in the same way a high frequency sinusoid is the carrier of an AM audio signal. Sometimes, such bodily messages might be easily perceived while in other cases they might be expressed in such a subtle way that they can only be detected using special equipment. Table 1.1 Effects of Sentic Modulation (taken from [Pic94]) Apparent effects Less apparent effects Facial expression Voice Intonation Gestures and Body Posture Pupilary Dilatation Respiration Heart rate, pulse Temperature Electrodermal Fluctuation Blood Pressure Nowadays, the richness of this information is simply discarded by computers. The most common way of transmitting information to a computer is either by using simple keyboard devices, which reduce messages to a stream of well-formatted characters, or through the clicks of a mouse or other similar pointing devices. Obviously, this state reduction seriously decreases the possibility of Human-Machine Interaction but it also reduces the possible intensity of communication between Humans made through computers. Certainly, computers able to recognize the user’s Emotional states would boost interaction possibilities. Such computers would be equipped with a wide variety of sensor systems ranging from video cameras, speech recognition modules with intonation analysis, body temperature sensors, blood pressure and electrodermal measurement devices, as well as other capture devices. Computers would then be able to detect the user’s Emotional state and react accordingly. Computers could react to user 24 frustration or disorientation (expressed, for example, through facial expression and body posture). Such a feature would be most useful in Interactive Tutoring Systems and, of course, in Human Computer Interaction in general [Zho00][Ell99]. Other interesting possibilities include having computers decoding our Emotional responses during a computer-mediated communication (chat, email...) and recreating the same Emotional behavior in the receiver terminal (for example, using a 3D character). There are several very complex implementation issues in this field. These issues range from building effective physical sensors, to extract usable information from the user, to developing algorithms that analyze that data and choose the best response from the system (possibly after a learning period). For example, recognizing Human Emotion from videos of the users is not currently feasible both because of the computational power required but also because it demands very advanced Human face recognition techniques (the interested reader is invited to [Zha00] and [Bar99]). Apart from these extremely complex technical implementation issues, there are several important aspects regarding Emotion recognition that will eventually have to be considered, although possibly only in a distant future: • Sentic Modulation can be controlled or simulated and even people may have a hard time recognizing other people’s Emotions. Will computers be able to do it, or will they be easily fooled? • On the opposite corner, there is a chance, although remote, that computers will become more efficient than people in recognizing Emotions. What possibilities does this bring? • Do we want to have a computer recognizing our Emotions and use them as programmed by another person? These are very motivating questions that, however, are beyond the scope of our work. 1.4.2. Expression of Emotional-like behavior Remarkably, computer expression of Emotion is, somehow, a less problematic issue. Since we were kids we have been in contact with objects and images created to “express” Emotion as if they were actually feeling that Emotion. Cartoons and animation films are a good example of how completely synthetic characters can effectively express and communicate Emotion and behave in an Emotionally believable way. There is a huge amount of experience in this field that can easily be adapted to computers, especially with the help of visual agents, usually called Synthetic Characters. A very famous example of a Synthetic Character capable of expressing Emotional-like behavior is the Microsoft Agent technology [Mic04]. A reasonable set of Emotional expressions may be simulated using small 2D Chapter 1General Introduction 25 animation sequences triggered by user and system events. Such a Synthetic Character can effectively express a wide range of Emotional states such as surprise, anger, happiness or virtually any other Emotion that can be expressed by a traditional 2D cartoon. Other more advanced systems involve using 3D models to create Synthetic Characters. 3D techniques have the advantage of allowing developers to create realistic characters whose appearance can be parameterized, allowing almost complete control over face expression, body posture and movements, in real-time if needed. Interesting examples of these systems include EMOTE [Cos00] or the system developed at MIT and described in [Rus99]. This type of technology is now widely established among developers and researchers. In fact, very similar technology is currently used in the computer game industry, and several vendors provide complete software packages that can be adapted for this purpose [Bos]. The major reason for trying to develop Synthetic Characters capable of expressing Emotions is concerned with believability [Bat94]: characters that act Emotionally and whose appearance changes with Emotion become more real, more engaging and look as if they were alive. As pointed out in [Kli99], the expression of Emotions helps to reveal a character’s inner side and to induce certain expectations on the user. Therefore, Synthetic Characters become particularly believable if their behavior conforms to those implicit expectations. For a good theoretical discussion concerning Believability and Emotions refer to [Ort01]. The issue of Believability becomes particularly important if we think people tend to spend a very significant amount of their time working with or using a computer. Synthetic Characters could help to increase the quality of such time not only by providing a user-friendlier interface but also by “humanizing” the interaction patterns between man and machine. It is important to note that although Synthetic Characters’ global behavior and visual appearance may look very complex, the underlying model of Emotions may not be as complex. In fact, a set of rules that maps events and conditions to plausible Emotional States2 and another set of rules that maps back Emotional States to behaviors (or changes in the visual appearance) may be enough to provide the Synthetic Characters with an Emotional-like behavior. Naturally, these rules are integrated in the global architecture that is usually complex and includes additional mentalistic concepts (drives, goals, beliefs, etc) and other advanced capabilities that provide greater depth to the Agent. For example, in order to adapt themselves to users or to their virtual environment, many of these Synthetic Characters require some learning capabilities. It is interesting to note that such learning methods need to cope with a complex environment as well as being able to deal with the existence of certain intentions or personality traits of the Agent [Yoo00]. Most of the work developed in the field of Synthetic Agent and Virtual Environments concerning Emotion has been based on simplified versions of the model of Emotions proposed by Ortony, Clore and Collins [Ort88], which is very convenient for rule representation. We will refer to 2 For now, let us assume that an Emotional State is simply a vector containing the values of certain internal variables. Chapter 2Human Emotion 33 CHAPTER 2. HUMAN EMOTION 2.1. Introduction In this chapter, we will address several subjects regarding Human Emotion. This will not, of course, be an exhaustive analysis of Human Emotion. We intend to present some of the properties of Human Emotion that have been described in literature and that are especially relevant for developing a functional approach to Emotion. Throughout this chapter, we will try to generalize the concepts introduced whenever we think it is possible to do so. We will use the word Agent if we think that is possible to generalize a given topic about Human Emotion in order to include it in a Synthetic Agent. In some sections we will be using the terms “Agent” and “Human” almost interchangeably. We will begin by analyzing the context in which Emotions play a functional role: the real world. Next, we will introduce the concept of Emotional State. We will then try to establish a connection between Emotion and three related aspects: Goals, Capabilities and Environment. Our notion of Emotional Agent will be introduced and we will explain the process of Emotional Elicitation. We will proceed by taking a closer look at Human Emotional phenomena and divide them into three different categories: Specific Emotions, Moods and Emotional Dispositions. We will then focus on the functional properties of Human Emotion according to what has been described in specialized literature. The aim of this Chapter is then to define those concepts and properties and try to define them in order to make them applicable to Autonomous Software Agents. Finally, we will present this Chapter’s summary and conclusions. 2.2. A Context for Emotion In order to clarify the reasons why Emotion may have functional properties, let us first analyze a context from where Emotion emerges: Human Agents in the real world. Using the taxonomy provided by [Rus95], the real world may be classified as an environment with the following properties: 34 • Inaccessible: the Agent’s sensors cannot detect all relevant information for deciding its actions. • Non-Deterministic: although there may be some philosophical debate about the deterministic or non-deterministic nature of the Universe, we consider that from a Human Agent’s point of view a real-world environment appears to be frequently non-deterministic. • Non-episodic: the environment does not evolve in well-contained time-slots where we can first perceive and then act. • Dynamic: real-world environment is always changing even if we do not act upon it. • Continuous: in a real-world environment, all parameters range over a continuous set of possibilities. In [Rus95], the authors also present some examples of environments in accordance with all, or at least with most, of the afore-mentioned properties: “taxi driving”, “medical diagnosis system”, “refinery controller”, or “interactive English tutor”. It is also interesting to observe that on the opposite end we would find environments such as “ chess”, “poker”, “backgammon” and “image analysis systems”. On the other hand, Humans are resource-bounded Agents. We have limited intellectual power - including limited memory and inference capabilities - limited perception mechanisms, limited physical capabilities, as well as reasonably severe physiological constraints. At the same time, we have multiple simultaneous goals to be achieved and needs to be satisfied. Besides our permanent goal of maintaining our biological health, numerous personal (physical and intellectual) and social goals emerge from our interaction with the world and need to be attained. With such an apparent mismatch between our many goals and our limited capabilities, it seems quite reasonable to admit that Human Agents should have little chances of being successful in the real world (for an interesting and deeper approach on this subject refer to [Mof00]). 2.3. Emotional State Before proceeding with the discussion on Human Emotion, we shall make some previous assumptions for the sake of generalization. We now introduce, at this point, the notion of Emotional State, fundamental for the subsequent sections. Let us first assume that an Agent operating in a complex environment requires a set of specific capabilities to deal effectively with such complexity. These capabilities allow the Agent to endure in the environment and address multiple other needs. Let us also assume that if such capabilities, although limited, are reasonably sophisticated (either as a result of a long process of evolution or as a consequence of thorough design and implementation), they are almost certainly related to various operating parameters that control their individual activity. Chapter 2Human Emotion 35 The results achieved by the Agent in the environment will depend on how the specific parameters of its individual Capabilities are instantiated. If several of such capabilities exist, intended to address multiple needs, the Agent will be required to achieve a global state where all operating parameters of all available Capabilities are optimally instantiated. Definition 1: An Agent’s Emotional State is a particular instantiation of operating parameters for the complete set of Capabilities (physical or cognitive) it possesses. An Emotional State defines a global operating context in which Agent’s available resources are concentrated in the achievement of a specific Goal or set of Goals. We will now proceed with a more specific discussion on Human Emotion where we will try to introduce some additional concepts to reinforce the functional perspective of Emotion. 2.4. Emotion and Situated Agents The work developed by Damásio [Dam94] established a strong relationship between the Human ability to deal with real world situations and Emotions. Damásio undertook several studies in his patients that had suffered damages in specific brain structures (pre-frontal cortex). As a result from such damages, these patients had developed major changes in their emotional behavior and were unable to respond to emotionally rich stimuli, such as, for instance, violent or sexual content images. Despite showing average or above average results in regular laboratory IQ tests, these patients were very unsuccessful when confronted with situations that required them to make real-world decisions, either at a personal, social, or professional level. Damásio concluded that Emotion should play a decisive role in allowing humans to deal with the complexity of the real world. Emotions3 would speed up Cognitive processes, creating various kinds of shortcuts for deliberation in a given situation. Without such shortcuts people seem to be unable to deal with the complexity of the real world with the usual time constraints: Emotions provide mediation mechanisms between cognitive capabilities and the environment. Interestingly, the notion of Emotions as a mediation mechanism between a Situated Agent and its environment had already been pointed out by Fridja: Many emotions can be defined by an intentional structure: that of maintaining or changing a given kind of situation, and to doing that in certain ways, with respect to certain aspects 3 At this stage, we will not be providing a formal description of Emotion. In fact, throughout this thesis we will avoid giving an exact definition of what an Emotion is because a consensual definition is yet to be found in literature. However, in Chapter 4 we will try to more carefully formalize some of the concepts related to Emotion. 36 of the given situation and with regard to given kinds of objects in those situations. [Frijda, The Emotions, p. 98] This perspective can be better understood by calling attention to the simultaneous link that Emotion establishes with the perception the Agent has of the environment and simultaneously of its own capabilities to deal with it. As mentioned in [Mof00], Emotions are connected to two essential issues. Firstly, Emotions are related to the identification of internal or external events/situations that are relevant to specific Agent’s Goals (or concerns[Fri86]). Such situations may be considered either positive (in the case of opportunities) or negative (in the case of threats), regarding the Agent’s chances of achieving those Goals. Additionally, Emotions are simultaneously related to the assessment of current capabilities and resources (or coping potential [Fri86]) that the Agent has or can make available to deal with the identified situation. According to the authors, a given Emotion will emerge depending on the relation established between the two previous parameters. For example, the Fear Emotion would result from the identification of a situation or event that is particularly dangerous to the Agent (i.e. to a Goal) and, simultaneously, from the fact that the Agent has limited capability (i.e.: physical or computational resources.) to respond appropriately to that situation. The emergence of Emotions according to certain conditions is a process known in literature as Emotional Elicitation, which we will address in the next section. We will now make our first generalization regarding Human Emotion based on the previous definition of Emotional State. Generalization 1: An Agent reaches a given Emotional State depending on its perception of how favorable the environment is for the achievement of its Goals, considering its own current resources. The Emotional State generates a global operating context appropriate for the achievement of a given set of Goals and that, at same time, optimizes resource allocation for that purpose. Based on this generalization we may now introduce another fundamental definition: Definition 2: An Emotional Agent is an Agent that is able to operate under different Emotional States. An Emotional Agent will always be operating under one specific Emotional State. The implications of such definition will become clear in the next sections. Nevertheless, it is possible to say that, according to this definition, Humans may be considered Emotional Agents. Let us now take a closer look at the process of Emotional Elicitation. Chapter 2Human Emotion 37 2.5. The Process of Emotional Elicitation Emotional Elicitation is a matter that generates great controversy among researchers [Ekm94, Pic97]. The main issue here regards the basic nature of the elicitation process. Some researchers, especially those coming from the field of psychology, advocate that Emotional Elicitation itself requires some form of high-level cognitive processing of the corresponding antecedents (appraisal theories [Fri86, Ort88]). On the other hand, researchers coming from the field of neurobiology [Led96] defend that some emotions can arise from physiological or lower level stimuli that do not involve any high-level cognitive circuitry (neocortex or hippocampus). However, there is much less dispute around this issue if a wider concept of “form of cognition” is considered, one that includes both high-level cognitive processes and also basic sensory information processing. From this perspective, it is possible to gather the two contending views and reach a comfortable compromise. We will also adhere to this broader concept of “ form of cognition”, believing that it is possible to devise a general functional model of emotional mechanisms irrespective of the nature (deliberative or physiological) of the Emotional Elicitation process. As we previously mentioned, Emotions are deeply related to the evaluation of the Agent’s own capabilities to achieve a given goal. Goal, in this context, includes not only explicit formulations of desirable future states that the Agent seeks to achieve, but also implicit states that must be reached or maintained. Emotions reflect the Agent’s capability (knowledge, action set, physical robustness, etc.) to cope with the current state of the environment when trying to achieve one or more of its goals. This concept is known as coping potential [Fri86] and is essential to a functional view of Emotions. For instance, Frustration may reflect a systematic inability to deal with the environment when attempting to achieve a certain goal. Fear may reflect the fact that the current environment situation is highly unfavorable to the accomplishment of at least one goal (probably a very important or even vital one), and that the Agent does not have the necessary capabilities to cope with the situation. It is interesting to note that Fear is considered to have a very strong neurobiological / sensory dependency (the fear generated by the approaching of a fast object or by a high volume sound) but it also has a highly cognitive dependency (the Fear of a major inversion in the market share). Nevertheless, we are obviously dealing with “ Fears” of very different nature. Therefore, and regardless of the nature of the eliciting process (cognitive or neurological), Emotion Elicitation involves the evaluation of the chances of achieving a given goal, taking into account both the state of the environment and the internal state of the Agent itself. Emotional Elicitation can be abstractly seen as a process based on an evaluation function involving: • a goal • the Agent’s capabilities • the environment state or an event. 38 The next figure illustrates these relations: Figure 2.1.Emotions may be seen as the center of a triangle formed by (i) Agent’s Goals, (ii) the State of the Environment and (iii) the Agent’s Capabilities to ensure its Goals under the current Environment. We may now introduce an additional definition that generalizes this concept in order to make it applicable to Synthetic Agents: Definition 3: The Emotional Elicitation is a process that promotes a given Emotional State after an evaluation based on an Emotional Evaluation Function (EEF). The evaluation depends on three parameters: the Agent’s set of Goals <G>, the state of the Environment <Es> and the Agent’s Internal State <Is>. If <E> denotes the change in the Agent’s current Emotional State, then: <E> = EEF(<G>,<Es>,<Is>) We will be developing this issue further in Chapter 4. For now, it suffices to say that such Emotional Evaluation Function may possibly be very simple (for example a comparison function), and the same may be said about the data contained in both the Internal State <Is> and the State of the Environment <Es>, i.e. direct sensory information instead of complex Belief-like structures. 2.6. A Simple Taxonomy of Emotional Phenomena Up to this moment, we have been referring to Emotion without paying attention to the fact that Emotion is a complex and broad concept and that there are many phenomena that we are used to indistinctly call Emotion. Emotions can be subdivided in, at least, three different categories of Emotional phenomena, namely: 1. Specific Emotions; 2. Moods; Relevance and Limitations Opportunity or Threat Emotional Phenomenon Goal Capabilities Environment Coping Potential Chapter 2Human Emotion 39 3. Emotional Dispositions. Divisions can be made according to the following criteria: • Object/Antecedent: cause or pre-conditions that trigger the emotional phenomena; • Intensity: how strong is the influence of the emotion on the Agent; • Duration: time span that includes the rise and fall of the emotional phenomena; • Consciousness: whether the Agent is conscious or not about the occurrence of the emotional phenomenon. It is important to make this division because, although all these phenomena share a common functional logic and high-level model, they have distinct impacts on cognition. Thus, for now, we shall try to differentiate them as clearly as possible, without alluding to their functional role. Specific Emotions Specific Emotions are, by definition, emotional phenomena that can be quite clearly differentiated. Although cognitive researchers are still debating which emotions exactly constitute the set of Specific Emotions [Ekm94], Anger, Fear, Joy, Surprise, Disgust are usually considered part of that set. Specific Emotions, as those we have just referred, have well-defined objects and are usually very intense. However, they usually occur over a reduced time span (few seconds or minutes). Because of their strong intensity, an Agent is, most of the times, clearly aware of its existence. Moods On the other hand, moods encompass a different set of emotional states such as Pleasant/Unpleasant, Anxious/Relaxed. Contrary to Specific Emotions, moods may have no clearly defined object or antecedent. In addition, because they are usually less intense, they may remain unconscious to the Agent for most of the time. Moods may be originated by an intense or recurrent occurrence of a Specific Emotion (e.g.: Anxiety after several fearful episodes) or simply by environmental factors (e.g.: weather). They may last for hours or even for a few days. We will also include in this category states such as Self-Confidence and Frustration, a choice that will be made clear shortly. Emotional Dispositions Emotional Dispositions (sometimes called temperament) represent yet another step regarding duration of emotional phenomena. Chronic Anxiety and Depression are examples of Emotional Dispositions. They may last for several months or years but tend to be inactive most of the time. They are assumed to be partially influenced by genetics and may appear early in life. Emotional Dispositions have also 40 been observed to arise in patients who have been prescribed certain medication or have been subjected to neuro-surgery [Dam94]. Table 2.1 Summary of Human Emotional Phenomena Emotional Phenomena Elicitation Time Duration Antecedent Intensity Specific Emotions Short Short Identifiable event or condition usually limited in time Strong Moods Medium Medium Environmental or recurrence of Specific Emotions Strong or Medium Emotional Dispositions Long Large Usually environmental or intrinsic. Difficult to identify Medium Some of the previous Emotional phenomena may occur combined: each Emotional phenomenon makes its contribution to the global Emotional State of the Agent. However, during a given time period, the influence of one single Emotional phenomenon usually prevails over the others, becoming the dominant Emotional phenomenon and, therefore, the major component of the Emotional State. This happens to create the internal context needed for the Agent to achieve the most relevant Goal at that moment. After some time, the dominant Emotional phenomenon will fade (or not), allowing other Emotional phenomenon, related to another Goal, to become dominant. In the next section, we will analyze some of the conditions that interact with these dominance changes. Some issues regarding the Dynamics of Emotional Mechanisms The typical activity period of emotional phenomena varies for each of the three categories, ranging from a few seconds (Specific Emotions) to, eventually, many years (Emotional Dispositions). Additionally, for each of those categories the elicitation process duration also has a typical value. For instance, a Specific Emotion such as Angriness may be elicited in just a few seconds (we all know that). On the other hand, Depression, which can be considered either a Mood or an Emotional Disposition, has usually much longer elicitation periods. Both the activity and duration period of emotional phenomena can be understood according to two functional requirements: urgency and temporal consistency. Fast elicitation times are functionally appropriate when the eliciting condition is an event/situation that demands an urgent response from the Agent. Note that this does not necessarily imply a reactive response from the Agent. It simply creates internal conditions, i.e. an Emotional State, to promote the appropriate answer (may it be braking a car after detecting an approaching object or quickly constructing the appropriate verbal answer to an insulting comment). On the other hand, long elicitation times are compatible with the need to keep up with slow environment drifts or with recurrent events that do not pose an urgent demand on the Agent but may have great influence in the Chapter 2Human Emotion 41 long run. As for the activity period of emotional phenomena, shorter activity periods seem appropriate when the corresponding Emotional State is not required to last, or may not be sustainable for long periods. They are also usually related with specific events or conditions that can be clearly identified in time. On the other hand, longer activity periods seem useful when there is a need to cope with situations, possibly internal ones, which may be difficult to specifically identify or whose duration may be undefined. In these cases, the Agent should keep a consistent response over a longer (probably undefined) time. 2.7. Functional Properties of Emotional Phenomena In the previous section we have briefly alluded to a functional property of Emotional phenomena, that of creating an appropriate operational context for the Agent. In this section we will try to analyze this and other different functionalities and formulate them in a more general way, so that they can be transposed to Synthetic Agents. 2.7.1. Emotion as Information Real-life environments are usually inaccessible, non-episodic, dynamic and continuous. People deal with this complexity everyday. However, when someone is asked to rate the overall life satisfaction, one will not trigger a multi-criteria evaluation process to reach a conclusion. This task would certainly involve a great amount of factors, some of them uncertain or difficult to identify and quantify. Instead, people will find the answer simply by asking themselves “ How do I feel about my life?” [Sch00]. This behavior, which may also be adopted in more specific questions, is based on the premise that our Emotional phenomena seem to be capable of condensing large amounts of dispersed information into a single and easily identifiable information unit, the Intensity of Emotionalphenomena. In complex environments, information may be difficult to obtain because sources are disperse and noisy. Alternatively, the amount of information available may be so large that Agents will simply not have enough computational resources to process it. Therefore, the information condensation provided by the feeling of Emotion can actually become a very functional property. The intensity of Emotional phenomena can be used as supplementary or alternative input to certain high-level cognitive processes, such as decision-making or learning, which would otherwise be impossible to compute or very unreliable, if based exclusively on external information. Since Emotions are essentially linked to Agents’ Goals and Capabilities, this condensation works as a very functional relevance filter. Emotional phenomena are capable of amplifying the most 48 Cognition and Affect Group (CogAff) located at Birmingham University. The CogAff group has been dedicated to the study of Agent Architectures for years and has had the involvement of several researchers with relevant work on Emotion in AI (notably Luc Beaudoin[Bea94] and Ian Wright[Wri97]). The work and ideas developed by Sloman and other members of the CogAff Group founded what is known as the “Birmingham School” in respect to the relation established between Cognition and Emotion. Sloman has addressed the study of Intelligent Agent Architectures using the “ Design-based Approach” [Slo00][Slo01]. Instead of trying to model particular Agent Architectures exclusively based on what is known about physiological structures and neurological processes involved in intelligent behavior, the “Design-based Approach” consists in exploring the space of all relevant Architectures that may explain intelligent behavior and Agent control. According to Sloman, this exploration will help us understand which Architectures are able to explain a specific form of intelligence, after being properly instantiated. In fact, Sloman seeks to develop Architectures that are capable not only of explaining Intelligence, both in Humans and in insects, but also Architectures that may effectively explain other observable phenomena, for example, in patients with brain diseases. 3.2.1. The Architecture The Agent Architecture proposed by Sloman involves the combination of two other traditional Architectures: the “three towers” model and the “ three layers” model [Slo01]. The “three towers” model comprises three parallel subsystems that are conceptually vertical within the Agent Architecture: 1. the Perception Subsystem that is responsible for extracting data from the environment where the Agent is operating; 2. the Action Subsystem that allows the Agent to act upon the environment; 3. the Central Processing subsystem that mediates perception and action and is capable of controlling both of these subsystems. In order to achieve a globally successful performance, there is usually a great interaction level among these three subsystems. For example, the Action Subsystem may provide direct feedback to the Central Subsystem (proprio-perception) or it may interact with the Perception Subsystem to allow effective coordination of some tasks (e.g.: hand-eye coordination). Moreover, despite the existence of the three conceptually different subsystems, the actual implementation of an Architecture using the “three towers” model may blur such well-defined distinctions. Some of the Architecture’s components (e.g.: visual system) may belong both to the Perception Subsystem and to the Central Subsystem. Chapter 3 Emotion-Based Agent Architectures: State of the art 49 The “three layers” model is based on the horizontal division of the Architecture in three layers, each of them capable of providing different forms of information processing and control functions. This model is deeply inspired in Evolutionary theories: each of the layers would have been developed in different stages of the evolutionary path of organisms based on the capabilities of the previous layer. The “ three layers” model consists of the following layers: 1. Reactive Layer. This layer includes reactive mechanisms, capable of delivering automatic responses when triggered by specific environment (internal and external) conditions. The reactive layer contains mechanisms capable of implementing pattern matching functions, condition-action rules and other direct input-output functions. They are also responsible for providing a basic set of Motivations. Mechanisms such as these may achieve great complexity levels and are usually implemented with great degree of parallelism (e.g.: neural nets). 2. Deliberative Layer. This layer would have resulted from the evolution of some of the mechanisms belonging to the Reactive Layer. The most important evolution in this layer is the development of reasoning capabilities about past, present and future events (what if reasoning). This layer also supersedes the previous one by adding more sophisticated memory systems and symbolic reasoning capabilities. In spite of these extra elements, the Deliberative Layer has limited processing resources. 3. Meta-Management Layer. This third layer includes mechanisms intended to monitor, evaluate and redirect processes executed in the Deliberative and Reactive Layers. This layer would have been developed in order to provide efficient control of all the machinery operating in the two lower layers. The mechanisms implemented at this layer may be reactive or deliberative. Processing resources at this level are also limited. The Architecture presented by Sloman, superimposes the “three towers” model and the “three layers model” forming a hybrid Architecture with nine distinct conceptual components (3x3 grid). The processes running in each of the three layers operate concurrently and deal with Perception, Central Processing and Action at different levels of abstraction. 3.2.2. Motives, Global Alarm Mechanisms and Variable Attention Filter Sloman observes that Agents operating in complex and real-time environments are usually compelled by several Motives (similar to Fridja “multiple concerns”). Some of these Motives will be active simultaneously and can be considered competitors regarding the Agent’s available processing resources. Conversely, other Motives will not be permanently active, requiring special motivegenerator mechanisms to be activated when certain conditions are found. Since the Agent has limited resource capabilities, some information processing mechanisms (in particular those located in the 50 Architecture’s upper layers) may not be able to respond fast enough to specific conditions occurring in real-time environments (either dangers or opportunities). Moreover, urgent Motives may require processing resources not available at that moment, possibly because they are being used to process less urgent Motives. Sloman tackles the problem of resource limitation by including in his Architecture “Global Alarm Mechanisms”. These Global Alarm Mechanisms receive information from every component in the system and, by using fast pattern matching procedures, they are able to detect situations whose urgency requires a change in the Agent’s internal processing strategy. When such a situation is detected, Global Alarm Mechanisms immediately send system-wide interrupt signals in order to stop current processes and trigger the whole system’s redirection to conveniently deal with the urgent Motive or situation. This may involve switching the Motive that is receiving processing resources or it may even require stopping computationally heavy processes, followed by the activation of faster reactive mechanisms that can provide a more efficient response to the urgent situation. However, Sloman does not clearly explain how these Global Alarm Mechanisms should be implemented. It is not clear if they are exclusively located in the reactive layer, or if they may be eventually located in higher layers. Global Alarm Mechanisms can help the Agent achieve a balanced use of its capabilities. However, they may also become very unproductive, if the conclusion of deliberative and metamanagement processes is systematically delayed or postponed as a result of frequent interruptions. In order to reduce this possible negative effect of Global Alarm Mechanisms, Sloman has devised the Variable Attention Filter mechanism, which is intended to filter some of the interruptions targeted at specific processes by raising the minimum urgency level that may interrupt the process. This will result in what can be considered as a state of “concentration”. Therefore, when Agents need to execute tasks that require continuous deliberative or metamanagement processing, the Variable Attention Filter will increase the minimum urgency threshold to ensure that such process will not be interrupted, unless the signal is generated in response to a very urgent condition or Motive. 3.2.3. Emotion Sloman draws a relationship between Emotion and the interactions established between the Architecture’s subsystems. In fact, Sloman argues that Emotional States arise naturally from interactions established between these subsystems, with no need for a dedicated Emotion-generating mechanism. Sloman divides Emotional Phenomena into three categories directly connected to the three layers in the Architecture. Following the definitions proposed by Damásio [Dam94] closely, Sloman agrees on the existence of Primary and Secondary Emotions but introduces an additional concept: Tertiary Emotions. Chapter 3 Emotion-Based Agent Architectures: State of the art 51 According to Sloman, Primary and Secondary Emotions result from the interactions established between Alarm Mechanisms and other subsystems located in the Reactive and Deliberative Layers [Slo98]. Primary Emotions, such as being startled, frozen with terror or sexually aroused [Slo01], are supported by Alarm Mechanisms located in the Reactive Layer, which are mainly concerned with processing sensory information (inside and outside the environment) and triggering fast Reactive Mechanisms. This interaction is believed to be similar to the role of the Limbic System in Humans. Secondary Emotions are supported by mechanisms in the Deliberative Layer and include emotions such as apprehension, relief and other semantically rich emotions that require deliberative capabilities. Secondary Emotions result mainly from Alarm Mechanisms concerned with evaluating internal cognitive responses (e.g.: the chances of success of a risky plan) that are not directly linked to the perceived environment. Tertiary Emotions, on the other hand, result from the mechanisms located in the MetaManagement Layer. Therefore, Sloman argues that they are probably exclusive to Humans. They include emotions related with thought and attention control such as infatuation, humiliation and thrilled anticipation. These Emotions interfere with Deliberative processes by diverting the attention from current tasks and triggering introspective processes, in spite of the Agent’s attempt to ignore such interruptions. 3.2.4. Overview and Comments It is important to note that the perspective followed by Sloman regarding Emotions is focused on explaining their occurrence as a result of Architectural requirements. Sloman’s works focus mainly on Agent Architectures. For the author, the concepts regarding Emotion are a consequence of the evolution of such Architecture. In spite of this, Sloman’s view on Emotion is very broad and the related concepts are supported by solid and coherent arguments. Sloman agrees with the general opinion that Emotional phenomena are mechanisms whose existence is inherently associated with resource-bounded Agents that need to operate in real-time environments. However, in [Slo00] Sloman disagrees with the point of view shared by several other researchers (notably Damásio), by stating that Emotions should not be considered an absolute requirement for Intelligence. In fact, Sloman argues that Emotions are simply side effects of the mechanisms needed to overcome an Agent’s resource limitation. The key concepts around Emotion are Alarm Mechanisms and Interruptions. Emotions are mainly seen as effects of existing control mechanisms intended to: • detect situations or motives that need urgent response from the Agent; • trigger the appropriate redirection of processing resources at different levels of abstraction. 52 In this way, Sloman seems to prefer a hard resource scheduling policy (interrupt and swap) instead of an alternative softer and finer-grain control of Agent’s processing resources. Sloman’s views about Emotions come from a more general perspective on cognition that we also share. However, Sloman is much more oriented towards a global descriptive theory of cognition and less focused on the specific functional properties of Emotion. Nevertheless, some of the underlying concepts behind Sloman’s theories are close to the ones we presented in the last Chapter, namely those regarding environment evaluation and resource control in complex architectures. 3.3. Velásquez Juan Velásquez’ work [Vel98a] [Vel98b] represents a very interesting approach to the study of Emotional Mechanisms because it includes a simple, modular and extendable Architecture that explicitly supports Primary and Secondary Emotions, as defined by Damásio [Dam94]. Additionally, Velásquez has also developed a physical robot whose decision-making process is based on the proposed Architecture, providing an alternative hardware implementation of Emotional Mechanisms. 3.3.1. The Architecture As discussed in [Vel98b], Velásquez follows a Biological perspective concerning the study of Emotions. The author considers Emotions as biological phenomena that have been conserved through various stages of evolution and that have a deep relation with survival and adaptation. Velásquez establishes as a basic assumption for the development of computational Emotional Mechanisms the need to understand and model the neurological structures that support Emotion. This goes in the opposite direction of other more descriptive approaches as those leading to Cognitive Appraisal Models. In this sense, Velásquez’ approach is quite different from many researchers’, who have based their Architectures exclusively on Cognitive Appraisal Models (such as the OCC Model [Ort88]), since it includes both Cognitive and non-Cognitive components of Emotion. Velásquez also stresses the importance of differentiating Emotions (i.e. Specific Emotions) from other Affective Phenomena (i.e. Emotional Phenomena) such as Moods or Temperament. Additionally, Velásquez argues that while developing Emotional Mechanisms the following conditions should be observed: • Emotional Mechanisms should be modeled within a broader Architecture that integrates Perception, Motivation, Behavior, Motor-Control, etc. • Development should be incremental and Architectures should be modular enough to accommodate new properties and functionalities as researchers develop deeper insight into Emotion. Chapter 3 Emotion-Based Agent Architectures: State of the art 53 • At each step of development, the Architecture should be able to generate complete Agents, capable of dealing with real-world situations. The Architecture proposed by Velásquez is extremely modular and establishes a very high-level relation between five groups of systems: 1. Perceptual Systems 2. Motor Systems 3. Behavior Systems 4. Emotional Systems 5. Drive Systems (Motivational System) The actual implementation of these systems is based on a network of non-linear processing units called Basic Computational Units (BCU), which are composed by three elements: 1. a set of Inputs; 2. an Appraisal Mechanism; 3. a set of Outputs. Higher level structures, such as the five groups of systems mentioned before, are built from the aggregation of these Basic Computational Units in complex networks. Interaction between high-level systems is also established by connecting the output of one or more BCU’s from one subsystem to the input of BCU’s in another subsystem. This way, the entire Architecture is a large network of BCU’s. The fundamental component of the Appraisal Mechanism inside BCU’s is what Velásquez calls Releasers. Releasers are described as “computational units that filter sensory data and identify special conditions which will provide excitatory (positive) or inhibitory (negative) input to the system they are associated with” [Vel98a]. Releasers can bee regarded as functions capable of evaluating (sensory) input and generating an output signal that depends on the relevance of the input. There are two types of Releasers, the Natural Releasers, which are hardwired from development, and Learned Releasers that can be learned by associating certain stimuli with Natural Releasers. 3.3.2. Motivations, Drive Systems, and Emotional System The Drive Systems that Velásquez proposes are conceptually similar (but not equivalent) to motivational systems: Drives are mechanisms that impel the Agent to Action. Drive Systems are composed by several BCU’s, whose Releasers keep track of the value of several motivational variables in order to maintain them within a specific range. Whenever the value of one of these 54 motivational variables falls outside the desired range, the appropriate Drive Releasers generate an error signal that will be used as input to other systems inside the Agent (e.g. Emotional and Behavior). Velásquez draws a clear distinction between motivations, Drive Systems and Emotion Systems. Within the proposed Architecture, Emotion Systems are the main motivational forces that lead the Agent to Action. In fact, even Drive Systems use this function to promote specific behavior. As an example, an Agent is motivated to get food by a combination of Hunger (Drive) and Distress (Emotion) caused by Hunger. 3.3.3. The Emotional System Velásquez also insists on combining both cognitive and non-cognitive factors of Emotion. He divides Emotional Releasers (i.e. Releasers of BCU’s inside the Emotional System) into four categories: 1. Neural. Neural Releasers are triggered by neuro-physiological conditions or compounds such as neurotransmitters, hormones, environment conditions, etc. 2. Sensorimotor. These Releasers are influenced by sensorimotor conditions, such as facial expression or body posture, that have the capability of triggering emotional mechanisms and eliciting Emotions. 3. Motivational. Includes all Releasers triggered by Motivational forces, which include both the Drive System and the Emotional System itself. 4. Cognitive. Cognitive Releasers account for Emotions triggered by cognitive processes, such as appraisal, comparisons, attribution, beliefs and memories. All Neural, Sensorimotor and Motivational Releasers implemented in the Architecture are Natural Releasers (i.e. pre-wired). Cognitive Releasers were implemented as Learned Releasers. It is interesting to note that the Cognitive Releasers Velásquez had implemented in previous versions of the Architecture were Natural Releasers, based on Cognitive Appraisal theories. However, Velásquez reimplemented Cognitive Releasers using Learned Releasers because he considered that Cognitive Appraisal theories had limited capability to explain the underlying brain processes that support Emotion. The Activation of BCU’s in Emotional Systems follows a different function from that of BCU’s in Drive Systems. It includes an extra term that accounts for the excitatory or inhibitory influence of other Emotional Systems, and also another term that introduces temporal decay behavior. Velásquez claims that these Activation functions enable the Architecture to support several different types of Emotional Phenomena, including: Chapter 3 Emotion-Based Agent Architectures: State of the art 55 • Primary Emotions. Primary Emotions are related with the activation of certain Emotional Systems, such as Disgust or Fear, by Natural Releasers. Primary Emotions are fundamental in providing the Agent with adaptation capabilities to immediate environment conditions. Velásquez brings the example of Fear Emotional System, which may be activated by a Natural Releaser that detects a dangerous situation, and generates the appropriate defensive context to cope with it. • Secondary Emotions. As we have seen, Velásquez has devised special releasers called Learned Releasers that are capable of learning relations between certain stimuli and the activation of Natural Releasers. The activation of Learned Releasers can thus be considered as the mechanism that supports Secondary Emotions (Damásio). Secondary Emotions tend to emerge after Agents develop a certain environment experience and start developing associations between events, objects and Primary Emotions, which are always present because they are hard-wired. The new Learned Releaser will be able to influence the action selection in future situations. Whenever the Agent reencounters the emotionally tagged stimulus (the person), the Learned Releaser will trigger the associated Emotional response (Fear). • Emotion Blends and Mixed Emotions. Although admitting that Emotion Blends and Mixed Emotions are not consensual matters within Emotion research, Velásquez claims that his Architecture may also support them [Vel98a]. According to Velásquez, in spite of the absence of an explicit model, Emotion Blends might emerge naturally by the simultaneous activation of two or more Emotional Systems. The co-activated Emotional Systems would subsequently be able to bias one or more non-conflicting perceptual or behavior systems. • Moods. Velásquez follows the view that Moods differ from Emotions mainly in what concerns the level of arousal. In this perspective, Moods would be explained by a state of low activation of particular Emotional Systems. Emotional Systems in such a low activation state would have the potential to become highly activated even in response to smaller stimuli. Velásquez describes this behavior as being consistent with the established theories about Moods. • Temperament. Architectural support for Temperament comes from the possibility of varying the parameters associated with Emotional Systems: thresholds, gains and decay rates. Velásquez gives the example of a “grumpy” Agent that would be configured by lowering the activation threshold and decay rate for Anger and increasing those for Joy. It is not clear, however, if such a variation is performed exclusively during the initial setup of the Agent, or if it may also be done automatically by the Agent during its interaction with the environment. This would represent another opportunity for adaptation. 56 3.3.4. The Implementation Something quite interesting and original about Velásquez work is the fact that the proposed Architecture was implemented in software Agents and also on a hardware Robot, Yuppi [Vel98a]. Yuppi Sensory System is composed of several different sensors, including: • Two CCD Cameras for Stereo Vision; • Two Microphones for Stereo Audio; • IR sensors for obstacle detection; • Air pressure sensor for sensing touch (Pleasant or Painful sensation); • Pyro sensor aligned to detect temperature changes (caused for example by the presence of people); • A simple proprio-perception system; Yuppi’s Drive System is constituted by four different drives (three of which related to physical goals) that control internal variables: • RechargingRegulation (Battery control); • TemperatureRegulation (Temperature control); • Fatigue (Energy Control); • Curiosity (Interest control); Yuppi’s Behavior System includes 19 behaviors, most of them concerning the satisfaction of its needs. Some of Yuppi’s behaviors are: Search-For-Bone, Approach-Bone, Cower, Recharge-Battery, Wander, Startle, Avoid-Obstacle, Approach-Person and Express-Emotion. Emotional Systems include Distress, Anger, Happiness, Disgust, Fear and Joy. Yuppi’s Emotional Systems support behavior-selection through simple Natural Releaser. They also enable the development of new emotional associations within each System through Learned Releasers. Let us first address the role of Natural Releasers within Emotional Systems that are the basis of a set of Primary Emotions associated with Drive Systems, Sensory Systems and with environment interaction in general. For instance, unsatisfied Drives will activate directly (i.e. through Natural Releasers) both Distress and Anger Emotional Systems. Conversely, satiation of drives will activate Happiness, whereas Distress is also activated if Drives are over-satiated. Sensory Information may also activate Emotional Systems. For example, every pink-reddish object spotted in the environment will activate Happiness, but yellow objects will activate Disgust. Darkness and some specific Blue objects will trigger Fear. Loud noises will activate Surprise. Interacting with people will also activate certain Emotional Systems. When people pet Yuppi, they Chapter 3 Emotion-Based Agent Architectures: State of the art 57 promote a Pleasure sensation (Air Pressure sensor) that will activate Joy. On the other hand, disciplining action will cause pain and activate Fear. Emotional Systems together with Drive Systems allow a simple selection of Behaviors. As an example, Velásquez describes a situation where the Activation level of Curiosity (Drive) reaches a high value, and Yuppi is motivated to wander around (Wander Behavior) looking for a pink bone. When Yuppi finds the pink bone the Happiness Emotional System will be activated which will promote the activation of behaviors such as Wag-Tail and Approach-Bone. However, if Yuppi fails to find the bone after a certain period of time, Distress (Emotion) will be activated which will then trigger the Droop-Tail behavior. The previous examples were focused on the activation of Emotional Mechanisms via Natural Releasers, which give support to Yuppi’s Primary Emotions. However, Yuppi can also develop Secondary Emotions through associations established by Learned Releasers. Another example given in [Vel98a] describes a situation where the Fear Emotional System acquires a new Releaser for loud sounds. In that situation, Yuppi is being disciplined and as a result is subjected to some Pain. Pain will promote the activation of Fear that in turn activates the Cower behavior. Initially, the loud sound does not activate the Fear Emotional System by itself, and therefore the Cower behavior will not be triggered. However, if both Pain and stimulus loud sounds start occurring simultaneously the Fear Emotional System develops a new (Learned) Releaser associated with loud sounds. After several simultaneous occurrences of both stimuli, the new Releaser will be able to activate the Fear Emotional System whenever loud sounds are sensed, promoting the subsequent activation of the Cower behavior. According to these results, Velásquez claims that the proposed Architecture is capable of supporting Emotional Conditioning, providing yet another mechanism for action-selection. 3.3.5. Overview and Comments Velásquez proposed a simple and modular model for Emotional phenomena, following a strong neuro-biological inspiration. The model makes a clear distinction between Primary and Secondary Emotion. The author addresses mainly the problem of behavior selection within a set of possible behaviors. However, it is not clear if behaviors change their operating mode depending on their activation level of the Emotional system or of other behaviors. There seems to be an all-or-nothing policy regarding the behavior selection. Another question that remains unanswered is how to extend the influence of Emotional Systems to additional systems, such as perception, or to modulate cognitive tasks, such as planning or goal generation. We believe that a richer scenario would provide a better test-bed for the Architecture but, nevertheless, Velásquez ideas have proven to be very useful for our work. 64 • Sensations: values obtained by using data collected from the Robot’s sensors and also from other events calculated using the Robot’s internal variables (e.g.: level of actuators). • Feelings: values obtained by combining the value of Sensations with the value of Hormones. • Emotions: The level of each Emotion results from combining the values of several Feelings. Although there are several Emotions at stake, the robot behavior will be defined by just one of them that is considered to be the Dominant Emotion. The Dominant Emotion is selected according to pre-defined threshold levels at every step of the simulation. • Hormone System: a System that combines the values of several Emotion levels and produces Hormones. These Hormones will be used to influence the level of the Feelings in combination with Sensations. The Hormone System acts as feed-back loop and it helps stabilize the value of Feeling and Emotions, in spite of possible quick changes in Sensations’ values. Using this architecture, the problem of state transition was solved by triggering transitions not directly from sensory information, which changes too quickly and is prone to instability, but from changes in Emotional levels instead. When the level of one Emotion suffers a change higher than a predefined threshold value, a new state transition is triggered. The authors have successfully employed this strategy in training a controller for the Robot, using Q-Learning. The Q-Learning algorithm was capable of converging and produced an efficient controller. The authors have also compared the performance of the controller developed using emotion triggered state transition with controllers developed using other strategies. Specifically, they have developed another controller using Q-Learning, but used a fixed interval of time to trigger state transitions. According to the authors, these two controllers showed similar performances in controlling the Robot, but the one developed using Emotion-triggered transitions was able to converge in significantly less steps (one sixth of the steps). This reduced computational effort is extremely important in real-time control situation such as the one in the experiment, because it releases processor power to other concurrent processes, if needed. In [Gad98], the authors also suggest a strategy that could be employed to reduce the number of steps needed to train a controller that uses time triggered state transition. Since Emotional Mechanisms should be able to reflect the relevance of the environment, (we will address the problem of choosing Emotion and their relations later) the controller should eventually learn more from situations where Emotions have substantial activation levels. Based on this Emotion Level/Environment Relevance correspondence, which is fundamental in Emotional Architectures, it seems that reducing the time between state transitions whenever the level of Emotion is high, i.e. increasing the frequency of state Chapter 3 Emotion-Based Agent Architectures: State of the art 65 transitions, may be advantageous for the learning process. Conversely, reduced levels of Emotion should signal less relevant periods in the environment and should not require such frequent state transitions. This decreased frequency would allow the controller to converge in less steps and release processing resources. 3.5.1. Comments and Overview for Gadanho This work shows a very interesting application of Emotional mechanisms to Autonomous Agents, specifically in connection with learning algorithms in complex environments. Two possible connections between Emotional mechanisms and learning algorithms are considered. In the first case, Emotional mechanisms are used to define the State of the environment to be learned, instead of using only sensory information. The author reports that this strategy actually results in faster learning times by reducing the influence of noise in data and the dimension of problem. This is a clear example of how to use the Emotion as Information paradigm (see Chapter 2). In the second case, the author proposes using Emotional mechanisms in order to adjust specific parameters of the learning algorithm (namely cycle of state transition), thereby achieving shorter convergence times. This is based on the assumption that relevant information should be signaled by intense Emotions and may be seen as an example of Emotion as a mechanism for Operating Mode configuration (also Chapter 2). 3.6. Summary and Conclusions In this Chapter, we made a brief review of some of the works that influenced most the development of our own ideas. The authors share the notion that Emotions are phenomena tightly connected to Agents that operate in complex worlds, and that help them in dealing with such complexity. Most of the work presented was concerned with the problem of Action-Selection and adaptation to the environment, especially the work by Velásquez and Cañamero. Emotional Mechanisms were involved in changing or complementing the Agent’s basic motivations that lead to action. Emotional mechanisms were also used as a way to control resources and schedule internal processing (interrupt, swap). This was shown to be particularly useful for Agents whose processing resources are limited and that have to deal with multiple and simultaneous Goals. We have also seen how Emotion could be used in improving Agent Learning capabilities by providing an alternative source of information for the learning process. Emotional Information is more stable and more compact than the information available exclusively through sensors, speeding up the learning process. Additionally, the process of learning itself could be modulated according to the activity of Emotional Mechanisms in order to improve convergence. All these Architectures seem to have a complementary view about the functional role of Emotion. However, in most cases the authors focus only on a particular possible use of Emotion, and 66 do not include other interesting perspectives that have been identified throughout the literature. We believe that there is still a lot of work to do in that direction, and that this is a challenging endeavor for AI. Chapter 4 – Modeling Emotion Mechanisms 67 CHAPTER 4. MODELING EMOTION MECHANISMS 4.1. Introduction In this chapter, we will try to formalize and model the concepts presented in Chapter 2, related both with Emotion and Emotion-Cognition interactions, with the purpose of enabling their integration within Agent Architectures. We will start by revisiting Emotional Elicitation as the first step in the process of modeling Emotional Mechanisms. We will try to integrate Emotional Mechanisms within other global concepts of Agent Architectures, namely Goals, Perception, Internal State and Proprio-perception. We will refocus on the concept of Emotional Evaluation Function, briefly presented in Chapter 2, and show its relation to Emotional Elicitation. Next, we will once again address the concept of Emotional State and introduce the notions of Emotional Accumulator and Basic Emotional Structures. These two concepts together with the Emotional Evaluation functions are the basic components of our Architecture. We will finish this Chapter presenting four models of Emotional Interactions, in particular, those regarding the concepts of: • Emotion as Information; • Emotion as a Process Control mechanism; • Emotion as a Resource Allocation mechanism; • Emotion as a mechanism for defining Agent’s Processing Strategies. Through these models, we formalize all important concepts relevant for our approach. 4.2. Elicitation revisited In Chapter 2 we described the process of Emotional Elicitation by which Emotional phenomena are activated or triggered. The process of Emotional Elicitation was basically concerned with the evaluation of the chances of a certain Goal being achieved (or not), considering both the state of the 68 environment (external state) and the Agent’s internal state. We will now take a closer look at this subject. 4.2.1. Multiple and Simultaneous Goals To conveniently model the process of Emotional elicitation, we will first need to consider some issues about Goals in complex environments. Let us first assume that an Agent operating in a complex environment will have multiple goals. This is a reasonable assumption, even if the Agent has only one basic fundamental goal: for that goal to be achieved, at some point in time, it will be necessary to unfold it into multiple sub-goals. Additionally, since the environment is only partially controlled by the Agent, situations representing new threats or imposing restrictions on current goals will certainly occur, motivating the Agent to create new Goals. Of course, such situations may also represent interesting opportunities for the achievement of certain Goals and, therefore, the Agent will generate new Goals to address those opportunities. In any case, it is perfectly reasonable to assume that the Agent will have multiple goals. Furthermore, we may also safely assume that in complex environments Agents will have simultaneous Goals. The environment evolves even if the Agent does not perform any action. More than one threat or opportunity may occur close in time. Therefore, even if the Agent is developing efforts to achieve a specific important Goal (which he must not abandon), situations that generate a new urgent Goal will eventually occur, forcing the Agent to deal with more than one goal at the same time. However, not every Goal needs to be immediately considered by the Agent. Goals have different priorities and distinct levels of importance for the Agent. Some goals have a high priority level while others may be postponed for some time or simply discarded. In addition, goals may have dependencies (sub-goals) to be observed which alters the priority of their execution. We will now try to formalize these concepts. Let gi denote a specific Agent Goal at instant t. Each goal g may be assigned a priority, pi, and a Set of Dependencies, Di, variable over time and composed of individual dependencies dik: gi(t) = {pi(t), Di(t)} Di(t) = {di1(t), di2(t)… din(t)} More urgent goals will have higher levels of priority. Dependencies dik establish relations between one goal gi and other goals. This implies that the achievement of Goal gi may only be reached when no more dependencies exist, i.e. the dependent goals have been achieved: achievement(gi) => Di(t) = ∅ Chapter 4 – Modeling Emotion Mechanisms 69 Let [G(t)] denote the Set of Goals that the Agent is pursuing at instant t. Then: [G(t)] = {g1(t), g2(t)… gn(t)} where gi(t) is a specific Goal that the Agent possesses at instant t. The Set of Goals is never an empty set because the Agent will always have at least one Goal, the Fundamental Goal, gF: ∀ t, [G(t)] ≠ ∅ or, more specifically, ∀ t, [G(t)] ⊃ {gF(t)} The Fundamental Goal is a Goal that exists during the Agent’s entire lifespan and its existence is intrinsically connected with the Agent’s own existence. Any other Goal requires (either implicitly or explicitly) the Fundamental Goal to be ensured, implying that some (or all) of Agent’s Capabilities and Resources need to be constantly allocated to the achievement of the Fundamental Goal. The Fundamental Goal does not change during Agent’s existence. However, depending on the specific environment conditions the Agent may spawn different sub-Goals that contribute to the achievement of the Fundamental Goal. The Fundamental Goal has priority pF and may have a non empty Set of Dependencies DF. No goal has higher priority than pF: ∀ gi ∈ [G(t)] : pi ≤ pF However, the priority of Goals belonging to the Set of Dependencies of the Fundamental Goal gF is equal to pF, since all dependent Goals must be accomplished previously to gF. Let as also introduce the SG operator. The SG returns the number of Simultaneous Goals occurring in a given Goal Set (at instant t): n = SG([G(t)]) For Agents operating in complex environments, SG([G(t)]) is almost always larger than one because the Agent will have at least the Fundamental Goal and the others resulting from the interaction with the environment. The Agent will be considering simultaneous goals by their priority. 70 However, since goals cannot be permanently postponed, Agent Capabilities and Resources will need to be shared between the processes leading to their achievement. 4.2.2. The Perception of Environment In order to have any chances of being successful, an Agent will need to capture a reasonable set of significant environment features. Each feature may be obtained either by direct inspection of sensory data or it may result from a more complex analysis, perhaps using previously stored information. In any case, the Agent will need to create a model of the external environment using such features. Let [E] denote the Environment Model that the Agent is able to obtain using a given set of significant features (extracted from the environment or produced). Then [E] is composed by a set of Beliefs5, ei, about multiple elements existing in the environment. [E(t)] = {e1(t), e2(t)… en(t)} Since sensory data is intrinsically noisy and the Agent has limited resources to analyze it, [E(t)] is only a rough approximation of the actual environment state. However, up to this moment, it is all the Agent has to derive its decisions and actions. One should note that not every Belief about the environment is relevant for all of the Agent’s Goals. For each goal gj belonging to the Set of Goals [G(t)], there is a subset of [E(t)], denoted by [E(t)]j, so that all elements of [E(t)]j are relevant for the achievement of goal gj: ∀ gi ∈ [G(t)] ∃ [Ei(t)] ⊂ [E(t)] : ek ∈ [Ei(t)] => isRelevant(ek, gi) where the relation isRelevant(e, g) is true if e, a specific Belief about the Environment, has information that is necessary for the achievement of Goal g. For each goal gi, it should be possible to define a matrix, the Relevance Matrix Rli, that relates [E(t)] with the corresponding [E i(t)]: [E i(t)] = Rli • [E(t)] As described in Chapter 2, Emotional Elicitation operates over a subset of [E(t)] that is relevant for a given goal, in order to detect situations that pose a threat (or opportunities) for the achievement of each goal. In a certain way, Emotional Elicitation processes should be able to perform relevance analysis as if they had intrinsic knowledge of the Relevance Matrix. 5 A Belief can be defined as a piece of information gathered through perception filters, produced by internal information processing mechanisms or obtained from other Agents not yet proved to be true. In the context of some Intentional Logics, Beliefs that are proved to be true become Knowledge. Chapter 4 – Modeling Emotion Mechanisms 71 Finally, since it should be possible to obtain a Relevance Matrix for each Goal, we may define the Set of Relevance Matrixes [Rl(t)] as: ∀ gi ∈ [G(t)] ∃ Rli : Rli ⊂ [Rl(t)] Almost certainly, Agents will not know accurately each Rli because the environment is complex and they have limited sensory and analysis resources. Consequently, they will only be able to obtain an estimate of the Set of Relevance Matrixes, [Rl’(t)]. We will pick up this issue again in the following sections. 4.2.3. Internal State of the Agent It is easy to understand the direct importance of the perception of environment state, [E(t)] in the process of Emotional Elicitation. However, we also need to capture the importance of the Agent’s Internal State. Let us consider that an Agent possesses a Set of Capabilities, [C(t)], that enable it to operate in the environment. These Capabilities include: • information processing mechanisms (e.g.: planning, inference, communication, algorithms); • action mechanisms capable of changing the environment up to some extent (e.g.: change its position or the position of other objects); We may assume that over time the Set of Capabilities may change as the Agent may acquire or lose certain Capabilities. However, the actual instantiation of these Capabilities is always bounded by the Set of Resources, [R(t)], that the Agent has available either intrinsically or at a given instant. Resources include: • the computational structure that supports information processing mechanisms (memory, computational power) • the physical (virtual) infrastructure that enables Agents to perform actions in the environment (e.g.: muscles, robotized arms, tools or machines that help in the execution of tasks) • other Agents that may help in the achievement of a given goal; • any resource that may be used to obtain, maintain or extend any of the previously mentioned resources. 72 Therefore, we may define the Set of the Effective Capabilities of the Agent, [EC(t)], as the Set of Capabilities that the Agent may effectively use considering the Resources available at that moment and their allocation: [EC (t)] = A([G(t)],[C(t)], [R(t)]) where A is a Global Allocation Function that assigns Resources to Capabilities according to the Set of Goals. The Global Allocation Function is able to globally coordinate the allocation of Resources among Agents Goals. Although the Global Allocation Function influences all Capabilities, it is not intended to provide a fine-grain control over each Capability. However, it may generate many different allocation contexts so that Agent Capabilities may be instantiated in multiple ways. Particular allocations of Resources to Capabilities will be related with different Process Operating Modes and a distinct global Processing Strategy. Let us then define the Agent’s Internal State [I(t)] as the combination of all these sets in order to explicitly include the set of Effective Capabilities at a given instant: [I(t)] = {[C(t)], [R(t)], [EC(t)]} 4.2.4. Proprio-Perception It is not guaranteed that the Agent knows exactly its own Set of Capabilities [C(t)] or its Set of Resources [R(t)]. Instead, the Agent has an estimate of all these sets over time, which we will denote respectively by [C’(t)] and [R’(t)]. In fact, for sufficiently complex Agents, the Agent’s entire Internal State might only be known approximately, even by itself. The Agent may estimate its own Set of Effective Capabilities taking: [EC (t)] = A’(t, [G(t)], [C’(t)], [R’(t)]) where A’ is an estimated Global Allocation Function that corresponds to the Agent’s estimate of how it might allocate its own perceived Resources to its perceived Capabilities. Let us now define Agent Proprio-Perception [P(t)] as the combination of the previous sets, the Modified Allocation Function, A’, and also an estimate of the Set of Relevance Matrixes defined in a previous section: [P(t)] = {[C’(t)], [R’(t)], [EC’(t)], A’, [Rl’(t)]} Chapter 4 – Modeling Emotion Mechanisms 73 We may also extend our previous definition of Internal State in order to include Agent’s Proprio-Perception: [I(t)] = {[C(t)], [R(t)], [EC (t)], [P(t)]} We have now defined and formalized all concepts needed for a systematic notion of Emotional Elicitation. 4.2.5. Emotional Evaluation Functions In Chapter 2, we established a relation between Emotional Elicitation and the evaluation of chances of Goal achievement, taking into account the perception of the environment and the Internal State of the Agent (effective and perceived). The key element in this process is the Emotional Evaluation Function that is able to evaluate these parameters and promote a change in the Emotional State of the Agent6. We shall now formalize this concept. Definition Let [E(t)] be the Environment Model, as perceived by the Agent, and [I(t)] be the Agent’s Internal State, which includes both the effective and perceived state. Then, for a gi belonging to the Set of Goals, [G(t)], there is an Emotional Evaluation Function, EEF, that is able to produce a change, [∆Em], in the Emotional State of the Agent, [Em]: [∆Em] = EEF(g i(t), [G(t)], [E(t)], [I(t)]) Note that we do not impose any restriction regarding the nature of the Emotional Evaluation Function itself: it may be a simple algebraic function that maps a combination of input values to a scalar output value (similar a neurobiological / sensory circuit),or it may quite possibly be a very complex inference or pattern matching procedure (similar to a high-level cognitive analysis). Figure 4.1 – Emotional Evaluation Functions produce changes in the Emotional State Expanding each of the parameters of EEF we obtain: 6 A deeper formalization of the concept of Emotional State will be developed later. [ ∆ Em] [Em(t)] EEF(g i(t), [G(t)], [E(t)], [I(t)]) [G(t)] [I(t)] [E(t)] 80 4.4.2. Modeling Emotion as Information In Chapter 2 we have described how Emotions may be advantageously used as sources of information, in order to complete or simplify data gathered by perception (or previously stored in memory). We shall now model this interaction. Let Pi be a Process related to Capability Ci. Let [IAi] be the set of Input Arguments of Process Pi. Then, [IAi] results from data belonging to: • the Model of the Environment, [E(t)] • Agent’s Proprio-Perception, [P(t)] • the Output Arguments of other Processes, [OA’i] • Agent’s Emotional State, [Em(t)] Alternatively, [IAi] = {[Ei(t)], [P(t)], [OA’i], [Emi(t)]} where: [Ei(t)] ∈ [E(t)] [Emi(t)] ∈ [Em(t)] ∀ [OAj] ∈ [OA’i], ∃ Pj , ∃ [IAj], ∃ [CA]j, ∃ Cj : Pj([IAj], [CA]j) -> {[OA]j , Cj} The key point in the previous formula is the inclusion of the Agent’s Emotional State as input to the Process, in much the same way as data from the Model of the Environment or from ProprioPerception. As mentioned before, the information about the state of the Emotional State is stored in the Emotional Accumulators that are dynamically updated. Figure 4.7. Emotion as Information. Values stored in Emotional Accumulators are included in the set of Input Arguments of Process Pi. [Ei(t)] : Model of the [OA’i ]: Output Arguments from other Processes [P(t)] : Proprio - Perception [Em i (t)] : Emotional Accumulators C i [OA i ] to other Process [CA i ] [IA i ] P i Chapter 4 – Modeling Emotion Mechanisms 81 4.4.3. Modeling Process Control We have also seen in Chapter 2 how Emotion helps to modulate each individual process by controlling several of its information processing parameters. Following our previous modeling effort regarding Agent’s Processes, we may now model this type of interactions. Let Pi be a Process related to Capability Ci. Let [CAi] be the set of Control Arguments of Process Pi. Then, [CAi] will be composed by data belonging to: • the Model of the Environment, [E(t)] • Agent’s Proprio-Perception, [P(t)] • a Global Allocation Function, A • Agent’s Emotional State, [Em(t)] Again, alternatively, [IAi] = {[Ei(t)], [P(t)], A, [Emi(t)]} where [Ei(t)] ∈ [E(t)] [Emi(t)] ∈ [Em(t)] We have now included the influence of the Agent’s Emotional State in the Set of Control Arguments of Process Pi. This can be done in two ways: 1. directly, by including the Emotional State [Em(t)] itself as a Control Argument 2. indirectly, by including the perception that the Agent has of its own Emotional State (included in Agent’s Proprio-Perception). The following picture illustrates this double influence of Emotional Mechanisms on the control of individual Processes. 82 Figure 4.8. Emotion as a Process Control Mechanism. Both the values stored in Emotional Accumulators and Agent’s Perception about its own Emotional State are included in the set of Control Arguments of Process Pi. 4.4.4. Modeling Resource Allocation In the model that we have been presenting so far, resource allocation is accomplished by a Global Allocation Function that assigns Resources to specific Capabilities, thereby generating a set of Effective Capabilities [EC(t)]: [EC (t)] = A([G(t)], [C(t)], [R(t)]) However, as we have seen, Emotions play an important role on global resource allocation. In order to model such interactions, let us simply extend our previous definition of Global Allocation Function to include the Emotional State, [Em(t)], as one of the parameters: [EC (t)] = A([G(t)], [C(t)], [R(t)], [Em(t)]) Figure 4.9. Values contained in Emotional Accumulators influence the way Global Allocation Functions assign resources to different Capabilities, which involve multiple processes C C [Rk(t)] Pj1 Pj2 Pj3 Pk1 ECk Pl1 Pl2 [Rj(t)] [Rl(t)] A([G(t)]) [R(t)] [C(t)] [Em i (t)] : Emotional Accumulators [Em’i(t)] : Perception of Emotional State [Em i (t)] : Emotional Accumulators Ci [OAi] to other Process [CAi] [IAi] P i [P(t)] : Proprio-Perception [Ei(t)] : Model of the Environment A: Global Allocation Function Chapter 4 – Modeling Emotion Mechanisms 83 Once again we are reinforcing the feedback loop around EEF’s with one more paths. Recall that one of the arguments of EEF’s was the Agent’s Internal State, in which we may find the set of Effective Capabilities. Furthermore, there is still another less direct loop that is established but whose importance must not be overlooked. In fact, the allocation of Resources to Capabilities will have a decisive impact on the Agent’s success in achieving its Goals. Since the Set of Goals is an input of EEF’s, another feedback loop is thus established. Figure 4.10. Another feedback loop is established through the influence of the Global Allocation Function, A([G(t)]), on the state of the Goals composing the Set of Goals, [G(t)] 4.4.5. Modeling Processing Strategies In Chapter 2 we have defined four basic Information Processing Strategies that showed different patterns regarding the amount of Emotional Information used as well as the computational resources employed: • Direct Access • Motivated Processing • Heuristic Processing • Substantive Processing When an Agent is operating under a specific Processing Strategy, all its processes tend to make a typical use of Emotion as information. Also, the control parameters of each individual process are set differently, resulting in distinct degrees of information processing complexity and, consequently, varying levels of the computational resources employed. At the same time, since computational resources are not infinite, a global scope mechanism is used to allocate the (limited) existing resources among capabilities, especially to those addressing more relevant Goals. As described in Chapter 2, Emotions play an important role at these three levels. Therefore, using the previous three models we may also model the concepts related to Information Processing Strategies and its interactions with Emotional Mechanisms. We will need to combine the models of: [ ∆ Em] [Em(t)] EEF(g i(t), [G(t)], [E(t)], [I(t)]) [G(t)] [I(t)] [E(t)] A([G(t)]) 84 • Emotion as Information • Process Control • Resource Allocation The next picture combines ideas from the three afore-mentioned models and presents a global perspective of our view of Emotional interactions within an Agent Architecture: Figure 4.11.The overall influence of Emotional Mechanisms over Cognitive Processes (Emotion as Information, Emotion in Process Control and Emotion in Resource Allocation) allowing different global Information Processing Strategies 4.4.6. Emotional Agent After all this modeling effort, we are now in position to complete the definition of Emotional Agent presented in Chapter 2. Firstly, let us restate the notion that Emotional Agents exist in complex8 environments that require multiple and sophisticated Capabilities in order to achieve various (simultaneous) Goals. Therefore, an Emotional Agent is an Agent with complex Capabilities, supported by an Architecture that will eventually have multiple operating subsystems. Each of these 8 The notion of complex environment we are considering has been explained in Chapter 2 Emotion in Process Control Emotion as Information Emotion in Resource Allocation 3 2 1 A([G(t)]) [Rj(t)] [Rl(t)] [R(t)] [C(t)] Pk1 ECk 1 2 3 [Em i (t)] : Emotional Accumulators Pl1 Pl2 C [Em i (t)] : Emotional Accumulators 1 1 2 2 3 Chapter 4 – Modeling Emotion Mechanisms 85 Capabilities, and consequently the underlying operating subsystems, should possess several parameters that control its information processing methods. Definition: An Emotional Agent is an Agent that is able to operate under different Emotional States that are reached through a process of Evaluation. Such Evaluation takes into account the Internal State of the Agent, the Environment Model and the chances of achievement of each individual Goal belonging to the Agent’s Set of Goals. The Emotional State may be used by other Processes within the Architecture (i) as input information, (ii) as a Process Control mechanism, (iii) as a Resource Allocation mechanism, and (iv) in the definition of the global information Processing Strategy. 4.5. Summary and Conclusions In this Chapter we analyzed the global operating and architectural contexts for Agents intended for complex environments. Next, we defined and modeled several concepts regarding Emotion and Emotion-Cognition interactions. The key concepts discussed were: • Emotional Elicitation • Emotional Evaluation Functions • Emotional State • Emotional Accumulators • Basic Emotional Structures We then described how these concepts could be applied in modeling Emotion-Cognition interactions. We presented models for the following paradigms described in Chapter 2: • Emotion as Information • Emotion as a Process Control Mechanism • Emotion and Resource Allocation Mechanisms • Emotion and Information Processing Strategies We finalized this Chapter with an extended definition of Emotional Agent. Chapter 5The Pyrosim Platform 87 CHAPTER 5. THE PYROSIM PLATFORM 5.1. Introduction In this Chapter we will describe the Pyrosim platform that we have been building to support the development of our Emotion-based Agent Architecture. The Pyrosim platform has provided the basis for many experiments and has been very useful in clarifying our ideas about Emotion. The Pyrosim platform simulates a forest environment where a team of Agents is placed to fight an ongoing fire. Each individual Agent can combat fire cells with a water jet that is connected to a limited capacity water tank. Agent mobility is constrained by its own simulated physical capacities (energy and acceleration) as well as by limitations imposed by characteristics of both the terrain and the fire. Fire propagation through the terrain, depends on the vegetation type and density, the terrain slope, and the wind, based on a realistic model. The Pyrosim also allows Agents to communicate in order to enable team efforts. It may seem awkward to describe the platform before even summarizing the Architecture we are proposing, but we think this will help to explain the Architecture’s underlying concepts. Although our Agent Architecture is not dependent on the Pyrosim Platform, some of its design requirements become more evident when focusing on this particular scenario. In fact, the Pyrosim Platform and our Agent Architecture have grown together. When we were developing our first Agents, we immediately identified specific points for improvements in the platform itself. Throughout the next sections, we will be focusing on the platform’s most relevant issues for Agent development. Although we will sometimes be forced to go into some detail about the implementation, we will try to describe the platform as much as possible in high-level. More detailed information about the platform and Agent programming is available in the “ Pyrosim Agent Developer Manual”, which is distributed with the platform. Issues about the mathematical modeling behind the simulation are described in Appendix A. 88 5.2. Motivation for Pyrosim Our first attempts to build an Emotion-based Agent Architecture were made using a simple simulator named RealTimeBattle (RTB)[Ouc02]. This platform provides a simulated real-time environment where softbots fight for survival in 2D scenarios. RTB allows the developer to program their own softbots in C/C++, as well as to create custom 2D scenarios. Simple physical properties (air resistance, friction, material hardness) are also implemented to enrich the simulation. Softbots perception is basically a set of radar events from which they can detect walls, other softbots, shots and randomly distributed energy sources and mines. Softbots can accelerate, break, rotate and shoot in a given direction. The experiments made using RealTimeBattle allowed us to develop a sharper understanding of emotional-like structures, namely Emotional Evaluation Functions coupled with Emotional Accumulators [0li02]. However, we realized that to effectively test other emotion-cognition interactions, we would need a more complex simulated environment that would impose higher demands on the Agent. Simple environments do not reveal the need for emotional-like mechanisms. In order to be an appropriate testbed for Emotion-based Agent Architectures, this simulation environment must meet the following requirements: • high complexity for the participating Agent • real-time requirements • multiple concerns at stake • autonomous decision-making coupled with multi-agent interaction • closeness to a real-world problem9 These requirements are similar to those Humans meet when dealing with the real-world. As we explained in Chapter 2, Emotions may play a useful role in such a context because they help resourcebounded Agents to deal with complexity and to achieve multiple simultaneous goals. Simpler environments or Agents with fewer goals will simply not need Emotional mechanisms because possible problems may be solved with the help of simpler, more straight-forward mechanisms. After trying to find a suitable environment simulation to test Emotional-based Architectures, we chose to build one from scratch. In our search for alternative simulation environments, most of the systems complex enough for Emotion-based Agents were based on a domain that we did not find appealing. Some of them were simulations of war, while others were based on toy environments. Since forest fires are a major concern in Portugal, we decided to develop a forest fire simulation environment. We knew that building a simulation environment was a considerable effort. However, 9 This is not really a requirement but any simulation environment that complies with this condition will almost certainly ensure the previous four conditions. Chapter 5The Pyrosim Platform 89 such a platform would also be useful for other studies in the field of autonomous Agents such as, for example, in multi-agent coordination. Furthermore, being able to control some of the parameters of the simulation environment would also be advantageous for experimenting new features in our Agents. The balance between effort and possible future advantages seemed positive enough to start Pyrosim’s development. 5.3. Overview of the Pyrosim Platform The Pyrosim Platform is a simulation environment composed by a set of applications and modules that communicate through a network infrastructure. The platform was designed to run as distributed as possible in order to allow a balanced scaling of the simulation. Each module of the platform can be run on a different computer to avoid heavy computational bottlenecks. The core of the Pyrosim Platform includes two basic elements: 1. the Pyrosim Server application, responsible for running all the simulation logic. It contains the environment’s model and updates the state of every entity in each simulation cycle. 2. the Agent Skeleton layer that provides a convenient interface for creating customized Agents. It manages all low level communication between a customized Agent and the Pyrosim Server. There is also another application that is very useful for monitoring the state of the simulation: the Pyroviz. The Pyroviz allows the developer to visualize almost in real-time what is happening during the entire simulation through a 3D rendered scene, as well as track all the communication activity between the participating Agents. Figure 1 illustrates the relationship between the three components of the Pyrosim Platform and lists some of their specific functionalities. Figure 5.1 - Overview of the Pyrosim Platform 96 5.5. Pyrosim Agents As we have mentioned before, the Pyrosim Platform provides a software layer, the Agent Skeleton, which helps developers to build higher-level Agents. The Agent Skeleton is not in itself an Agent, it does not have any behavior. However, it provides a basic set of capabilities for perceiving and acting on the Pyrosim World that will be described throughout this section. It is also important to refer that the Agent Skeleton does not possess any intrinsic “emotional” capability: all the Emotional mechanisms were implemented on top of this layer, as will be shown in Chapter 6. In this section we will focus on the following issues concerning the Pyrosim Agents and Agent Skeleton: • Proprio-perception • Visual Perception • Action Set • Communication For further details about the Agent’s programming, please refer to the Pyrosim Developer Manual, which is distributed with the Pyrosim Platform. 5.5.1. Proprio-perception The most elementary data provided by the Agent Skeleton is the state of the Agent, which comprises several parameters intrinsically perceivable. These parameters can be considered the Agent’s proprio-perception since they are always known and no special query action needs to be executed in order to find its value. The Agent’s proprio-perception includes the following (read-only) parameters accessible through the Agent Skeleton: • X and Y coordinates of the Agent’s location. Range depends on the simulated terrain dimensions (default range: 0 to 128). Chapter 5The Pyrosim Platform 97 Figure 5.7 – Pyrosim coordinate system • height (Z coordinate) of the Agent. Range depends on the terrain dimensions but is always a positive value. • energy level of Agent, ranging from 0 to 100. • skin temperature of the Agent in degrees Celsius. Values over 50º will result in damage to the Agent and in fast decrease of the energy level. • rotation in degrees of the Agent’s direction, measured clockwise in the XY plane. Figure 5.8 – The rotation angle LVPHDVXUHGFORFNZLVHIURPWKH;D[LV • the tangential acceleration the Agent is trying to produce. This value ranges from –1.73 m/s2 to 6.92 m/s2. The 6.92 m/s2 value corresponds to the acceleration needed to climb a 45º slope terrain. This means that the Agent will not be able to proceed forward when the slope of the terrain is higher than 45º. • the speed in m/s along the X axis and along Y axis. • the slope of the terrain along the X axis and Y axis. • The Agent Skeleton also provides access to information about the Agent’s water jet: • operating state of the water (on or off). 98 • the distance the Agent intends to reach with the water jet. Depending on the water flow employed, the desired distance may be reached or not. In practice, it corresponds to controlling both the orientation and the flow of the water jet. The maximum absolute value is 40 meters. • the flow in lt/s that the Agent intends to eject. The maximum absolute value is 16 lt/s. • the distance the water jet is effectively reaching. The Agent does not control this value. • the effective water flow of the water jet. • the angle made between the water jet direction and the forward direction of the Agent. • X and Y coordinates of the cell being targeted by the water jet. The Agent does not have direct control of this value. • the level of the tank in lts. 5.5.2. Visual Perception The Agent Skeleton also provides information about what the Agent would see in its surroundings. This information is not an image, as if it was captured by the Agent’s eyes, but a structured description of the features that would be extracted from such an image. Therefore, we may consider this information as the visual perception of the Agent. Visual perception is organized into several 2D maps called Float Maps. These maps store the properties of a given element in the environment. Changes in the environment may be detected by comparing two consecutive Float Maps. The Agent Skeleton has several Float Maps available that numerically describe the properties of the surroundings: • terrain geometry; • type of vegetation; • intensity of fire; • level of damage caused by fire; For example, the Float Map concerning fire intensity would store the values of fire intensity in each of the cells around the Agent. A value of 0 would mean that the given cell is not burning and a positive value would describe the intensity of the ongoing fire. In order to account for the expected loss of detail of more distant objects two specific Float Maps are supplied for each of the categories listed before: 1. a Close Range Map; 2. a Medium Range Map; Chapter 5The Pyrosim Platform 99 Close Range Float Maps are related with the Agent’s immediate surroundings and, therefore, describe the properties of a given category with a higher level of detail. Each Close Range Float Map consists of a 3x3 array of float values describing a certain property both of the terrain cell where the Agent is located and of the eight surrounding terrain cells. The next sketch shows the 3x3 cell area covered by a Close Range Map (the Agent is located in the center cell). Figure 5.9 – The cells comprised by a Close Range Map. On the other hand, the purpose of a Medium Range Float Map is to provide a description of a much wider (and distant) area, namely 21 x 21 terrain cells. However, the increased area coverage is obtained at the cost of detail. In fact, instead of being composed of 21 x 21 float values (one for each terrain cell covered), a Medium Range Float Map is composed of just 7x7 elements: each element of a Medium Range Float Map describes the average value of property over a 3x3 cell area. The next picture tries to describe these correspondences. Figure 5.10 – The cells inside a Medium Range Float Map. The Medium Range Float Map stores a 7x7 grid of the average values of the 3x3 checkered areas. As illustrated above, the 7 x 7 checkered pattern covers an area of 21 x 21 terrain cells. The Agent is located at the center of this 21 x 21 area. This particular terrain cell and the eight terrain cells 100 surrounding it, comprise the Agent’s close range. As we have seen before, a Close Range Map describes these nine cells in much more detail. However, all Float maps are subjected to noise, altering the numerical descriptions they store. The further away the cell is from the Agent, the more significant will be the influence of noise distortion. 5.5.3. Visual Occlusion Pyrosim server takes into account the possibility of occlusion during visual inspection. Whenever the geometry of the terrain (e.g. a hill) intersects with the Agent’s line of vision to the center of a given terrain cell, the Agent will not be able to receive visual information about that cell. Figure 15 tries to explain this limitation imposed by the server: Figure 5.11 – The fireman located in cell 3 will be able to receive information about cells 2, 3 and 5. Both cell 1 and cell 4 are occluded by the geometry of the terrain. For all Close Range Float Maps, occlusion is ignored to simplify the task of the Agent at a local stage. However, for Medium Range Float Maps, the server checks how many of the nine cells that compose each 3 x 3 cell area covered are occluded. If more than four cells are in fact occluded, then, the whole 3 x 3 area is also considered occluded. In this case, the value stored in the Medium Range Float Map to describe the occluded 3 x 3 area will be –1. Note that none of the Medium Range Float Maps presented so far can store negative values to describe the corresponding property, because all properties have always non-negative values (even the height of the terrain). A negative value will always represent occlusion. 5.5.4. Perception of other Agents The Agent Skeleton also stores information about other Agents in the simulation. Our Agent is able to see all other Agents inside the Medium Range Area (21 x 21 cells) if these Agents are not occluded, as described in the previous section. The information gathered about other Agents is: • the name of the Agent; Chapter 5The Pyrosim Platform 101 • the X coordinate of Agent location; • the Y coordinate of Agent location; • an estimate of the current energy of the Agent; • a description of the current action of the agent; 5.5.5. Action Set The Agent Skeleton layer has several methods available for Custom Agents to act upon the World. At the Agent Skeleton level, the set of possible actions comprises only basic low-level actions for movement and for control of the water-jet. More complex tasks, (e.g. walking to a target point), are not considered basic low-level actions as they may involve multiple criterion decisions. Therefore, this type of actions should be implemented in higher-level classes. Controlling moves Agents may control two basic parameters of their movement: the acceleration effort and the direction of movement. Agents are able to produce positive acceleration efforts up to 6.92 m/s2, which is equivalent to the acceleration effort needed to climb a 45º slope limit. Therefore, Agents will not be able to climb hills with slopes higher than 45 degrees, unless they accelerate sufficiently before starting to climb the hill. In addition, Agents will not be able to stop their movement when descending hills with slopes higher than 45 degrees, which represents a risk to avoid. Backward (negative) acceleration is also limited to 1.73m/s2 (25 % of the maximum positive acceleration), which results in Agents not being capable of moving backwards as fast as they move forward. Controlling Water Jet Agents can control four parameters of their water jet: 1. the On/Off switch; 2. the intended distance at which the water is ejected (absolute limit 40 meters); 3. the direction of the jet; 4. the flow of the jet in liters/s (absolute limit 16 liters/s). However, since the water jet has limited power, there is a dependency between the flow of the jet and the distance at which the Agent intends to throw water: higher flow will reduce the possible distance range. The Agent may control the flow of the jet directly but he has no guarantee that the water will reach the intended distance. This implies that the Agent may have to deliberately reduce the flow of ejected water to ensure that the distance he wants to throw water at is actually reached. 102 However, this will decrease the volume of water that the Agent can throw in a fire cell. Therefore, in order to achieve maximum fire fighting efficiency, the Agent has to be as close to fire as possible, although this may become dangerous. 5.5.6. Communication During the simulation, the Agent may send other Agents two different types of messages: 1. broadcast messages: messages sent simultaneously to all Agents. All Agents in the simulation will receive the sent message. 2. 1-to-1 messages: messages sent exclusively to one Agent. Only the selected target Agent will receive the message. To optimize message processing, the Agent Skeleton has an internal event-driven mechanism that signals new incoming messages. The platform does not constrain by the structure of the message content. Pyrosim simply delivers a string between two Agents whose structure is irrelevant for the message passing mechanisms. Any message protocol has to be implemented in the Agent-Skeleton. 5.6. Pyrosim and Emotional Agents Let us now analyze Pyrosim in the perspective of the requirements that we considered essential for any environment where Emotions might be useful. We will be pointing out situations where Emotional-mechanisms may come into play. High level of complexity for the participating Agent Pyrosim is a complex environment. Using the taxonomy from [Rus95], the Pyrosim world is: • Partially Accessible. Agents only have partial perception of the environment. They can only visualize accurately objects in their close surroundings (3x3 cells area). Information about objects located further away has an inferior level of detail and is altered by noise. Additionally, there is always the chance of visual occlusion. After a certain point, the Agent is not able to get any more “visual” information. Reports about the global state of the fire are available only from time to time and have a limited degree of detail and accuracy. Agents may have to find a way to compensate for a possible absence of information. Chapter 5The Pyrosim Platform 103 • Non-Deterministic. Although the world evolves in accordance to a defined set of rules, the number of unknown or uncertain parameters (terrain geometry, wind, vegetation) is large enough to make it very difficult to predict. Furthermore, the outcome of an action is not certain, at least in an indirect way. The Agent is not sure if pointing its water jet to a fire cell will extinguish it. Actions may be carried out according to a variable set of parameters that establish a compromise between performance and resource cost. The Agent might require a method to adapt these parameters according to its resources. • Dynamic. The world is constantly changing, even if the Agent does not intervene. Firefronts progress in the terrain, wind speed and direction change, vegetation is destroyed. The Agent itself is always changing (e.g.: loosing energy, slipping from a hill). Other Agents in the environment will take action (change position, choose another target cell) and change the world according to their own criteria. The Agent will need a mechanism capable of detecting the most relevant changes and situations where more attention should be given to environment analysis. • Continuous. This is, of course, a computer simulation run on a discrete machine but we may consider that for the Agents the simulated world is continuous. We may consider that the world may be in one of an infinite number of states. Agents may need a mechanism to reduce the state dimension of the environment in order to make it more tractable. In order to act in such a complex world, the Agent will need multiple capabilities and those capabilities will naturally be dependent on several parameters. Managing all the capabilities with resource constraints is essential for Agent’s success. Real-time requirements The environment demands real-time action from the Agent. The fire is not stopping. The world is both dynamic and dangerous for Agent Goals (e.g.: survival). The Agent needs to be constantly perceiving the environment and to act accordingly. However, there might not be enough time/resources to perform an exhaustive analysis to decide the “best” action to be taken. Agents might need a mechanism to adapt their response time to environment requirements and a method to balance the amount of time spent on environment analysis and action control. Multiple concerns at stake Each Agent has to ensure its own safety while fighting the fire. At the same time, the Agents need to help the team to remain safe, which implies that they comply to certain restrictions (remain close, remain in line-of view, etc). Additionally, certain points in the map may be considered vital (e.g.: houses or industrial installations) and the team has to ensure that fire will not damage them. 104 There are various simultaneous (sub)goals to be attained. The Agent will need a mechanism to help him decide which is the most relevant or urgent goal at each moment. Multi-Agent System No single fireman is able to put out a reasonably large fire by himself, so, there is an obvious need for cooperation. However, since the Agent has both individual and team goals to attain, a balance between individual action and cooperation must be established. Furthermore, decisions taken by a group leader affect the entire team and need to consider very complex team factors (previous performance, current state of the team), besides the threats imposed by the environment. Besides autonomous decision-making and interaction capabilities, a robust mechanism that ensures stability to leadership decisions would be advantageous. Closeness to a real-world problem This is not a requirement per se but closeness to a real world situation may help understand the role of Emotional-mechanisms. This does not mean that the Emotional mechanisms used should mimic those of Humans. However, since the real world is a very complex world that we know reasonably, we may use this knowledge to decide where Emotional-mechanisms should be employed more advantageously. These properties point towards the need for Emotional Mechanisms, as those described in the previous Chapter. Naturally, Emotional Mechanisms are only a part of a more global and complex Architecture. Therefore, implementing an Agent with Emotional capabilities is by no means simpler than implementing an Agent with no Emotion-based concept present in its Architecture. An Emotional Agent possesses a regular set of functionalities that would be found in Agents of other type. Additionally, an Emotional Agent uses Emotional Mechanisms to interact, control or help other mechanisms in their usual functions. Emotional mechanisms are not intended to substitute other more standard functionalities but may be useful in improving their global performance. In the next chapters we will describe our Architecture and explain how Emotional Mechanisms help to solve some of these issues. 5.7. Summary and Conclusions In this chapter we presented the Pyrosim platform that has been especially built to support the development of Emotion-based Agent Architectures. We presented the motivation for building our own simulation platform instead of using other simulators available. Next, we made a brief overview of the Pyrosim platform and showed how it is composed by three elements: the Pyrosim simulation server, the Agent Skeleton layer and the Pyroviz visualizer. Chapter 5The Pyrosim Platform 105 We focused on the simulated environment and explained several factors that contributed to its complexity. We have also described the basic functionalities provided by the Agent Skeleton in order to build higher-level Agents, namely those functionalities regarding proprio-perception, “ visual” perception, action and communication. We proceeded to analyze why the Pyrosim environment is appropriate for building Emotional Agents. The environment’s main characteristics were enumerated and we pointed out some problems that suitable Agent Architectures will have to deal with in order to be successful in the Pyrosim environment. We finished making some suggestions about where Emotional mechanisms may be advantageously used to help solve those problems. 112 reactive or hard-wired mechanisms, while higher-level modules will all make use of deliberative processes. Of course, this may eventually be the case, but if it happens to be so, it is a result of the requirements imposed by certain goals (e.g.: response time) and not because of an a priori criteria imposed by the designer. The significant feature of the proposed Architecture is that the management of processing time is done upwards, from lower-level to higher-level layers. Modules related to higher-level layers will only run after those located in lower-level layers, due to the relative importance of goals at each layer. In the best possible case, higher-level modules will run as frequently as lower-level ones, but, usually, they will be given processor resources only a fraction of the number of times that lower-level modules will. In fact, we may consider that a higher-level layer is simply an additional “Module” of a lower-level layer that is chained to run along with the other Functional Modules. The scheduling frequency of such a “Module” should depend on how well the active Goals of the lower-level layer (the most basic ones) are being ensured. Figure 6.5. Allocation of processor time among Functional Modules in the two layers. The general idea of this Architecture, thought for environments where Agents have to deal with multiple and simultaneous concerns, is that all modules run in a pseudo-parallel fashion, sharing the existing computational resources according to the current importance of the Goal(s) that they help to achieve. The role Emotions play in this process will become clearer throughout the next sections. The Basic Control Agent Layer As we have mentioned before, in our current implementation of the Architecture we have developed two layers. On top of the Agent Skeleton layer, we have placed a layer named Basic Control Agent (BCA) Layer which was built around one major Goal: to ensure that the Agent will “survive” in its environment. The notion of “survival” depends, of course, on the environment but, in our very Processor Processor Time Chained Processor Time Basic Deliberative Agent Layer Mj Mn Mo BDA Execution Cycle Basic Control Agent Layer Mi Mj Mk BCA Execution Cycle Chapter 6An Emotion-based Agent Architecture 113 anthropomorphic scenario, it simply means that the Agent must ensure its safety and not get caught by the fire. In order to do so effectively, several Functional Modules need to be placed at this layer. Starting with Perceptors, the BCA Layer includes the following specialized Perceptor modules: • Pain Perceptor – tracks Agent “energy” to detect rapid decreases that result from harmful events. • Temperature Perceptor – analyzes temperature and its variations. • Fire Perceptors – analyzes close range and medium range visual maps to extract data about the surrounding fire (intensity of burning cells; distances; starting of new fire spots). The processing tasks associated with these Perceptors are executed during each BCA Execution Cycle, producing (or revising) Beliefs that become globally available through the Working Memory. Also located in this layer is the Goal Manager module that is responsible for scheduling all of the Agent’s Goals. The main Goal at this layer is the Survival Goal. It is constantly monitoring sensory data (temperature, pain), Beliefs (intensity and spreading speed of nearby fire) and the value of Emotional State (the value of “Fear” Accumulator), i.e. parameters that are relevant or connected with the Agent’s survival. Whenever there is a situation that may be very dangerous for the Agent, i.e. to its survival, the Survival Goal will generate another Goal that will make the Agent run to a safer position. These situations are identified by simple (and fast) comparisons made with certain thresholds, such as the maximum sustainable temperature, the maximum safe temperature increase, the maximum safe fire spreading speed or the maximum level of Fear the Agent is willing to reach. Whenever these thresholds are surpassed, the Survival Goal will simply generate a new goal in order to escape to a safer position. The new Goal will be added to the Goal Manager queue and run in parallel with the already existing ones. It is important to emphasize that the Survival Goal has several configurable parameters. Each of the thresholds used to decide whether a new goal should be generated can be changed, promoting different behaviors towards the environment. The Survival Goal is relaxed if thresholds (e.g.: maximum sustainable temperature) are expanded. This will promote risky attitudes from the Agent, as for example, keeping a shorter distance to fire, which may also be important in certain situations. For this particular case, keeping shorter distances to fire-fronts will enable the Agent to fight the fire more effectively. However, these parameters need to be set appropriately so that the Survival Goal will still be ensured. Another essential module placed at this layer is the Action Manager. The Action Manager is responsible for pipelining the commands issued by Goals (by means of Controller Modules) to the Agent Skeleton Layer. The current implementation of the Action Manager does not test for conflicts among commands, such as for example two simultaneous commands coming from different Goals, one requesting the Agent to move backwards while the other is requesting it to move forwards. This will be 114 the subject of future work. For now, there has been great care in avoiding conflicts using the Goal Manager to ensure proper coordination between Goals. Finally, the BCA Layer is also responsible for providing processing time to the upper layer. The Execution Cycle of the upper layer, the Basic Deliberative Agent Layer, is chained in the Execution Cycle of this layer as if it was another functional module. Therefore, the Basic Deliberative Agent Layer will only be given a share of BCA Layer processing time, which it will have to divide among its own Functional Modules. The following image illustrates the Execution Cycle at the BCA layer: Figure 6.6. Execution Cycle at the BCA layer. The Basic Deliberative Agent Layer While the BCA layer is mainly concerned with the most essential Goal of the Agent, survival, the layer placed immediately above, the Basic Deliberative Agent (BDA) Layer, is essentially focused on specific task related Goals. This layer’s Functional Modules are aimed at providing the Agent with the mental capabilities needed to achieve the Goals related to fire fighting. Of course, these Goals are only reasonable if the Agent fulfils the basic Goal of surviving. Consequently, they should be given processing time depending on the Agent’s success in achieving its primary Goal. This may be achieved in two ways: 1. directly, by ensuring that the priority assigned to these goals is never higher than the one assigned to Goals in the BCA layer. This way, the Goal Manager will never allocate more processor time to Goals at this layer than it does to Goals at the BCA layer. Get Agent State from Skeleton Layer Working Memory Basic Perception Check Agent State. Produce Beliefs about Pain, Temperature, Fire at close and medium ranges Beliefs Beliefs Goal Manager Schedule Active Goal by Priority. Update Goal Scheduling Count G G Action Controller Generate Low Level Actions from high Level Commands Basic Deliberative Agent Execution Cycle Chapter 6An Emotion-based Agent Architecture 115 2. indirectly, by chaining the execution of the BDA Layer in the Execution Cycle of the BCA Layer. Thus, additional processing (e.g.: using Reasoners) required for Goal achievement at this layer will only be performed after the processing is executed in the BCA layer, and using only the remaining processor time. Goals in this layer may be divided in two different groups. In the first group we have included those Goals that are more concerned with direct fire-fighting operations, such as for example goals like “Move to a specified location”, “ (Re)Approach a given fire segment”, “Fight a fire in a close range location”. These Goals usually follow naturally from each other and compose the basic fire-fighting sequence. The following picture illustrates these relationships. Figure 6.7. Goal Generation flow at BDA Layer. Goals in the second group are connected with global monitoring of the situation and the corresponding decision-making. In most cases, these goals are responsible for producing the information required by operational Goals. In this second group, we include goals that are constantly being sought, such as “Track fire evolution in close range locations”, “Track Fire Evolution in Medium Range Location”, “Check if repositioning is needed” or “analyze global fire estimates”. Despite their almost permanent nature, these goals’ priority is not very high and their scheduling is highly flexible. In fact, the Goal Manager does not need to schedule them in every Execution Cycle because they are concerned with global or distant realities that do not change significantly in short intervals. However, they are important for strategic decisions such as changing current combat position or ensuring that there is always a runaway corridor available. This kind of information is very important for leadership decisions, if we consider a multi-agent approach. The priority of these Goals is configurable according to the role the Agent may have in a fire fighting team. In order to account for all these goals, the BDA Layer is equipped with Reasoner modules capable of processing more complex information and producing higher-level Beliefs. These Reasoners operate on the data produced by lower-level modules and on information distributed by the Pyrosim simulator, such as a terrain map (pocket map) or a periodic estimate of the fire’s global evolution. They also share Global Fire Combat Go To fire Segment Approach Fire in Medium Range Combat Fire in Close Range Go To X Y Location Re-approach Fire Front The Goal to Reapproach a Fire Front may be generated when the Agent succeeds in achieving a safer position after being forced to run away from an uncontrolled fire front. 116 Beliefs with each other creating a horizontal flow of information. The BDA Layer includes the following Reasoners: • Pocket Map Reasoner: this module is capable of generating path plans between two points in the map, taking into account factors such as the geometry of the terrain, distance to fire, visual occlusion, etc. • Fire Evolution Reasoners: analyze close range and medium range visual maps to extract data about the evolution of the surrounding fire (speed of spread, most dangerous or safest areas, etc.). • Fire Map Reasoner: this module analyzes the global fire estimate in order to detect fire segments (continuous areas), and extract several features about them (size, average intensity, distance, etc.). • Fire Segment Evolution Reasoner: tracks the evolution of fire segments, calculating how fast and where to they are evolving. Finally, the BDA Layer also includes the Execution Memory module to allow reasoning about previous actions. The information stored in Execution Memory is used to decide new goals coherently (e.g.: re-approach a location the Agent had to run from) and it is important for the Agent in performing evaluation of its own success (e.g.: How many time did I need to run away up to this moment?). 6.5. Emotional State and Emotional Mechanisms The Agent’s Emotional State includes three Emotional Mechanisms, which we labelled using the name of Human Emotional Phenomena, for a better understanding of their functional role: 1. “Fear”: Specific Emotion that will be elicited whenever a dangerous or uncontrollable condition (temperature too high or fire too close) is detected. “Fear” is tightly connected to the Survival Goal; 2. “Anxiety”: Specific Emotion (or Mood) should be elicited when the Agent faces a possibly difficult situation (e.g.: running out of water, fire-front approaching an important resource). 3. “Self-Confidence”: Mood elicited as a result of a successful performance. (e.g.: has successfully extinguished the last fire cells). This Emotion promotes optimistic behaviors. For each of these Emotional Mechanisms there is an Emotional Accumulator that functions exactly as described in Chapter 4. The values of Emotional Accumulators are globally accessible for inspection throughout the Architecture. Each layer and their corresponding Functional Modules have constant access to the global Emotional State. Before beginning an Execution Cycle, the values stored in Chapter 6An Emotion-based Agent Architecture 117 the Emotional Accumulators may be used to configure parameters related to the Execution Cycle itself, the operating mode of the associated Functional Modules, and several parameters of corresponding Goals. Figure 6.8. Values stored in Emotional Accumulators may be used for several purposes in Layers and Functional Modules. Emotional Accumulators are updated by Emotional Evaluation functions dispersed all over the Architecture. These Emotional Evaluation Functions are located in both layers and inside Goals, Perceptors, Reasoners and Managers, each contributing to the global Emotional State, according to their scope of influence. For example, the elicitation of “Fear” is deeply connected with local perception. The Functional Module responsible for the perception of nearby fire, the Fire Perceptor, is able to inject impulses in the “Fear” Emotional Accumulator. Therefore, this module contains one of the Emotional Elicitation Functions related to the Fear Emotional Mechanism. However, this is not the only Emotional Evaluation Function connected to “Fear”, since it is possible to have others all over the Architecture, for instance, in the Temperature Perceptor or directly in the Survival Goal. The following picture illustrates the various sources of Emotional Evaluation that provide stimuli over a single Emotional Accumulator. Figure 6.9. Perceptors, Pi, Reasoners, Ri, and Goals may contain Emotional Evaluation Functions that interact with a single Emotional Accumulator Emotional State Basic Control Agent Layer Mi Mj Mk Basic Deliberative Agent Layer Ml Mn Mo G G Pi Pi Ri Ri 118 6.6. Architecture Overview The next figure gives a final overview of the Architecture. The boxes represent generic Functional Modules of a given category. The two shaded regions correspond to the BCA and the BDA layers. Figure 6.10. Overview of the Emotion-Based Agent Architecture Chapter 6An Emotion-based Agent Architecture 119 6.7. Summary and Conclusions In this Chapter, we have introduced our Emotion-based Architecture focusing mainly on its components. We have also presented all the Functional modules and explained how they are grouped in two layers, the Basic Control Agent Layer and the Basic Deliberative Agent Layer, according to the Goals they help to achieve. The Basic Control Agent Layer is mainly concerned with basic Goals, such as ensuring Agent survival in the environment. The Basic Deliberative Layer, which is built on top of the previous one, is related to higher-level Goals, and related to fire combat. Each Layer includes several Functional Modules and is responsible for managing its own processor time. However, in this highly distributed Architecture, processor time has not only to be divided between Functional Modules that run inside each layer, but it also has to be shared by the layers themselves. The Basic Deliberative Layer will be set to run chained on the Execution Cycle of the Basic Control Layer, ensuring that the lower layer basic Goals will always have direct (and indirect) priority in accessing processor time. Additionally, the Architecture holds several configurable parameters. Functional Modules, Goals and both layers have parameters that may be changed. We have also briefly explained how the Emotional Mechanisms, described in Chapter 4, are integrated within the Architecture. We introduced the three Emotional Mechanisms that are used in the Architecture: “Fear”, “Self-Confidence” and “Anxiety”. We have described how Emotional Evaluation Functions are distributed between all components and how they stimulate individual Emotional Accumulators simultaneously. However, we did not address the details of the interactions established between Emotional Mechanisms and Functional Modules, Goals and Layers. This will be the subject of the next Chapter. Finally, we have presented a global perspective of the Architecture in which the interaction between all Functional Modules and the layer may be identified more easily. Chapter 7 – The Emotional Mechanisms 121 CHAPTER 7. THE EMOTIONAL MECHANISMS 7.1. Introduction In this Chapter we will describe the Emotional Mechanisms included in our Architecture. We will start by revising and emphasizing some of the Architecture’s properties that enable the inclusion of Emotional Mechanisms. In particular, we will focus on various possibilities of configuring the Architecture in order to shape different and adapted behaviors, such as those resulting from different Emotional States. We will consider the configuration possibilities at three different levels: Goal, Functional Modules and also Layers. We will then explain the three Emotional Mechanisms that operate inside the Architecture: “Fear”, “Anxiety” and “Self-Confidence”. For each of these Emotional Mechanisms, we will address issues regarding Emotional Elicitation, which is made through various Emotional Evaluation Functions, distributed by the several components of the Architecture. We will then address the interaction of Emotional Mechanisms in the configuration of the various parameter types available in the Architectures. Specifically, for each Emotional Mechanism, we will show how the values stored in the corresponding Emotional Accumulator may be used to dynamically adapt the Agent’s behavior. We will also relate these interactions with the information processing Strategies, described in Chapter 2. Finally, we will summarize the major points about Emotional Mechanisms presented in this Chapter. 7.2. A Configurable Architecture Before starting to explain the precise functionality of Emotional Mechanisms, it is important to emphasize the configurability properties of our Architecture at three different (but interconnected) levels: Functional Modules, Goals and Architectural Layers. The operation of individual Functional Modules is usually highly configurable and dependent on several parameters. For example, the Fire Map Reasoner calculates the path plan using an “oriented” depth-first search in the action-state space. In addition to input data such as terrain maps, 128 Table 7.4 Summary of eliciting conditions for “Anxiety” Nature Condition Wind High wind speed. Irregular direction. Geography and Agent Location Irregular Geography. Being located in a point above the level of a nearby fire. Fire Analysis and Reasoning Large areas burning (even far away). Fire spreading very quickly in medium range locations. Fire approaching strategic locations Help Availability Small team for the size of the fire Proprio-Perception Low levels of Energy. Running out of water “Fear” Emotion Successive Fearful episodes As shown in the previous table, “Anxiety”, as “Fear”, has multiple elicitation sources. However, “Anxiety” is connected to more indirect conditions that do not pose immediate threats. It may thus be considered an Emotional Mechanism closer to Capabilities and Goals, located in the Basic Deliberative Layer, contrary to the Fear Emotional Mechanism that is closer to the Basic Control Agent Layer. It is interesting to note that “Anxiety” is also influenced by the “Fear” Emotional Mechanism. In fact, if the Agent is going through successive fearful episodes, it is very probable that it did not adapt to the current environment and, therefore, it is very natural that it might find another threatening situation in the near future. Successive high levels of fear are therefore conditions that should contribute to increase the “Anxiety” level. Figure 7.4 Elicitation sources for the “Anxiety” Emotional Mechanism 7.4.2. The Interactions In a certain way, “ Anxiety” works as a “preventive” mechanism. High “Anxiety” levels will promote a very intensive processing state so that the Agent will be able to detect possible Goal threatening situations or favorable opportunities. The Agent will invest a great deal of its processing resources in analyzing perception data and producing higher-level Beliefs to use in the decision of new “ Anxiety” Emotional Accumulator BCA Proprio - perception Fire Evolution Reasoner Pocket Map Reasoner Vision Maps Other Agent Medium Range Fire Perceptor “ Fear” Chapter 7 – The Emotional Mechanisms 129 Goals. This is generally achieved by increasing the execution frequency of de Basic Deliberative Layer, where Reasoners are scheduled to run. As a result, the Agent will be configured to produce updated Beliefs about the areas where the fire is progressing faster, which are the most dangerous ones (where fire seems to be converging), and which seem more worth fighting (e.g.: less damaged). The Agent will also increase the breadth and depth of the path planning procedure in order to increase the final plan’s quality. At the same time, plans are revised frequently to integrate the knowledge of the updated Beliefs produced by Reasoners. The increase in the processing load needs to be compensated by reducing the processor time allocated to other Functional Modules. Since the Agent is not going through a very urgent situation, which would be signaled by “Fear”, it is possible to decrease the frequency of the Perceptors and Controllers and thus relax the load of the Basic Control Layer. This extra computational power may now be spent in the Basic Deliberative Layer and be allocated to Reasoners. Clearly, this situation may not be sustainable for long periods as the Agent will eventually need to focus all its resources on a nearby fire spot. However, as we have seen previously, “Fear” will take care of that. Table 7.5 The effects of “Anxiety” Level Object Changes Effect Pocket Map Reasoner (Path Planner) Increase depth and breadth of planning procedure. Increase the weight of safety parameter in the cost function. Test a great number of possibilities to obtain the “best” plan Modules Fire Evolution Reasoners Turn on available processing options. Increase detail in the analysis procedure; Produce estimates about fire progression. BCA Decrease all Perceptors Frequency; Increase BDA Scheduling. Release resources at the level of BCA, Give processing Time to BDA Layers BDA Increase all Reasoners Scheduling Increase the Priority of the Goal “Track Fire Evolution in Visible Range” Keep the estimate of fire evolution reasonably updated There is a certain amount of overlap in the effects of “Anxiety” and “Fear”. However, for example, the planning procedure is changed in opposite ways by “Fear” and “ Anxiety”. The behavior of the BCA layer in the case of “Fear” is also almost the opposite of the one in the case of “Anxiety”. Obviously, it will not be possible to change simultaneously all available parameters in opposite ways. However, since the Emotional State is globally available to all elements in the Architecture, some additional rules may be imposed when incompatible cases arise. And because “ Fear” is directly connected with urgent situations, it should always subsume “Anxiety” in its effects. The following snippet of code illustrates such condition: IF (ANXIETY==HIGH) AND (FEAR < MEDIUM) THEN PLAN_BREADTH = HIGH 130 The value stored in the “Fear” Emotional Accumulator will naturally decay with time and, when it drops below a certain level, “Anxiety” will be able to produce its effects. 7.4.3. Additional remarks “Anxiety”, in a certain way, is a negative Emotion as well. Like “Fear”, it should promote a certain conservative behavior in the Agent because, although no direct threat is at stake, the situation will probably become difficult to deal with in a close future. Among the four information processing strategies described in Chapter 2, Anxiety would be related to Substantive Processing. When performing under Substantive Processing information processing strategies, Agents operate in a systematic fashion with complex information processing structures, without having a specific goal to attain at that moment. As described in the previous sections, when the “Anxiety” level rises, the Agent responds by increasing the amount of processor time allocated to the BDA Layer and, at the same time, increasing the complexity of planning procedures and fire evolution analysis. In our opinion, this clearly corresponds to adopting a Substantive Processing strategy. Additionally, Substantive Processing strategies make a significant use of Emotional Information in the overall processing. However, that is not so visible in our Architecture because the value of the “Anxiety” Accumulator itself is not directly used as an input for any process. We may identify a relationship between “Anxiety” and “Fear” in the elicitation process of “Anxiety”. This should not be seen as an example of the Emotion as Information paradigm. On the other hand, the subsumption rules described at the end of the previous section may be seen as an indirect application of such paradigm. 7.5. Self-Confidence 7.5.1. The Elicitation “Self-Confidence” is an Emotional Mechanism directly related with the success achieved in previous Goals. If the Agent is successfully accomplishing Goals, the level of Self-Confidence should be increased to reflect that the Agent has enough capabilities to cope with the environment. There are a few situations that might indicate that the Agent is having success in its main goal, which is fire fighting. Firstly, an Agent (or group of Agents) that is able to diminish the intensity of fire, or even extinguish the fire, at the cell that it is currently fighting, should receive a positive indication about its fitness to cope with the fire. Such an indication should be materialized in an increase in the “Self-Confidence” level. Similarly, if fire at close and medium range locations is receding due to successive fire extinguishments recently achieved, the Agent should also get an Chapter 7 – The Emotional Mechanisms 131 increment in its “Self-Confidence” to reflect its capability to cope with the environment. The level of “Self-Confidence” should be changed, even if the Agent is not having direct influence in a positive outcome, such as for example a significant decrease in the intensity of a distant fire segment as a result of a wind change or specific terrain geometry. In this case, the level of “Self-Confidence” should also increase because the environment has globally become more tractable, even if this does not represent an immediate advantage for the Agent. Additionally, if other external conditions arise, helping or enabling the Agent to be more successful in fire fighting, the level of “ Self-Confidence” should also be increased. For example, whenever the concentration of firemen Agents increases in the surroundings of a certain location, the global fire fighting potential in that area grows. Consequently, each of those Agents should feel more comfortable to face the current situation and their level of “Self-Confidence” will increment. On the other hand, situations such as moving back or running away from a fire-front should have opposite effects on the level of “ Self-Confidence” than the ones mentioned before. If, when combating a given fire cell, an Agent constantly needs to move back, this may mean that he is not able to cope with the fire-front. This is especially true if the Agent is forced to run away from its location often. In both cases, the Agent is obviously unable to cope with the situation. Therefore, the level of “Self-Confidence” should reflect this fact and be decreased. Another indirect indication of the possible inability to cope with the environment may be drawn from the value of the “Fear” Emotional Mechanisms. Multiple fearful episodes mean that the Agent is frequently going through very difficult situations and this will happen when the Agent is not capable of dealing with the situations that are constantly arising. Thus, the level of “Self-confidence” should decrease whenever the level of “ Fear” reaches a given threshold. It is interesting to note that the detection of most of these situations requires the ability to track the evolution of the environment. Therefore, “Self-Confidence” is an Emotional Mechanism that is tightly connected with Reasoning Capabilities, i.e., with the Basic Deliberative Agent Layer. We had already suggested this relationship when we presented a global overview of our Architecture, in Chapter 6. The “Self-Confidence” Emotional Mechanism is placed inside the Basic Deliberative Agent Layer. The next picture tries to summarize the factors that contribute to changes in the Emotional Accumulator’s values: Figure 7.5 - Elicitation sources for the “Self-Confidence” Emotional Mechanism “ Self-Confidence” Emotional Accumulator Vision Maps Other Agent “Fear” Fire Evolution Reasoner Execution Memory Medium Range Fire Perceptor Close R ange Fire Perceptor 132 7.5.2. The Interactions Therefore, high-levels of “ Self-Confidence” signal that the current environment poses no significant difficulties to the Agent. In such a situation, the Agent could simply relax its information processing strategies. The processing resources thereby released may be used in other tasks or Goals that are not urgent but that may become advantageous later. For example, if the Agent is not having problems in extinguishing successive fire cells, some of the processor time that was being spent in analyzing local data could be allocated to analyzing the fire progression in the medium range surroundings. This will help the Agent to find other possible threats or opportunities. Another possible use for the released processor time is to apply it in Learning. This is a promising possibility that we are still experimenting [Mou03]. Moreover, if the Agent is actually having success systematically, then it should adopt a more optimistic approach towards the environment and stretch its own previous limits. In our Architecture, high values of “Self-Confidence” increase the temperatures that the Agent is willing to withstand before starting to move back or run away. As a result, the Agent will remain closer to fire and will be able to fight it much more efficiently. In this way, “Self-Confidence” helps the Agent in having a more “aggressive” behavior in the search for favorable, yet unknown, opportunities. Additionally, Agents will also loose the normal constraints regarding distance to fellow firemen. Usually, firefighters should keep close to each other to protect themselves better and to concentrate their efforts in extinguishing one fire cell quickly. High levels of “Self-Confidence” will increase the maximum allowable distance between firemen resulting in a wider combat front, which is preferable, if possible. This also suggests that high levels of Self-Confidence are useful for experimenting new possibilities. For example, if firemen are feeling comfortable with the current team behavior (signaled by high levels of SelfConfidence), then it seems appropriate to try at that moment a more risky, but possibly more effective, combat tactic. This could lead to quicker fire extinguishments, or to the “discovery” of new effective tactics to control fire. However, during this work, we have focused mainly on the behavior of individual Agents and we did not explore the realm of team coordination. On the other hand, “ Self-Confidence” decreases whenever the Agent is forced to run away. Several unsuccessful attempts to extinguish fire cells mean that the Agent is not capable of coping with the current situation. “Self-Confidence” will be decreased which will have opposite effects to those described before, but will also trigger a Goal revision process. A new Goal, more adapted to current Agent possibilities, needs to be generated. For example, the Agent might change the combat strategy or move to a more favorable location. In our current implementation, the Agent will move to a region where the fire is progressing slower. Note that there is a significant cost regarding locomotion as the Agent will not be able to fight fire when moving, causing it to spread faster. Therefore, “SelfConfidence” is an interesting trigger for Goal revision because it includes information about the Agent’s previous successes and not just about the immediate state of the environment. The following table tries to summarize the effects of high “ Self-Confidence” levels. Chapter 7 – The Emotional Mechanisms 133 Table 7.6 The effects of (high) “Self-Confidence” Level Object Changes Effect Goals “Combat Close Range Fire” Increase temperatures (TOPT and TMAX) Increase distance to fellow firemen Approach Fire front; widen fire combat front Resist more time before running away. Modules Fire Evolution Reasoners Turn on available processing options. Increase detail in the analysis procedure; produce estimates about progression of fire BCA Decrease the all Perceptors Frequency of all Perceptors; Increase BDAthe Scheduling Frequency. Release resources at the level of BCA, Give processing Time to BDA Layers BDA Increase all Reasoners Scheduling or allocate time to Learning procedures10 Do not give up current Goal (current firefront). Explore new possibilities (different team positioning). 7.5.3. Additional remarks Contrary to the other two cases, “Self-Confidence” is a positive valence Emotion. Therefore, it should promote one of the two minimum-effort processing strategies mentioned in Chapter 2, either Direct Access or Heuristic Processing. This is a justifiable response, because if the Agent is having success with the environment, it may relax its information processing strategies (e.g.: Motivated or Substantive Processing) and assign the released resources to other less urgent tasks or Capabilities. The difference between Direct Access and Heuristic Processing lies in the amount of Emotional Information used and the complexity of the environment where the Agent is operating. Heuristic Processing makes use of a large amount of Emotional Information and is intended for complex environments where the Agent may not be able to employ pre-existing knowledge structures or behaviors. On the other hand, Direct Access relies much more on previous experiences that it tries to apply directly without making use of Emotional Information. In the case of our Agent, the effects of the “Self-Confidence” Emotional Mechanism would place it somewhere closer to Heuristic Processing. In this case, we are using the value of the Emotional Accumulator for triggering (not modulating) a Goal revision process, so there is a clear example of the Emotion as Information paradigm. In fact, since the environment is quite complex, it would be very difficult to apply other more direct decision techniques, such as rules about the state of the environment, because they would be too complex to obtain and possibly too expensive to check in real-time. It would be very difficult to define the exact conditions that indicate that the Agent should generate a new goal to move to another fire-front or alternatively keep its current positioning. 10 In this work, we have not addressed directly the issue of Learning, although some simple experiments have been made using Pyrosim platform. For a preliminary study, please refer to [Mou03]. 134 Additionally, although we have not implemented it yet, the released processing resources should be used to promote an explorative behavior. Agents under the influence of “Self-Confidence” should be able to try variations of tactics, even if they result in less safe options. Of course, these decisions would have to be taken by a leader in the context of a team effort. We have not explored the issue of team coordination in this work. 7.6. Summary and Conclusions In this Chapter, we have presented three Emotional Mechanisms included in our Architecture: “Fear”, “Anxiety” and “ Self-Confidence”. We have described the conditions by which such Emotional Mechanisms are activated, namely through various Emotional Evaluation Functions spread all over the Architecture. Each Emotional Evaluation Function is able to detect specific situations that may pose a threat (in the case of “Fear” and “Anxiety”) or may suggest an opportunity (in the case of “SelfConfidence”), indicating thereby that the Agent behavior should be readapted to the environment. For each Emotional Mechanism we have demonstrated how such conditions could be detected from data gathered through Perception. We have then demonstrated how the values stored in Emotional Accumulators, which resulted from the contributions of Emotional Evaluation Functions, could be used to change specific parameters of the Architecture at three different levels, Goal, Functional Modules and Layers, in order to adapt Agent’s behavior to the changing environment. Modification of Architecture parameters is achieved in real-time and allows the Agent to dynamically adapt its Resources and Capabilities to the current situation. This adaptation aims at getting the most out of the Agent’s Capabilities, and revealing therefore a very functional property of Emotional Mechanisms. Additionally, for each Emotional Mechanism, we have tried to establish a relationship between the information processing pattern they generate and Human Information Processing Strategies, described in Chapter 2. We have identified that “Fear” promotes a processing strategy similar to Motivated Processing, and “Anxiety” to Substantive Processing. Both of these strategies are extremely resource consuming which is compatible with the need to employ all available resources when difficult situations arise. On the other hand, “Self-Confidence” promotes a processing pattern that is closer to Heuristic Processing. In this case, information processing procedures are relaxed and there is a considerable use of Emotional Information in the decisions taken (e.g.: move to another firefront if “Self-Confidence” drops below a given value). Finally, the essential point that needs to be emphasized is that Emotional Mechanisms belong to the realm of highly configurable Agent Architectures, which are usually related with very complex environments. The role of Emotional Mechanisms is mainly that of adapting the global information processing capabilities of the Architecture in real-time, as well as its individual Functional Modules. Chapter 8 – Conclusions 135 CHAPTER 8. CONCLUSIONS 8.1. Overview The work reported in this thesis addresses a complex topic: Emotional Mechanisms. Our goal was to develop a Software Agent Architecture in which Emotional Mechanisms could be used as a functional advantage to Agents, especially for those operating in complex environments. A great deal of our work has been devoted to exploring a vast amount of dispersed bibliography about Human Emotion so that a deeper understanding of Emotional-Mechanisms and their interaction with Cognition could be achieved. Much of this research about Human Emotion has been summarized in Chapter 2. At the same time, we investigated other Models and Architectures of Emotional Mechanisms that have been developed in the field of AI. As described in Chapter 3, a significant amount of work has been produced in which Emotional Mechanisms play a functional role. However, most of the work done so far, usually addresses one or only few very specific topics regarding Emotional Mechanisms. To our knowledge a complete approach has still not been developed. All this research allowed us to identify the building blocks of Emotional Mechanisms and understand how they could interact with other elements that compose generic Software Agent Architectures. Our perspective has always been that all the modeling effort should have a functional purpose in mind. Using this approach, we have developed a general model of Emotional Mechanisms and of their functional interactions, introduced in Chapter 4. We believe that this general model is one of the most important results of our work. In order to test our models, we decided to develop a software platform for simulation upon which we would develop the Emotional Agent Architecture. The platform developed enables the simulation of a fire-fighting environment, which is complex enough to justify the employment of Emotional Mechanisms. This platform also offers a software layer (the Agent Skeleton Layer) that provides all the basic functionalities needed to build specialized Agents. We described the platform and the simulated environment in Chapter 5. The simulator allowed us to develop a concrete Emotional-based Agent Architecture with a specific scenario in mind. The Architecture developed is rather general and can be transposed to other application scenarios. It is composed of several Functional Modules that are grouped in two layers, 136 according to the Goals they help to achieve. The Basic Control Agent Layer is built on top of the Agent Skeleton Layer (provided by the simulation platform) and is connected to the Agent’s Fundamental Goal in the fire-fighting environment, survival. This layer possesses all the necessary capabilities to address this Goal. At the same time, this layer is responsible for managing and providing processor time to the layer placed immediately above, the Basic Deliberative Layer, which is connected with higher-level goals and Capabilities. An important conclusion is that this specific layered Architecture is highly configurable and can be set to operate under many possible modes. Configuration may be done at three levels by changing the parameters associated to (i) individual Functional Modules, (ii) Goals and (iii) Architectural Layers. Details about the Architecture may be found in Chapter 6. The Emotional Architecture includes three different Emotional Mechanisms, namely “Fear”, “Anxiety” and “Self-Confidence”. Each Emotional Mechanism tries to address specific environment situations, either threats or opportunities, and alters some of the Architecture’s parameters available at the three aforementioned levels. These changes are intended to adapt the Agent’s current Capabilities to the specific state of the environment. The exact conditions that lead to the elicitation of a given Emotional Mechanism are identified by several Emotional Evaluation Functions, distributed throughout the Architecture’s elements. The values stored in the corresponding Emotional Accumulators are used to guide the parameters of the Architecture configuration. Some experiments with simple scenarios lead us to the conclusion that considering Emotional Mechanisms really matters regarding the Agent performance. These Emotional Mechanisms were detailed in Chapter 7. In summary, the main results of this work are: • a generic Functional Model of Emotional-Mechanisms; • an Emotion-based Agent Architecture where Emotional Mechanisms have a functional purpose; • the establishment of a clear connection between Emotional Mechanisms and specific components of distributed mentalist-like Agent Architectures; • the application of three Emotional-Mechanisms, whose functionalities are strongly related with Agent’s performance, in a specific application scenario: “Fear”, “Anxiety” and “Self-Confidence”; • a complete simulation platform based on a relevant real-world problem. 8.2. Current State of Implementation and Limitations As of this writing, we have implemented most of the Architecture and most of the Emotional Mechanisms. All the Functional Modules described in Chapter 6 were implemented. Additionally, methods for configuring the parameters of Functional Modules, Goals and Layers have been provided. Chapter 8 – Conclusions 137 We have also implemented the mechanisms related with the elicitation of “Fear” and with most of the interactions described in Chapter 7. We have also implemented the most significant features about the “Self-Confidence” Emotional Mechanism, specifically those related with the Goal revision process. However, we have not yet implemented the “Anxiety” Emotional Mechanism, nor have we developed means to clearly visualize Emotional interactions. We are currently trying to solve this last issue. As for the Pyrosim platform, it is globally functional and all the features described in Chapter 5 have been implemented. In addition, an Agent Programming Manual has been produced and is provided with the Platform to help programming new Agents in Java. 8.3. Future Research Direction There is still a lot of work to be done in order to refine and sophisticate the proposed Emotionbased Architecture. Besides refining current Emotional Mechanisms and the set of their possible interactions, it would be interesting to study and develop other Emotional Mechanisms such as “Frustration” and “Angriness”. “Frustration” could absorb some of the functionalities that have now been included in “Self-Confidence”, namely those related with the negative situations, i.e. those that promote a decrease in “Self-confidence”. This would make our model cleaner by avoiding the use of bi-directional Emotional Mechanisms, whose Emotional Accumulators are both increased and decreased by corresponding Emotional Evaluation Functions. Another interesting Emotional Mechanism that could lead to good results is “Angriness”, which is usually related to blocking the environment’s negative effects. “Angriness” is important in motivating the Agent to maintain a consistent (“aggressive”) behavior towards the environment, even when the situations are clearly negative. This could be useful in the discovery of sudden environment drifts. Supplementary work on Emotional Mechanisms may be developed if we choose to enlarge the scope of such Mechanisms to Agent Coordination. If we consider teams of Agents instead of individual Agents, what will be the function of Emotional-Mechanisms? We have already explained that certain Emotions could be useful in the context of team coordination. For example, “SelfConfidence” is important in promoting Goal revision and, if applied to the decisions of a team coordinator, could involve moving an entire team from its current fire fighting positioning to another possibly less dangerous fire-front. However, how this and other Emotional Mechanisms could be used to enable more subtle tactical variations remains an unanswered question. Finally, it would also be interesting to address some issues regarding Personality. If we consider that Personality is the set of possible behavior variations among Agents sharing the same Architecture, it seems easy to implement this notion in the context of Emotional Agents. In fact, as shown in Chapter 4, we included several parameters in the model of Emotional Accumulators. These parameters are able to control how fast a given Emotional Mechanism is elicited and how long it remains active. Appendix A – The Mathematical Model of Pyrosim 145 APPENDIX A. THE MATHEMATICAL MODEL OF PYROSIM A.1. Introduction In this Appendix we will present the model developed to build the Pyrosim Platform. We will start by describing a fire-fighting environment and try to extract the main features that need to be modeled in order to develop a believable simulation. Pyrosim is not intended to be a realistic simulation in the sense that it will run as close as possible to the real world. Instead, we are only interested in replicating the reality approximately, just enough to create situations with similar complex requirements. For this reason, we do not need to develop very complex models. A high degree of complexity could even be problematic because the consequent computational effort probably would not allow the simulation to run in real-time for reasonably sized scenarios. We will then present the derivation of the fire propagation and extinction models. Many simplifications will be made in order to develop simple models. Nevertheless, the propagation model incorporates the influence of vegetation, terrain geometry and wind. Next, we will present the model that supports the calculation of Agent mobility. Agents are able to control their acceleration and direction while suffering from constraints imposed by terrain geometry. A.2. Some facts about forest fires A.2.1. The Process of Ignition The process of ignition of a combustible solid material is a complex phenomenon, involving both chemical and physical reactions. In a first state, it involves the chemical decomposition of the material surface into flammable volatiles that are then ignited and start the actual surface combustion. These volatiles are released from the solid’s surface when the temperature increases, however, they only ignite when their temperature reaches a certain threshold point – the flashpoint, Tf. The ignition of these volatiles will increase the solid’s temperature that will eventually ignite if it reaches another threshold temperature known as the fire point, Tig. Depending on its nature, the solid will possibly attain a sustained combustion state after ignition. 146 The temperature increase in the solid’s surface, which leads to the ignition of the volatiles, may have three different causes that give rise to three forms of ignition. Namely: 1. Pilot Ignition: due to the presence of a pilot flame; 2. Spontaneous Ignition: by heat transference from the atmosphere by effect of radiation or convection; 3. Surface Ignition: by a combination of the two previous factors. In forests, fire-fronts develop and spread through Surface Ignition. Vegetation immediately ahead of the fire-front is ignited not only because of direct exposure to the fire-front itself, but also due to the high temperatures in its surroundings, which may reach values above 500ºC within a few meters. On the other hand, fire usually starts by the spontaneous ignition of certain fuel elements in the forest with lower fire points. This happens frequently in dry seasons when the reduction of moisture in certain organic elements covering the ground may decrease their flashpoint / fire point temperature, thus increasing the probability of spontaneous ignition. Figure A. 1 The typical progression of a fire-front. A.2.2. The Spreading of the Fire-Front As seen in the previous section, the spreading of a fire-front is based on the surface ignition of fuel elements in its close surroundings. Despite this apparently simple process, the overall fire-front behavior results from a complex interaction between vegetation, topography and meteorological conditions. Let us briefly analyze each one of these items. A.2.2.1. Vegetation Vegetation is the major source of unpredictability in fire spreading. It introduces several variables whose value and impact are very difficult or impossible to determine. For instance, during their life span, plants are exposed to stress caused by biological agents and meteorological conditions (draught, wind… ), which greatly influence both their resistance and behavior towards fire. Other important factors include soil conditions, the growth rate, the amount of dead material present in the plant as well as on the ground, and also the existence of physical damage. Appendix A – The Mathematical Model of Pyrosim 147 When considering the fire behavior of a particular plant, it is important to take into account six basic variables in order to correctly understand the spreading of fire in a forest: 1. Moisture. Water is the most important element in preventing combustion of organic elements because it acts as an effective heat sink. With increasing temperatures, the water in the plants will volatilize at a rate that depends inversely on the square of the size of plants constituents. A branch with 6 mm will lose water twice as fast as a 12 mm branch. 2. Extractives. Extractives inside a plant present a wide variety of possible fire behaviors. Some extractives in plants will volatilize at temperatures inferior to that of the moisture while others will only volatilize near the point of thermal degradation of wood. The combination of moisture and some extractives produces an interesting fire delaying mechanism that diminishes the combustion process. However, when the vegetation is substantially dry, extractives can contribute significantly to combustion by producing 30% more heat than the heat produced by dry wood itself. 3. Foliage. Foliage is the most fire critical component of the plant. In fact, the thin material that composes most foliage favors a very quick loss of moisture, making foliage a fast fuel. Dry or dead foliage is especially flammable, contributing to a rapid spread of flame to the rest of the plant and, eventually, to other plants. 4. Twigs and Branches. Both twigs and branches, depending on their width and volume may represent a significant portion of the fuel. When they are still green they do not contribute very much to the combustion but dry or dead twigs and branches may burn with intermediate intensity. 5. Stem. The main stem is usually the major part of the plant’s biomass. It is quite resistant to flash combustions, as the ones resulting from burning foliage or grass. On the other hand, when ignited, they provide a large amount of fuel that will burn continuously for a long period. Generically, plants with the following characteristics will favor a rapid spread of fire: • High surface to volume ratio; • Low moisture content; • Presence of a high percentage of dead material. Dry grass and foliage are, therefore, extremely dangerous in fire propagation. 148 A.2.2.2. Topography It is a well-know fact that fire propagates faster in the upward direction. Therefore, the local topography deeply influences the spreading of the fire-front: fire will tend to advance in the direction of positive slope, climbing up hills faster than it proceeds over flat regions (S1 > S2). Figure A. 2. The speed of the climbing fire-front (S1) is higher than the speed of the fire-front progressing in the flat area (S1). The exact relation between the terrain slope and the speed of spread of fire is difficult, if not impossible, to obtain. In spite of this, laboratory tests may help us develop a finer picture of this relation. The following table presents some laboratory results about the speed of spread of fire in strips of filter paper [Roh93]. Table A. 1. Spreading Speed as a function of the burning surface slope. Values refer to burning strips of filter paper. Orientation (º) Spreading Speed (m/s) 0 3.6 22.5 6.3 45 11.2 75 29.2 90 46-74 Although not shown in Table 1, it is interesting to note that for negative slope values (downward direction) below -30º the speed of spread reaches a down limit corresponding to a slow, yet stable, spreading. Considering the entire slope range (-90º, 90) the expected speed of spread would be something similar to the exponential curve presented in Figure 3. The values presented refer to the spread over strips of filter paper. Speed of Spread in mm/s for Slope Values between -90º and 90º 0 10 20 30 40 50 -90 -75 -60 -45 -30 -15 0 15 30 45 60 75 90 Figure A. 3. The speed of spread of the fire-front over the entire slope range (-90 to +90). Appendix A – The Mathematical Model of Pyrosim 149 A.2.2.3. Meteorological Conditions Weather conditions have direct impact on the spread of fire. As far as the ignition process is concerned, air temperature and relative humidity are crucial factors since they have direct impact on fire point temperature. Thus, high air temperatures, combined with reduced humidity, will favor a rapid spread of fire. Once again, this relation is very difficult to establish and depends heavily on the existent type of vegetation. For analysis purposes, and despite their fundamental role in the fire process, these factors are usually greatly simplified. Another very important factor is wind influence. Besides providing transportation for burning or incandescent elements (which may start fires in nearby locations), wind dramatically alters the firefront’s speed and direction. When blowing from behind the fire-front, the wind will enrich the combustion process. This will increase heat production and its transfer to the region immediately ahead contributing to greater fire spreading speeds. On the other hand, if the wind velocity reaches certain high speeds, the heat around the firefront will be dissipated, significantly slowing down the speed of spread or even extinguishing the flame. Figure 4 shows the typical spreading behavior of a fire-front for increasing wind speeds. Figure A. 4. Spreading Speed Vs. Wind Speed A.2.3. The Point of Extinction Extinction is basically the inverse process of ignition, again involving two critical temperature thresholds. By cooling a flame below the fire point temperature, the essential condition for a sustained combustion is suppressed and the flame will go out. The flammable vapors, however, will continue to be expelled, keeping the possibility of a later re-ignition. By further reducing the temperature it is possible to stop the release of the flammable volatiles, and thus effectively prevent the ignition of a new flame. There are several ways to achieve cooling. When fighting forest fires, the most frequent cooling process consists in throwing water over the burning surface. The water will cool the burning surface by absorbing its heat during its transformation into vapor. At 25ºC, water can absorb 2.4 kJ/g of energy in the evaporation process. The rise of the resulting vapor into the atmosphere will also favor the dispersion of heat. 150 Using water to fight fire has important advantages over the use of chemical suppressants. Besides being much more economical, water does not pose any threat to the fire fighting personnel nor to the environment. This is very relevant as some highly effective chemical suppressants often create extremely dangerous atmospheres for humans, which can greatly complicate firefighting procedures. A.3. Pyrosim Simulator All the simulation in Pyrosim is made within a quadrangular grid containing N X N cells. Cells are the simulation’s basic entities and all its content is considered homogeneous along the cell. Each cell has its own atmosphere temperature and moisture variables. A cell may be populated by several types of vegetation elements and also by underground fuel. Fire propagates from cell to cell according to a process that will be described later. Figure 5 tries to illustrate a Pyrosim cell in a simplified fashion. Figure A. 5. The basic elements of a Pyrosim cell. A.3.1. Topography The N x N grid used in Pyrosim takes into account the information about the relative altitude of each cell. In this way, the overall grid may represent realist regions that include all types of topographic elements. The altitude information is essential to simulate different speeds of fire spreading, as described before. Currently, Pyrosim does not support data from GIS. Instead, and for now, it generates its own topography according to user parameters. Figure 6 shows a 3D representation of a terrain generated by Pyrosim. Appendix A – The Mathematical Model of Pyrosim 151 Figure A. 6. An example terrain generated by Pyrosim A.3.2. Vegetation As explained in the previous section, vegetation takes the leading role in the fire-front progression. To produce a realistic simulation, with an interesting set of fire behaviors, several types of vegetation must be considered. As we have seen, different vegetation species show distinct burning properties. Pyrosim explores: • Temperature of Ignition – the temperature at which the vegetation will start burning: TMAX (ªC). • Burning Rate – the amount of heat produced at each second by the burning plant: BR (kJ/s). • Overall Energy – the overall energy that a vegetation element has to burn initially: ETOTAL (J). A Pyrosim cell supports simultaneously four types of vegetation: underground vegetation, grass, twigs and trees. Table 2 presents a brief comparison between the mentioned types of vegetation in respect to the three burning properties listed before. Table A. 2. Vegetation types in Pyrosim and its basic properties. TMAX BR ETOTAL Underground Medium Low Medium Grass Low High Low Twigs Medium Medium Medium Trees High Medium High Each vegetation element has two possible states: burning or not burning. Consider a vegetation element VEG(i) with a given TMAX(i), BR(i) and ETOTAL(i). Let TENV be the environment temperature 152 of the corresponding cell and let EBURNED(i) be the amount of energy already consumed by the fire. The Ignition Rule used by Pyrosim is: IF (TENV >= TMAX(i)) AND (EBURNED(i) < ETOTAL(i)) THEN IGNITE VEG(i) Extinction of a ignited vegetation element VEG(i) can happen according to the Extinction Rule: IF (TENV < TMAX(i)) OR (EBURNED(i) >= ETOTAL(i)) THEN EXTINCT VEG(i) During an interval of ∆t seconds of combustion, a vegetation element will transfer to the environment an amount of energy given by: EPRODUCEDd(i,∆t) = BR(i) * ∆t Equation 1 The energy produced will increase the cell’s global temperature. The properties on table 2 suggest that the predictable spreading behavior of the fire will possibly be: • With the rising of the cell temperature, grass will be the first vegetation element to start burning. • The heat produced may ignite both underground vegetation and twigs. • Eventually, the heat being produced at that moment will ignite the trees, which have a huge amount of energy to consume. During this period, it is possible that grass and twigs have been completely consumed and have now stopped burning (because of their small overall energy and high burning rate). The ignition temperature of vegetation elements may change according to the supporting cell’s moisture. Dryer cells will naturally promote lower ignition temperatures, whereas highly moisturized ones will make ignition temperatures of vegetation elements rise. Figure A. 7. A close look over a Pyrosim terrain showing the generated vegetation. Appendix A – The Mathematical Model of Pyrosim 153 A.3.3. Fire Propagation In Pyrosim, fire propagation between cells takes place only by means of heat exchange. That is, it is the diffusion of heat from the burning cell to the neighbor cells that may ignite one of these. Therefore, most of the modeling effort of Pyrosim is done over this single issue: heat transfer. There are several issues that must be considered, namely: • Heat Production and Vegetation Type • Heat Transfer and Dissipation • Influence of Topography • Wind Influence A.3.3.1. Heat Production In Pyrosim, vegetation combustion is the only source of heat. We have already seen that each cell may have up to four vegetation elements, one of each supported types (refer to table 2). Consider a vegetation element VEG(i) with defined TMAX(i), BR(i) and ETOTAL(i). The production of heat and the corresponding rise of cell temperature, ∆TCELL, can be described by the following cycle: WHILE VEG(i) isBurning { EPRODUCED(i,Dt) = BR(i) x ∆t ∆TCELL = EPRODUCED (i, ∆t) x KAIR TCELL = TCELL + ∆TCELL } For each burning vegetation element in the cell a similar cycle is performed. KAIR is a constant that relates the increase of air temperature in the cell with the amount of energy released during combustion. KAIR is expressed in ºC/J. A.3.3.2. Heat Transfer and Dissipation During combustion, a portion of the heat produced in a cell is transferred to the neighbor cells while the other portion is dissipated to the environment. Figure 8 depicts these processes.