BioSonics - Interactive Growth System

GENERATIVE ART 2003 BioSonics – Interactive Growth System Daniel Bisig Artificial Intelligence Laboratory, University of Zurich, Switzerland.. e-mail: [email protected] Abstract BioSonics is an interactive art installation that explores the transformation of human interaction into spatial and temporal patterns by a dynamical and self-organized system. It both looks at aesthetic qualities of pattern formation processes and explores means of intuitive interaction with complex systems. In BioSonics, users interact with the growth processes of an organism. These processes are controlled by a chemical reaction network whose underlying dynamics eventually leads to changes in the organism’s appearance and behavior. BioSonics produces both visual and acoustic feedback. Each chemical in the system controls its own sound synthesis engine whose acoustic output depends on the characteristics of the chemical, its concentration and its spatial position. Users interact with the organism by means of several microphones. The sounds picked up by the microphones are converted back into chemicals, therefore leading to changes in the dynamics of the chemical reactions and the growth processes they control. By using an acoustic interface, the user’s influence on the system is embedded within the acoustic properties of the environment of which the interactive system as well as other users are a part. Interaction therefore becomes a collaborative endeavor to which the growing organism continually responds by changing its appearance as well as by modifying its acoustic feedback and sensitivity. 1. Introduction Complexity theory has become one of the most influential scientific disciplines since the end of the 20th century [2]. As a result, complex system research nowadays plays an important role in many scientific disciplines. More recently, application oriented areas such as engineering, entertainment and art are becoming influenced by this research. Of particular interest to art and entertainment are the capabilities of complex systems to create patterns at a variety of spatial and temporal scales that constantly change, adapt and evolve [8]. The highly dynamic aesthetics of these patterns form an important motivation for artistic explorations. Complex systems respond to external influences in a variety of ways ranging from an immediate return to the previous state to a drift into a qualitatively different regime. In an interactive setup, complex systems therefore tend to respond in non-trivial and surprising ways to user input. This fact renders interactive complex systems both attractive and problematic at the same time. Interaction can quickly become frustrating and boring for an inexperienced user if no causality between input and feedback seems to exists. Consequently, one of the major challenges in applying complex systems to interactive art and entertainment consists in the development of a suitable combination of feedback and interaction. The system BioSonics which we present in this paper tries to address these issues. p age # 211 GENERATIVE ART 2003 2. Concept BioSonics explores both the aesthetics of a particular complex system as well as means of intuitive interaction with such a system. It draws inspiration from biological principles and concepts in Human Computer Interaction (HCI). Biological systems represent particularly interesting complex systems with regard to their flexibility, robustness, adaptivity and wide range of morphological and behavioral patterns. All biological organism arise through a process of growth which in itself is a combination of several complex systems acting in parallel on different organizational levels (molecular, cellular, tissue). Some of these processes are sensitive to environmental influences. This sensitivity results in the formation of a complex adult morphology which is adapted to the properties of its habitat. BioSonics implements an abstract form of a growth process by combining an artificial chemistry [3] with a simple physical simulation of a body morphology. Growth results from the mutual influence between these two organizational levels. The chemical system is sensitive to user input. Therefore, the user activity forms part of the dynamics which ultimatively controls the appearance of shape. In HCI research a body of literature exists that looks at issues of interaction with artificial systems. The exploration of adequate forms of sensory feedback constitutes an important aspect of HCI research. The concepts of direct manipulation [7] and multimodal interaction [10] try to transform abstract representations into direct sensations. In many situations the interactive experience benefits from the combination of several sensory modalities that complement each other [1]. Dynamic visualization and computational steering concepts [5] address issues of real time interaction with computer programs that generate a large number of temporally changing data. BioSonics tries to combine some of these HCI concepts. It does so by providing simultaneously visual and acoustic feedback about the ongoing growth process. The artificial chemical system is transformed into sound whereas the organism’s morphology is visualized using 2D graphics. These two modalities complement each other: vision excels at discriminating spatial details while audition is particularly good in the detection of temporal patterns. Interaction with the system happens in real time and relies on input via several microphones. This input directly affects the chemical system. By this way acoustics is used as the same modality for both interaction and feedback. 3. Implementation The implementation of BioSonics involves the following aspects: installation hardware, morphological model, chemical model, growth , visual and acoustic feedback, and interaction. 3.1. Installation Hardware BioSonics was developed as an interactive art installation. As can be seen on figure 1 the installation consists of a horizontal plexiglas plate held in position by four aluminum rods and a black cube. The plate is positioned at about 1.3 meters above ground and has a size of one square-meter. The cube contains all electronic components such as a computer, a beamer and p age # 21 2 GENERATIVE ART 2003 Figure 1: BioSonics Installation. Depicted is the installation which was shown at the exhibition “Abstraction Now” in Vienna. two acoustic mixers. One set of speakers is built into each vertical face of the cube. These four speakers constitute a quadraphonic sound system (see 3.6). The computer generated image is back-projected through a circular hole in the cube’s top face onto the plexiglas plate. Since the plexiglas plate is slightly transparent the displayed image seems to float in midair. This gives spectators the impression of looking down into a pond containing a growing organism. A microphone protrudes from the upper end of each aluminum rod. These four microphones serve as acoustic input devices for interaction with the growing organism (see 3.7). Figure 2: Extended Mass Spring System. Mass-Points are indicated by filled circles. Lines between these circles represent springs. The following forces are indicated in the figure: 1) spring force 2) angular force 3) Brownian force 4) chemotaxis force. p age # 21 3 GENERATIVE ART 2003 3.2. Morphological Model The structure of the artificial organism is implemented as an extended mass-spring system (see figure 2). In addition to the spring and damping forces modeled in standard mass-spring systems, BioSonics implements an angular, Brownian and chemotaxis force. The angular force causes two successive springs to assume a preferred relative orientation. The Brownian force contributes a random component to the overall force vector. The chemotaxis force points into the direction of a chemical gradient (see 3.3). The simulation of the morphological model proceeds by using a simple Eulerian integration scheme. The paper should be completed (if possible) using Microsoft Word . The paper size should be A4 (210 mm x 297 mm). Line spacing should be 1. 3.3. Chemical Model BioSonics implements a very simple artificial chemistry consisting of a total of six different types of chemicals. These types differ with regard to their diffusion coefficients, their initial concentration at the beginning of the simulation, the current concentration and the reactions they participate in. Reactions are always of the following type: Reactions are unidirectional. The direction of the reaction depends on the sign of the reaction rate. The three reaction partners can be the same chemical, a different chemical or a null chemical (e.g. no chemical). Therefore, any of the following reactions can be represented: Each reaction is specified by the three chemicals involved, its rate, yield, threshold and saturation. Chemical concentrations and reaction parameters are in arbitrary units and always range either between 0 and 1 (concentration, yield, threshold, saturation) or between -1 and 1 (rate). Chemical concentration changes are calculated as follows: p age # 214 GENERATIVE ART 2003 Each structural element contains either one (spring) or three (mass-point) chemical compartments. Every compartment stores its own set of chemicals. Chemicals can be exchanged between neighboring compartments by diffusion (see figure 3). Reactions between chemicals occur within these compartments. This chemical system represents a sophisticated variant of reaction diffusion systems [9] in which the organization of the organisms morphology determines the shape and neighborhood relationships of the chemical dish grid. Figure 3: Chemical System. Part A depicts reactions among chemicals. Circles represent the different types of chemicals. Positive reaction rates are represented by filled arrows, negative rates by outlined arrows. In Part B chemical compartments are indicated as small outlined circles containing a set of six chemicals each. Arrows between chemical compartments indicate diffusion of chemicals. Part C hints at how the arrangement of chemical compartments in Part B forms a small subset of all the chemical compartments which are embedded in the entire structure. p age # 21 5 GENERATIVE ART 2003 Figure 4: Reaction Network. Each compartment contains the same reaction network. Circles represent the different types of chemicals. Structural elements are depicted as rounded rectangles. Arrows represent reactions: filled arrows indicate positive effects, empty arrows negative effects. 3.4. Growth Growth of the simulated organism results from the interactions between its artificial chemical system and its body morphology. These interactions are implemented as a special kind of reactions in which both chemicals and structural components form the reaction partners.Essentially, any chemical parameter can control any structural parameter and vice versa. By this way the structure and the chemicals form a reaction network (see figure 4). Possible reactions include: creation and deletion of structural elements, consumption of chemicals in order to sustain structural elements, modification of structural parameters, and fusion of structural elements due to proximity. Growth always starts from an single masspoint containing a set of initial chemical concentrations. In the absence of any user input the internal dynamics of these chemicals lead to a mostly deterministic growth process (see figure 5). For the moment all reactions are hard-coded and somewhat arbitrary. The intention in designing the reactions was to create a system that shows both a fairly interesting behavior when left on its own but at the same time is very responsive to user input. p age # 21 6 GENERATIVE ART 2003 Figure 5: Growth Process. This time sequence depicts the graphical representation of the structure (top) and the spatially averaged chemical concentrations (bottom). Time runs from left to right. Numbers indicate corresponding positions in simulation time. For information concerning the visualization of the structure refer to 3.5. 3.5. Visual Feedback BioSoncis displays the structure of the growing organism graphically as a collection of simple 2D shapes such as rectangles for mass-points and lines for springs. Each mass-point and spring possesses in addition to its physical parameters a set of values controlling its display characteristics (color hue, color saturation, color alpha, diameter and line width). These characteristics are subject to change based on the reactions that take place between chemicals and structural parameters (see figure 4). To convey the spatial dynamics of the growing structure, corresponding mass-points between consecutive time frames are connected by lines. The length and direction of these lines visualize the direction and magnitude of the structure’s motions. At the same time successive images representing consecutive structural states are displayed on top of each other with older images gradually fading into the background. . Depending on the amount of fading applied BioSonics displays a more or less densely entangled meshwork of points and lines. This mesh-work creates the illusion of a highly complex morphology despite the fact that due to computational performance issues the actual morphology is never larger than about 100 mass-points and springs. During the visualization the simulation automatically switches between different fading values. These fading values in combination with the changing display characteristics of the structure create a wide variety of graphical patterns (see figure 6). p age # 21 7 GENERATIVE ART 2003 Figure 6: Visual Representations of the Growing Organism. During the growth process it is not only the body structure of the organism that changes but its graphical representation as well. 3.6. Acoustic Feedback BioSonics relies on acoustics in order to render the dynamics of the simulated chemical processes perceivable. The acoustic output is generated by means of additive synthesis [6]. The frequencies and amplitudes of the combined sine waves are characteristic for each type of chemical. Each visible chemical compartment continuously plays all six chemical sounds at an amplitude which is proportional to the concentration of these chemicals within the compartment. In the current setup (see 3.7) a quadraphonic sound system is built into the installation. Chemical sounds are positioned within the quadraphonic sound space depending on the relative screen position of the corresponding compartments. Compartments at the center of the screen play their chemical sounds at equal loudness on all four speakers. Shifting a compartment away from the screen center results in an equally asymmetric spatial audio output. By this way the temporally and spatially changing sound patterns directly reflect changes in the chemical system. 3.7. Interactivity Interaction with a complex system constitutes one of the key aspects of BioSonics. Since BioSonics relies on sound both as a means of providing feedback and modality of interaction, it implements a unique form of the direct manipulation concept [7]. In the current setup, four microphones pick up all the sounds provided by users and the environment. The frequency p age # 21 8 GENERATIVE ART 2003 spectra of the recorded sounds are compared to the sounds associated with the various chemicals. If the similarity is sufficiently hight a certain amount of the corresponding chemical in fed into the organism thereby changing the state of the chemical system (see figure 7). The dynamics of the chemical system is therefore both the result of its internal reaction network and the activity of the users and the environment. The amount of chemical which is infused in a particular compartment depends sound similarity, amplitude and distance between compartment and sound source. Each microphone acts as a chemical source possessing its own position in screen coordinates. The distance is calculated according to the following formula: and clipped to positive values. In this equation, CP is the position of the chemical compartment, CM is the center of mass of the organism and S is the position of the chemical source. The chemical concentration infused into the chemical compartment is multiplied by the squared inverse of this distance. By this way the effect of interaction on the chemical processes is strongest within those compartments that are closest to the interacting user. By choosing a particular microphone users decide which parts of the organism are exposed most strongly to interactivity related changes. When interacting with BioSonics, users not only infuse chemicals into the growing organism but also create chemical gradients within the simulated environment of the organism. These gradients result from the differing amounts of chemicals which are produced by each chemical source. The organism exhibits chemotactic behavior within these gradients. Each type of chemical possesses a certain attractiveness for the organism. Chemicals which lead to an increase in the growth rate of the organism have a positive attractivity value. On the other hand, chemicals that delay growth or cause a reduction in the size of the organism have negative attractivity values. Depending on this attractiveness a force vector pointing towards or away from the steepest gradient slope is applied to all mass-points of the organism’s structure (see figure 8). The sum of all these force vectors is multiplied by the squared distance of the organism center of mass to the edge of the screen. The resulting chemotaxis force causes the organism to move towards users which cause the production of attractive chemicals. Apart from exerting influence on the dynamics of the chemical system interactivity also affects the acoustic properties of the chemicals. Whenever the system evaluates the spectral similarity between the recorded sounds and the chemical sounds it tries to improve the best match by slightly changing the additive synthesis parameters of the corresponding chemical sound. Over an extended period of time this mechanism causes the acoustic feedback of BioSonics to match the acoustic properties of its environment. As a result, the growth process becomes increasingly sensitive to frequent environmental sounds. p age # 219