Human Technology, 2006 VOLUME 2, NUMBER 2 (The entire issue)

Volume 2, Number 2, October 2006 Pertti Saariluoma, Editor ISSN: 1795-6889 An Interdisciplinary Journal on Humans in ICT Environments Volume 2, Number 2, October 2006 Contents From the Editor in Chief: Looking at the Nature of Ideas Through pp. 154-157 New Lenses Pertti Saariluoma Original Articles: Probing a Proactive Home: Challenges in Researching and pp. 158-186 Designing Everyday Smart Environments Frans Mäyrä, Anne Soronen, Ilpo Koskinen, Kristo Kuusela, Jussi Mikkonen, Jukka Vanhala, and Mari Zakrzewski Evaluations of an Experiential Gaming Model pp. 187-201 Kristian Kiili Creating a Framework for Improving the Learnability of a pp. 202-224 Complex System Minttu Linja-aho An Acceptance Model for Useful and Fun Information Systems pp. 225-235 Thomas Chesney Book Review: Taking ICT to Every Indian Village: Opportunities and Challenges pp. 236-237 Atanu Garai & B. Shadrach Reviewed by Pertti Saariluoma Human Technology: An Interdisciplinary Journal on Humans in ICT Environments Editor in Chief: Pertti Saariluoma, University of Jyväskylä, Finland Board of Editors: Jóse Cañas, University of Granada, Spain Karl-Heinz Hoffmann, Center of Advanced European Studies and Research, Germany Jim McGuigan, Loughborough University, United Kingdom Raul Pertierra, University of the Philippines and Ateneo de Manila University, the Philippines Lea Pulkkinen, University of Jyväskylä, Finland Howard E. Sypher, Purdue University, USA Human Technology is an interdisciplinary, scholarly journal that presents innovative, peer-reviewed articles exploring the issues and challenges surrounding human-technology interaction and the human role in all areas of our ICT-infused societies. Human Technology is published by the Agora Center, University of Jyväskylä and distributed without a charge online. ISSN: 1795-6889 Submissions and contact: [email protected] Managing Editors: Barbara Crawford and Terhi Pennanen www.humantechnology.jyu.fi An Interdisciplinary Journal on Humans in ICT Environments ISSN: 1795-6889 www.humantechnology.jyu.fi Volume 2 (2), October 2006, 154–157 154 From the Editor in Chief LOOKING AT THE NATURE OF IDEAS THROUGH NEW LENSES What sets humans apart from other animals is not the use of technology: Many mammals are innovative in making simple tools to assist in life. But it is the sheer scale of technological development that distinguishes humans. Over the millennia, people have invented technologies, used them, and enhanced them. The once-innovative technologies become mundane elements of everyday contemporary life as human societies progress. The technological developments of the last decades have dramatically altered most humans’ way of life and perceptions of the myriad elements of the immediate and distant environment. It would not be an exaggeration to view humans as standing at the cusp of profound social changes that are in line with those following the invention of writing or the steam engine. Therefore, now is a good time to stop for a moment and ponder the forces that make such new developments possible. What should we pay specific attention to when we attempt to make sense of where we have succeeded as a species, and where we have failed? Certainly this complicated, multifaceted, and intangible question cannot be answered in the next three pages, or even in a thousand times that many: There are simply too many interrelated forces that form the necessary conditions for progress. But I can isolate one particularly relevant force to contemplate, one that underscores the human role amid the multitude of other factors: That force is the reception of new ideas. Humans are a creative sort, continually imagining new ideas to address common and uncommon problems in daily life. But the success of an idea depends not solely on its conception: An equal partner of the potential of an idea is its social acceptance. The lack of ideas is certainly not an ideal situation, and one must remember that even a bad idea is better than no idea at all. Many bad ideas have been rethought, reworked, and reinvented into pretty good ideas. But new ideas are also a double-edged sword: While innovative thinking may propose a solution to a perceived problem, the inventor often finds that his or her “big idea” © 2006 Pertti Saariluoma and the Agora Center, University of Jyväskylä URN:NBN:fi:jyu-2006516 Pertti Saariluoma Cognitive Science, Department of Computer Science and Information Systems University of Jyväskylä, Finland Looking at the Nature of Ideas Through New Lenses 155 causes problems too. For example, the questioning begins with the assessing the originality of the idea, and then moves to include logistical questions such as how to make the idea a reality, to economic and philosophical questions such as whether there is commercial or human value for it, as well as environmental questions such as whether this solution harms existing biological, interpersonal, or mechanical systems, and on and on. These questions arise, however, only if the idea has some greater outlet than the inventor himor herself. For example, the revolutionary ideas on genetics outlined in Mendel’s laws could not assist farmers in their hit-or-miss hybrid farming practices of the mid-1800s because the concepts weren’t generally known (O’Neil, 2006). Decades later, Mendel’s work was rediscovered and, through experimentation over the last century, has been refined into common practices that allow for successful and replicable cross-breeding practices. Other times, it is simply a matter of others not being intellectually sophisticated or astute enough to understand the value of the idea. Centuries before the Renaissance, the idea of experimental variation was invented. The study of phenomenon by means of systematic variation to and measurement of the effects on the phenomenon was devised by the Pythagoreans of the 5th century B.C. to prove that numbers are the essence of the world. This may have been revolutionary thinking, but no one understood what to do with it before Galileo Galilei (1638/1954) adopted it and began his study of the behavior of a pendulum using systematic variation. Thus a very old idea applied within a new context helped open the path to modern science and industry. Unfortunately, many generations of potential creativity built upon the Pythagoreans’ inspiration have been lost. Certainly ideas are not good simply because they have been created. The history of humankind is littered with instances of engineering and social science ideas that failed or never rose beyond disappointing levels (Petroski, 1994). As a result, many people remain skeptical about new ideas. On the other hand, if all new ideas were deemed valuable simply because they are new, our modern societies would be quite troubled and dangerous places to live. So, what should we do about new ideas? The ultimate challenge, of course, is deciding whether an idea is good, is not good but has potential for development, or is simply inappropriate or invalid. Some of the decisions are relatively minor; all of us make these nearly every day, occasionally without much thought. Some decisions are larger, conscious, and can involve other people. Sometimes we find the decision on an idea difficult, and are happy to let others be responsible for deciding its goodness. And some decisions are so large that only a few people can play a role in their outcome. Yet, our general attitudes toward ideas, as individuals within a society, have substantive impact on every assessment of an idea by decision makers within our society. Our laziness toward the process of considering ideas from various perspectives can doom otherwise useful and beneficial ideas, which can have a long-lasting social impact. The example of Galileo remains valid today: Progress can move onward if we develop the right ideas at the right time. Had Galileo not accepted his responsibility to view the appropriateness of an idea—past or present—perhaps our world might still be awaiting a new Galileo, but awaiting from within a far more primitive society. One of the benefits of modern ICTs is that they enable us to communicate faster and further than at any time in human history. The good news in this is that ideas—the good, the bad, the undeveloped—can reach new “Galileos” around the world perhaps in minutes, as compared to centuries. The bad news is whether modern societies are truly prepared— Saariluoma 156 mentally, critically, alertly—for this new culture of discovery. If we turn blind eyes and deaf ears to new ideas, if we are unable or unwilling to seek out new concepts and visions, if we cannot be imaginative in exploring new applications for old or underdeveloped ideas, then progress is slowed and we may miss an opportunity to develop our societies and our futures. Surely if a society is unable to recognize, evaluate effectively, and adopt in various ways new ideas and new ways of thinking, then improved communication is of little use. An ICT society can be seen as simply a technical revolution and little else if its members cannot understand that the technologies themselves are only part of the equation. Equally important is the mental revolution that must accompany technology: the creative ability to use the mechanisms to enhance social well being. ICT societies are new idea societies only when the new ideas are allowed to make progress possible. However, to make practical and creative use of new ideas, some old attitudes toward ideas must fall away. For centuries, some have viewed knowledge (i.e., augmented true opinions) as eternal truths. All of science has pointed toward discovering these truths and to evaluating anything new within a framework built around these pillars of our culture. Whatever did not coincide with what we held as truth was promptly discarded. Yet this approach limits the potential for innovation and progress. Perhaps what is needed today is simply a new approach, a new way of thinking. Without rejecting the established laws, we can look at ideas more dynamically. By using multiple lenses we can begin to imagine different possibilities for innovation, potential solutions for currently unsolvable problems (Laudan, 1977). But most importantly, we must be able to look at ideas with an eye toward tomorrow. This presupposes that we are wise enough to recognize that not all ideas are in usable form today. We must be able to see the potential in an idea: The decision should not be “This idea is useless to us today,” and then not only allowing the idea to die but also become forgotten; rather, the decision should be “This idea is okay,” and so it is allowed to progress. We must allow for the evolution of ideas, for the retooling of ideas, for the taking of current ideas to new levels, for seeing how more than one underdeveloped idea can be united with other ideas to form a greater good, and even allowing an impractical idea for today to survive long enough for it to have value and use in a more receptive and appropriate future. We must make decisions about ideas, but we must do so from a more open-minded, imaginative, and thoughtful stance. Our societies are progressing at an incredible pace: We must find a way to capture the potential of ideas of today that will provide the necessary potential for development and progress in our societies of tomorrow. Our current issue of Human Technology: An Interdisciplinary Journal on Humans in ICT Environments shows how looking at current practices and research a bit differently can enhance new knowledge and create new advantages. Each of the articles reflects the authors’ inspired thinking in raising the understanding of a concept to a new level or different application. The first article, by Mäyrä, Soronen, Koskinen, Kuusela, Mikkonen, Vanhala, and Zakrzewski, looks at the human experience of smart home technologies of the future. However, since these technologies currently do not exist, they innovatively created small experiences to help the users gain a feel on a limited scale of what embedded smart technologies could be, especially in the comfort of home environments that the study’s informants have. And they approached this research from multiple scientific disciplines, Looking at the Nature of Ideas Through New Lenses 157 thereby allowing new ideas and their potential to be collaborative. Looking at the concept of flow in relation to games is the focus of the article by Kiili. Building on prior research in the gaming world, he seeks out the elements of flow that might have implications for creating educational games. The third article by Linja-aho looks at the learnability of complex systems. She posits that the process for learning is more complex than the current literature indicates, and provides guidelines to assist developers in creating systems and training that are more learnable, particularly for novices. Finally, Chesney extends the current research on the technology acceptance model (TAM) by testing the relationships between perceived enjoyment, ease of use, usefulness, and intention to use for “dual” systems, those information systems used for both utilitarian and pleasurable purposes. Research such as this demonstrates the social benefit of looking at current science and current human needs through the lenses of many disciplines, as well as creativity, openmindedness, and the potential for the future. Good ideas are needed for human progress, but even good ideas can be enhanced, rethought, and taken to a new level when society looks at the ideas from a new stance. REFERENCES Galilei, G. (1954). Dialogues concerning two new sciences (H. Crews & A. de Salvio, Trans.). New York: Dover Publications. (Original work published in 1638) Laudan, L. (1977). Progress and its problems: Towards a theory of scientific growth. Berkeley: University of California Press. O’Neil, D. (2006). Mendel’s genetics. Retrieved on October 17, 2006, from http://anthro.palomar.edu/mendel/mendel_1.htm Petroski, H. (1994). Design paradigms: Case histories of error and judgment in engineering. Cambridge, UK: Cambridge University Press. All correspondence should be addressed to: Pertti Saariluoma University of Jyväskylä Cognitive Science, Department of Computer Science and Information Systems P.O. Box 35 FIN-40014 University of Jyväskylä, FINLAND [email protected]u.fi Human Technology: An Interdisciplinary Journal on Humans in ICT Environments ISSN 1795-6889 www.humantechnology.jyu.fi An Interdisciplinary Journal on Humans in ICT Environments ISSN: 1795-6889 www.humantechnology.jyu.fi Volume 2 (2), October 2006, 158-186 158 PROBING A PROACTIVE HOME: CHALLENGES IN RESEARCHING AND DESIGNING EVERYDAY SMART ENVIRONMENTS Abstract: Based on the results of a 3-year interdisciplinary study, this article presents an approach in which proactive information technology was introduced into homes, and discusses the derived design principles from a human-centered perspective. The application of proactive computing in homes will face particularly sensitive conditions, as familiar and reliable household elements remain strongly preferred. Since there is considerable resistance towards the increase of information technology in homes, both the calm system behaviors and the degree of variety in aesthetic designs will play major roles in the acceptance of proactive technology. If proactive technology will be an embedded part of a home’s structures and furniture, it needs to blend with the normal, cozy standards of a real living environment and aim to enhance the homeyness or the key social and aesthetic qualities of homes. Keywords: proactive computing, user-centered design, home technology. © 2006 F. Mäyrä, A. Soronen, I. Koskinen, K. Kuusela, J. Mikkonen, J. Vanhala, & M. Zakrzewski, and the Agora Center, University of Jyväskylä URN:NBN:fi:jyu-2006517 Anne Soronen Hypermedia Laboratory University of Tampere, Finland Frans Mäyrä Hypermedia Laboratory University of Tampere, Finland Ilpo Koskinen Department of Product and Strategic Design University of Art and Design Helsinki, Finland Kristo Kuusela Department of Product and Strategic Design University of Art and Design Helsinki, Finland Jussi Mikkonen Institute of Electronics Tampere University of Technology, Finland Jukka Vanhala Institute of Electronics Tampere University of Technology, Finland Mari Zakrzewski Institute of Electronics Tampere University of Technology, Finland Probing a Proactive Home: Challenges in Research and Design 159 INTRODUCTION: CHANGING ECOLOGIES IN HOMES In a way, it could be a quite nice idea that there would be coffee ready and waiting when you wake up, or if the lights would be automatically switched on. But on the other hand, there is a certain enjoyment in doing it yourself: closing the curtains, lowering the Venetian blinds, and switching off all the contraptions. And, in a way, when you think about it, I have no need for any change. (M, 351) Modern homes are becoming increasingly laden with various technologies, ranging from new-generation kitchen utensils and domestic appliances to home computers, digital televisions, and wireless media servers. The sales of consumer electronics in industrialized countries like the USA appear to rise to record heights every year (Consumer Electronics Association [CEA], 2006.). One vision of the future repeatedly evoked by the electronics industry is the creation of the smart home, a new kind of technologically enhanced living environment. Yet, there are different versions of what “smartness” means in this context, depending upon whom you ask. Planning a home around a complex entertainment center may represent smart for some, whereas others emphasize home security systems, or even more ambitious home automation solutions, where numerous home elements, such a lighting, door locks, or window shades, are programmed to behave in certain ways. Home automation is not near reality in most homes, not even in the highly technological West. Additionally, there is also some resistance towards the whole idea, as the quote above from one individual from our study illustrates. One can question whether there exists an actual need for which a smart home (as it is currently marketed) would be a solution. Perhaps, therefore, the issue should be approached from a different angle. It appears that we already are living in relationship with various devices and technologies, and our living is influenced by them even while we make decisions and apply these technologies in ways that shape their value and usefulness to us. These kinds of interdependent connections, and the networks they form, can be conceptualized as ecologies. While ecology is traditionally defined as a study of organisms and their environments, this concept has revealed its usefulness beyond the field of biology to encompass other entities and their environments, such as community ecologies, information ecologies, and media ecologies, among others. For example, Nardi and O’Day (1999, p. 49) define information ecology as “a system of people, practices, values, and technologies in a particular local environment.” They emphasize that, when studying information ecologies, the spotlight is not so much on the technology as it is on human activities that are served by the technology. One of the main conclusions from our research is that because the relationship between humans and their technology is complex, we need to develop a multidisciplinary approach to study our increasingly intensive and intimate relationship with technology. It is insufficient to regard the people who are adopting or rejecting new technologies as just passive consumers, since their attitudes and practices have a powerful effect on the success or failure of particular devices or services. It also would be a failure to overlook the important ways in which the design, distribution, and marketing of new technologies are affecting the relationship between the humans and the technology. As research and development practices become more closely informed by user studies, the clear-cut separation and opposition of the realm of production from that of consumption is no longer necessarily valid. For example, assuming that producer Mäyrä, Soronen, Koskinen, Kuusela, Mikkonen, Vanhala, & Zakrzewski 160 roles are distinctive from consumer roles might have been appropriate for an industrial society, but as much of modern production involves designing information systems and media content that is collaboratively produced, involving networks of people in various roles, the opposition between consumer and producer does not stay as clear. As participation and interaction are becoming the new standard of design, there is an increasing need for evolving further the practices for codesign and coproduction, where users and designers are conceiving and developing new concepts and products in a more collaborative and interactive manner. The starting point of our research was the need to provide a human-centered view on the development of proactive technologies for homes. Proactive technology is related to a particular information-technology-industry-driven vision of the future, where omnipresent computing, sensors, and other technologies have been developed to the point where they anticipate our needs and act on our behalf (Tennenhouse, 2000; Want, Pering, & Tennenhouse, 2003). There are obvious commercial reasons for companies like Intel and IBM to focus on such a processor-saturated view of the future. But when such views are raised to the agendas of researchers and developers, these visions also may carry some selfrealizing power. It was our aim to confront the concept of proactive computing, adapt it to concrete local environments in real homes, and thereby produce a better understanding about the related acceptability, usability, and feasibility issues should such technologies indeed be adopted and installed in homes. In this way, our research is both a contribution to the critical studies of science and technology, as well as a call for more ethical and sustainable ways of developing new home technologies. Actually, certain reasons exist for why we might have need for such technologies in the future. Some claim that the aging of population will necessitate the development of smart home environments (e.g., Baillie & Schatz, 2006; Dewsbury, Taylor, & Edge, 2001). However, as we argued in our book, The Metamorphosis of Home (Mäyrä & Koskinen, 2005), there are serious ethical considerations that must be taken into account if human contact, independence, and autonomy are becoming replaced by proactive technologies, as compared to assistive technologies, where humans themselves take actions with the help of technology. We claim that the perhaps most crucial need for proactive technologies in homes will be related to the information ecologies themselves and with their evolution. It is already becoming an observable reality and common problem that the omnipresent media and communication technologies also create stress and increase the complexity of life rather than just help us to cope (Edmunds & Morris, 2000). As information network connections become more prevalent in such ubiquitous devices such as televisions, stereo systems, and games consoles, as well as in mobile phones and cars, there also will be a related surge in e-mail, instant messaging, and other communications, much of it likely unsolicited (spam) or otherwise undesirable. As a result, the overall cognitive load on individuals must be taken into account in every context. Essentially, our information ecologies are rapidly becoming over-saturated or even polluted by nonessential information (Koski, 2001), and perhaps most needed will be proactive technologies to control and supervise all the other technologies that are fighting for our limited time and attention. Thus, one of our directives for proactive home technology design was that, if adopted, these technologies should enhance the homeyness of homes: to support and protect those qualities that are central for people in their homes, including peace, relaxation, intimate human relationships, and shelter from the pressures of modern life. Probing a Proactive Home: Challenges in Research and Design 167 domestic technologies. Media technologies were perceived as authentic technologies while kitchen and bathroom appliances were regarded more as fittings of those rooms than as technology per se. This can be explained by the various presumptions and experiences that people associate with these technologies. Domestic appliances are often perceived as simple devices that one can use without effort or the study of manuals, even though many of them involve complex electronic and digital controls. People also expect that these stand-alone appliances do not crash easily (as do computing systems), and this reliability has enabled people to forget that these technologies are complex entities (Edwards & Grinter, 2001). Perhaps the most important thing, however, is that media technologies are perceived as status devices that tell about the technological standard of one’s home. This relates also to the stereotypical notions about “white goods” (referring to most appliances) as feminine and “brown goods” (referring to most electronics) as masculine. As time-saving technologies related to domestic work and hygiene, white goods are typically associated with cleanliness, simplicity, transparency, and utility. Alternately, brown goods are for leisure and entertainment, and they seem to signify complexity, cleverness, opacity, and rich content (Cockburn & Ormrod, 1993, pp. 100-104). Because of its elusiveness, a person’s experience of the domestic atmosphere is challenging to study empirically (Pennartz, 1999). In our interviews, people frequently had no words for describing relevant elements of their domestic atmosphere, but the tasks of the probes package made the process easier to approach. By means of the probes kit, people could concretize and illustrate which aspects produced homeyness in their homes. Tasks also encouraged people to consider both the personal and familial significance of domestic technologies and their uses. Thus, the tasks illuminated shared and personal meanings within the domestic environment. Further, the probes made people question some taken-for-granted aspects of domestic life or technologies. In this respect, the probes together with the interviews opened up new ways for researchers not only to perceive the domestic technologies in the informants’ existing contexts but also to ideate promising directions that proactive technology could take in order to support a cozy ambience and sociality within the home. The Pillow Study While the probes study was underway and the researchers’ understanding of the homes was getting deeper and more multidimensional, the first prototype study phase was started. After establishing that technology use to enhance the sense of homeyness would be a key design goal, our team decided to experiment by introducing smart technology in the shape of a pillow. This was based on our analysis of pillows and cushions as intimate and personal elements, ubiquitous in homes, and, in their softness, also as things that appear to be situated at the opposite end of the mental spectrum of stereotypical conceptions of the high-tech home of the future (Mäyrä & Koskinen, 2005) that we were interested in challenging. Rather than stressful and hard, pillows are associated with comfort, relaxation, and softness. On the other hand, many traditional smart home concepts rely on the use of screens and other explicit interaction interfaces to facilitate the control of these complex environments. Based on our prestudy and probes investigation, the decision was made to take the design research into a direction that would explore ambient and tangible interfaces. Cushions and pillows were perfect objects from this perspective. Mäyrä, Soronen, Koskinen, Kuusela, Mikkonen, Vanhala, & Zakrzewski 168 A simple technical prototype was implemented, which operated as an embedded contextaware interface. It consisted of a pillow fitted with hidden electronics: batteries, power supply, microcontroller, amplifier with voice input and output (loudspeaker) connected to a recording and playback circuit, and a serial (RS-232) transceiver. The last component was essential for the operation of a RFID (Radio Frequency Identification) connection that was used to provide the pillow with a crude means for sensing its surroundings. As soon as a RFID tag was within range of the reader, the embedded electronics emitted a prerecorded sound. The pillow was covered by fake animal fur, and the sounds it produced imitated animal sounds. This was related to the hypothesis that the limited sophistication level of the test system would be suited better by a perception of animal intelligence rather than by human intelligence, which the use of human voices for interaction would have suggested. The test users were provided with several things. First, they were given several beanbags with embedded RFID tags, each of which elicited a different sound associated with it from the reader when it was brought within range. A pillow with the embedded reader was also provided. The participants also received a loose set of instructions detailing various ways of interacting with the beanbags and pillow. And, finally, they were provided a video camera to record the run of events (see Figure 3). The pillow was field-tested with three families with children. There were some technical issues in the testing that limited the sensitivity and range of RFID reader, and it was not possible to combine the different sounds as freely as was originally intended. Nevertheless, some basic interaction between the subjects and the prototype was possible. The main finding from the testing in real homes was that integrating interactions with smart home technologies can indeed be perceived with positive affect if they are embedded in familiar and soft home elements such as cushions or pillows. The informants appeared quite creative in their uses and ideas for further development of such technologies. When interviewed, the child informants suggested uses where a smart pillow could become the “emotional companion” for the occupants of their home. Such an interface for a smart home could comfort its user and provide companionship and access to Figure 3. A child informant uses a beanbag to experiment with the sounds that the pillow prototype makes. Probing a Proactive Home: Challenges in Research and Design 169 house services as the occupant relaxes, hugs, or rests on the pillow while watching television or reading. In this concept, touch and sound, and the mere proximity of the pillow, provided rather natural and nonintrusive modalities for control in the shape of a pillow. The adult informants suggested that a proactive system, in general, should provide services as a secretary or manager, assisting the family members in the challenges of organizing their daily lives. For example, a future version of the pillow companion could make sounds to remind or motivate children to do their homework before their favorite television show starts, or even somehow communicate more complex messages, like alerting them when books are due to be returned to the library. Such typically messy everyday information management systems that consists of different reminders, notes, calendar markings and mobile phone calls could be simplified if a smart home could offer itself as a helpful companion for this kind of uses. The First Iteration of Design Principles After the probes and pillow studies, we had enough experience and information to formulate an initial set of proactive home technology design principles. These served as a basis for further research, as we pursued to implement them in scenario and prototype studies, and to collect feedback about them from our informants. The principles are presented in Table 1. Following the creation of these principles, we determined two basic directions our research could have taken: focus on the interactions and cohabitation in a home augmented with Table 1. The Design Principles for Proactive Home Technology (Mäyrä & Koskinen, 2005). 1. The principle of consistency. If a function or element is delegated to be controlled by a proactive system, that function or element should demonstrate similar behaviors consistently. Main Principles 2. The principle of personalization. Smart home technology should follow the “rules of the house,” reflecting practices and preferences adopted and followed by this particular individual or family within their private space. 3. The principle of embedded media interface. The main goal and task for proactive technologies in homes are providing filtering and control in negotiating the charged boundary between the home-as-shelter and the need for staying in contact with the world “out there.” 4. The design principle of animism for advanced proactive functions and services. The easiest and most natural way to interact with a proactive home would be to treat it as if it had some kind of persona or other social interface of its own. Additional Principles 5. The principle of open-ended tangible designs. Proactive services are joined with physical objects to afford multimodal, sensory-rich interactions, as well as to provide usable and aesthetically pleasing interactions for future homes. Mäyrä, Soronen, Koskinen, Kuusela, Mikkonen, Vanhala, & Zakrzewski 170 strongly proactive technology, or follow the “weak” interpretation of proactivity. A strongly proactive home system operates in the background and completely without human awareness, combining input from various sensor systems, applying computation into the situation, and advancing from these into autonomous actions. As a human interface design research issue, this was not as interesting a case as the “weak” alternative, which is a bit closer to the situation of interactive computing. Here, the state and operations of smart technology need to be conveyed to the human occupant: The system will notify the user and offer alternatives, but the choice of accepting or cancelling actions remains with the occupant, rather than completely removing the user “from the loop.” Weak proactivity is not as efficient as its alternative if the primary consideration is reducing the users’ cognitive load. However, based on our interviews and other user studies, the human-supervised direction of smart home technologies was considered more acceptable and ethically sound than the totally unseen and autonomous operation of technologies in homes. The design of weakly proactive home technologies is related to the research into “calm technology,” as approached from within the field of ubiquitous computing (see Weiser, 1993; Weiser & Brown, 1996). The challenge can also be phrased in terms of an ambient display of and access to information: The increasing computing power and complexity of distributed and networked smart components of a future home are counterbalanced by the design principle of the “disappearing computer,” an environment where collections of artifacts link together and provide new behaviors and functionalities to users while also supposedly easing the everyday life and demanding only peripheral awareness (see The Disappearing Computer, 2002-2003). The requirements, however, appear to be partly contradictory towards each other, at least in the current phase of development in technology and related user cultures. A Scenario Study of Light and Sound Light and sound were chosen as the focus areas for the second phase of our research, based on the users’ responses in our earlier probes, prototype, and scenario studies. In the scenario method, possible proactive home designs and applications were discussed with the help of illustrations that described various use situations in the future. Twelve households participated in the scenario study phase. One of scenarios presented a concept where the smart home would monitor the sound levels in the home and inform occupants, via changes in the home lighting, when the noise rises to a certain level. By increasing the inhabitants’ awareness of sound levels, the process also would guide them to change their behavior and lower the sound level (see Figure 4). In this phase, a home technology system that takes actions related to the lighting and soundscape of home was perceived as a more easily acceptable way of implementing proactive behaviors than a scenario in which a system would try to infer human intentions or to provide, for example, entertainment suitable for the given situation. To some degree, this can be related to the reluctance or aversion of the subjects towards change in familiar and reassuring contexts. But equally important was the subjects’ general lack of confidence capacity in a computing system perceived as too limited to start making deductions about the human mind and intentions, particularly in complex and intimate social situations involving several people and their (sometimes conflicting) preferences. The assessment of our informants, based on previous experience, could be described as realistic. Probing a Proactive Home: Challenges in Research and Design 171 Figure 4. An illustrated scene from a late night social, with the smart home providing sound level feedback via changing colors of a table lamp. The Light and Sound Prototype Studies Based on the results from the scenario study, the research group decided to experiment with home lighting as a potential field for an ambient interface design for smart homes. The first constructed prototype was a large standard lamp2 (see Figure 5). The lamp was reconstructed around two pairs of 36W fluorescent tubes, each pair chosen from opposite color temperatures. The tubes were aligned in opposite internal corners to emit an even light when all tubes were lit. The fluorescent tubes were built with a dimming capacity and the on/off switch was operated by the microcontroller inside the lamp. In addition, multicolored light-emitting Figure 5. The large lamp prototype in use in an informant’s home. Mäyrä, Soronen, Koskinen, Kuusela, Mikkonen, Vanhala, & Zakrzewski 172 diodes (LEDs) were installed in the interior. The fluorescent tubes and most of the electronics other than some control electronics visible at the top of the lamp were covered by the paper shade. This study involved testing in two households. A light level sensor was installed on top of the lamp so the light output could be adjusted better to the changing light levels in the environment. When in use, the LEDs would light up simultaneously and in intensity directly proportional to the sensed sound level. The LEDs faded away within few seconds if further loud sounds were not measured. The microphone connected to the microcontroller at the top of the lamp prototype sensed the surrounding ambient and direct sounds, which the microcontroller then used to light the LEDs. The concrete research question at this point was focused on the interface between the smart environment and its occupants. Our hypothesis was that a familiar design (the wellknown lamp style) would ease the adoption of new technologies, while new functionalities related to light reacting to the sound level would promote new behaviors. In actual use, however, the sound-reacting behavior of the prototype proved so subtle that it did not provoke strong reactions or new behaviors among our informants. We realized that in order to derive interesting answers to our research questions, the prototype needed to have more diversity both in terms of its design and behavior. Still, this first-round lamp-shaped prototype had demonstrated that smart functionalities could be hidden in, or made more easily adaptable into, a regular home environment when embedded in familiar forms (Kuusela, Koskinen, Mäyrä, & Soronen, 2005). After analyzing the users’ experiences and lessons from the design of the first soundlevel reactive lamp experiment, a new collection of lamp prototypes was designed and implemented. The design-related research questions were made easier to control and focus on by applying clearly distinct lamp designs while the basic behavior of sound levels causing lighting changes was kept the same. The four lamp designs (Figure 6) reacted to sound levels by changing the intensity and color of the light. These systems were installed in two homes in Tampere and one home in Helsinki. Each lamp stayed one week in each home, one lamp at a time. To collect informants’ experiences and see how presuppositions changed with real contact with this kind of technology, the people were interviewed before and after the study. In the earlier scenario study phase, most of the participants assumed that a sound-reacting lamp system’s red color indicating the loudest sound level could be obtrusive because it would draw a lot of attention, and informants claimed that sometimes it would be impossible to avoid loud voices or noises at home. However, during the 4-week lamp-testing period, none of the informants perceived the red color as too obtrusive, even though the four prototypes differed in their design and intensity of light. In fact, some participants thought that if there are powerful voices at home, the lamp should come to the center of one’s awareness and, in that sense, the red color worked well. Their point was that lamps remained in their usual role until, by becoming red, they effectively functioned as decibel meters for a while. The lamp prototypes indicating an approximate volume level were interesting in the sense that they made invisible information visible. The participants told how surprised they were during the first test days when the lamps turned red when they were laughing or sneezing. Their expectation had been that the prototype would indicate only steady sound levels in the home, and its reaction to sudden loud voices was a surprise. However, as lighting artifacts, the prototypes became visible parts of the spatial order and technological ecology of the home and simultaneously operated as experience prototypes, providing the Probing a Proactive Home: Challenges in Research and Design 173 participants with an idea how it feels when technology steers your attention to invisible sensorial issues. The role of domestic technology is often ambiguous because domestic appliances and media technologies dominate the domestic space. Yet their roles as aesthetic elements are not typically established in decoration magazines (Routarinne, 2005). Our lamp prototypes blurred the distinction between decoration and technology items: They were interpreted as both. Some participants considered the lamp prototypes primarily decorative elements while others perceived them more as decibel meters. The appearance and placement of the prototypes were felt much more important in one home whereas informants from another home focused mostly on the ways the prototypes reacted to different voices and noises. A playful attitude to interior decoration was prominent in the first case, whereas more conventional attitudes towards metering devices were central among the informants from the latter home. In any case, if the visual design of the lamp was felt pleasing, it also increased to some extent the participants’ interest in the decibel measuring action. Figure 6. The four different sound-reacting lamp designs. Clockwise, from top left: “IKEA,” “Granny,” “Giger,” and “Glow.” Mäyrä, Soronen, Koskinen, Kuusela, Mikkonen, Vanhala, & Zakrzewski 174 A young couple determined, already in the scenario phase, that they would like to use a decibel lamp system in their home and, after the testing period, this opinion strengthened. However, their results demonstrated that they would not take just any smart lamp, but only those fitting in their interior decor. For instance, they argued that the Granny version represents a dated style they do not want in their home. Although they felt the technology interesting, the visual design of this prototype made it inappropriate for their home. Reasons for disliking certain domestic technologies were diverse and people’s mode of living, phases of life, and socio-historical backgrounds played a central role in their reasoning. Although there were some differences in preferences of style among household members, they all shared an opinion about the prototype they wanted least. Ambient Home Automation Study In trying to obtain actual user information about proactive home systems, we found that researching different ways of implementing smart home interfaces is not enough. We needed to set up a larger scale test environment, where real homes were augmented with sensors and programmable behaviors that would provide residents with an overall experience of what it means to be living in a proactive home environment. At the same time, numerous technological, resource, and even ethical constraints set limits on how strong and active a hold on people’s lives our prototype system could have. The key focus was on the acceptability of proactive technology in real homes, which was studied by providing our informants concrete and personal experiences of the functionality of a larger proactive system within their homes. Primarily we wanted to provide our informants with an example of how different devices could autonomously interact with each other in their homes, thereby highlighting proactivity as a feature of technology that acts on our behalf and anticipates our needs. We also wanted insight into how the experience of domestic space potentially changes with new ambient elements. The starting point for implementation of this research phase was that it had to be able to be installed as straightforwardly as possible into real homes. We wanted to minimize the need to install new apparatuses in homes, so the idea was to use existing lighting and other devices that are familiar elements to the users. Also, the design of devices or their acceptability was not the focus of this phase; rather we emphasized the new functionalities and how they are perceived and accepted when combined with the familiar existing devices within the home. One effect of this decision was that it decreased the set of possible functions that could be used in the prototype. We chose only very basic tasks and functions for proactive augmentation, such as lighting control and the waking and retiring routines. Furthermore, all the devices had to be removed without a trace after the test, which presented the team with a further challenge in research design. Since all permanent mounting methods had to be rejected, we were forced to use a set of temporary mounting methods (such as suction cups and adhesive pads). The control interface (Figure 7) was designed to resemble a clock radio and thereby to fit easily in a bedroom. This study involved two households. We chose a commercial home automation system known as X103 to meet our requirements since it offers the possibility for using existing technology and for retrofitting some compulsory new devices. One advantage in the X10 is that it uses existing electrical power lines for communication between devices. However, the commercial software of X10 Probing a Proactive Home: Challenges in Research and Design 175 Figure 7. The control interface unit developed for the X10-based home automation prototype system. The unit was a black box, approximately the shape and size of a common clock radio, with several buttons and a LCD screen available for users to make changes to the morning and evening time presets of the home automation system. appeared to be too rigid, so we replaced it with an open-source software called Misterhouse.4 By combining the X10 hardware with a PC, Misterhouse offered a simple user interface, as well as some basic means for programming and necessary object and method libraries (the key elements needed for object-oriented programming). The logic of events and functions were programmed with Perl.5 The basic functionalities of the system were lighting control and routines assisting in waking up and going to sleep. These were performed by adjusting the lighting levels of the home according to the time of day and motion sensor information. (See Figure 8 for an illustration of the setup.) In addition to light, ambient sound was used both in the morning and evening: the sound of birds singing in the morning, and the sound of the sea in the evening. Our philosophy for choosing sleep as the part of life subjected to proactive control was related to the fact that people already use sound and light as part of technologies for controlling their state of awareness and arousal, as the ubiquity of alarm clocks proves. The going-to-sleep sequence was the more experimental part of our setup, based on the premise that future home technology will adopt a more strongly proactive stance towards the health of users as well. The relaxing, ambient sounds and dimming lights that became activated when a preset “sleeping time” arrived were designed to have a double function: First, to signal the inhabitants that it is time now to go to bed and, second, to create a relaxing and sleep inducing effect in the atmosphere of the home. The lighting of the home was adjusted according to motion sensor information. The time of day also affected the lights in the bathroom and hallway: In the daytime, the lamps operated at their maximum, but at night, the lamps could be brightened to only half of the maximum power. The purpose was to avoid the blinding effect that occurs when the user enters these areas from a dark bedroom. On the basis of our earlier interviews, subjects emphasized the extreme importance that the atmosphere of a home be warm and homey. Finnish homes are often furnished with warm Mäyrä, Soronen, Koskinen, Kuusela, Mikkonen, Vanhala, & Zakrzewski 176 Figure 8. An imaginary floor plan showing the placement of devices attached to the proactive home system. The operational elements are named in the floor plan. colors, soft textiles, and light wood furniture. Especially in the evenings or when people expect guests, they wish that the lighting of the home has a warm tone. In that sense, the home environment differs immensely from, for example, an office environment. When we think of the visions of smart home as popularized in the media and advertisement, the atmosphere is often pictured to be rather cold and centered on a hard technological, almost businesslike, element (see Jokinen & Leppänen, 2005). The visionary illustrations of smart homes are dominated by various electronic components enclosed in black or grey boxes, large displays, and gleaming glass surfaces. We see here a contradiction between the visions of smart home interiors presented to the public and the actual appearance of today’s Finnish homes. In our research, we sought to challenge this stereotypical image of smart homes and to look into whether bringing new functionalities to the home necessarily means that the atmosphere of the home has to change. We believe that new devices can give the user the feeling that these technologies are designed and intended to be used precisely in common, everyday home environments. This is an important perspective because, in our study, the interviewees were not willing to compromise the cozy feeling in their homes. Therefore, this was and should be taken into account when designing novel devices and smart services for homes. In the beginning of 1990s, Mark Weiser (1993) presented the idea of ubiquitous computing. It is unlikely that our informants were familiar with the principle, yet, the attempt to embed technology was well known among them. The interviewees expressed the wish to have technology only if was implemented as embedded, unobtrusive devices, as is demonstrated by the following quote: Probing a Proactive Home: Challenges in Research and Design 183 technologies. Therefore, the technologically robust, fail-safe, and nonintrusive character of smart home technologies is a key priority. We also found that some functionalities in homes are currently more feasible for proactive implementation than others. For example, ambient elements, such as air conditioning, heating, security, and, to a certain extent, lighting and ambient sound, are features that inhabitants have a rather low threshold for delegating to proactive technology’s control. However, our informants were skeptical about the potential of smart technology taking a strongly proactive, intention-anticipating role in their personal lives. When a particular real-life situation needs to be interpreted and reacted to in a correct way, even knowledgeable humans such as family members sometimes have problems in deducing the right way to act. Misunderstandings are a common part of human life. Whether people would indeed be able to accept such applications if the technologies actually were accurate in their predictive operations remains for future research to solve. Using a team of professionals operating a specifically rigged house remotely and covertly would be a “Wizard of Oz” approach (Gould, Conti, & Hovanyecz, 1982) into studying human-level intelligence as experienced in a proactive home setting prototype. But this kind of research, of course, would include its own considerable challenges. The main derived lessons for research practice focus particularly on the necessity of interdisciplinary collaboration and multiple methodologies if changes to and developments in technologies are investigated. A study that utilizes only interviews as its method, for example, and tries to deduce some conclusions about the acceptability of future technologies from informants who have experienced only current technologies is inherently unreliable. The preconceptions of the subjects and various popular ideas will have a dominating effect on results of such a study. But if human science researchers, designers, and engineers work together to realize some concrete experiences of such future technologies for users, and the users have enough time to live with these technologies and thereby domesticate the prototypes as parts of their lives, then the results will have much more relevance for all parties involved. (For a fuller explanation of the domestication of technology, see Pantzar, 1996, and Silverstone & Hirsh, 1992.) The subject of proactive technology has proved to be a complex and controversial issue to study. Methodologically, it was challenging to investigate because the phenomena needed are indisputably intelligent services that would be able to deduce human needs and intentions and thereby genuinely anticipate and take action in a proactive manner on our behalf. Yet, most of these intelligent services remain beyond the capabilities of current state-of-the-art information technologies. Rather than attempting to implement such highpowered computational systems, the research goal here was focused on the human interface and coexistence of humans and “living” technologies in the context of real homes. Embedded processors, sensors, and network capabilities were applied to everyday objects such as pillows, lamps, and alarm clocks in order to learn more about the acceptability of various smart functionalities, the relation between design and technology within a home context, and about the applicability of our methodology. From a research angle, the results appear promising, and apparent benefits are to be gained by involving real users in the different stages of a research process, both as informants and codesigners, by inviting and eliciting their ideas for the potential applications of emerging technologies. The combination of cultural probes, scenario studies, minidesigns, and implemented prototype systems provided the Mäyrä, Soronen, Koskinen, Kuusela, Mikkonen, Vanhala, & Zakrzewski 184 interdisciplinary research team with a suitably wide set of tools from which to derive rich data and to build the basis for knowledge and theory formation. For a developer or designer of smart technology, the lessons of this research particularly focus on the proactive home technology design principles and their underlying case studies that we have created during our research. It would be most welcome to see examples of industry approaches where the users’ key priority of “feeling homey” that we have reported here are implemented as the driving principle for smart home designs. A different kind of finding is derived from a more action-research-oriented angle. As the informants became more familiar with the opportunities offered by contemporary home technology during their participation, one family actually decided to purchase and install a home automation system. Thus, in at least one case, the participation in research led to changes in informants’ lives. In more general terms, the increasing speed of the development and complexity of home automation and electronics has raised an apparent need for a “home technology consultant,” who would help people to make informed decisions, based on their unique needs, about which technologies would be genuinely valuable in their case. There is also a level of “techno-politics” that can be derived from this research, which concerns most directly the decision and policy makers. Contemporary citizens are in sharply unequal situations concerning the marketing and availability of home automation and proactive technologies. The possibility exists that, without public discussion and proactive measures by means of recommendations or even regulations, there might be developments that are either unethical or provide various groups in society unequal opportunities for taking advantage of technology’s benefits. There has been active interest and encouragement from public research policies towards technical and commercial exploitation of opportunities opened up by ambient intelligence and advanced computer systems. Our research points out how important it is to listen to actual users, both on the technological and regulatory levels, regarding the development of new technology, and involve them when deciding on the directions and uses of these technologies for the future. The consequences, after all, are going to influence everyone in the society. ENDNOTES 1. Quotations are cited with the informant’s gender and age. All interviews were conducted in Finnish with native-speaking Finns. Informants’ quotes have been translated into English by the authors. 2. The lamp is a model from IKEA, an international chain of home furnishings. 3. X10 home page (2006). Downloaded August 24, 2006, from http://www.x10.com. 4. MisterHouse home page (2006). Downloaded August 24, 2006, from http://www.misterhouse.net. 5. Perl is a programming language that has become particularly popular in implementations of different Internet services; see http://www.perl.org/about.html. REFERENCES Baillie, L., & Schatz, R. (2006). A lightweight, user-controlled system for home. Human Technology, 2, 84– 102. Retrieved August 24, 2006, from http://www.humantechnology.jyu.fi/articles/volume2/2006/baillieschatz.pdf Probing a Proactive Home: Challenges in Research and Design 185 Bell, G., Blythe, M., & Sengers, P. (2005). Making by making strange: Defamiliarization and the design of domestic technologies. ACM Transactions of Computer-Human Interaction, 12, 149–173. Buchenau, M., & Suri, J. F. (2000). Experience prototyping. In D. Boyarski & W. A. Kellogg (Eds.), Proceedings of the conference on Designing interactive systems: Processes, practices, methods, and techniques (pp. 424–433). New York: ACM Press. Chandler, D. (1995). Technological or media determinism. Retrieved October 3, 2006, from http://www.aber.ac.uk/media/Documents/tecdet/tecdet.html Cockburn, C., & Ormrod, S. (1993). Gender and technology in the making. London: Sage Publications. Consumer Electronics Association [CEA]. (2006). Retrieved August 24, 2006, from http://www.ce.org/Research/Sales_Stats/275.asp Dewsbury, G., Taylor, B., & Edge, M. (2001). Designing safe smart home systems for vulnerable people. In R. Proctor & M. Rouncefield (Eds.), Dependability in healthcare informatics, Proceedings of the First Dependability IRC [Interdisciplinary Research Collaboration] Workshop (pp. 65–70). Lancaster, UK: Lancaster University. Disappearing Computer, The. (2002-2003). Retrieved August 24, 2006, from http://www.disappearingcomputer.net/ Edmunds, A., & Morris, A. (2000). The problem of information overload in business organisations: A review of the literature. International Journal of Information Management, 20, 17–28. Edwards, K., & Grinter, R. E. (2001). At home with ubiquitous computing: Seven challenges. In G. D. Abowd, B. Brumitt, & S. A. N. Shafer (Eds.), Ubiquitous Computing 2001 (pp. 256–272). Berlin, Germany: Springer-Verlag. Garvey, P. (2003). How to have a “good home”: The practical aesthetic and normativity in Norway. Journal of Design History, 16, 241–251. Gaver, B., Dunne, T., & Pacenti, E. (1999). Cultural probes. ACM Interactions, 6(1), 21–29. Gould, J. D., Conti, J., & Hovanyecz, T. (1982). Composing letters with a simulated listening typewriter. In Proceedings of ACM CHI’82 [Human Factors in Computing Systems Conference] (pp. 367–370). New York: ACM. Habermas, J. (2004). Knowledge and human interests. (J. J. Shapiro, Trans.). Oxford, UK: Blackwell. (Original work published in 1968) Harper, R. (Ed.). (2003). Inside the smart home. London: Springer. Intille, S. S. (2002). Designing a home of the future. IEEE Pervasive Computing, 1(1), 80–86. Jokinen, M., & Leppänen, S. (2005). Älykäs koti: Utopiaa vai arjen ihme? [Intelligent home: Utopia or everyday miracle?] In A. Kasvio, T. Inkinen, & H. Liikala (Eds.) Tietoyhteiskunta: Myytit ja todellisuus (pp. 205– 225). Tampere, Finland: Tampere University Press. Koski, J. T. (2001). Reflections on information glut and other issues in knowledge productivity. Futures, 33, 483–495. Kuusela, K., Koskinen, I., Mäyrä, F., & Soronen, A. (2005, May). A metamorphosis of the home: Proactive information technology as a design challenge. Paper presented at the 2005 Nordic Design Research Conference, Copenhagen, Denmark. Retrieved August 24, 2006, from http://www.tii.se/reform/inthemaking/files/p43.pdf Leppänen, S. (2001). Kodistako älykäs? [Home to become intelligent?] In Digitalisoituvan viestinnän monet kasvot (pp. 113–131). Teknologiakatsaus 118. Helsinki, Finland: Tekes, National Technology Agency. Mäyrä, F. (1999). Demonic texts and textual demons: The demonic tradition, the self, and popular fiction. Tampere Studies in Literature and Textuality series. Tampere, Finland: Tampere University Press. Mäyrä, F., & Koskinen, I. (Eds.). (2005). The metamorphosis of home: Research into the future of proactive technologies in home environments. Studies in Information Sciences series. Tampere, Finland: Tampere University Press. Mäyrä, Soronen, Koskinen, Kuusela, Mikkonen, Vanhala, & Zakrzewski 186 Mozer, M. C. (1999). An intelligent environment must be adaptive. IEEE Intelligent Systems and Their Applications, 14(2), 11–13. Nardi, B. A., & O’Day, V. (1999). Information ecologies: Using technology with heart. Cambridge, MA: MIT Press. Pantzar, M. (1996). Kuinka teknologia kesytetään: Kulutuksen tieteestä kulutuksen taiteeseen [How to tame technology: From the science of consumption to the art of consumption]. Helsinki, Finland: Hanki ja Jää. Pennartz, P. J. J. (1999). Home: The experience of atmosphere. In I. Cieraad (Ed.) At home: An anthropology of domestic space (pp. 95–106). Syracuse, NY: Syracuse University Press. Rapaport, R. N. (1970). Three dilemmas of action research: With special reference to the Tavistock experience. Human Relations, 23, 499–513. Routarinne, S. (2005, May). Domestication and context: Studying objectification in print media. Paper presented at the 2005 Nordic Design Research Conference, Copenhagen, Denmark. Retrieved August 24, 2006, from http://www.tii.se/reform/inthemaking/files/p92.pdf Silverstone, R., & Hirsch, E. (Eds.). (1992). Consuming technologies: Media and information in domestic spaces. New York: Routledge. Soronen, A., & Kuusela, K. (2005). Domestic technologies of the future: Approaching through visual scenarios. In A. Sloane (Ed.), Home-oriented informatics and telematics, proceedings of the HOIT [Home-Oriented Informatics and Telematics] 2005 Conference (pp. 51–62). New York: Springer. Soronen, A., & Sotamaa, O. (2005). Adapting a probes approach for exploring domestic environments and practices. In F. Mäyrä & I. Koskinen (Eds.) The metamorphosis of home: Research into the future of proactive technologies in home environments (pp. 43–69). Studies in Information Sciences series. Tampere, Finland: Tampere University Press. Suchman, L. A. (1987). Plans and situated actions: The problem of human-machine communication. Cambridge, UK: Cambridge University Press. Tennenhouse, D. (2000). Proactive computing. Communications of the ACM, 43(5), 43–50. Want, R., Pering, T., & Tennenhouse, D. (2003). Comparing autonomic and proactive computing. IBM Systems Journal, 42, 129–135. Weiser, M. (1993). Some computer science issues in ubiquitous computing. Communications of the ACM, 36(7), 75–84. Weiser, M., & Brown, J. S. (1996). The coming age of calm technology. Retrieved August 24, 2006, from http://www.ubiq.com/hypertext/weiser/acmfuture2endnote.htm Authors’ Note The authors want to thank Academy of Finland for funding this research, as well as colleagues in this and related research projects, particularly Katja Battarbee, Olli Sotamaa and Tere Vadén, who made significant contributions during the first year of this research. This research has been conducted as part of the Proactive Computing Research Programme (see www.aka.fi/proact). All correspondence should be addressed to: Frans Mäyrä Hypermedia Laboratory FIN-33014 University of Tampere FINLAND frans.m[email protected] Human Technology: An Interdisciplinary Journal on Humans in ICT Environments ISSN 1795-6889 www.humantechnology.jyu.fi An Interdisciplinary Journal on Humans in ICT Environments ISSN: 1795-6889 www.humantechnology.jyu.fi Volume 2 (2), October 2006, 187–201 187 EVALUATIONS OF AN EXPERIENTIAL GAMING MODEL Abstract: This paper examines the experiences of players of a problem-solving game. The main purpose of the paper is to validate the flow antecedents included in an experiential gaming model and to study their influence on the flow experience. Additionally, the study aims to operationalize the flow construct in a game context and to start a scale development process for assessing the experience of flow in game settings. Results indicated that the flow antecedents studied—challenges matched to a player’s skill level, clear goals, unambiguous feedback, a sense of control, and playability—should be considered in game design because they contribute to the flow experience. Furthermore, the indicators of the actual flow experience were distinguished. Keywords: flow experience, educational games, game design, engagement. INTRODUCTION Computer games are a quite new form of media (Salonius-Pasternak & Gelfond, 2005), but today they have already established themselves as an everyday phenomenon. In addition to providing entertainment and diversion, games satisfy the basic requirements of learning environments that have been identified by Norman (1993) and can provide an engaging environment for learning as well. Unfortunately, educational games have been used primarily as tools for supporting the practice of factual information learning. In fact, it can be argued that most educational games too often resemble digital exercise books and do not utilize the power of games as interactive context-free media. The reason for this may be that the field of educational technology lacks research on how to design game environments that foster knowledge construction and deepen understanding (Moreno & Mayer, 2005) and problemsolving while engaging and entertaining the user at the same time. However, Kiili (2005a) proposed an experiential gaming model that may end this trend since it helps designers to understand the learning mechanism in games by integrating © 2006 Kristian Kiili and the Agora Center, University of Jyväskylä URN:NBN:fi:jyu-2006518 Kristian Kiili Tampere University of Technology Pori, Finland Kiili 188 pedagogical elements into the design process and distinguishes the factors that make game playing enjoyable. The flow theory is emphasized as a design principle because it provides a universal model of enjoyment, detailing the common aspects of the process that takes place when anyone experiences enjoyment. Kiili (2005b) evaluated the experiential gaming model through the IT-Emperor game, which was employed in a usability course. Kiili (2005c) revised the experiential gaming model to better address the needs of educational game designers. The revised version of the model is illustrated in Figure 1. The experiential gaming model can be used to design and study educational games and gaming in general. It consists of a gaming cycle and a design cycle. The gaming cycle provides a description of the gaming process and the learning process in games. It aims to focus the efforts of designers toward enhancing the most important factors that influence the gaming experience and learning with games. Meanwhile, the design cycle describes the main phases of game design and works as a guideline in the design process. The design process is presented abstractly because it may vary among the different game genres. The model emphasizes the importance of considering several flow antecedents in educational game design: challenges matched to the skill level of a player, clear goals, unambiguous feedback, a sense of control, playability, gamefulness, focused attention, and a frame story. The ambition of designing the sort of games that enhance experiencing flow is justifiable because previous research indicates Challenges (problems) Active experimentation Reflective observation Schemata construction Preinvative idea generation Idea generation Flow -Challenges- -Gamefulness- -PlayabilityActive experimentation Reflective observation Schemata construction Preinvative solution generation Solution generation Flow (Need analysis) Situated learning objectives -Frame story- -FocusedattentionPositive attitudes Reflective evaluation Implementation Design knowledge -Cleargoals- -Senseof control- -UnambiguousfeedbackDesign Cycle Figure 1. The experiential gaming model developed to bridge the gap between game design and pedagogy. (A fuller description is provided in Kiili, 2005c). Evaluations of an Experimental Gaming Model 189 that flow has a positive impact on learning, exploratory behavior, and the attitudes of players (Ghani, 1991; Kiili, 2005b; Skadberg & Kimmel, 2004; Webster, Trevino & Ryan, 1993). A more detailed description of the model is provided in Kiili (2005c). In this paper, the usefulness of the experiential gaming model is studied through a problem-solving game. Two main goals can be distinguished. The first goal is to validate the main flow antecedents included in the experiential gaming model and to study their influence on the flow experience. Second, this study aims to operationalize the flow construct in a game context and to start a scale development process for assessing the flow experience in game settings. Although Csikszentmihalyi (1991) defined flow as a multidimensional construct consisting of nine dimensions, he relied primarily on the challenge–skill balance to measure flow (Jackson & Eklund, 2002), which is not an adequate measurement method alone. Thus, this paper focuses on developing a flow scale that takes all relevant flow dimensions into account. This paper begins with a brief discussion of the flow experience and the methodology used to measure flow before turning to the evaluations of the experiential gaming model. FLOW EXPERIENCE Flow describes a state of complete absorption or engagement in an activity and refers to the optimal experience (Csikszentmihalyi, 1991; Ghani & Deshpande, 1994). During the optimal experience, a person is in a psychological state where he/she is so involved with the goaldriven activity that nothing else seems to matter. Csikszentmihalyi (1991) defined the phenomena of flow state as having nine dimensions. The first five dimensions can be considered flow antecedents and the rest indicators of flow experience (Kiili, 2005c). 1) Challenge–skill balance. When experiencing flow, a person perceives a balance between the challenges of the activity and his or her skills, with both operating at a personally high level (Jackson & Marsh, 1996). In the other words, a person’s skill is at just the right level to cope with the situational demands. 2) Action–awareness merging. The flow state is so involving that, during it, activity becomes spontaneous and automatic. This dimension is problematic from the point of view of educational games because the ultimate aim of educational games is to support knowledge construction, which requires cognitive processing (Kolb, 1984; Winn, 2004). Thus, in this context, the action–awareness dimension should be applied to the playability of the game rather than to the entire gaming activity. Pilke’s (2004) argument that the goal of flowinducing interface design is to design good usability and vise versa supports this view. 3) Goals of an activity. The goals should be clearly defined in order to be able to achieve flow (Novak, Hoffman, & Duhachek, 2003). However, the goals of some activities cannot be always clear, as in the case of creative activities. Still, a person can develop a strong personal sense of what he/she intends to do. 4) Unambiguous feedback. Unambiguous feedback is related to the goal dimension because it allows a person to know how he/she is succeeding in a specific goal. A reasonable feedback system is easier to develop if the main goal is divided to subgoals. 5) Control. A sense of control is experienced without the person actively trying to exert it. Csikszentmihalyi (1991) has stated that this is more a sense of the possibility of control Kiili 190 rather than the actuality of having control. A person senses when he/she can develop skills sufficient enough to reduce the margin of error to close to zero, which makes the experience enjoyable. According to Ghani and Deshpande (1994), this sense of control is one of the most important flow antecedents in games. 6) Concentration. Concentration on the task at hand is the most frequently expressed flow dimension (Csikszentmihalyi, 1991). While in flow, a person concentrates totally on the activity and is able to forget all unpleasant things beyond the game. Because flow-inducing activities require a complete focusing of attention on the task at hand, the person has no cognitive resources left for irrelevant information processing. 7) Loss of self-consciousness. The self disappears from one’s awareness during flow because when a person is thoroughly engrossed with an activity, few cognitive resources are available to allow the person to consider either the past or the future. In other words, flow allows no mental room for self-scrutiny (Csikszentmihalyi, 1991). 8) The transformation of time. According to Csikszentmihalyi (1991), the sense of time during the flow experience tends to bear little relation to the actual passage of time as measured by the absolute convention of a clock. Time seems either to “fly” or to “drag.” Csikszentmihalyi (1991) argued that losing track of the clock is not a major antecedent of flow and it may be just a by-product of the intense concentration required for the activity at hand. 9) Autotelic experience. Autotelic experience refers to an activity that is “done, not with the expectation of some future benefit, but simply because the doing itself is the reward” (Csikszentmihalyi, 1991, p. 67). According to Kiili (2005c), this is the most important final result of flow in educational gaming: Students undertake studying activities not necessarily with the expectation of some external future benefit, but simply because playing the game is enjoyable, a reward in itself. This nature of the flow experience supports the ideology of lifelong learning and is a priceless goal in education. Whenever people reflect on their flow experiences, they mention some and often all of these characteristics (Csikszentmihalyi, 1991). The combination of these elements causes a sense of deep enjoyment that is so rewarding that people feel it’s worthwhile to expend a great deal of energy to experience it. Measuring Flow Experience Flow has been studied in previous research using several methods. These methods can be divided into two main approaches. 1. The activity–measurement method begins with involving participants in a selected activity. Afterward, participants evaluate their experience either through an interview or by completing a survey instrument (Ghani & Deshpande, 1994; Pilke, 2004; Skadberg & Kimmel, 2004; Webster et al., 1993). 2. The Experience Sampling Method (ESM) gathers information during certain activities. Participants are interrupted for a short period throughout the day activity to evaluate their experience with a survey instrument (Csikszentmihalyi, Larson & Prescott, 1977; Csikszentmihalyi & LeFevre, 1989; Csikszentmihalyi & Nakamura, 1989; Havitz & Mannell, 2005). In spite of some criticism, both approaches have been successfully utilized in flow studies. An important question in the first approach is whether the respondents can reliably evaluate Evaluations of an Experimental Gaming Model 191 flow after, rather than during, an activity. On the other hand, the ESM can be criticized for interrupting a participant’s experiences and normal behavior, which may decrease the ecological validity of the study (Loomis & Blascovisch, 1999). However, it is apparent that different activities and contexts require different methods for use. Although the ESM provides continuous information about the experiences of the participants during an activity, it is not the appropriate approach for short experiments like the small problem-solving games utilized in this study. For that reason, the first method was selected in this study. The most significant challenge for this study was to operationalize the flow experience appropriately. Operationalization of the Flow Experience For the past two decades, researchers have strived to understand how the flow model fits the experiences of people (Voelkl & Ellis, 1998). The nine dimensions of flow outlined by Csikszentmihalyi (1991) have been used as a framework to operationalize flow in various contexts. In spite of that, most of the formed operationalizations of flow branch off quite distinctively from one another. For example, Ghani and Deshpande (1994) used a 15-item scale measuring only the dimensions of enjoyment, concentration, challenge, control and exploratory use. Exploratory use refers to amount of experimentation with tools available. On the other hand, Webster et al. (1993) studied the experiences of an accounting firm’s employees who attended a course with a 12-item flow scale measuring the amount of control, focused attention, curiosity, and intrinsic interest they experienced. In sport and physical activity settings, flow experience has been assessed using the Flow State Scale (FSS) questionnaire developed by Jackson and Marsh (1996). This 36-item instrument provides scales of all nine dimensions of flow outlined by Csikszentmihalyi (1991). Internal consistency estimates for the nine FSS scales were reported to be reasonable. Although the operationalizations of flow diverge from one another, almost all flow measuring instruments include the challenge–skill dimension that has been argued to be the most important flow antecedent (Csikszentmihalyi, 1991). However, Chen, Wigand, and Nilan (1999) have argued that researchers studying the flow phenomenon in a Web environment too often operationalize the perceived challenge too generally. Researchers tend to ignore the original concept of flow as a construct that induces human beings to grow in the sense of fulfilling potentialities and going beyond those limits (Csikszentmihalyi, 1975). Thus, it is not reasonable to assume that we can develop digital environments where users experience flow throughout the entire time they are interacting with or in the virtual environment. In fact, the states of anxiousness and frustration should be understood more as the triggers or driving forces that motivates a user to strive for the flow state rather than as a plague that should be eliminated entirely. Another important question is how valid users’ evaluations of perceived challenge are. In fact, Chen et al. (1999) found in their study that a great number of the participants were confused with the questions measuring challenge. These results indicate that the ways of measuring the skill–challenge balance should be studied more exhaustively. In spite of efforts to operationalize flow in different contexts, several researchers maintain that much work remains to be done in the operationalization of the key concepts of flow before valid empirical research can be conducted (Chen et al., 1999; Novak, Hoffman, Kiili 192 & Yung, 2000). One aspect that should be considered, in particular, is the partition of flow dimensions into flow antecedents and the flow state. It can be argued that it is not always appropriate to blindly use all nine dimensions of flow before considering the aims of one’s study. Is the aim to study the flow state or the factors contributing to the flow experience? In this paper, both flow antecedents and the flow state are studied. The flow dimensions are divided to antecedents and flow state according to previous research (Kiili, 2005c). METHOD Participants Participants (N = 221) were recruited by e-mail from university students and staff and their families. The gender breakdown was 56% males and 44% females. The ages of participants were distributed as follows: 9% were over 30 years old, 72% were 21–30 years old, 14% were 16–20 years old, and the rest were 11–15 years old. Fifty percent of the participants played digital games almost daily, 22% played once a week, and the rest played rarely or not at all. All were native speakers of Finnish. Materials The game used in this study was based on a Japanese crossword, which is a puzzle also known as nonogram, griddler, and paint-by-numbers. The puzzle genre was selected for this study because solving mental puzzles is one of the oldest forms of enjoyable activities (Csikszentmihalyi, 1991). Generally, the aim in Japanese crossword is to solve the image encrypted with numbers. The numbers are clues that can be interpreted by using logical deduction in order to color the correct squares of the grid. As Figure 2 shows, clue numbers are located at the left and top of the grid. Each number indicates the number of contiguous cells to be colored (length of the filled block). The blocks of cells to be colored are arranged from left to right and from top to bottom according to clues. At least one empty cell must exist between the filled blocks. The experiential gaming model’s design phase was utilized to extend the traditional Japanese crossword into a new game called Day Off. The crossword was embedded within a story line that gives meaning to the puzzle to be solved. The actual game space consisted of 15 columns and 9 rows (Figure 2). The image to be revealed was the Finnish steamship Piiparinen. The game, which was conducted in Finnish, starts with an introduction implemented as an animation that describes the ordinary world of the main character (hero), who is a professional chess player. He has just won the world championship title in chess and is enjoying his vacation by fishing in a national park. Suddenly military troops kidnap him and transport him to their base. The hero is compelled to help military officers solve an encrypted message that contains information about the location of a bomb that terrorists have primed to go off in 20 minutes. The hero hesitates but decides to cooperate because the officers inform him that his son is working in the very harbor where the bomb is located. Evaluations of an Experimental Gaming Model 199 activities are not undertaken by the player with the expectation of some future benefit, but rather because the playing of an educational game itself is the reward. This type of attitude supports the ideology of life-long learning and is a priceless goal in education. This study is a part of an ongoing attempt to develop a usable and valid scale for assessing the flow experience of players in educational games. The results of the experiment described in this paper demonstrate that the constructed FSG instrument provides a satisfactory tool for assessing the gaming experiences of players. However, this work is still in its very initial stages and the FSG instrument needs further development and validation with more complex educational games. REFERENCES Chen, H., Wigand, R., & Nilan, M. S. (1999). Optimal experience of web activities. Computers in Human Behavior, 15, 585–608. Csikszentmihalyi, M. (1975). Beyond boredom and anxiety. San Francisco: Jossey-Bass. Csikszentmihalyi, M. (1991). Flow: The psychology of optimal experience. New York: Harper Perennial. Csikszentmihalyi, M., Larson, R., & Prescott, S. (1977). The ecology of adolescent activity and experience. Journal of Youth and Adolescence, 6, 281–294. Csikszentmihalyi, M., & LeFevre, J. (1989). Optimal experience in work and leisure. Journal of Personality and Social Psychology, 56, 815–822. Csikszentmihalyi, M., & Nakamura, J. (1989). The dynamics of intrinsic motivation. In R. Ames & C. Ames (Eds.), Handbook of motivation theory and research (pp. 45–71). New York: Academic Press. Ghani, J. A. (1991). Flow in human–computer interactions: Test of a model. In J. Carey (Ed.), Human factors in management information systems: Emerging theoretical bases. Ablex, NJ: Ablex Publishing Corp. Ghani, J., & Deshpande, S. (1994). Task characteristics and the experience of optimal flow in human–computer interaction. The Journal of Psychology, 128, 381–391. Havitz, M. E., & Mannell, R. C. (2005). Enduring involvement, situational involvement, and flow in leisure and non-leisure activities. Journal of Leisure Research, 37, 152–177. Jackson, S. A., & Eklund, R. C. (2002). Assessing flow in physical activity: The Flow State Scale-2 and Dispositional Flow Scale-2. Journal of Sport and Exercise Psychology, 24, 133–150. Jackson, S. & Marsh, H. (1996). Development and validation of a scale to measure optimal experience: The flow state scale. Journal of Sport & Exercise Psychology, 18, 17–35. Kiili, K. (2005a). Digital game-based learning: Towards an experiential gaming model. The Internet and Higher Education, 8, 13–24. Kiili, K. (2005b). Content creation challenges and flow experience in educational games: The IT-Emperor case. The Internet and Higher Education, 8, 183–198. Kiili, K. (2005c). On educational game design: Building blocks of flow experience. Tampere, Finland: Tampere University of Technology Press. Kolb, D. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall. Loomis, J., & Blascovich, J. (1999). Immersive virtual environment technology as a basic research tool in psychology. Behavioral Research Method, Instruments, & Computers, 31, 557–564. Miller, G. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81–97. Kiili 200 Moreno, R., & Mayer, R. E. (2005). Role of guidance, reflection, and interactivity in an agent-based multimedia game. Journal of Educational Psychology, 97, 117–128. Norman, D. A. (1993). Things that make us smart: Defending human attributes in the age of the machine. New York: Addison-Wesley. Novak, T. P., Hoffman, D. L., & Duhachek, A. (2003). The influence of goal-directed and experiential activities on online flow experiences. Journal of Consumer Psychology, 13, 3–16. Novak, T. P., Hoffman, D. L., & Yung, Y. F. (2000). Measuring the flow construct in online environments: A structural modeling approach. Marketing Science, 19, 22–42. Pilke, E. M. (2004). Flow experiences in information technology use. International Journal of Human Computer Studies, 61, 347–357. Salonius-Pasternak, D. E. & Gelfond, H. S. (2005). The next level of research on electronic play: Potential benefits and contextual influences for children and adolescents. Human Technology, 1, 5–22. Skadberg, Y. X., & Kimmel, J. R. (2004). Visitors’ flow experience while browsing a web site: Its measurement, contributing factors, and consequences. Computers in Human Behavior, 20, 403–422. Webster, J., Trevino, L. K., & Ryan, L. (1993). The dimensionality and correlates of flow in human-computer interaction. Computers in Human Behavior, 9, 411–426. Winn, W. (2004). Cognitive perspective in psychology. In D. H. Jonassen (Ed.), Handbook of research on educational communication and technology (79-112). Mahwah, NJ: Lawrence Erlbaum. Voelkl, J. E., & Ellis, G. D. (1998). Measuring flow experiences in daily life: An examination of the items used to measure challenge and skill. Journal of Leisure Research, 30, 380–389. Author’s Note This research was funded by the Academy of Finland (201879). All correspondence should be addressed to: Kristian Kiili Tampere University of Technology, Pori P.O. Box 300 FIN-28101 Pori, Finland [email protected] Human Technology: An Interdisciplinary Journal on Humans in ICT Environments ISSN 1795-6889 www.humantechnology.jyu.fi Evaluations of an Experimental Gaming Model 201 Appendix Flow Scale for Games (translated from Finnish to English) Please answer the following questions in relation to your experience with the Day Off game you just played. These questions relate to the thoughts and feelings you may have experienced during playing. Think about how you felt and answer following questions. When you have answered all the questions, press the Send Form button. Thank you! Agree Disagree 1 I was challenged, but I believed my skills would allow me to meet the challenge. 5 4 3 2 1 2 I could use the user interface of the game spontaneously and automatically without having to think. 5 4 3 2 1 3 I knew clearly what I wanted to do and achieve. 5 4 3 2 1 4 I was aware how I was performing in the game. 5 4 3 2 1 5 My attention was focused entirely on playing the game. 5 4 3 2 1 6 I felt in total control of my playing actions. 5 4 3 2 1 7 I was not concerned with what others may have been thinking about my playing performance. 5 4 3 2 1 8 My sense of time altered (either speeded up or slowed down). 5 4 3 2 1 9 I really enjoyed the playing experience. 5 4 3 2 1 10 The challenge that the game provided and my skills were at an equally high level. 5 4 3 2 1 11 The use of the user interface was easy to acquire. 5 4 3 2 1 12 The goals of the game were clearly defined. 5 4 3 2 1 13 I could tell by the way I was performing how well I was doing. 5 4 3 2 1 14 It was no effort to keep my mind on game events. 5 4 3 2 1 15 I had a feeling of control of my actions. 5 4 3 2 1 16 I was not worried about my performance during playing. 5 4 3 2 1 17 The way time passed seemed to be different from normal. 5 4 3 2 1 18 I loved the feeling of playing and want to capture it again. 5 4 3 2 1 19 I had total concentration while playing the game. 5 4 3 2 1 20 The playing experience left me feeling great. 5 4 3 2 1 21 I was totally immersed in playing the game. 5 4 3 2 1 22 I found the experience extremely rewarding. 5 4 3 2 1 23 Read the description of flow experience and answer to the following statement: I experienced a clear flow experience during playing. 5 4 3 2 1 Description of flow: The word flow is used to describe a state of mind sometimes experienced by people who are deeply involved in some activity. For example, a football player may experience flow when nothing else matters but the game itself and it is going very well. Activity that induces flow totally captivates a person for some period of time, in which case time seems to distort and nothing else but the activity seems to matter. Flow may not last for a long time on any particular occasion, but it may come and go over time. Flow has been described as being an intrinsically enjoyable experience. 24 If you experienced flow, what factors in the game contributed to flow experience? 25 If you did not experience flow, what factors in the game disturbed achieving a flow experience? An Interdisciplinary Journal on Humans in ICT Environments ISSN: 1795-6889 www.humantechnology.jyu.fi Volume 2 (2), October 2006, 202–224 202 CREATING A FRAMEWORK FOR IMPROVING THE LEARNABILITY OF A COMPLEX SYSTEM Abstract: When designing complex systems, it is crucial but challenging to make them easy to learn. In this paper, a framework for improving the learnability of a complex system is presented. A classification of factors affecting the learnability of a building modeling system as well as guidelines that refine the factors into practical ways of action are introduced. The factors and guidelines include issues related to the user interface, conformity of the system to user’s expectations, and training. The classification is based on empirical research during which learnability was assessed with several methods. The methodology and the classification of learnability factors can be used as references when analyzing and improving the learnability of other systems. System developers and training providers can utilize these guidelines when striving to make systems easier to learn. Keywords: learnability, ease-of-learning, complex systems, grounded theory, guidelines. INTRODUCTION As complex systems get more and more common in various problem domains, it becomes necessary to make them easily learnable. Good learnability will lead to acceptable learning times, sufficient productivity during the learning phase, and greater satisfaction in new users. However, designing complex systems that are easy to learn is challenging. Complex systems need to provide a wide variety of functionality and to support complex task flows and object structures. There is a danger of complexity leading to long and unproductive learning times. Another challenge with improving the learnability of complex systems is that the changes made in the system must not decrease the efficiency of use (Santos & Badre, 1995). It has been discussed whether learnability and efficiency actually support each other or rather, in fact, contradict. Several studies have indicated that learnability and efficiency are congruent. Whiteside, Jones, Levy, and Wixon (1985), for example, noticed in their study concerning several command, menu, and iconic interfaces that the best system for novice users was also the best for expert users, and the worst system for novices was the worst for experts. However, some researchers (e.g., Goodwin, 1987) have pointed out that experts and novices may have different requirements for a system: Abbreviations and shortcuts, for example, © 2006 Minttu Linja-aho and the Agora Center, University of Jyväskylä URN:NBN:fi:jyu-2006519 Minttu Linja-aho Tekla Corporation Finland Improving the Learnability of a Complex System 203 will improve the performance of experts but may slow down the learning of novices. Thus, balancing learnability and efficiency requires careful consideration. In any case, novices are an important user group and therefore the learning dimension should be taken into account when designing a system. Compacting the learning process and reducing the length of training needed and the number of problems that new users face will save costs for the organization that has taken the system into use and, in many cases, the system provider as well. If users consider the system easy to learn, they are more likely to pass through the learning stage and continue using the system regularly. Satisfied learners may also tell other prospective users about an easily learned system and thus perform efficient peer-to-peer marketing. To improve the learnability of a system, a general understanding of the factors affecting learnability is needed. In this paper, a classification of learnability factors related to a building modeling system is introduced. Practical guidelines that can be used by product developers who design new systems or redesign existing ones are presented as well. I believe that the classification of factors and the guidelines are useful for developing complex systems that are easy to learn. LEARNABILITY In this article, the word learnability signifies how quickly and comfortably a new user can begin efficient and error-free interaction with the system, particularly when he or she is starting to use the system. It can be seen from this definition that both objective and subjective facets of learnability are considered: the speed of learning (quickly) and the subjective satisfaction of the learner (comfortably). The goal of the learning process is efficient and error-free interaction. In the literature, the terms ease-of-learning and learnability often have been used interchangeably. Multiple other definitions for learnability exist in the literature, and they differ from each other slightly. For example, Bevan and Macleod’s (1994) definition of learnability comprises the usability attributes of satisfaction, effectiveness, and efficiency that are evaluated within a certain context, namely the context of a new user. In the ISO 9241 standard (International Organization for Standardization [ISO], 1998a and 1998b), learnability is also defined through the three attributes of efficiency, effectiveness, and satisfaction. Dix, Finlay, Abowd, and Beale (1998) define learnability as the ease with which new users can begin effective interaction and achieve maximal performance. In summary, what most of the definitions have in common is that they address the initial usage experience and include a criterion such as effectiveness or efficiency that can be used to measure the learning results. In addition, some researchers have emphasized that the term learnability should also cover expert users’ ability to learn functions that are new to them (Sinkkonen, 2000). While this perspective is important, I considered it feasible to concentrate on one group of users, namely new users, in this research. The importance of learnability in determining system acceptability has been noticed early (e.g., Butler, 1985). Lin, Choong, and Salvendy (1997) found that learnability is correlated with user satisfaction. The learnability of complex systems is especially critical, as the complexity tends to make the unproductive learning period longer than what is desired by the user and the managers in the organization. Linja-aho 204 The Relationship of Learnability and Usability There are contradicting views of how learnability relates to usability. Some researchers consider learnability to be a subconcept of usability (e.g., Elliott, Jones, & Barker, 2002). Nielsen (1993) presents five subattributes of usability: learnability, efficiency, memorability, errors, and satisfaction. In the same book, Nielsen presents 10 usability heuristics that should be considered when designing user interfaces. Dix et al. (1998) in turn divide usability into the three attributes of learnability, flexibility, and robustness. Lin et al. (1997) list eight attributes: compatibility, consistency, flexibility, learnability, minimal action, minimal memory load, perceptual limitation, and user guidance. Elliott et al. (2002) have discussed the relationship of learnability and usability in their publication. They refer to several studies indicating that the concepts of learnability and usability are strongly related and even congruent. Roberts & Moran (1983), for example, found that procedural complexity underlies both the performance of experts and the learning of novices. Whiteside et al. (1985) have also stated that the concepts of usability and learnability are congruent. Based on these studies, Elliott et al. (2002) made the conclusion that elements from models for usability can be incorporated to models of learnability as well. However, other researchers (e.g., Paymans, Lindenberg, & Neerincx, 2004) have noted that sometimes learnability and usability may be contradictory: that issues that improve learnability actually reduce usability. This is related to the question of how learnability and efficiency relate to each other, which I discussed earlier in this article. Based on the literature review and my experiences, I expected learnability and usability to have several issues in common. However, I expected during the study that I would also notice issues that affect learnability but are not included in common models of usability. I will discuss the relationship of learnability and usability later in this article after presenting the empirical results. Aspects of Learnability Learnability studies have often concentrated on the effect of the user interface design on learnability (see Elliott et al., 2002; Lin et al., 1997). Naturally, the user interface is crucial for learnability, as it essentially forms the link between the user and the system. Different researchers stress various issues as determinants of user interface learnability. Rieman, Lewis, Young, and Polson (1994) emphasize the effect of consistency. Green and Eklundh (2003) in turn emphasize the naturalness of interaction. Dix et al. (1998) have presented five principles that support user interface learnability: predictability, synthesizability, familiarity, generalizability, and consistency. Elliott et al. (2002) found four factors that determine the learnability of a system: transparency of operation, transparency of purpose, accommodation of the user, and the sense of accomplishment. The two first elements are determined by the user interface design and, according to Elliott et al., (2002), the accommodation of the user and the sense of accomplishment follow them causally. Applying these principles to user interface design helps in designing systems that are easy to learn. However, to improve learnability, the correspondence between the system and users’ expectations must be analyzed too, as expectations have a remarkable effect on learning. Users’ expectations may cover the scope, underlying concepts, and basic functionality of the system. Improving the Learnability of a Complex System 205 Kellogg and Breen (1987) among others, have stated that differences between the users’ expectations and the actual system can cause learning difficulties. I decided to use the theory of mental models as a basis for analyzing these differences. Mental models are internal representations of entities with which we interact. According to Fein, Olson, and Olson (1993), a mental model of a computerized system may contain information on the system functionality, components of the system, related processes, and their interrelations. Fein et al. (1993) write that learning can be viewed as a process in which the user processes information and thereby his or her mental model is changed. According to Shayo and Olfman (1998), a user’s mental model helps him or her to plan how to interact with the system, interpret the behavior of the system, and perform correctly when problems occur. As the goal of this study was to provide tools to make the learning process faster, I needed to analyze the entire learning process, from the first interaction with the system, through the training process, and into the post-training phase. In this study, I paid special attention to the training arrangements, as changes in training are a rather quick and easy way to improve learnability. To analyze the effect of training, it is useful to know something about the human learning process and different learning theories. Multiple theories of learning exist, developed by different schools of scientists. The current HCI (human-computer interaction) research has tended to adopt a cognitive perspective on learning (Elliott et al., 2002). Cognitive theorists stress the importance of internal thought processes and mental structures, as opposed to behavioral scientists’ emphasis on behavioral patterns, reinforcement, and conditioning. In this study, I adopted a cognitive perspective to learning and adjusted it with the ideas presented by constructivists. Constructivism is based on cognitive science, and cognitive scientists and constructivists see learning rather similarly. According to constructivists, learning can be defined as a process of building and reorganizing mental structures. Constructivism also states that knowledge is never independent of the learner and the learning context. The learner combines new information with his or her existing knowledge to form a more accurate model of the subject (Marton & Booth, 1997). This view of learning is closely related to the theory of mental models, as both stress the importance of changes in a human’s internal knowledge structures. I saw the constructivist learning theory combined with the concept of mental models as a good basis for analyzing the learning process and the effect of training. In this study, I concentrated on analyzing the three aspects of learnability that were mentioned above: user interface design; differences between users’ expectations and the system, which can be analyzed through the theory of mental models; and the effect of training on the learning process. These aspects are later referred to as user interface, conformity to user’s expectations, and training. Figure 1 illustrates this approach to learnability. Figure 1. A definition of learnability and the aspects addressed in this study. Learnability = How quickly and comfortably a new user can begin efficient and error-free interaction with the system User interface Conformity to user's expectations Training Linja-aho 206 THE BUILDING MODELING SYSTEM In this study, I analyzed the learnability of the Tekla Structures program, a building modeling system that has been developed by the Tekla Corporation. The primary users of the Tekla Structures system are structural engineers. With Tekla Structures, structural engineers can create a three-dimensional model of steel and concrete parts, connections, and other details of a building. Structural analysis can be done using the information contained in the model. The system is very complex in that it provides a wide selection of functionality and supports complex task flows and object structures. My expectation in undertaking a case study on the Tekla Structures program is that it would provide information that could be used to improve the learnability of this particular system as well as be used as a reference when improving the learnability of other complex systems. A typical user interface state of the Tekla Structures system is shown in Figure 2. Figure 2. User interface of the Tekla Structures system. (Model by Antti Pekkala, A-Insinöörit, 2003) Improving the Learnability of a Complex System 207 To support learning, the Tekla Corporation organizes a three-day training course. However, because of the complexity of the system, only a small subset of its features can be addressed in the training and the learning period continues after the formal training. Improving learnability would result in a desired reduction in the learning time. The training course was a good opportunity to observe the beginning of the learning process. I also observed and interviewed users before and after the training. I describe these research activities in the following section. RESEARCH METHODS The purpose of the empirical learnability research was to identify the factors that affect the learnability of the Tekla Structures system and to develop ways to improve learnability. This research was spread over a 3-month period in order to obtain information on different phases of the learning process. Six novice users who had an engineering or technical drawing background were chosen as subjects. Two of them had worked in the building-modeling domain for only a few months, two of them for about 2 years, and two of them for more than 20 years. All of them had some experience with CAD (computer-aided design) systems but five of them had no experience with Tekla Structures and one of them had tried the system for only a day. Four research methods were used at different phases of this study in order to collect versatile information and to capture as many different issues affecting learnability as possible. The four research methods are presented in the following sections. The choice of the research methods was highly dependent on the definition of learnability presented in the beginning of this article. I wanted to address both the objective and subjective facets of learnability and to observe how efficient and error-free the users could be in performing tasks with the system in each learning phase. Pre-Training Interviews The purpose of this research method was to acquire information on the mental models that users had before interacting with the system. This information is useful because differences between users’ mental models and the system may explain learning difficulties. An interview method similar to the one employed in this study was used by DykstraErickson and Curbow (1997). They studied the learnability of a document management platform called OpenDoc. In the interviews that they conducted, they asked users to comment on user interface prototypes. Their goal was to address users’ expectations on how to use certain system features. In this study, the six subjects were interviewed individually and in-person for about 45 minutes. Interviews were conducted during a two-week period before the training. During the interviews, the user interface of the Tekla Structures system was shown to the users and questions were asked about the user interface elements. Subjects were also asked how they expected certain basic modeling tasks to be performed. They were allowed to test some procedures briefly with the system and comment on them. Interview questions included, as a sample, the following: • Which icons do you find familiar? What do you think the others represent? Linja-aho 208 • Which do you expect to be the biggest differences between this system and the software you used before? • How would you start creating columns and beams? • How do you think you can copy and mirror elements? • How do you expect changes in the model to affect drawings? The interviews were audio recorded. The comments were transcribed to a written form after the interview. The interview language was Finnish and I translated users’ comments into English for this article. Training Observation A basic training course organized for new users was observed to acquire information on the beginning of the learning process. The purpose was to see which functions were difficult to learn, what kind of problems users faced when learning to use the system, what training methods were used, and how training affected the learning results. Training observation as a method for studying learnability has not been widely discussed in literature. However, it has been mentioned by Karn, Perry, and Krolczyk (1997) as one method for collecting learnability data. Because training sessions are organized regularly for new Tekla Structures users, training observation was an easily arranged and efficient method for evaluating learnability. The training course that I observed lasted 3 days. All six users who attended the training course had been interviewed prior to the course. The training course consisted of demonstrations given by the instructor and exercises that the subjects performed according to the instructions in the training material. The training material was available in both printed and electronic form. The instructor helped the subjects with the problems they faced while doing the exercises. I observed the six subjects while they performed the exercises and took notes on an observation template, which was a table with the following columns: • main topics covered in the training (which corresponded to chapters in the training material) • time that was spent with each main topic • subtopics covered in the training (corresponded to subsections in the training material) • teaching methods • concepts that were explained • concepts that were not explained • references in the training material to additional learning resources (the references were available as links in the electronic version of the training material) • questions that the subjects asked • behaviors of the subjects. Usability Tests The purpose of the scenario-based usability tests was to assess the outcome of the training and the self-learning phase that followed. The tests were expected to reveal issues that are problematic for new users. Improving the Learnability of a Complex System 215 Figure 3. Overview of learnability factors. Factors Related to the User Interface The supporting data for the factors related to the user interface arose from situations in which the user interface was misleading or not understandable. Some of these factors are familiar from usability checklists (e.g., Nielsen, 1993). However, the factors in my classification concentrate specifically on the issues that are important for a novice user. It is indicated in Table 8 how many of the 237 observations support each of the factors. A summary of the supporting observations is presented as well. Each factor is presented in more detail below. Table 8. Learnability Factors and Observations Related to the User Interface. Learnability factor # of observations Summary of observations Visibility of operations 68 Subjects had problems finding commands that were not clearly visible near the object with which they were interacting. In addition, the subjects did not necessarily remember command names or locations of commands in the user interface. Feedback 23 Subjects were often unsure about whether they succeeded with a certain operation in the absence of a confirmative feedback message. They would also have needed feedback about the current system state. Continuity of task sequences 16 Discontinuities in task sequences were problematic for the subjects. They often did not recognize the way to proceed and, as a result, failed to complete the task. Design conventions 14 User interface elements that were designed conventionally were easy to understand but unconventional ones caused problems. Information presentation 45 Graphical presentations or fields without explanations caused problems for the subjects. User assistance 10 In many problematic situations that were observed in the training and learnability tests, users sought properly designed user assistance to help them overcome the problem. Error prevention 6 A large portion of the observed errors were made by many, and in some cases all, of the six subjects. Learnability = How quickly and comfortably a new user can begin efficient and error-free interaction with the system User interface Conformity to user's expectations Training Visibility of operations Feedback Continuity of task sequences Design conventions Information presentation User assistance Error prevention Differences in functionality Differences in interaction styles Concept clarity Completeness of information Conceptual information Excercises Instructions for basic interaction Instructions for solving problems Motivational content Coverage of system functionality Material types Linja-aho 216 • Visibility of operations. An essential requirement for a learnable user interface is the visibility of possible operations. Whereas expert users can rely on experience, novice users must deduce possible operations and inputs from the hints given by the interface. • Feedback. Feedback is useful for experienced users but especially important for novices. They need feedback on the results of operations and the system’s state. • Continuity of task sequences. A desirable situation is that when users start a command from a menu or by clicking on an icon they are directed by the system until the desired end result is reached. Users should not be required to jump from one menu item or dialog box to another while performing a single task. • Design conventions. If design conventions are followed, users can easily grasp the meaning and usage of one program’s elements from those they have seen in other applications. Design conventions arise from user interface standards and the most common office, Web, or domain-specific software. • Information presentation. Novice users need more detailed descriptions for commands, input fields, and image details than experts do. Special attention should be given to the amount and clarity of information as well. • User assistance. The system should instruct the user and provide additional information on the user interface elements and the related tasks. Current technologies allow user assistance to exist as part of the user interface rather than as a separate help system. • Error prevention. A large portion of the most common errors could be prevented by making small changes in the user interface. In general, the most common causes of errors can be identified by observing new users interacting with the system. Factors Related to Conformity to User’s expectations The learnability factors in this group reflect the effect of differences between the users’ existing mental models and the actual system. These differences may cause learning difficulties. The evidential data for these learnability factors arose from situations in which the subjects expected the system to function differently than it did, and therefore faced problems. In those situations, the subjects had formed their mental model mainly on the basis of a system they had used earlier. The numbers of observations supporting each learnability factor as well as summaries of the observations are presented in Table 9. Next, each factor is then described in more detail. Improving the Learnability of a Complex System 217 Table 9. Learnability Factors and Observations Related to the System Structure. Learnability factor # of observations Summary of observations Differences in functionality 9 When the subjects described their expectations for new software, it turned out that they based their expectations on their experiences with software they are familiar with. Differences in the functionality between the old and new software caused problems for the subjects. Differences in interaction styles 16 It could be deduced from the subjects’ comments that mental models concerning interaction styles were based on the subjects’ experiences with other applications, most commonly office software or operating systems. Subjects expected interaction styles to be domain-independent. Concept clarity 30 Concepts that had not been used elsewhere caused problems unless they were very self-explanatory, communicated clearly in the user interface, and contained familiar terminology. Completeness of information 60 Lack of information about the user interface elements, system concepts, and causes and effects of operations caused difficulties with using the system. • Differences in functionality. The functionality of different software applications naturally varies. Usually, it is not desirable to avoid those differences; instead, users should be supported in learning the new functionality. • Differences in interaction styles. Interaction styles of various software applications also vary. Some of this variation may be necessary because of the different nature of the applications; however, some of it is avoidable. Designing the software so that it supports common interaction styles makes the software easier to learn. • Concept clarity. When starting to use a new software application, the user usually needs to learn new concepts. To support learning, new concepts should be communicated clearly with familiar and understandable terminology. • Completeness of information. The change in user’s mental model can be facilitated by providing enough information about user interface elements, concepts that are present in the system, and causes and effects of operations. Factors Related to Training In this section, training factors that were noticed to affect learnability are presented. The information was extracted from the training observation and comments that the subjects made in the usability tests after the training. When designing training courses to support learning as best as possible, these issues should be considered. The number of observations supporting each of the factors and a summary of the observations are presented in Table 10. Each factor is then described in more detail next. Linja-aho 218 Table 10. Learnability Factors and Observations Related to Training. Learnability factor # of observations Summary of observations Conceptual information 45 Missing conceptual information made the subjects face problems when completing tasks. Exercises 44 Several subjects commented that they learn best by completing exercises. However, completing a task according to step-by-step instructions provided did not always lead to a persistent learning result. Instructions for basic interaction 14 Subjects were not familiar with all the basic interaction strategies even after the training, which caused problems. Instructions for solving problems 16 Subjects were not very well prepared for solving problems themselves but asked for external help when facing problems. Motivational content 3 It could be deduced from the subjects’ comments that they were weighing the advantages of learning the software against the effort spent using it. Coverage of functionality 9 Some tasks that are central to users’ work had received only a little attention in the training and thus the subjects had problems with performing them in usability tests. Material types 13 Several observations concerning the appropriateness of different material types were made. Users’ opinions on the usefulness of different material types varied in different phases of the learning process. • Conceptual information. Conceptual information helps the user to build a revised mental model of the system. For skill learning, mere memorization of procedures is not enough; it is desirable that one truly understands the procedure on a conceptual level as well. Therefore, conceptual information should be included in the training process. • Exercises. For skill learning, it is necessary to practice operations by completing exercises. However, the nature of the exercises also matters. Training should contain exercises that encourage users to process new information and to apply it to new situations. • Instructions for basic interaction. Teaching basic interaction strategies thoroughly in the training will raise productivity during the post-training learning period. This is because users will not need to spend time with simple interaction problems. • Instructions for solving problems. Users will usually face problems when starting with a new software application. To moderate this, users should be equipped with problem solving skills during training. This would help them to use the application competently and independently when no instructor is available to help. • Motivational content. Motivational content is important because it affects the learning behavior of users both during and after the training. Motivational content encourages the users to devote effort to learning more persistently. • Coverage of functionality. Training should concentrate on the system functions that are essential for the users. This can be done only after carefully analyzing user needs. • Material types. The type of the material that is used in training and provided for additional support should be carefully considered. The quality of the material also naturally affects users’ perception of its appropriateness. Improving the Learnability of a Complex System 219 Learnability Guidelines Based on the observations and the learnability factors, 64 guidelines for improving learnability were created. They cover issues related to the user interface, conformity to user’s expectations, and training. The guidelines are presented next. Guidelines Related to the User Interface Altogether, 28 guidelines were formulated for improving the learnability of the user interface. The guidelines are presented in Table 11, and can be used as a checklist when designing new user interface elements. Existing parts of the user interface can also be compared against the Table 11. Guidelines Related to the User Interface. Factor Learnability guidelines 1.1 Place related operations within the same location. 1.2 Make all controls visible. 1.3 Distinguish visually the items that cannot be used in a certain situation. 1.4 Support direct manipulation. 1.5 Direct the user to give the right input. Visibility of operations 1.6 Avoid modes, or if that is not possible, then indicate the mode clearly. 2.1 Provide a system response when the user performs an action. 2.2 Provide a directive system feedback if the user tries to perform an operation that is not possible in a certain situation. Feedback 2.3 Indicate the existence of hidden information. 3.1 Provide links between the different steps of a task. 3.2 Integrate the tasks if they need to be completed sequentially. Continuity of task sequences 3.3 Make the basic steps of a task easily visible and do not complicate them with advanced options. 4.1 Use controls that are familiar from other applications. 4.2 Use familiar task sequences for operations that are not domain specific. Design conventions 4.3 Provide templates to direct the user to the desired design style. 5.1 Organize menus so that they support user tasks. 5.2 Design descriptive labels. 5.3 Avoid system-oriented symbols or abbreviations. Information presentation 5.4 Avoid any unnecessary information. 6.1 Provide information on existing objects. 6.2 Inform the user about errors. 6.3 Give instructions for solving a problem. 6.4 Design clear instructional texts. 6.5 Provide advanced and beginner modes. 6.6 Provide several forms of user assistance. User assistance 6.7 Integrate user assistance into the system interface. 7.1 Automate operations that do not require user action. Error prevention 7.2 Change errors to alternative paths of operation. Linja-aho 220 guidelines and necessary adjustments can be made. Naturally, applying the guidelines requires careful consideration of the user interface elements in question and possibly some expertise in human-computer interaction. Guidelines Related to Conformity to User’s Expectations Ten guidelines concerning conformity to user’s expectations were formulated and they are summarized in Table 12. The guidelines can be referred to when designing new features or introducing new concepts to the system. The guidelines address the issues that may affect the adaptation of users’ mental models. As these guidelines are related to the system’s structure, underlying concepts, and basic functionality, they must be taken into account early in the system development process. The problem with creating guidelines for the learnability factor Differences in Functionality was that those differences can seldom be avoided. The very reason to have a new software application is that it meets distinct needs not met by other software applications. Therefore, it is desirable to make the new software application different from others. Clarity in instruction can help bridge the differences between the former mental model and the new mental model. Table 12. Guidelines Related to Conformity to User’s Expectations. Factor Learnability guidelines Differences in functionality 1 Do not avoid introducing new kinds of functionality but assist the user in learning them. 2.1 Follow design conventions for controls and task sequences. 2.2 Allow the user to interact with objects as in other similar software applications. Differences in interaction styles 2.3 Use menu titles that are familiar from other software applications. 3.1 Use terminology that is familiar from the real world or other software applications. 3.2 Avoid terminology that may be cause incorrect associations. 3.3 Avoid system-oriented terminology. Concept clarity 3.4 Clarify concepts with symbols and images. 4.1 Provide explanations for new concepts in the interface. Completeness of information 4.2 Help the user to perform actions. Guidelines Related to Training Table 13 summarizes the 26 learnability guidelines related to training that were formulated on the basis of the observations. They are expected to cover the training issues that have the most significant effect on learning results. The contents and organization of existing training setups can be compared against the guidelines to find the necessary adjustments. Training sessions differ from each other in terms of the type and number of participants, the duration of the training, the complexity of the subject, practical and physical arrangements, as well as many other dimensions. Therefore, some of the guidelines presented here are intentionally left on a rather abstract level. They present issues that should be checked to assure effective training but the training organizer must also adapt them, as needed, to find the best solution for each training context. Improving the Learnability of a Complex System 221 Table 13. Guidelines Related to Training. Factor Learnability guidelines 1.1 Clarify the meaning of unfamiliar terms. 1.2 Explain the relationship between concepts. Conceptual information 1.3 Clarify the underlying principles that determine how the system is used. 2.1 Introduce the basic form of an operation and require the learner to apply it to new situations. 2.2 Encourage the learner to actively process the information. 2.3 State the goal of each exercise clearly. Exercises 2.4 State the conditions in which the operation can be performed. 3.1 Demonstrate how to interact with objects. 3.2 Demonstrate how to adjust the basic settings. Instructions for basic interaction 3.3 Demonstrate how to use the basic controls. 4.1 Instruct about the available documentation. 4.2 Demonstrate how to use the documentation. 4.3 Instruct how to contact support personnel. Instructions for solving problems 4.4 Address the most common causes of error. 5.1 Summarize the contents of the training at the beginning of the session. 5.2 Concentrate on practical issues that each learner will need in his/her work. Motivational content 5.3 Follow up with learners, if possible. 6.1 Get to know the learners and their needs. 6.2 Adjust the material to cover all the core tasks. Coverage of functionality 6.3 Adjust the time that is spent on each core task according to the difficulty and importance of the task. 7.1 Provide help that is integrated into the user interface and can be easily accessed from within the system. 7.2 Provide printed material or dual monitors in training. 7.3 Provide a limited amount of material to be covered in detail, and supplemental material to be referred to later. 7.4 Design a clear layout for material. 7.5 Provide material in the native language of the learner, if possible. Material types 7.6 Provide search possibilities for digital material. Comparing the Learnability Factors and Guidelines to Previous Research Several classifications exist on the factors that affect the usability of a system. In many of those studies, learnability is seen as a subfactor of usability. However, the classifications of factors affecting learnability are less common. My learnability guidelines and the usability guidelines that have been presented in the literature have some issues in common. For example, I have Error Prevention in the list of user interface related learnability factors, and Nielsen (1993) includes it in his list of usability heuristics. One of my user interface-related learnability factors is Visibility of Operations, whereas Nielsen stresses the visibility of system status in his heuristics. However, the classifications of usability attributes seldom address the issues that I have in the categories of Conformity to User’s Expectations and Training. In the beginning of this article, I discussed how usability has been divided into subattributes by Nielsen (1993), Dix et Linja-aho 222 al. (1998), and Lin et al. (1997). All of these researchers concentrate on attributes of the user interface and not on user’s expectations or training. There may be situations in which training is not available and it is not possible to change the underlying system concepts to correspond to user’s expectations. Then, it may be sufficient to evaluate only the effect of user interface on learning. However, in most cases, it is beneficial to take a multifaceted view of the learning process and address also user’s expectations and training, as has been done in this study. Nevertheless, the classifications of usability attributes presented in the literature and my classification of factors affecting learnability do not contradict each other, but rather, in fact, are complementary. My detailed classification can be used to analyze the learnability of complex systems corresponding to the building modeling system, and to identify ways to improve learnability. General usability classifications, such as the one presented by Nielsen (1993), can be applied to a wider range of systems from consumer products to software applications, as it has been left on a more general level than the classification presented in this article. CONCLUSIONS In this paper, 18 factors affecting the learnability of a building modeling system have been presented. These factors can be used as a general framework for understanding the learnability of this system. In addition, 64 guidelines for improving learnability have been introduced. By following these guidelines in system development and training, the learnability of the building modeling system can be improved. Throughout the study, three aspects influencing learnability were addressed: the user interface, conformity to user’s expectations, and training. Learnability studies have often concentrated on the effect of the user interface, but I believe that a classification addressing the other two distinct aspects of learnability as well helps to improve the learning process and system learnability as a whole. The classification of learnability factors and guidelines was based on a body of empirical data collected via several research methods. The classification was created with the grounded theory method that is intended for creating a theory that fits the available set of data. The classification should have practical relevance to other developers of complex systems as well. The learnability factors and guidelines can be used as a reference when analyzing and improving the learnability of any systems. However, it must be noted that the factors and guidelines are based on the empirical data concerning a building modeling system. Thus, some of the factors and guidelines may not even apply to a system whose scope differs radically from the scope of the building modeling system I studied. Furthermore, the emphasis put on the different factors and guidelines may vary for different systems. However, the grounded theory methodology that was used for analyzing the learnability of a building modeling system can be applied to other systems as well. This would produce corresponding classifications of learnability factors and guidelines that take into account the particularities of each system. I expect that the results concerning learnability are of interest not only for system developers but also for the body of HCI researchers. Not many classifications of factors affecting the learnability of complex systems have been introduced in the HCI literature. This is true for learnability guidelines as well: Several sets of usability guidelines have been Improving the Learnability of a Complex System 223 presented in the literature, but sets of learnability guidelines are less common. In the future, it would be especially interesting to study in more detail the effect of differences between users’ mental model and the actual system. Another future research topic would be to validate the learnability factors and guidelines. This could be done by implementing changes to real systems according to the guidelines and measuring the effect of the changes on the performance of new as well as expert users. REFERENCES Bevan, N., & Macleod, M. (1994). Usability measurement in context. Behavior and Information Technology, 13, 132–145. Butler, K. A. (1985). Connecting theory and practice: A case study of achieving usability goals. In L. Borman & R. Smith (Eds.), Proceedings of the SIGCHI [Special Interest Group on Computer-Human Interaction] conference on human factors in computing systems (pp. 85–88). New York: ACM Press. Dix, A., Finlay, J., Abowd, G., & Beale, R. (1998). Human-computer interaction (2nd ed.). Hertfordshide, UK: Prentice Hall International. Dykstra-Erickson, E., & Curbow, D. (1997). The role of user studies in the design of OpenDoc. In S. Coles (Ed.), Proceedings of the conference on designing interactive systems: Processes, practices, methods, and techniques (pp. 111–120). New York: ACM Press. Elliott, G. J., Jones, E., & and Barker, P. (2002). A grounded theory approach to modeling learnability of hypermedia authoring tools. Interacting with Computers, 14, 547–574. Fein, R. M., Olson, G. M., & Olson, J. S. (1993). A mental model can help with learning to operate a complex device. In S. Ashlund, K. Mullet, A. Henderson, E. Hollnagel, & T. White (Eds.), INTERACT ’93 and CHI [Computer-Human Interaction] '93 conference companion on human factors in computer systems (pp. 157–158). New York: ACM Press. Goodwin, N. C. (1987). Functionality and usability. Communications of the ACM, 30, 229–233. Green, A., & Eklundh, K. S. (2003). Designing for learnability in human-robot communication. IEEE Transactions on Industrial Electronics, 50, 644–650. International Organization for Standardization [ISO]. (1998a). Standard ISO 9241: Ergonomic requirements for office work with visual display terminals (VDTs), part 10: Dialogue principles. Geneva: ISO. International Organization for Standardization [ISO]. (1998b). Standard ISO 9241: Ergonomic requirements for office work with visual display terminals (VDTs), part 11: Guidance on usability. Geneva: ISO. Karn, K. S., Perry, T. J., & Krolczyk, M. J. (1997). Testing for power usability. In S. Pemberton (Ed.), CHI [Computer-Human Interaction]’97 extended abstracts on human factors in computing systems: Looking to the future (p. 235). New York: ACM Press. Kellogg, W. A., & Breen, T. J. (1987). Evaluating user and system models: Applying scaling techniques to problems in human-computer interaction. In J. Carroll & P. Tanner (Eds.), Proceedings of the SIGCHI/GI [Special Interest Group on Computer-Human Interaction/Graphics Interface] conference on human factors in computing systems and graphics interface (pp. 303–308). New York: ACM Press. Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7, 57–78. Lin, H. X., Choong, Y.-Y., & Salvendy, G. (1997). A proposed index of usability: A method for comparing the relative usability of different software systems. Behaviour & Information Technology, 16, 267–278. Marton, F., & Booth, S. (1997). Learning and awareness. Mahwah, NJ: Lawrence Erlbaum Associates. Nielsen, J. (1993). Usability engineering. San Francisco: Morgan Kaufmann. Paymans, T. F., Lindenberg, J., & Neerincx, M. (2004). Usability trade-offs for adaptive user interfaces: Ease of use and learnability. In J. Vanderdonckt, N. Nunes, & C. Rich (Eds.), Proceedings of the 9th international conference on intelligent user interface (pp. 301–303). New York: ACM Press. Rieman, J., Lewis, C., Young, R. M., & Polson, P. G. (1994). Why is a raven like a writing desk? Lessons in interface consistency and analogical reasoning from two cognitive architectures. In B. Adelson, S. Dumais, & J. Olson (Eds.), Proceedings of the SIGCHI [Special Interest Group on Computer-Human Linja-aho 224 Interaction] conference on human factors in computing systems: Celebrating interdependence (pp. 438– 444). New York: ACM Press. Roberts, T. L., & Moran, T. P. (1983). The evaluation of text editors: Methodology and empirical results. Communications of the ACM, 26, 265–283. Salter, W. J. (1988). Human factors in knowledge acquisition. In M. Helander (Ed.), Handbook of humancomputer interaction (pp. 957–968). Amsterdam: Elsevier Science Publishers B.V. Santos, P. J., & Badre, A. N. (1995). Discount learnability evaluation. Graphics, Visualization & Usability Center, Georgia Institute of Technology. Retrieved December 2, 2004, from ftp://ftp.gvu.gatech.edu/pub/gvu/tr/1995/95-30.pdf Shayo, C., & Olfman, L. (1998). The role of conceptual models in formal software training. In R. Agarwal (Ed.), Proceedings of the 1998 conference on computer personnel research (pp. 242–253). New York: ACM Press. Sinkkonen, I. (2000). Things that facilitate learning in products. Unpublished licentiate thesis, Helsinki University of Technology, Espoo, Finland. Whiteside, J., Jones, S., Levy, P., & Wixon, D. (1985). User performance with command, menu, and iconic interfaces. In L. Borman & R. Smith (Eds.), Proceedings of the SIGCHI [Special Interest Group on Computer-Human Interaction] conference on human factors in computing systems (pp. 185–191). New York: ACM Press. Author’s Note I thank all the employees at Tekla Corporation who gave their input into the research project. I also thank the participants of the empirical research. In addition, I present thanks to the researchers at the Helsinki University of Technology for the discussions that contributed to the writing of this paper. All correspondence should be addressed to: Minttu Linja-aho Tekla Corporation P.O.Box 1 FI-02131 Espoo, FINLAND minttu.linj[email protected]m Human Technology: An Interdisciplinary Journal on Humans in ICT Environments ISSN 1795-6889 www.humantechnology.jyu.fi TAM for Useful and Fun Information Systems 231 The research model shown in Figure 1 was tested by multiple regression analysis using SPSS 11. This is consistent with methods used in similar previous studies, such as Davis et al. (1992) and Moon and Kim (2001). The results are shown in Table 2. Consistent with Hypothesis 1, a positive relationship was found between perceived ease of use and perceived enjoyment. Perceived enjoyment and perceived usefulness both impact positively on intention to use, which provides support for Hypotheses 2 and 4. A positive relationship was found between perceived ease of use and perceived usefulness, which is consistent with Hypothesis 5. No positive relationship between perceived ease of use and intention to use was found, meaning no support was found for Hypothesis 3. To provide quantitative estimates of the relationships between intention, perceived ease of use, usefulness, and enjoyment, a path analysis of the path diagram shown in Figure 2 was conducted. Figure 3 shows the path coefficients that were computed. Since these coefficients are standardized, it is possible to compare them directly (Bryman & Cramer, 2005). It can be seen that perceived usefulness has a stronger effect on intention to use than perceived enjoyment, Table 2. Results from running regression analyses to test the hypotheses. Model R2 Beta t p Standard error Result 1. PE = PEOU + errors 0.124 0.352 2.551 0.014 0.661 H1 was supported 2. I = PE + errors 0.242 0.492 3.831 0.000 0.877 H2 was supported 3. I = PEOU + errors 0.063 0.251 1.755 0.086 0.975 H3 was not supported 4. I = PU + errors 0.379 0.616 5.297 0.000 0.794 H4 was supported 5. PU = PEOU + errors 0.239 0.489 3.805 0.000 0.611 H5 was supported P < 0.05 PE = perceived enjoyment, PEOU = perceived ease of use, PU = perceived usefulness, I = intention to use. Perceived Ease of use Perceived usefulness Perceived enjoyment Intention to use e1 e2 e3 0.872 0.353 0.571 0.700 0.936 -0.153 0.352 0.489 Figure 3. Acceptance model supported by the analysis including the standardized beta path coefficients and error terms. Chesney 232 and that perceived ease of use has a slightly negative direct effect. The indirect effect of perceived ease of use is 0.403; the overall impact is therefore 0.250. Clearly an appreciation of the intervening variables perceived usefulness and perceived enjoyment is essential to an understanding of the relationship between perceived ease of use and intention to use. DISCUSSION Hypotheses 1 (There is a positive relationship between perceived ease of use and perceived enjoyment) and 2 (There is a positive relationship between perceived enjoyment and intention to use)—about the relationships between ease of use, enjoyment, and intention, as shown at the bottom of Figure 2—were derived from other studies involving recreational systems. Neither was rejected by the results of this study: Perceived ease of use is significantly related to perceived enjoyment and perceived enjoyment is significantly related to intention to use. Hypotheses 3 (There is a positive relationship between perceived ease of use and intention to use), 4 (There is a positive relationship between perceived usefulness and intention to use), and 5 (There is a positive relationship between perceived ease of use and perceived usefulness) all concern relationships predicted within the original TAM. Hypotheses 4 and 5 were confirmed: There is a positive relationship between perceived usefulness and intention to use, and there is a positive relationship between perceived ease of use and perceived usefulness. However hypothesis 3 was rejected: A positive relationship between perceived ease of use and intention to use was not found. The acceptance model that these results support is shown in Figure 3. The empirical data show that perceived usefulness does achieve dominant predictive value over perceived enjoyment and perceived ease of use. Further research is needed to see if this result is replicated with other dual systems, although the finding is consistent with Davis et al. (1992). Perceived ease of use loses any direct impact on intention to use but plays an important part in influencing perceived usefulness and enjoyment. Clearly, given the strength of the error terms in Figure 3, there are other unknown factors impacting intention to use, perceived usefulness, and enjoyment, and further work may attempt to identify these. The results also suggest that there may be value in exploring alternative ways to make dual systems more acceptable to users other than by merely increasing ease of use. Increasing enjoyment is one of them. Although ease of use has an impact on enjoyment, identifying the other factors that impact enjoyment would allow investigation into whether these could be exploited to increase acceptance. This study agrees with the finding of Van der Heijden (2004) that purpose of use is important in determining the factors that predict acceptance, and that progress in user acceptance models can be made by focusing on the nature of use. The grid shown in Figure 1 is a useful way of doing this. This study has a number of limitations. First, almost all of the respondents were male. Future work should repeat the study with a dual system that has an even gender mix. Second, the system studied is very different from a more mainstream system, such as a word processor, not least in the technical ability of the user. Therefore, future work should study more common dual systems. Also, this study, like many other studies, is biased toward users of the technology: The reasons for how and why a technology-minded individual might use a system, or view its context of use, may be quite distinct from someone who is less TAM for Useful and Fun Information Systems 233 technology-minded. Relatedly, the important factors in choosing to use a system may be different from the important factors in choosing not to use a system. These aspects of use should be considered in future studies. Lastly, although the results are consistent with other findings, they cannot be applied to purely utilitarian systems. For instance, the results do not suggest that acceptance of productivity-oriented systems can be increased by adding a fun dimension. The systems studied here were specifically used in part for fun and in part for productivity; for many users, the fun was as important or more so that the productivity. In any case, trying to increase acceptance of utilitarian systems by increasing enjoyment may encourage users to spend their time on frivolous use. ENDNOTE 1. For more information about Lego Mindstorms see http://mindstorms.lego.com/ REFERENCES Adams, D. A., Nelson, R. R., & Todd, P. A. (1992). Perceived usefulness, ease of use and usage of information technology: A replication. MIS Quarterly, 16, 227–247. Ajzen, I. (1991). The theory of planned behavior. Organisation Behavior and Human Decision Processes, 50, 179–211. Atkinson, M., & Kydd, C. (1997). Individual characteristics associated with World Wide Web use: An empirical study of playfulness and motivation. The Database for Advances in Information Systems, 28, 53–62. Babin, B. J., Darden, W. R., & Griffin, M. (1994). Work and/or fun: Measuring hedonic and utilitarian shopping value. Journal of Consumer Research, 20, 644–656. Bryman, A., & Cramer, D. (2005). Quantitative data analysis with SPSS 12 and 13. New York: Routledge. Chesney, T. (2004). “other people benefit. i benefit from their work”: Sharing guitar tabs online. Journal of Computer-Mediated Communication, 10(1). Available from: http://jcmc.indiana.edu/vol10/issue1/chesney.html, accessed 17/10/06. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 319–340. Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1992). Extrinsic and intrinsic motivation to use computers in the workplace. Journal of Applied Social Psychology, 22, 1111–1131. Dennis A., Nelson, R., & Todd, P. A. (1992). Perceived usefulness, ease of use, and usage of information. MIS Quarterly, 16, 227–247. Doll, W. J., Raghunathan, T. S., Lim, J., & Gupta, Y. P. (1995). A confirmatory factor analysis of the user information satisfaction instrument. Information Systems Research, 6, 177–189. Hendrickson, A. R., Massey, P. D., & Cronan, T. P. (1993). On the test-retest reliability of perceived usefulness and perceived ease of use scales. MIS Quarterly, 17, 227–230. Hsu, C., & Lu, H. (2004). Why do people play on-line games? An extended TAM with social influences and flow experience. Information & Management, 41, 853–868. Igbaria, M., Zinatelli, N., Cragg, P., & Cavaye, A. L. M. (1997). Personal computing acceptance factors in small firms: A structural equation model. MIS Quarterly, 21, 279–302. Chesney 234 King, C. V., Fogg, R. J., & Downey, R. G. (1998, April). Mean substitution for missing items: Sample size and the effectiveness of the technique. Paper presented at the 13th Annual Meeting for the Society of Industrial and Organizational Psychology, Dallas, TX. Moon, J., & Kim, Y. (2001). Extending the TAM for a World Wide Web context. Information & Management, 38, 217–230. Rogers, E. (1995). Diffusion of innovations (4th ed.). New York: The Free Press. Segars, A. H., & Grover, V. (1993). Re-examining perceived ease of use and usefulness: A confirmatory factor analysis. MIS Quarterly, 17, 517–525. Starbuck, W. H., & Webster, J. (1991). When is Play Productive? Accounting, Management, and Information Technology, 1, 71–90. Van der Heijden, H. (2004). User acceptance of hedonic information systems. MIS Quarterly, 28, 695–704. Venkatesh, V. (1999). Creation of favorable user perceptions: Exploring the role of intrinsic motivation. MIS Quarterly, 23, 239–260. Venkatesh, V., & Brown, S. A. (2001). A longitudinal investigation of personal computers in homes: Adoption determinants and emerging challenges. MIS Quarterly, 25, 71–102. All correspondence should be addressed to: Thomas Chesney Nottingham University Business School Jubilee Campus Nottingham, UK NG8 1BB [email protected] Human Technology: An Interdisciplinary Journal on Humans in ICT Environments ISSN 1795-6889 www.humantechnology.jyu.fi TAM for Useful and Fun Information Systems 235 Appendix SURVEY INSTRUMENT 1. Please rate on a scale of one to ten how much you are programming robots because the programming itself is fun. 2. Please rate on a scale of one to ten how much you view programming as a way of getting the job of building robots done. Perceived usefulness (6 point Likert scale - strongly agree, agree, slightly agree, slightly disagree, disagree, strongly disagree) 1. Using my CHOSEN LANGUAGE enables me to build robots quickly 2. Using my CHOSEN LANGUAGE improves my performance at building robots 3. Using my CHOSEN LANGUAGE increases my productivity at building robots 4. Using my CHOSEN LANGUAGE enhances my effectiveness at building robots 5. Using my CHOSEN LANGUAGE makes it easy to build robots 6. I find my CHOSEN LANGUAGE useful in building robots Perceived ease of use (6 point Likert scale - strongly agree, agree, slightly agree, slightly disagree, disagree, strongly disagree) 1. Learning to use my CHOSEN LANGUAGE was easy for me 2. I found it easy to get my CHOSEN LANGUAGE to do what I want it to 3. Interaction with my CHOSEN LANGUAGE is clear and understandable 4. It was easy for me to become skilful at using my CHOSEN LANGUAGE 5. I find my CHOSEN LANGUAGE easy to use Perceived enjoyment (6 point Likert scale – respondents were asked to select where their CHOSEN LANGUAGE lies between each of the two terms) EnjoyableDisgusting ExcitingDull PleasantUnpleasant Interesting-Boring Intention to use 1. I intend to keep using my CHOSEN LANGUAGE An Interdisciplinary Journal on Humans in ICT Environments ISSN: 1795-6889 www.humantechnology.jyu.fi Volume 2 (2), October 2006, 236–237 236 BOOK REVIEW Atanu Garai & B. Shadrach (2006). Taking ICT to every Indian village: Opportunities and challenges. New Delhi, India: One World South Asia; 133 pages. Reviewed by Pertti Saariluoma Department of Computer Science and Information Systems, University of Jyväskylä Finland Information and communication technology (ICT) activities can easily be seen as a sort of technocracy, which is not surprising because the focus of attention is often dominated by issues such as the bandwidth, new devices, or the fierce competition between technological companies and their innovative products. In short, the discussion often is restricted to Habermasian technical interest of knowledge. At the end of the day, however, everything in ICT is about people and, more specifically, about the emancipatory application of knowledge for and by the people. This latter perspective on ICT development comes to the fore in a very interesting and thought-provoking way in a book by Garai and Shadrach, titled Taking ICT to Every Indian Village. The book discusses ICT developments in hundreds of thousands Indian villages, presented through four somewhat independent texts that shed light on various practical and research aspects of the status of ICT in India. The book opens with an analysis of Martha Nussbaum’s ideas about central human functional capacities, and how these ideas relate to the vision of technological benefit in India. Nussbaum’s important humanistic goals—such as bodily integrity, cognitive faculties, emotions, affiliation and control over one’s own environment—need to serve as an essential beacon for how technology should benefit human development on both sides of the digital divide. One simply needs to search the Web for any of Nussbaum’s humanistic themes to discover how far some technological uses have strayed from obvious benefit to human cognitive and emotional development. In the balance of the book, the authors develop their themes through assisting the reader to understand the challenges and opportunities for ICTs in India, and through good factual argumentation. They raise important people-centered themes for technological use, such as education, health, governance, community, and business. Garai and Shadrach provide a snapshot of the ICT diffusion in a country of more than a billion people, where ICT access is © 2006 Pertti Saariluoma and the Agora Center, University of Jyväskylä URN:NBN:fi:jyu-2006526 Book Review 237 challenged by the geography, economy, literacy rate, multilingualism, rural poverty, and so on. The abundance of rural villages that are quite socially, economically, politically and culturally diverse underscores the need for tailored solutions to unique situations. Thus the text presents a concrete picture about the relationship between research and practice, and it discusses with strong expertise the vital issues regarding how technology is applied in rural— sometimes remote—settings. As a result, this book presents a good guide to the ICT development in India—with possible implications for other rural and developing environments—encompassing both the reality and the opportunities. The realities of rural life in India, and the implications for technology implementation, require solutions to technological needs that, while perhaps quite different from highly technological societies, are obviously very practical for India. For example, information kiosks are commonplace in rural areas, offering calling and Internet services to the public, an effective solution to make limited ICT facilities accessible to many. The lesson provided, of course, is the need for technologies—and, more specifically, technological solutions—to conform to the realities of the people in a particular setting and with particular needs. This book serves well technology designers and strategists who envision technology that is adaptable to and in harmony with the great variation in human need and circumstances throughout the world. Garai and Shadrach do not limit their discussion to the social aspects of ICT: They raise issues and concerns about technology infrastructure and ICT functionality, which have equal implications for the implementation and use of any technology. In this way, the authors provide an important insider’s view to all who are interested in the opportunities and challenges for ICTs in the developing world. On the whole, this small book offers valuable insight on the multidimensional human element of ICTs, and specifically on the unique needs and solutions required for rural communities in developing countries. Human Technology: An Interdisciplinary Journal on Humans in ICT Environments www.humantechnology.jyu.fi