A creative prototype illustrating the ambient user experience of an intelligent future factory

Journal of Ambient Intelligence and Smart Environments 9 (2017) 41–57 41 DOI 10.3233/AIS-160417 IOS Press A creative prototype illustrating the ambient user experience of an intelligent future factory Tiina Kymäläinen a,*, Eija Kaasinen a, Jaakko Hakulinen b, Tomi Heimonen b,**, Petri Mannonen c, Maiju Aikala a,***, Hannu Paunonen d, Jouni Ruotsalainen dand Lauri Lehtikunnas d aHuman Factors in Complex System, VTT Technical Research Centre of Finland Ltd, Tampere, Finland bTampere Unit for Computer-Human Interaction, School of Information Sciences, University of Tampere, Finland cSchool of Science, Aalto University, Espoo, Finland dR&D Valmet Automation Inc, Tampere, Finland Abstract. This article introduces user experience research that has been carried out by evaluating a video-illustrated science fiction prototype with process control workers. Essentially, the prototype ‘A remote operator’s day in a future control center in 2025’ was aimed at discovering opportunities for new interaction methods and ambient intelligence for the factories of the future. The theoretical objective was to carry out experience design research, which was based on explicit ambient user experience goals in the nominated industrial work context. This article describes the complete creative prototyping process, starting from the initial user research that included evaluations of current work practices, technological trend studies and co-design workshops, and concluding with user research that assessed the final design outcome, the science fiction prototype. The main contribution of the article is on the ambient user experience goals, the creation process of the video-illustrated science fiction prototype, and on the reflection of how the experience-driven prototype was evaluated in two research setups: as video sequences embedded in a Web survey, and as interviews carried out with expert process control workers. For the science fiction prototyping process, the contribution demonstrates how the method may employ video-illustration as a means for future-oriented user experience research, and how complementary user-centered methods may be used to validate the results. Keywords: Industry 4.0, science fiction prototyping (SFP), (ambient) user experience (UX) design, user-centered design (UCD), process control work 1. Introduction Enabled by advanced digitalization, industrial internet and intelligent technologies, such as ambient intelligence (AmI), it is expected that the 4th industrial revolution, often referred as to Industry 4.0, will soon be on its way [28,31]. In general, it is expected that Industry 4.0 will benefit from the AmI technologies and result e.g. in shorter development periods, individualization in demand for the customers, flexibility, decentralization and resource efficiency. In *Corresponding author. Tel.: +358 400399511; E-mail: [email protected]. **Present address: Department of Computing and New Media Technologies, University of Wisconsin-Stevens Point, USA. ***Present address: Pilot Plant, Oy Keskuslaboratorio – Centrallaboratorium Ab, Espoo, Finland. the production processes, there will be significantly greater demands made of all members of the workforce, in terms of managing complexity, abstraction and problem-solving [19]. For the industrial workers, the revolution is expected to provide opportunities by the qualitative enrichment of the factory work: a more interesting working environment and the greater autonomy and opportunities for self-development. Subsequently, the employees are likely to act much more on their own initiative, to possess excellent communication skills and to organize their personal work flow; i.e. in the future factories they are expected to act as strategic decision-makers and flexible problemsolvers [10]. The industry transformation is anticipated to be most relevant in the manufacturing industry, but it will also affect such industrial sectors as process control This article is published online with Open Access and distributed under the terms of the Creative Commons Attribution Non-Commercial License (CC BY-NC 4.0). 1876-1364/17/$35.00 © 2017 – IOS Press and the authors. 42 T. Kymäläinen et al. / A creative prototype illustrating the ambient user experience of an intelligent future factory work, which is the main context environment of this article. The design research introduced presents an idea that the future control workers will monitor and supervise AmI systems with new interaction methods, and that the work tasks will be shared flexibly between the systems and human workers. The research has taken a stance that, in order to succeed, the Industry 4.0 requires more than merely introducing new technologies on the factory floor. In essence, for developing sustainable solutions there is a need for a shared vision of the future, which requires a clear and extensive view of how the new technologies will be utilized and what kind of work roles and practices will emerge as a result. So as to create a shared vision of the future process control work, the research has focused on user experience (UX) design and employed a method called science fiction prototyping (SFP) [17] in delivering its design outcome. The article explains in detail the UX investigations that have followed the SFP framework [12,13], yet the main focus is on illustrating how the UX research has been implemented in a videoillustrated science fiction prototype, entitled ‘A remote operator’s day in a future control center in 2025’, in which AmI, computing and design play a critical role. For the AmI community, this contribution is essential, as it demonstrates UX research resulting from the user-centered design (UCD) approach to ambient intelligence, which is an important topic raised by Aarts and De Ruyter [1] in their call for new research perspectives on AmI. In addition, the design research contributes to the second order ambient intelligence, which advocates new forms of experience, curiosity and engagement in AmI solutions [5]. Essentially, the article tackles a challenging problem in the barrier reduction for Industry 4.0. in the inclusion of smart intuitive systems explicitly targeted at process control work in future process plants. The main objective is to provide a practical example of the creative UX design methodology for early engagement with participants in future AmI systems. 2. Background of the study and key literature The user experience (UX) research was seen to be timely and relevant in this case study, as, overall, the UX approach is currently receiving growing attention in the development of industrial working environments and services [18,44]. Basically, the UX approach suggests that, in contrast to problemor technology-driven design, user experience should be the main force in driving the design [9,24,39]. In general, the aim is to guide the design towards positive and satisfying experiences that help in communicating important objectives, as proposed e.g. by [9,15,40,45]. In an industrial work context, the investigations have usually focused on a thorough understanding of what the employees want to achieve in their work, and how this can best be supported. Accordingly, in this domain, UX has been interpreted explicitly as: “The way a person feels about using a product, service, or system in a work context, and how this shapes the image of oneself as a professional” [18]. So far, the main difficulty in adopting the UX-driven design approach in a technologydriven industry has been the fact that the technological skills in a company often dictate the design space [38]. Therefore, the core proposition in UX design, ‘experience before product’ [15, p. 63], has not been realized. That is the reason why the research in this article has encouraged the idea that UX design investigations and outcomes should initially be in balance with the brand and image of the company and involve the characteristics of the services that are generally valued by its customers. To achieve this, the important experiences has been pursued by defining explicit ambient UX goals to which the industrial partner and the research group have commit themselves. As for the primary means to create, deliver and evaluate the UX goals, the study has employed the science fiction prototyping (SFP) method that B.D. Johnson originally introduced as a tool for intelligent environment (IE) research [16,17]. After the method launch, the great majority of the SF-prototypes were published within the IE domain, although the method has later been widely adopted also by other fields, such as futures studies, foresight and business studies (a full literature synthesis on the SF-prototype topics can be found in [25]). It should be noted that the relation between science fiction and science fact has also been identified simultaneously by scholars, technology designers and researchers from diverse disciplines, e.g. by [4,6,7,14,30,37,41,42]. Principally, the prototypes created by the SFP method are stories grounded in current science and engineering research that are written for the purpose of acting as prototypes for people to explore a wide variety of futures [17]. In order to justify the use of the method in a UX research context, it is referred here to Forlizzi and Battarbee [9], who have confirmed that stories and storytelling provide a solid basis for UX research. They explain that, as a repository of experience, stories T. Kymäläinen et al. / A creative prototype illustrating the ambient user experience of an intelligent future factory 43 contain almost everything that is required for a deep, appreciative understanding of the strengths and weaknesses of a service, as well as what needs to be redesigned for the future. In this context, the SFP method seemed particularly expedient, since, as a design tool, it allows one to study alternative, potential futures and illustrate how to interact with AmI and emerging technologies after the transformation of the industrial work environment. The earlier SF-prototypes that have referred to UX design can be found e.g. from the work of Egerton et al. [8] and Graham [11], but the role of the UX design has not been systematically described in either of those. Correspondingly, many other prototypes make reference to the broader concerns of UCD, e.g. [36,43,47], but do not explain in detail how these findings are converted into prototype creation. The most suitable previous example of a UX-driven SFP process was found from [23]. The approach in the UX research has nevertheless been introduced in a very different research context, and as it does not include the critical involvement of the company brand in the process, the background for the design process was sought from elsewhere, namely business sciences. In that domain, Wu, has introduced “imagination workshops” [46] and Zheng & Callaghan “Diegetic Innovation Templating” (DiT) [49] as potential SFP creation processes. The disadvantage of those for this research, however, seemed to be that they employed existing Science Fiction as their primary source of inspiration. Ultimately, the research for this article found solid ground for the UX-driven process from the framework introduced by Graham et al. [12,13]. This SFP framework intensifies the method by “expanded consumer experience development”, which seemed to be the most convenient approach to ambient UX design research. 3. Methodology – Design of the study and used methods Accordingly, the research for this article has employed the SFP framework by Graham et al. for its UX design process and Johnson’s SFP method for creating its main design outcome, the SF-prototype. In brief, the SFP framework includes four tasks: –Research and experimentation (what) –Exploration and planning (who) –Development (how), and –Deployment The first task of the framework is intended to support the world building process of the prototype by research and experimentation. In the practical UX research, during this task the research group investigated trend studies (business, technology and societal trends) and organized co-design workshops for selecting most suitable technologies and topics to be explored in the prototype. The second task involves people in the process by continuing the exploration and planning with that focus. In order to determine the specific people and locations in the nominated process control work, the research group conducted preliminary user studies on location. In the framework, this task concludes with an experience specification, which in the study has been understood as the defining of explicit ambient UX goals for the process control work. In contemplating the third task, the development of the SF-prototype, the research group arranged several iterative workshops, in which the manuscript for the prototype was accomplished. The creation process followed rigorously Johnson’s SFP method, which consists of five fundamental steps [16]: 1. Select a technology, science or issue to be explored using the prototype. Set up the world in question; introduce people and locations. 2. Introduce the scientific inflection point. 3. Explore the science’s implications and ramifications for the world. 4. Introduce the human inflection point with the technology; modifications or fixing the problem; a new area of experimentation. 5. Explore the implications, solution or lessons learnt. In general, the use of the method results in an SFP that takes a written form, but in this case the outcome was decided to be video sequences and supportive interaction demos. The final step of the method “exploring the implications, solution or lessons learnt” may also be understood as the last, fourth step in the SFP framework, the deployment. In the study, this part received special attention, as there were two complementary user evaluation setups organized for evaluating the SFP: a Web survey and expert interviews. To summarize, in the pursuit of creating, delivering and evaluating the SFP the design process was constructed from the following actions (see Table 1). Figure 1explicates how the timeline of the case study is attached to the SFP framework and SFP method. 44 T. Kymäläinen et al. / A creative prototype illustrating the ambient user experience of an intelligent future factory Table 1 A summary of how the study was designed Task 1. Research and experimentation Trend studies Co-design workshops Task 2. Exploration and planning User studies in the nominated industrial work context Defining of the ambient UX goals Task 3. Development Workshops for creating the manuscript Creating of the SFP video Creating the supportive interaction demos Task 4. Deployment User evaluations by a Web survey User evaluations by expert interviews Fig. 1. Design process: on top are the tasks of the SFP framework, in the middle the case study settings, and at the bottom, the steps of the SFP method. As the objective of the research was to carry out UX investigations by creative prototyping, the research questions were formed around this problem-space. Consequently, the design research was pursued in order to answer the following questions: –How to carry out UX-driven research by employing the SFP method and SFP framework –How to create and deliver video-illustrated SFP with ambient UX goals –How to evaluate the ambient UX goals embedded in the SFP and by that, validate research The remainder of this article will follow the structure of the SFP framework, at first, with a detailed description of the process, methods used and data sets of each of the tasks, and subsequently, by delivering the results and outcome of the process. 3.1. Research and experimentation In the initial phase of the process, the research group arranged a series of co-design workshops for deliberating the world building procedure of the first framework task. The process began with a trend analysis that studied business, technology and the general societal trends of the nominated process control work domain. After that, the results were shared within the first co-design workshop, which involved researchers and company representatives. The participants assessed the importance of the trends presented, and the most popular of them were used as the basis for a discussion in groups that identified important themes relating to user experiences and possible technical solutions that supported them. The most interesting concepts, from academic and business perspectives, were in the concluding workshops further developed into usage scenarios. As a reference, in the SFP framework, the conceptual prototyping is encouraged as the outcome of the first task. 3.2. Exploration and planning The first user research in an actual location focused on what kinds of experiences people currently had during the process control work, and what kinds of positive experiences they expect to have in the future. The research was carried out in an oil refinery focusing on the production of advanced, low-emission traffic fuels in Finland (details omitted to guarantee participant anonymity), in the autumn of 2012. The research was conducted in situ in the control center where the operators worked; 23 operators participated in the research (20 male/3 female; with work experience ranging from 1–15 years). The research consisted of a contextual inquiry [2], a user experience significance questionnaire, and critical experience interview [32]. There were altogether six contextual inquiry sessions and five critical experience interviews. The research methods were selected so that the user and work experiences were handled both directly and indirectly during the information gathering. 3.3. Development During the workshops, the research group considered alternative means for describing and illustrating T. Kymäläinen et al. / A creative prototype illustrating the ambient user experience of an intelligent future factory 45 the UX goals by using the SFP method. To support this aim, the means were selected primarily to be video sequences and, secondarily, interaction demos, which were hypothesized to provide an effective delivery of the content for the evaluation participants. Videos were chosen to illustrate, in particular, experiences relating to AmI, new technical possibilities for remote control work, the remote presence of employees and new collaboration practices. The videos were also considered to be an effective way to communicate about the overall design, which was aimed to be a comfortable and flexible working space that supported collaboration. The interactive demo was created for speechand gesture-based interaction. For this work, the fundamental framework came from [29]. The screenplay was collaboratively developed by means of visual scripts and early test videos, as these techniques were identified as being relevant during the trend analysis. The video-illustrated SFP ‘A remote operator’s day in a future control center in 2025’ included altogether six video sequences. The iterative work was carried out by a team comprising researchers, company representatives and video production professionals. To make the videos, the research group first organized a workshop for creating the screenplays for the video sequences. For this work, the grounding framework came from [35]. The sequences were filmed in an interactive collaborative environment that was staged with a set of monitors, wall-sized projected displays and large touchscreens. The imaginary visual content was exclusively created for all the displays seen on the video. Technically, the videos were shot by using a combination of minimally interactive prototypes and green screen technique, where envisioned screen content was added to the video at the post production stage. Display contents were added by using digital compositing and animations, e.g. an on-screen cursor following hand pointing, were synchronized to the filmed material. Spoken dialogue and a narrator’s voice were added into the video at post production. The speechand gesture-based interaction demo was implemented by adding new interaction modalities into an existing process automation system and, consequently, the prototype did not exactly match with the interactions seen on the video-illustrated SFP. However, the same automation system was in use in the organization where the on-site evaluations took place, so the participants were already familiar with the system and could, therefore, focus simply on the new interaction techniques. Technically, the supportive interaction demos were implemented by using Microsoft Kinect and Microsoft Speech Recognition. A wireless clip on a microphone attached to the user’s clothes was used for the speech input. The prototype enabled window and view manipulation; participants could point at windows using hand gestures and, using an on-screen pointer, grab and move windows by closing a pointing hand into a fist, resize windows by grabbing with two hands, close and resize windows with speech, and point at and change views and open new ones by using speech commands. The actual process control operations were omitted from the supported functionality. 3.4. Deployment For evaluating the video-illustrated SFP, two complementary user research setups with expert process control operators and workers were established. In the first setup, the participants were introduced to the SFP via the videos uploaded to YouTube and embedded in a Web questionnaire. The questionnaire included a discussion space that was active for a two-month period, in late 2014. In all, 58 experts participated in the Web survey, 16 of whom were active commentators. The participants were selected from among the customer companies of the project’s participating company. The participants’ background in process control work was diverse, as the domains related to the chemical industry, energy distribution, energy production, food industry, forest industry, manufacturing and nuclear power. The participants had work experience of up to 41 years; all were interested or very interested in new technologies. The second evaluation setup included interviews conducted in situ in a municipal power plant in a city in southern Finland (details omitted to guarantee participant anonymity), during October 2015. In addition to seeing the SFP via YouTube videos, the participants were also able to try out the speechand gesture-based interaction demo. The evaluations included six operators (all male) aged 27–34, who described their occupational titles as: automation manager, process operator, power plant operative, service engineer, automation engineer, and electricity instrument manager. The participants had experience of working in a control center environment ranging from 1–7 years; all were interested or very interested in new technologies. The Web survey consisted of both closed and openended questions; the interview setup consisted of a video interview with user analysis [48] and a semi- 46 T. Kymäläinen et al. / A creative prototype illustrating the ambient user experience of an intelligent future factory structured interview [22]. In both groups, the participants assessed six video scenes, one at a time; the main difference between the evaluation setups was that, in the Web survey, the participants could choose which of the six scenes they wanted to see and comment on first. In order to gain quantitative data about the UX goals, the users were, after seeing each video scene, requested to assess whether they could identify with the UX goals by answering a UX significance questionnaire (using a 5-point Likert scale) specifically created for this project. The users in both research setups answered the same open-ended questions relating to the SFP; in addition, they were requested to analyze the new interaction methods and deliver new ideas. As a final part, the participants were allowed to provide overall feedback on the presented future control environment. The Web survey and interview data were transcribed and qualitatively analyzed. 4. Analysis and findings 4.1. Results of the research and experimentation The aim of the first workshops was to identify and discuss the results of the trend studies, analyze the user research results and to create future control center usage concepts for the SFP. Visions of the new user interaction tools were of especial interest, as it was seen that they may improve the work processes and support new ways of working. In the first workshop, the following ideas concerning business renewal and related development possibilities were identified for further consideration: –Distributed production: smart mobile interaction tools, –Highly automated production: a centralized remote expert competence center, –Temporary plant with less well educated users: extremely intuitive control with safety ensured, –Novice operators: help from a social network of other operators, intelligent agents and knowledge management, –Quickly changing production plans: co-creation with the customer. In addition, during the first co-design workshops the potential ambient UX goals that based on trend analysis and current understanding of the operator work in the participating company were developed. 4.2. Results of the exploration and planning The initial user research focused on defining the building blocks of the user experiences in the nominated process control work environment. The contextual inquiry focused on work characteristics, as defined in the core-task analysis method [20,33,34]. The analysis was thorough; it included the definition of the work environment by inspecting special occasions, events, feelings and experiences related to it. The current state of UX was also polled with a user experience significance questionnaire, especially created for this case study, which requested the operators to rate 19 user experience goals developed in the co-design workshops. The critical experience interview [32] was developed by adapting the critical decision method by Klein et al. [21]. While the method in general focuses on decisions and elements of decision-making, the critical experience interviews focused on the feelings and experiences and their development during the participants’ work activities. The structure of the critical experience interview mainly followed the structure of the critical decision method, i.e. obtaining an unstructured incident account, constructing an incident timeline, experience moment identification, and experience moment probing. In addition, the operators were asked about the tools used in different phases of the incident, possibilities of new technologies for improving the operators’ work during similar incidents, and information sharing needs and practices relating to the incidents. The rich field research data was further analyzed by identifying operators’ remarks concerning the UX goals and their categorization. Altogether, 216 excerpts were identified and all of them, excluding 7 general product improvement suggestions, were also classified into 19 predefined UX goals. The UX goals were again analyzed and their dynamics, context of emergence and building blocks described. Consequently, as a result of the initial user studies, there was a relatively extensive list of relevant UX goals for the explicit AmI domain; however, the information was not yet focused enough to guide the concept video phase. Therefore, it was essential to narrow the list down into more distinct ambient UX goals. 4.3. Results leading to SFP development Together with the user study findings, the results of the initial workshops were employed in determining the UX goals and for inspiring further ideation for T. Kymäläinen et al. / A creative prototype illustrating the ambient user experience of an intelligent future factory 47 the SFP. Subsequently, it was defined that the one primary, higher level ambient UX goal in the process control work should be “Peace of mind”, which was interpreted as: “The operator knowing what is going on in the production process and how to intervene when needed”. In essence, “Peace of mind” included the following seven broader-spectrum UX goals that guided the final SFP creation: –Sense of control –Trust in human-automation cooperation –Sense of freedom –Ownership of the process –Relatedness to the work community –Meaningfulness of the work –Success and achievement. The ideas gradually begun to form into scenarios that all related to the topic: ‘A remote operator’s day in a future control center in 2025’. The selected usage scenarios introduced a future plant where: –The control center changed into a flexible space, which could be shaped to support different work tasks and responsibilities, such as group or individual work; –The AmI system was able to predict the process disturbances through simulations; and, as a result, many disturbances could be prevented before they took place; –The AmI system enabled proactive and futureoriented process control work; –User interfaces provided visual and situationaware descriptions of current and upcoming process events; –The AmI system enabled collaborative problem solving and continuous development. Consequently, each scenario was written as a UXdriven scene: the starting point was the targeted user experience, and essentially the scene described how the experience was facilitated by design. 5. The science fiction prototype As a storyline, the SFP video “A remote operator’s day in a future control center in 2025” has been built from six scenes that take place during an operator’s work shift. The live action scenes and voice-over explanations deliberately aimed to be provocative in order to stimulate discussion between the participants. In what follows, there is a brief explanation of each Fig. 2. Production monitoring. SFP scene and a link to the YouTube videos (presented in Finnish, the original language, with English subtitles). The scenes introduce: routine production supervision, operator guidance, preparation for a production change, carrying out the production change, incident management, and a concluding shift change. 5.1. Production monitoring ‘Production monitoring’ depicts process control operation settings where an operator monitors the status of the process from their personal workstation (see Fig. 2). The AmI system is presented as a responsive and intelligent partner assisting the operator in predicting and reacting to changes in process status. The goal is to emphasize the user’s sense of freedom and control by allowing them to use interaction methods of choice. These qualities are exemplified through speech-based interactions, which allow the operator to issue commands to retrieve information and enable intelligent agents to monitor the process on behalf of the operator. This frees the operator to focus on other tasks. Link to the scene: https://www.youtube.com/watch? v=wy-3AwfiY-A. 5.2. Guidance ‘Guidance’ focuses on portraying how new interaction techniques enable operators to move freely around the control center and collaborate with one another (see Fig. 3). Their tasks are supported by the intelligent AmI system. In the scene, an experienced operator provides guidance to a novice operator on a specific part of the production process. The guidance is given by using gestural and spoken interaction on a wall-sized display. The use of the system blends in as a natural element of the collaboration. Link to the scene: https://www.youtube.com/watch? v=JRjRJIpwQN8. 48 T. Kymäläinen et al. / A creative prototype illustrating the ambient user experience of an intelligent future factory Fig. 3. Guidance. Fig. 4. Preparation for a production change. 5.3. Preparation for a production change Major production changes are procedures that require careful planning in order to avoid disturbances or unnecessary deviations in production quality. This scene illustrates how the future control center facilitates collaboration between the center and field personnel through improved situational awareness, such as personnel locations and ongoing maintenance operations (see Fig. 4). The personnel are going through a production change plan, which they verify by communicating with the field personnel. The technologies include speech-based interaction, large touchscreen displays, synchronous voice communication and live video with augmented reality and 3D elements. Link to the scene: https://www.youtube.com/watch? v=9rnB96tq_QY. 5.4. Production change ‘Production change’ focuses on operator’s tasks during an ongoing production change (see Fig. 5) and the use of internal social media tools. The main focus is on the role of the AmI system, which, as an active partner, supports the human operator in situations that arise during production change. The system is able to anFig. 5. Production change. Fig. 6. Disturbance management. ticipate potential disturbances, records the operator’s troubleshooting activities, and suggests various solutions that the operator can choose from. Successful operations shared by operators allow the system to make future recommendations. The new technologies demonstrated in the scene include speech commands and intelligent and proactive information collection, prediction and recommendation functionalities. Link to the scene: https://www.youtube.com/watch? v=guMgYnPoI1Y. 5.5. Disturbance management ‘Disturbance management’ illustrates how mobile interaction technologies may be used to access the automation system remotely, outside of the control center, and enable collaboration onand off-site (see Fig. 6). The scene demonstrates how dialogue with the AmI system helps operators manage troubleshooting and disturbance situations. By tracking best practices used in operating the automation system, operators can be connected to available resources and other – even remotely located – operators to enable collaborative analysis and the resolution of problems. The trou- T. Kymäläinen et al. / A creative prototype illustrating the ambient user experience of an intelligent future factory 49 Fig. 7. Shift change. bleshooting actions are automatically stored by the system so that they can be of use should the same situation arise in the future. Link to the scene: https://www.youtube.com/watch? v=8BiNS8dA-xo. 5.6. Shift change The final scene portrays the opportunities for mobile work afforded by new technologies (see Fig. 7). The shift change between operators is facilitated by the automation system, which automatically tracks operators’ locations within the facility. The system ensures that responsibility for the process is seamlessly transferred without a face-to-face handover. The new technologies illustrated in the SFP include the use of mobile devices to monitor and control the process, new sensor technologies that recognize and track people, and the functionality to monitor operators’ activities and facilitate the handover process – without compromising safety or operators’ control over the system. Link to the scene: https://www.youtube.com/watch? v=fz2wyn6Xzgw. 6. Results of the deployment In Johnson’s method, the SFP creation process concludes with an exploration of the implications and revisits the lessons learnt. In this article, the reflection part presents the main findings of the user research and UX goal evaluations, and, as such, it also demonstrates the final step of the SFP framework. In what follows will be the most important findings from both evaluation setups in detail. The findings are based on the written feedback (Web survey) and verbal discussions (interviews) that took place after the participants had seen each of the video scenes. 6.1. Scene 1: Production monitoring This scene illustrates the use of speech commands and smart automation system and its user interface. The AmI system was described as supporting freedom of choice, and, according to the participants, the system presented certainly provided interesting opportunities for interaction. In general, the control center was stated to be pleasant in appearance, and the user interface color scheme well composed. Speech commands were generally seen to be suitable for navigation, opening appropriate views, monitoring of plans and predictions; but not for operations, except in critical situations. Especially in the field work, the speech commands were considered helpful for searching relevant information. The commands were expected to be very simple and customizable for each operator’s preferences. One respondent in the Web questionnaire nevertheless speculated critically: “As speech-based interaction doesn’t work very well between humans, it could be even worse in the communication between humans and machines”. The potential for speech recognition errors also raised some concerns. Still, it was believed that the smart automation system would be able to pick out some important keywords from conversations and provide relevant, context-aware information. In general, the AmI system was considered to make practical work more effortless and efficient, and consequently reduce the probability of human errors in the operation work. The greatest value resulting from the AmI was expected to be achieved in fieldwork. According to the participants, automation is currently not at the level illustrated in the video, and quite a number of the tasks presented are carried out manually. One concern was that, when the AmI offered information about alarms, it only presented the extremes, although there would also be a need for more comprehensive alarm information. In the interviews, some participants stated that they were used to “fixed” alarm notifications, but even currently some preferred to customize them. In the Web survey, some respondents criticized that too much automation might erode the professional skills of the workforce; nevertheless, this could be compensated with simulation training. 6.2. Scene 2: Guidance This scene introduced the gesture control and use of different types of displays. In general, gesture con- 56 T. Kymäläinen et al. / A creative prototype illustrating the ambient user experience of an intelligent future factory [12] G. Graham, A. Greenhill and V. Callaghan, Creative prototyping, Technological Forecasting and Social Change 84 (2014). [13] G. Graham, A. Greenhill, G. Dymski, E. Coles and P. Hennelly, The science fiction prototyping framework: Building behavioural, social and economic impact and community resilience, Future City & Community Resilience Network, 2015. [14] A. Greenfield, Everyware – The Dawning Age of Ubiquitous Computing, New Riders, USA, 2006. [15] M. Hassenzahl, Experience Design – Technology for All the Right Reasons, Morgan and Claypool Publishers, 2010. [16] B.D. Johnson, Science fiction prototypes. Or: How I learned to stop worrying about the future and love science fiction, in: Proceedings of the 5th International Conference on Intelligent Environments, IOS Press, 2009, pp. 3–8. [17] B.D. Johnson, Science Fiction Prototyping: Designing the Future with Science Fiction, Morgan and Claypool Publishers, 2011. [18] E. Kaasinen, V. Roto, J. Hakulinen, J.P. Jokinen, H. Karvonen et al., Defining user experience goals to guide the design of industrial systems, Behaviour & Information Technology 3001 (2015), 976–991. doi:10.1080/0144929X.2015.1035335. [19] H. Kagermann, W. Wahlster and J. Helbig, Securing the future of German manufacturing industry. Recommendations for implementing the strategic initiative INDUSTRIE 4.0, Final report of the Industrie 4.0 Working Group, Forschungsunion, 2013. [20] H. Karvonen, I. Aaltonen, M. Wahlström, L. Salo, P. Savioja and L. Norros, Hidden roles of the train driver: A challenge for metro automation, Interacting with Computers 23 (2011). [21] G. Klein, R. Calderwood and D. MacGregor, Critical decision method for eliciting knowledge, IEEE Transactions on Systems, Man and Cybernetics 19 (1989), 462–472. doi:10.1109/ 21.31053. [22] M. Kuniavsky, Observing the User Experience, a Practitioner’s Guide to User Research, Morgan Kaufmann, 2003, pp. 419–437. [23] T. Kymäläinen, Science Fiction Prototypes as Design Outcome of Research – Reflecting Ecological Research Approach and Experience Design for the Internet of Things, Aalto ARTS Books, Aalto University, 2015. [24] T. Kymäläinen, The design methodology for studying smart but complex do-it-yourself experiences, Journal of Ambient Intelligence and Smart Environments (2015), IOS Press. [25] T. Kymäläinen, Science fiction prototypes as a method for discussing socio-technical issues within emerging technology research and foresight, Athens Journal of Technology & Engineering (ATINER) (2016). [26] T. Kymäläinen, P. Perälä, J. Hakulinen, T. Heimonen, J. James and J. Perä, Evaluating a future remote control environment with an experience-driven science fiction prototype, in: 11th International Conference on Intelligent Environments, IEEE Computer Society, 2015, pp. 81–88. [27] T. Kymäläinen, E. Kaasinen, M. Aikala, J. Hakulinen, T. Heimonen, P. Mannonen, H. Paunonen, J. Ruotsalainen and L. Lehtikunnas, Evaluating future automation work in process plants with an experience-driven science fiction prototype, in: 12th International Conference on Intelligent Environments, IEEE Computer Society, 2016. [28] H. Lasi, P. Fettke, H.G. Kemper, T. Feld and M. Hoffmann, Industry 4.0. Business & Information Systems Engineering 6(4) (2014). [29] L. Lehtikunnas, Puheja eleohjaus prosessinhallinnassa (Speech and gesture interaction in process control work, in Finnish), Diploma thesis, Tampere University of Technology, Tampere, Finland, 2014. [30] C. Linehan, B.J. Kirman, S. Reeves, M.A. Blythe, J.G. Tanenbaum, A. Desjardins and R. Wakkary, Alternate endings: Using fiction to explore design futures, in: CHI’14 Extended Abstracts on Human Factors in Computing Systems, ACM, 2014, pp. 45–48. [31] W. MacDougall, Industrie 4.0: Smart manufacturing for the future, Germany Trade & Invest (2014). [32] P. Mannonen, M. Aikala, H. Koskinen and P. Savioja, Uncovering the user experience with critical experience interviews, in: Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design (OzCHI ’14), ACM, New York, NY, USA, 2014, pp. 452–455. [33] L. Norros, Acting under uncertainty, the core-task analysis in ecological study of work, Espoo, VTT, Finland, 2004. [34] M. Nuutinen and L. Norros, Core task analysis in accident investigation: Analysis of maritime accidents in piloting situations, Cognition, Technology & Work 11 (2007), 129–150. doi:10.1007/s10111-007-0104-x. [35] H. Paunonen, Roles of informating process control systems, Ph.D. Dissertation, Tampere University of Technology, Tampere, Finland, Publication 225, 1997. [36] R. Peldszus, Surprise payload rack: A user scenario of a conceptual novelty intervention system for isolated crews on extended space exploration missions, in: Workshop Proceedings of the 7th International Conference on Intelligent Environments, IOS Press, 2011, pp. 290–300. [37] P. Purdy, From science fiction to science fact: How design can influence the future, User Experience Magazine 13(2) (2013). [38] V. Roto, Y. Lu, H. Nieminen and E. Tutal, Designing for user and brand experience via company-wide experience goals, in: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems,ACM, 2015, pp. 2277–2282. [39] E.B.N. Sanders and U. Dandavate, Design for experiencing: New tools, in: Proceedings of the First International Conference on Design and Emotion, C.J. Overbeeke and P. Hekkert, eds, Delft University of Technology, Delft, The Netherlands, 1999, pp. 87–91. [40] N. Shedroff, Experience Design, New Riders Publishing, Indianapolis, 2001. [41] N. Shedroff and C. Noessel, Make It So: Interaction Design Lessons from Science Fiction, O’Reilly Media Inc., 2012. [42] J. Tanenbaum, Design fictional interactions: Why HCI should care about stories, Interactions 21(5) (2014), 22–23. doi:10. 1145/2648414. [43] K. Tassini, The magician’s assistant, in: Workshop Proceedings of the 7th International Conference on Intelligent Environments, IOS Press, 2011, pp. 267–278. [44] H. Väätäjä, T. Olsson, P. Savioja and V. Roto, UX goals – How to utilize user experience goals in design, Tampere University of Technology, Department of Software Systems Tampere, Finland, 2012. [45] P. Wright and J.J. McCarthy, Technology as Experience,MIT Press, Cambridge, 2007. [46] H.Y. Wu, Imagination workshops: An empirical exploration of SFP for technology-based business innovation, Futures 50 (2013), 44–55. doi:10.1016/j.futures.2013.03.009. T. Kymäläinen et al. / A creative prototype illustrating the ambient user experience of an intelligent future factory 57 [47] H.Y. Wu and V. Callaghan, The spiritual machine, in: Workshop Proceedings of the 7th International Conference on Intelligent Environments, IOS Press, 2011, pp. 155–166. [48] S. Ylirisku and J. Buur, Designing with Video, Focusing the User-Centred Design Process, Springer Science & Business Media, 2007. [49] P. Zheng and V. Callaghan, Diegetic innovation templating, in: Workshop Proceedings of the 12th International Conference on Intelligent Environments, IOS Press, 2016.