A close look into the storytelling process: the procedural nature of interactive digital narratives as learning opportunity

Entertainment Computing 41 (2022) 100466 Available online 26 October 2021 1875-9521/© 2021 Elsevier B.V. All rights reserved. A close look into the storytelling process: The procedural nature of interactive digital narratives as learning opportunity ☆ Maitˆ e Gil a , * , Cristina Sylla b , 1 a Research Centre on Child Studies, University of Minho, Braga, Portugal b Research Centre on Child Studies, ITI/LARSyS, University of Minho, Braga, Portugal ARTICLE INFO Keywords: Interactive Digital Narratives Children Learning Storytelling Interactive process Digital manipulatives ABSTRACT Differently from traditional narratives, which focus on the output, i.e. the oral or written text, interactive digital narratives provide a more holistic view of the storytelling process, considering as integral part of it the system, the user, the process and the output. In this framework, the procedural nature of IDN as a reactive and generative system becomes prominent. Such an approach is particularly interesting when considering educational applications of IDN and how they can support early literacy practices in pre-and primary school children. Here, we take a close look into the procedural nature of IDN, presenting observations and results from two pilot studies carried out with six to seven-years old children, arguing that interactive digital narratives can provide a window into (i) how the children plan their story, (ii) how, along the storytelling process, the children learn the rules and constraints provided by the IDN system, which they appropriate and incorporate in their storytelling to achieve a certain output, (iii) how the children empathize with the story characters, diving into the story world and (iv) how the system provides opportunities for mediating new knowledge in a meaningful way, which was visible e.g. in the way the children immediately appropriated and used the new conveyed vocabulary. 1. Introduction Storytelling may be the most ancient way to communicate ideas, thoughts and events, contributing to and being a key dimension of children’s social, cognitive and affective development [1–4]. According to Bruner [5], we organize our experience in the form of narratives and at the same time narratives operate as a tool for the construction of reality. This is particularly relevant for children as they are in the process of creating their own identity and finding their place in the world [4,6,7]. A fundamental part of this process is to become literate. Research has shown that children’s early exposure to narratives, i.e., hearing or telling stories, plays a fundamental role in the development of children’s early literacy skills [8,9,4,1,10]. Engaging with stories also promotes the development of a “memory framework”, i.e., the ability to remember and effortlessly analyze new stories, anticipating information, facilitating the understanding and retelling of new stories [9], which in turn facilitates the construction of meaning, as well as the creation of new stories [11]. Since the nineties, storytelling has been a topic of investigation across different fields, such as Human-Computer Interaction (HCI), Child-Computer Interaction (CCI), Artificial Intelligence [12] and Games, which have explored the potential of interactivity, multimedia and embodiment to foster engagement, fun, playful learning and creativity [13]. Giannakos et al. [14] identified storytelling as one of the major research themes in CCI for the period 2013–2018, which in connection with collaboration, creativity, tangibles, and interactive surfaces has become one of two motor themes in CCI research [14:8,9]. Another review [15] that analyzed all full papers from the Interaction Design and Children Conference (IDC) from 2003 to 2016, identified ‘collective storytelling’ as a strong concept in CCI. Overall, these investigations have resulted in the development of a variety of storytelling tools for children, targeting the development of various skills, such as language and literacy skills [16], collaboration and social skills [17–19] or creativity [20,21], among others. Storytelling systems have also become popular in public spaces, such as libraries [22] or science [23,24] and museum contexts [25,26]. This well-established body of research has provided valuable contributions, here we focus particularly on the potential of Interactive ☆ This paper has been recommended for acceptance by Christos Gatzidis. * Corresponding author at: Institute of Education, Universidade do Minho, Campus de Gualtar, 4710-057 Braga, Portugal. E-mail addresses: [email protected] (M. Gil), [email protected] (C. Sylla). 1 Address: Institute of Education, Universidade do Minho, Campus de Gualtar, 4710-057 Braga, Portugal. Contents lists available at ScienceDirect Entertainment Computing journal homepage: www.elsevier.com/locate/entcom https://doi.org/10.1016/j.entcom.2021.100466 Received 9 February 2021; Received in revised form 15 September 2021; Accepted 22 October 2021 Entertainment Computing 41 (2022) 100466 2 Digital Narrative (IDN) storytelling systems for literacy development. Our investigation is driven by two central aspects: (i) a great part of research in CCI has primarily focused on technical development and on artefacts-centered evaluations [13,14], which are often developed for single use [12,27]; and on the other side, (ii) most studies on literary studies focus on the product, paying relatively little attention to the interactive storytelling process. In this context, researchers and practitioners, at the crossroad between IDN, narratology and HCI, have described and designed computer-mediated narrative experiences by integrating formal and structural theories [28]. Research in Interactive Digital Narratives has also called out the need for updated theoretical discussion on newer forms of interactive narrative construction [29]. Here, we present an expanded version of the paper which appeared in the ICIDS 2020 proceedings [30]. Our aim is to investigate the potential of the procedural nature of IDN as a learning opportunity. The research question that guides this paper is: To which extent can a close look at the IDN narrative process inform early literacy development? To answer it, we analyse the process that takes place between the users and an IDN system while creating a narrative (i.e. the output). Outgoing from our results, we argue that this is a major aspect to be explored in the design of IDN educational applications. Concerning the scope of this paper, it is relevant to mention that our goal is not to diminish the value of the well-established approaches that focus on the pedagogical possibilities and contributions of the narrative itself and its structure, aligned with the classical narratology theory and therefore mainly related to narrative in traditional media. Rather, we argue that investigations about the specific characteristics of IDN and its relation to education are a fruitful area for new insights, thus the main contribution of this paper is to better understand the pedagogical potential of IDN. We begin by discussing the key points in which a specific theory of IDN differs from the classical narratology theory and narratives in traditional media. We then characterize the procedural nature of IDN as an element which promotes young children’s linguistic competences during storytelling. Finally, we present an IDN authoring tool, and discuss the results of two pilot studies carried out with it to illustrate how the procedural nature of IDN plays a relevant role in the creation of stories by children supported by the authoring tool. 2. Narratives in traditional media versus Interactive Digital Narratives According to Montford [31], an Interactive Fiction (or IDN) artefact/ system is more complex than the narrative itself, as it contains the system, the process and the output (i.e. the narrative). Differently from classical narratology models that focus on the output, a fundamental part of an IDN output is shaped and results from the interactive process that takes place between the user and the system. In view of this, an IDN framework needs to consider both the system (i.e. software/hardware) and how the user interacts with it (i.e. the process), which together produce the output [32]. The procedural nature of IDN results from a “reactive and generative system” [32:97], composed of “system, process, and product”. Although we do not explore the notion of ‘procedural rhetoric’ proposed by Bogost [33], his definition of procedures or processes as a set of constraints based on rule-based models (algorithms) that generate behaviours and “create possibility spaces, which can be explored through play” [33:122] is useful here to highlight the role of the system and the possibilities that it provides for narrative creation, in the IDN process. More specifically, an IDN system comprises the hardware, the digital artefact, which includes the executable programming code and the virtual assets. The system is a container of potential narratives, which are structured through the ‘protostory, the narrative design and the narrative vectors’ [32]. The protostory entails potential narratives; the narrative design refers to the structure of the protostory that enables a flexible presentation of a narrative; and the narrative vectors are the substructures that provide a specific direction to the story [32]. The second defining element of an IDN system is the interactive process that takes place between the system and the user/s, and which is defined by the opportunities the system provides and shaped by the user’s actions. The interactions performed by the user with the system are fundamental to the output and represent a key distinction between IDN and narrative in traditional media. The third defining element of an IDN system, the product, is classified as an instantiated product, since the participatory process and the procedural nature of IDN make different narrative outputs possible [32]. In sum, an IDN can be defined as “an expressive narrative form in digital media implemented as a computational system containing potential narratives and experienced through a participatory process that results in products representing instantiated narratives” [32:98]. Building on this model, we highlight the central role of the user as a key element in generating an IDN output. In this sense, a full analysis of IDN needs to include the three discussed elements: system, process and product, as well as the user, who is a cross-cutting element in IDN [30]. Thus, the shift from an output-centred view to a procedural account is the key difference between the applications of traditional narrative theories to IDN [34,35, among others] and a specific theory of IDN. Based on this shift, we argue that this procedural nature represents a new learning opportunity, from which educational applications of IDN can benefit. 3. Key contributions of the procedural nature of IDN to early literacy The shift from an output-centred view to a procedural account discussed in the previous section moves the core contribution of IDN to early literacy from the narrative itself to the interactive creation process. In this sense, we argue that the design of educational applications to foster early literacy can benefit from considering the aspects in which IDN differs from other narrative forms. While in more traditional narrative forms only the output (in the form of an oral or written text) is available for analysis - and it is only possible to speculate about the author’s thoughts and her writing process - with IDN, it is possible to record and analyse the output, as well as to analyse two other elements: the system and the interaction process that takes place between the users and the system. So, a specific IDN framework is composed by system-interaction-output (which can be fixed) [32], as opposed to the established theoretical narrative framework used in traditional media, which is composed by an authorwriting-output (fixed). In this context, an IDN analysis would reveal not only aspects of the narrative structure but also how particular interactions guide the construction of new learnings and shape the output. This becomes possible since the procedural nature of IDN allows to look at the system’s configurations and observe the participant’s exploratory process of creating a narrative. Within this framework, the procedural nature of IDN as a reactive and generative system expands the possibilities of IDN educational applications, representing a convergence point between the IDN theoretical framework and the educational context that might lead to new insights. One relevant contribution, for instance, is that the obligatory interactive process when producing the output of a specific IDN, as well as the possibility to record and analyse this process makes the activity of planning and creating stories more concrete and transparent, just as the activity of teaching how to plan and how to create stories. The participatory process concretely guided by the system’s configurations that takes place during the storytelling supported by IDN systems brings up two crucial aspects of young children’s language development. The first one is the embodied cognition, which is supported by the system in the pedagogical uses of IDN. The second is the interaction, promoted by the process through which a narrative is created [30]. Considering the embodied cognition, it is known that children refer M. Gil and C. Sylla Entertainment Computing 41 (2022) 100466 3 back to their own experience of reality, as well as to what they know from their repertoire of narratives to understand and create narratives. As narrators, children place themselves at a distance from the momentary situation, imagine something, remember, and participate in the portrayed reality. This is a demanding process, and research has called attention to children’s need for structures that scaffold them in these tasks, which are intrinsically related to the embodied cognition theory [36]. The claim that system configurations in line with the user’s bodily and social interactions foster the development of storytelling skills is supported by the understanding that children assimilate and produce narratives in connection with their own embodied experience and with other narratives. In this context, an IDN system can provide scaffolding structures to young children by: (i) structuring the space of potential narratives through the protostory; (ii) guiding flexible presentations of a narrative through the narrative design; and (iii) determining possible directions to the story through the narrative vectors [30]. Considering the obligatory interaction (whether with the system or with the system and other individuals at the same time) to produce the output, IDN educational applications enhance the mediative nature of learning advocated by socio-cultural theories. Considering that the development of thought is determined by language, that is, by the linguistic instruments of thought and by the socio-cultural experience of the child [37], it is possible to argue that learning occurs through a mediative process, either by means of signs or by means of human mediation. In consonance with this approach, we argue that the IDN interactive process accentuates the role of mediation as an important semiotic mechanism of learning. Here, it is particularly relevant to consider the role of the user not just in the creation process, but also in her interaction with the system. The user is, therefore, the central element that articulates the IDN as a meaningful whole and potentiates IDN as a learning tool [30]. This is a relevant aspect to be considered in IDN educational applications, supported by recent research that has identified the need of fostering interaction with and through the artefact as a more knowledgeable other [38]. In order to highlight this process, in the following section we present and discuss two pilot studies carried out with an IDN digital manipulative. We start by presenting the tool, and then describe the studies. 4. The IDN authoring tool The digital manipulative is a storytelling authoring tool directed to young children that uses physical blocks to promote the creation of collaborative intercultural narratives. Based on the assumption that it is of major importance to develop child centered learning materials [39], its development followed an iterative and participatory design methodology, involving children and teachers along its development. The blocks communicate with a computer or tablet via Bluetooth, and with each other through magnets embedded on the sides of each block. The physicality of the input devices (the blocks) allows their sharing among the users, promoting collaboration and generating a form of democratic interactions, giving children equal power to manipulate and interact with the device [40], this way encouraging social interaction and collaboration. At the same time, the physical blocks contribute to the children’s awareness, control and accessibility to different kinds of actions [41]. Each physical block embodies a story element having the respective visual representation on the upper face (see Fig. 1). Connecting the blocks to each other triggers its digital representation on a device’s screen. Presently, the blocks comprise eight cultural sets, i. e., India, Brazil, China, Portugal, Germany, Angola, Turkey and Cape Verde. Each set aims at representing elements from the respective culture and is composed of a landscape, a girl and a boy protagonist, an animal, an antagonist, a musical instrument and a magical object. There are also five blocks that simulate different atmospheric conditions (rain, snow, wind, thunder, night), as well as music, sounds of the characters, the instruments and ambient sounds. Each element has specific animations that display different actions. The visual narratives unfold according to the combination of blocks that the users connect to each other. Children can change the scene, mix and remix the elements, try out different plots, shift direction and start all over again. As the system provides visual and auditory feedback in the form of sounds from the characters and music from the instruments, children can imagine and create their own spoken narratives. According to the IDN narrative framework proposed by Koenitz [32], it is possible to argue that the protostory of this authoring tool is defined by the space of possible stories embedded in the narrative elements, as well as by the constraints provided by the programming code and the interactive interface. Each narrative element behaves according to a behaviour tree, a method often used in the field of computer games to model character behaviour, reactive decision-making and control of virtual characters [42]. The behaviour trees define the actions of all narrative elements; i.e. each element interacts with the environment according to a set of predefined rules that define its behaviour and the relations to the other elements. As the behaviour triggered for each element depends on the other active elements and the properties of those elements, there is a certain degree of unpredictability in the outcome of a given situation. For instance, if there are more than two “good elements”, i.e. two protagonists and one animal, and just one antagonist on the scene, the antagonist will become afraid and won’t attack, however this changes if another antagonist is added to the scene. This way the narrative design opens up a space for experimentation and agency. Similar to other IDN systems, the storytelling process with this tool is intrinsically interactive and it can engage one or several participants. The tool offers a space for exploring storytelling within a multimodal (tactile, verbal, visual and auditory) embodied (through manipulation) collaborative environment. Users may also record the output of the Fig. 1. The blocks (left), the interface prepared for use at school (right). M. Gil and C. Sylla Entertainment Computing 41 (2022) 100466 4 interactive digital narrative walkthrough, i.e., the instantiated narrative. In this sense, another relevant contribution of this model is that the instantiated narrative can be analysed considering the specificity of the digital manipulative’s narrative design, which is defined by certain narrative vectors, instead of applying “the output-centred view of legacy theoretical frameworks’’ [32:97]. 4.1. Pilot study: A close look into the procedural nature of IDN In this section, we present two pilot studies carried out with the authoring tool. The study aimed at gathering observational data and information about children’s exploratory process of creating a narrative supported by the IDN system in order to analyse not only the output (that is, the narrative itself), but also how particular interactions performed during the storytelling process may shape the output. 4.1.1. Methods We conducted two pilot studies with a total of 22 children. All children were first graders, aged between six and seven years old. The first pilot took place within a home context in Brazil. The second pilot took place in a primary school in Portugal. All participant children use tablets and different educational digital media regularly. The studies were presented to the children’s parents and their teacher respectively. All parents gave their informed consent and signed a statement of agreement. The children were informed that they were free to withdraw from the activity at any time. We have translated children’s verbalisations from Portuguese into English, thereby we have tried to be as faithful as possible to the original language. In the following we discuss the intervention. Given the detailed analysis of each narrative process, we present excerpts from just four children (two pairs) that are representative of children’s narrative construction during both pilot studies. That is, the patterns of interaction highlighted here were identified across the groups. We selected one pair from each pilot in order to have a more diverse sample, considering the constraints of length of this text. We will refer to the children as A. (girl) and P. (boy) - from the home context in Brazil, and L. (boy) and M. (girl) - from the school context in Portugal. 4.1.2. Procedure The children interacted in pairs with the digital manipulative. One researcher conducted the studies, the other stood in the background collecting the data. The facilitator researcher started by giving a brief explanation of the functioning of the IDN system. After that, the children interacted and explored the tool for around 15 min. Following the exploratory phase, the researcher invited them to tell a story using the tool (some pairs wanted to tell more than one story). 4.1.3. Data collection and methodology The pilot studies followed a qualitative, explorative and inductive methodology. Thus, the children were able to collaborate with each other, without interruptions from the researchers, and learn how to use the system. This method was selected in consonance with the intent of describing and analyzing the process of narrative creation in this context of application. Since our goal has a descriptive nature, there was no preliminary hypothesis to be validated or rebutted. The data was collected through observations and field notes. All the interactions were audio–video recorded using a video camera on a tripod with a fixed focus and zoom. The video camera was placed behind the children focusing on the manipulation of the blocks. The children were informed about and shown the camera and did not pay any further attention to it. In order to keep a fresh record from the observations, the researchers noted impressions of the intervention directly after the sessions, this way contributing to a more reliable analysis of the data [43]. The video recordings were later transcribed and analysed (see Fig. 2). 4.2. Zooming into the storytelling process In order to analyse how particular interactions with the digital manipulative during the process of the storytelling may shape the narrative output, we (i) begin by presenting the written version of two narratives recorded during the pilot studies, (ii) select passages from the texts to be used as a window to look into the interactive process, and (iii) describe the interactions that may have motivated, shaped or influenced such narratives’ passages. The interactions are illustrated through vignettes selected for their relevance as representative of children’s experience with the digital manipulative in the process of creating their narratives. 4.2.1. Narrative 1 “Once upon a time, there was a city, there were two boys…no, one boy and a girl…then it appeared…a squirrel! They were friends, but one day the squirrel disappeared. They were laughing and laughing and, suddenly, it appeared a gnome; the gnome that had captured the squirrel! And then they found a weapon and they attacked the gnome and then they won, and the squirrel came back! Then a weird person appeared, someone that was not from our country. They were scared, but suddenly they attacked him, and he was transformed into a boy. And they lived happily ever after! The end!” Fig. 2. Pilot studies: Intervention session with the children (left: home context, right: at school). M. Gil and C. Sylla Entertainment Computing 41 (2022) 100466 5 A range of different analyses can be performed of this small narrative created by the two six-years-old children. If we look exclusively at the output, it is possible to analyse the sequence of narrated events, the connectors used by the authors, the vocabulary chosen, and so on. However, we argue that the change of the focus of the analysis from the product itself to the process, made possible by the procedural nature of IDN, reveals some competences and learnings that are not perceived through a traditional output-centred approach. The vignettes presented below zoom into the storytelling process, reproducing situations related to key passages of the narrative and, to a certain extent, highlight and convey the contributions of the interactive process and of the system to the instantiated narratives, disclosing the procedural nature of IDN as a learning opportunity. 4.2.2. Narrative 1Vignette 1: Once upon a time, there was a city… The researcher invites the children to tell a story together, using the digital manipulative. P. starts by selecting some blocks, while saying “I know, I know…”. A. says [and makes a stop gesture with her hand] “calm down, let’s first select which ones we are going to use…” [she starts selecting some blocks] “we are going to use this one at the end, this one too…”. P. connects the German landscape [which was one of the blocks selected by A.] and starts “once upon a time, there was a city…”. A. interrupts him, disconnects the block and says “no, no, no, disconnect this…wait…” and continues separating the blocks. Relevance and relation to the output: This vignette illustrates the strategy adopted by A. to plan the storytelling. She looks at all the available blocks to think about which ones will be used during the narrative. So, when P. connects the German landscape and starts narrating “once upon a time, there was a city…”, A. asks him to stop and wait, indicating that she wanted to plan the whole text before starting the storytelling (for this she wanted to select some specific blocks). This is an example of how the activity of planning and creating stories can become more visible and concrete with the support of an IDN system. This is relevant, since (i) the process of creating a narrative can be a challenging and abstract task for young children, and (ii) this understanding is only possible in an analysis that goes beyond the narrative itself. 4.2.3. Narrative 1Vignette 2: It appeared a gnome P. connects the German antagonist and laughs. A. asks “what is this?”. The researcher answers “This is a gnome. It is a very common character in German stories”. A. “Oh, he is afraid!” [referring to the German boy in the scene] “Oops!” [looking for something to ‘help’ the protagonists]. The researcher says “try this one” [giving them the instrument]. A. connects the instrument, and the German boy starts playing the accordion. A. laughs and sighs relieved, saying “that’s funny”, and after some seconds she says “go away evil gnome!” [disconnecting the German antagonist], (see Fig. 3). Relevance and relation to the output: This vignette illustrates (i) the process of knowing the digital manipulative and learning its rules (discovering that the protagonists are afraid of the antagonist, that the gnome is an antagonist, and that they can use the instrument to avoid attacks), (ii) children’s empathy/solidarity with the protagonists (concern with their reactions), (iii) children’s immersion into the story world (they speak to the characters), (iv) an opportunity to learn about unknown elements (the gnome). All these aspects can be related to the instantiated narrative, specifically when they introduce the “gnome”. The fact that this character is introduced as the one who “captured the squirrel” can be related to the previous discovery that he was an antagonist, as well as the fact that the other characters fight with the gnome to help the squirrel (with which children are sympathetic). So, the child defines the introduction of new elements scaffold by the possible directions of the story, which is determined by the narrative vectors within the protostory. The appropriate use of the vocabulary they have learnt during the interaction is another relevant aspect to highlight, since this linguistic choice in the instantiated narrative is not random, but rather it is shaped by the interactive process. 4.2.4. Narrative 1Vignette 3: Someone that was not from our country A. continues the storytelling “they were friends, but one day the squirrel disappeared and…”. P. connects the antagonist and completes A.’s sentence “and a gnome appeared!”. A. disconnects the antagonist and complains “No! Not this one! Wait” [talking to P.], then she continues, now talking to the characters on the screen “Calm down, Maria and Jo˜ ao!” [She looks at P., smiles and says “I gave names to them”], (see Fig. 4). Relevance and relation to the output: This vignette is interesting, because it helps to understand the use of the pronoun ‘our’ in the final part of the narrative. Although the text has a third person narrator, in this passage, A. includes themselves in the narrative by saying ‘our country’, creating a connection between them and the story world. The interaction described in this vignette happened some minutes before the creation of the final narrative and illustrates children’s involvement with the characters along the process. A. gives names to the characters and asks them to ‘calm down’ after seeing that they were scared when the antagonist was connected. So, the fact that they have included Fig. 3. Narrative 1 - Vignette 2: Elements on scene during vignette 2. M. Gil and C. Sylla Entertainment Computing 41 (2022) 100466 6 themselves into the story world by using the expression ‘our country’ may be related to their involvement and sympathy with the characters. Therefore, a use of the language that could even be evaluated as an error according to an output-centred approach, when analyzed from a procedural perspective, acquires another meaning. It is also interesting that the children refer to the notion of “country”, which indicates that (i) they have identified the German scenario as representing a certain place, (ii) and this place is different from the ones they know. Again, this aspect provides opportunities for learning. 4.2.5. Narrative 1Vignette 4: Then they found a weapon A. is telling the story “the gnome that had captured the squirrel! And then they found a weapon…” [P. connects the German object and both look at the screen to see what happens]. The researcher asks “do you know what this is? It is a ‘cone’ that children in Germany receive on their first day at school; it is a cone full of candies”. A. and P. listen to the information and continue looking at the animation on the screen. [Some minutes later] P. is actively telling another story and A. is helping him. He is using the German Set. The following blocks are connected: scenario, boy, girl and antagonist. P. connects the object and says “she has got the weapon”. The researcher asks “do you remember what this object is?”. A. quickly answers, “this is a thing full of candies that they receive on their first day at school”. P. complements “It’s a cone!”. The researcher asks “and are they using it to defend themselves?”. Both answer “Yes!” and go back to the story (see Fig. 5). Relevance and relation to the output: This vignette illustrates how the interactive process creates a meditative learning environment, accentuating the role of mediation as an important semiotic mechanism of learning. Thus, it provided an opportunity for a more knowledgeable person (here the researcher) to naturally introduce new information (about the German object). The interaction during the process of creating the story opened up an opportunity for new learnings. Again, the possibility to look at this kind of process represents a relevant aspect to be considered in IDN educational applications, since it helps teachers and educators to understand and to access the students’ learning process and at the same time it provides opportunities to introduce new learnings naturally. 4.2.6. Narrative 2 “Once upon a time, there was a dragon that was walking and then it started flying and then a boy came and the boy got scared, and then the thunderstorm started and the boy fell to the floor, he died and disappeared. What a strange thing! Then the boy came back and he started to turn into a skeleton and then the fan appeared, the night came and it got dark and they slept.” 4.2.7. Narrative 2 - Vignette 1: Once upon a time, there was a dragon M. starts choosing the blocks they will use in their storytelling, after a while she says: “I’ve already put mines here” (referring to the elements she has chosen for her story). L. starts narrating. M. connects the blocks Fig. 4. Narrative 1 - Vignette 3: Elements on scene during vignette 3. Fig. 5. Narrative 1 - Vignette 4: Blocks connected and elements on scene during vignette 4. M. Gil and C. Sylla Entertainment Computing 41 (2022) 100466 7 as L. mentions them. M. puts the elements on stage, L. watches the interactions on screen and narrates them, using them as a scaffold to construct the narrative. At some moments during their story telling, they also exchange ideas about how to proceed: e.g., L. asks: “Shall we use rain, storm, snow or the night?”. Then they start planning another narrative: M. says:” Shall we change the landscape?”; L. “Yes!… Istanbul?”; M. You have another one here [pointing to a landscape block]; L. “There… there are the other landscapes” [pointing to the other landscape blocks]. Relevance and relation to the output: This vignette illustrates the strategy adopted by L. and M. to construct their storytelling. It is interesting to highlight that it was a different strategy than the one adopted by A. and P. in the previous narrative, however the originally complex and abstract process of creating a story becomes more visible and concrete with the support of an IDN system. Thus, the output is shaped by this strategy, in which the narrative process is supported by the narrative design. 4.2.8. Narrative 2 - Vignette 2: He started to turn into a skeleton L. is narrating, he says “Then the boy came back and he started to turn into a skeleton”. M. interrupts him: “But we don’t have a skeleton block!”; L. says “I know, look” [pointing to the screen - when the characters are struck by lightning, their silhouette turns black and their skeleton appears] (see Fig. 6). Then, they continue the narration. Relevance and relation to the output: This vignette illustrates children’s discovery of one of the system’s rules. L. included the skeleton in his story motivated by the characteristics of an element (the thunderstorm) within the protostory, a propriety of the system that he already knew. When M. was surprised by his narration, he asked her to pay attention to the interactions in order to understand his choice of words. On the one hand, the rules of the system support the storytelling, on the other, the way children use the story elements is a trace of the learnings developed during the interaction with the system. 4.2.9. Narrative 2 - Vignette 3: The fan appeared M. and L. start exploring the digital manipulative. L. takes the chinese object block (a fan) and says “I’ll connect the fan!”; the researcher asks “Do you like this object?”; and L. answers “Yes! I’ve already made many paper fans with my mom!”. Relevance and relation to the output: This vignette describes the moment when L. related a story element with his personal experience. During the free interaction with the system, he used the fan more than once and, finally, during the narrative creation, he chose this object again. So, although the presence of the fan seems to be out of context in the narrative (why would the character need a fan during a thunderstorm?), it is possible to understand the reason why L. included it in the story when we look at the whole process. The narratives and vignettes illustrate how a more comprehensive analysis of the system, the process and the output represents an opportunity to intensify the dialogue between the theoretical model of IDN and early literacy practices. We argue that a close look into the storytelling process as presented here opens a window into the student’s learning process, supporting researchers and educators to guide and to better understand the development of the students’ language and narrative skills. 5. Conclusion The above presented vignettes allow us to zoom into the storytelling process mediated by the system and carried out by the children. Differently from a narrative analysis focused on the output, the articulation of system, user, process and output provides a window into children’s thinking, i.e. in the way they construct their narratives, discovering and appropriating the rules and constraints of the system and incorporating new acquired knowledge. Together the vignettes reveal: (i) how the children planned their story, an abstract process, which here was facilitated by the physicality of the narrative elements (in the form of blocks) as well as by the immediate visual feedback provided by the system. Before starting their story or even during the storytelling, the children looked at the available story elements, sorted, divided, grouped and decided which ones they would use in their narrative (embodied cognition supported by the system). While doing this, children reflect about the narrative construction, i.e. the structure and the content of their stories, therefore developing meta-narrative awareness [9]; (ii) how along the storytelling process, the children learned the rules and constraints provided by the IDN system, which they appropriately incorporated in their storytelling to achieve a certain output (interaction, promoted by the process); (iii) how the children empathized with the story characters, personalizing the characters, giving them names, and talking to them as if they were part of the same world. This also provides insights into the process of diving into the story world and stepping out (when taking distance and planning the story) strongly contributing to intrinsic learning [44]; (iv) how the system provided opportunities for mediating new knowledge in a meaningful way, which was visible in the way the children immediately appropriated and used the new vocabulary. In sum, our investigation showed how the procedural nature of the IDN tool enabled a collaborative, participatory process, which was guided by the system’s configurations. The IDN system scaffolded young children’s storytelling by structuring the space of potential narratives through the protostory, guiding a flexible narrative through the narrative design, and determining possible directions to the story through the narrative vectors. As discussed above, this process brings to light two crucial aspects of young children’s language development: embodied cognition (supported by the system) and interaction (promoted by the process). In this sense, a myriad of pedagogical applications can be explored. For instance, IDN systems could be incorporated into formal education settings through early literacy projects in which the process of narrative Fig. 6. Narrative 2 - Vignettes 2, 3: The elements hit by the thunderstorm (left); the fan appeared (right). M. Gil and C. Sylla Entertainment Computing 41 (2022) 100466 8 creation is more important than the narrative itself, so the teacher could use the system’s properties to support students and to foster interaction among them. IDN can also be used as a playful activity to introduce new content (e.g., intercultural awareness) and new vocabulary. IDN systems may be used to support dramatic play, which is one of the primary ways through which children engage with the world. These are only some possibilities that need to be investigated and detailed in future work. Furthermore, we are aware that the insights presented in this paper need to undergo an in-depth analysis, which requires long-term studies with a larger group of participants (such studies are underway). Finally, the results of the pilot studies presented here allow us to conclude that the system’s configurations aligned with the user’s bodily and social interactions establish the procedural nature of IDN as a learning opportunity, fostering the development of storytelling competences. Therefore, the shift from the product to the process represents a fertile ground to the dialogue between the IDN framework and the educational context. Such an approach is particularly interesting when considering educational applications of IDN and how they can support early literacy practices in pre-and primary school children. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgements This work has been financed by national funds through the Portuguese Foundation for Science and Technology (FCT) - and by the European Regional Development Fund (ERDF) through the Competitiveness and Internationalization Operational Program under the reference POCI/01/0145/FEDER/032580. References [1] K. Speaker, D. Taylor, R. Kamen, Storytelling: Enhancing language Acquisition in Young Children, Education 125 (1) (2004) 3–14. [2] J.S. Bruner, H. Haste, Making sense: the child’s construction of the world, Rutledge, 1988. [3] S. Eagle, Learning in the early years: Social interactions around picture books, puzzles and digital technologies, Comput. Educ. 59 (1) (2012) 38–49. [4] V.G.A. Paley, Child’s Work: The Importance of Fantasy Play, Chicago University Press, Chicago, 2004. [5] J. Bruner, The Narrative Construction of Reality, Critical Inquiry 18 (1) (1991) 1–21. [6] J.S. Bruner, The Culture of Education, Harvard University Press, Cambridge, MA, 1996. [7] J. Zipes, Creative storytelling, Routledge, NY, 1995. [8] F. Collins, The Use of Traditional Storytelling in Education to the Learning of Early Literacy Skills, Early Child Devel. Care 152 (1) (1999) 77–108. [9] S. May, Story in its rightful place, in: J. Miller (Ed.), Eccentric propositions: essays on literature and the curriculum, Rutledge and Kegan Paul, London, 1984. [10] K. Speaker, The art of storytelling: A collegiate connection to professional development schools, Education 121 (1) (2000). [11] L.M. Morrow, Literacy development in the early years: Helping children read and write, 5th ed., Allyn and Bacon, Boston, 2005. [12] F. Garzotto, Interactive storytelling for children: a survey, Int. J. Arts Technol. 7 (1) (2014) 5–16. [13] F. Garzotto, P. Paolini, A. Sabiescu, Interactive Storytelling for Children, in: Proceedings of the Conference on Interaction Design and Children New York, NY, ACM Press, 2010, pp. 356–360. [14] M. Giannakos, Z. Papamitsiou, P. Markopoulos, J. Read, J.P. Hourcade, Mapping child-computer interaction research through co-word analysis, Int. J. ChildComput. Interact. (2020) 23–24, https://doi.org/10.1016/j.ijcci.2020.100165. [15] W. Barendregt, O. Torgersson, E. Eriksson, P. B¨ orjesson, Intermediate-Level Knowledge in Child-Computer Interaction: A Call for Action, in: Proceedings of the 2017 Conference on Interaction Design and Children (IDC ’17), Association for Computing Machinery, New York, NY, USA, 2017, pp. 7–16, https://doi.org/ 10.1145/3078072.3079719. [16] C. Sylla, I. Pereira, C. Coutinho, P. Branco, Digital Manipulatives as Scaffolds for Preschoolers’ Language Development, IEEE Trans. Emerg. Top. Comput. (2016), https://doi.org/10.1109/TETC.2015.2500099. [17] C. Sylla, C. Coutinho, P. Branco, A digital manipulative for embodied “stagenarrative” creation, Entertain. Comput. 5 (4) (2014) 495–507, https://doi.org/ 10.1016/j.entcom.2014.08.011. [18] J.A. Fails, A. Druin, M.L. Guha, Mobile collaboration: collaboratively reading and creating children’s stories on mobile devices, in: Proceedings of the 9th International Conference on Interaction Design and Children (IDC ’10), Association for Computing Machinery, New York, NY, USA, 2010, pp. 20–29, https://doi.org/ 10.1145/1810543.1810547. [19] T. G¨ ottel, Reviewing children’s collaboration practices in storytelling environments, in: Proceedings of the 10th International Conference on Interaction Design and Children (IDC ’11), Association for Computing Machinery, New York, 2011, pp. 153–156, https://doi.org/10.1145/1999030.1999049. [20] J. Cassell, K. Ryokai, Making space for voice: technologies to support children’s fantasy and storytelling, Person. Ubiquit. Comput. 5 (3) (2001) 169–190. [21] A. Catala, M. Theune, C. Sylla, P. Ribeiro, Bringing Together Interactive Digital Storytelling with Tangible Interaction: Challenges and Opportunities, in: N. Nunes, I. Oakley, V. Nisi (Eds.), Interactive Storytelling, ICIDS 2017, Lecture Notes in Computer Science, vol. 10690, Springer, 2017, https://doi.org/10.1007/978-3319-71027-3_51. [22] E. Eriksson, A. Lykke-Olesen, StorySurfer: a playful book browsing installation for children’s libraries, in: Proceedings of the 6th international conference on Interaction design and children (IDC ’07), Association for Computing Machinery, New York, NY, USA, 2007, pp. 57–64, https://doi.org/10.1145/ 1297277.1297289. [23] M.S. Horn, A. Banerjee, D. Bar-El, I.H. Wallace, Engaging families around museum exhibits: comparing tangible and multi-touch interfaces, in: Proceedings of the Interaction Design and Children Conference (IDC ’20), Association for Computing Machinery, New York, 2020, pp. 556–566, https://doi.org/10.1145/ 3392063.3394443. [24] D. Long, T. McKlin, A. Weisling, W. Martin, H. Guthrie, B. Magerko, Trajectories of Physical Engagement and Expression in a Co-Creative Museum Installation, in: Proceedings of the 2019 on Creativity and Cognition (C&C ’19), Association for Computing Machinery, New York, 2019, pp. 246–257, https://doi.org/10.1145/ 3325480.3325505. [25] L. Pujol, A. Katifori, M. Vayanou, M. Rossou, M. Karvounis, M. Kyriakidi, S. Eleftheratou, Y. Ioannidis, From personalization to adaptivity – Creating immersive visits through interactive digital storytelling at the Acropolis Museum. Workshop Proceedings of the 9th International Conference on Intelligent Environments, IOS Press, 2013. [26] T. Sezen, I. Iurgel, N. Fisch¨ oder, R. Bakker, K. Van Turnhout, D. Sezen, A Top-Down Narrative Design Approach for Networked Cultural Institutions, in: Proceedings of the 11th International Conference on Interactive Digital Storytelling, ICIDS 2018, Dublin, Ireland, 2018. https://doi.org/10.1007/978-3-030-04028-4_73. [27] D. Harley, J. Ho Chu, J. Kwan, A. Mazalek, Towards a Framework for Tangible Narratives, in: Proceedings of the TEI ’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’16), Association for Computing Machinery, New York, NY, USA, 2016, pp. 62–69. https://doi.org/1 0.1145/2839462.2839471. [28] G. Ferri, Narrative Structures in IDN Authoring and Analysis, in: H. Koenitz, G. Ferri, M. Haahr, D. Sezen, T. Sezen (Eds.), Interactive Digital Narrative: History, Theory and Practice, Routledge, New York, 2015. [29] D. Echeverri, H. Wei, Letters to Jos´ e: A Design Case for Building Tangible Interactive Narratives, in: A.G. Bosser, D.E. Millard, C. Hargood (Eds.), Interactive Storytelling - ICIDS 2020, Lecture Notes in Computer Science, vol. 12497. Springer, 2020. https://doi.org/10.1007/978-3-030-62516-0_2. [30] C. Sylla, M. Gil, The Procedural Nature of Interactive Digital Narratives and Early Literacy, in: A.G. Bosser, D.E. Millard, C. Hargood (Eds.), Interactive Storytelling. ICIDS 2020. Lecture Notes in Computer Science, vol. 12497, Springer, Cham, 2020, https://doi.org/10.1007/978-3-030-62516-0_23. [31] N. Montfort, Toward a Theory of Interactive Fiction, 2003. http://nickm.com/if/to ward.html (accessed 04 February 2021). [32] H. Koenitz, Towards a Specific Theory of Interactive Digital Narrative, in: Koenitz (Ed.), Interactive Digital Narrative, Routledge, NY, 2015. [33] I. Bogost, The Rhetoric of Video Games, in: Salen, et al., (Eds.), The Ecology of Games: Connecting Youth, Games, and Learning. MacArthur Foundation Series on Digital Media and Learning, The MIT Press, Cambridge, MA, 2008. http://dx.doi.1 0.1162/dmal.9780262693646.117. [34] E. Aarseth, A narrative theory of games, in: FDG ‘12 Proceedings of the International Conference on the Foundations of Digital Games, ACM, New York, NY, 2012, pp. 129–133. https://doi.org/10.1145/2282338.2282365. [35] B. Laurel, Computers as Theater, 2nd ed., Addison-Wesley, Reading, 1993. [36] G. Lakoff, M. Johnson, Philosophy in the flesh: the embodied mind and its challenge to western thought, Basic Books, New York, 1999. [37] L.S. Vygotsky, El desarrollo de los procesos psicol´ ogicos superiores [The Development of Higher Psychological Processes], Crítica, Barcelona, 1979. [38] A. Fadeev, Vygotsky’s theory of mediation in digital learning environment: Actuality and practice, Punctum 5 (1) (2019) 24–44, https://doi.org/10.18680/ hss.2019.0004. [39] L. Plowman, J. McPake, C. Stephen, Extending opportunities for learning: the role of digital media in early education, in: S. Suggate, E. Reese (Eds.), Contemporary Debates in Child Development and Education, Rutledge, London, 2012. [40] C. Sylla, E. Brooks, L. Tümmler, Blocks as Symbolic Tools for Children’s Playful Collaboration, in: A. Brooks, E. Brooks, N. Vidakis (Eds.), Interactivity, Game Creation, Design, Learning, and Innovation. ArtsIT 2017, DLI 2017, Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications M. Gil and C. Sylla Entertainment Computing 41 (2022) 100466 9 Engineering, vol. 229, Springer, 2018, https://doi.org/10.1007/978-3-319-769080_40. [41] S. Wright, Graphic-Narrative Play: Young Children’s Authoring Through Drawing and Telling, Int. J. Educ. Arts 8 (8) (2007) 1–27. [42] M. Kapadia, F. Zund, J. Falk, M. Marti, R.W. Sumner, M. Gross, Evaluating the authoring complexity of interactive narratives with interactive behaviour trees. Proceedings of the 10th International Conference on the Foundations of Digital Games, FDG’15. Pacific Grove, 2015. [43] J. Flanagan, The critical incident technique, Psychol. Bull. 51 (4) (1954) 327–358. [44] E. Ackermann, Perspective-Taking and Object Construction: Two Keys to Learning, in: Y. Kafai, M. Resnick (Eds.), Constructionism in Practice: Designing, Thinking, and Learning in a Digital World, Lawrence Erlbaum Associates, Mahwah, New Jersey, 1996, pp. 25–35. M. Gil and C. Sylla