Multimodal Communication and Peer Interaction during Equation-Solving Sessions with and without Tangible Technologies

This is a self-archived version of an original article. This version may differ from the original in pagination and typographic details. Author(s): Title: Year: Version: Copyright: Rights: Rights url: Please cite the original version: CC BY 4.0 https://creativecommons.org/licenses/by/4.0/ Multimodal Communication and Peer Interaction during Equation-Solving Sessions with and without Tangible Technologies © 2023 by the authors. Licensee MDPI, Basel, Switzerland Published version Lehtonen, Daranee; Joutsenlahti, Jorma; Perkkilä, Päivi Lehtonen, D., Joutsenlahti, J., & Perkkilä, P. (2023). Multimodal Communication and Peer Interaction during Equation-Solving Sessions with and without Tangible Technologies. Multimodal Technologies and Interaction, 7(1), 6. https://doi.org/10.3390/mti7010006 2023 Citation: Lehtonen, D.; Joutsenlahti, J.; Perkkilä, P. Multimodal Communication and Peer Interaction during Equation-Solving Sessions with and without Tangible Technologies. Multimodal Technol. Interact. 2023,7, 6. https://doi.org/ 10.3390/mti7010006 Academic Editor: Mu-Chun Su Received: 12 December 2022 Revised: 29 December 2022 Accepted: 9 January 2023 Published: 11 January 2023 Copyright: © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). Multimodal Technologies and Interaction Article Multimodal Communication and Peer Interaction during Equation-Solving Sessions with and without Tangible Technologies Daranee Lehtonen 1,* , Jorma Joutsenlahti 2and Päivi Perkkilä 3 1Faculty of Information Technology and Communication Sciences, Tampere University, Kanslerinrinne 1, P.O. Box 300, 33014 Tampere, Finland 2Faculty of Education and Culture, Tampere University, Åkerlundinkatu 5, P.O. Box 700, 33014 Tampere, Finland 3Kokkola University Consortium Chydenius, University of Jyväskylä, P.O. Box 567, 67701 Kokkola, Finland *Correspondence: [email protected] Abstract: Despite the increasing use of technologies in the classroom, there are concerns that technology-enhanced learning environments may hinder students’ communication and interaction. In this study, we investigated how tangible technologies can enhance students’ multimodal communication and interaction during equation-solving pair work compared to working without such technologies. A tangible app for learning equation solving was developed and tested in fourthand fifth-grade classrooms with two class teachers and 24 students. Video data of the interventions were analysed using deductive and inductive content analysis. Coded data were also quantified for quantitative analysis. Additionally, teacher interview data were used to compare and contrast the findings. The findings showed that the tangible app better promoted students’ multimodal communication and peer interaction than working only with paper and pencil. When working in pairs, tangible-app students interacted with one another much more often and in more ways than their paper-and-pencil peers. The implications of this study are discussed in terms of its contributions to research on tangible technologies for learning, educational technology development, and the use of tangibles in classrooms to support students’ multimodal communication and peer interaction. Keywords: mathematics classroom; tangible user interface; multimodal communication; peer interaction; computer-supported collaborative learning; equation solving; primary school 1. Introduction Different technologies are increasingly being employed in mathematics classrooms for various purposes, such as supporting teaching and learning, improving learning outcomes, and increasing motivation and enjoyment. The potential benefits of technologies for classroom communication and social interaction have also received attention from education and research communities [ 1 , 2 ]. Despite growing interest, there are still concerns that technology-enhanced learning environments may hinder students’ communication and interaction [1,3]. Screen-based technological solutions (e.g., computers and tablets) typically used in the classroom have some constraints that obstruct collaborative activities [ 2 ]. While working together in front of a computer or tablet, the students’ attention is primarily on the screen, which potentially decreases their communication with each other [ 4 ] and their awareness of groupmates [ 5 ]. Moreover, screen-based technologies usually allow only single-user keyboard-and-mouse or touchscreen control, thus reducing collaboration among students [ 1 ]. Tangible technologies are emerging, with technological solutions offering unique interfaces that allow digital information to be intuitively operated through the manipulation of physical objects [ 6 ]. A growing body of research has explored the Multimodal Technol. Interact. 2023,7, 6. https://doi.org/10.3390/mti7010006 https://www.mdpi.com/journal/mti Multimodal Technol. Interact. 2023,7, 6 2 of 17 possible contributions of tangible technologies to educational contexts, including classroom communication and interaction [5,7–9]. A traditional school mathematics lesson typically consists of teacher-led instruction and students’ independent practice, in which students individually and silently complete exercises in their textbooks [ 10 ]. Nevertheless, it has been acknowledged that students are likely to learn mathematics better through multimodal (e.g., verbal, visual, mathematical, and gestural) communication [ 10 ] and peer interaction [ 11 , 12 ]. When used appropriately, technologies can benefit mathematics education [ 13 , 14 ], but their use in the classroom is still relatively limited [ 15 ], particularly for supporting discussion and collaboration [ 13 ]. Previous studies (e.g., [ 14 , 16 ]) have demonstrated that technology use within a social context can promote student achievement in mathematics. Therefore, it is important to explore the potentials of technologies for supporting multimodal communication and peer interaction in mathematics classrooms. To investigate how tangible technologies can enhance students’ multimodal communication and interaction, the current study utilised research data collected during the first author’s doctoral dissertation [ 3 ], in which a tangible app for equation solving was developed and tested in schools. In our previous work [ 16 ], we analysed parts of the dissertation data to examine the benefits of tangible technologies in mathematics classrooms with regard to impacts on learning, learning support, and usability. In this study, we largely relied on the dissertation class intervention data to explore the potential role of tangible technologies in students’ multimodal communication and interaction. We compared how primary students collaboratively learned to solve linear equations using either the developed tangible app or paper and pencil, a dominant way of working in typical mathematics classrooms [17,18]. In doing so, we attempted to answer the following questions: 1. What are the characteristics of students’ multimodal communication and peer interaction during mathematics pair work sessions with and without tangible technologies? 2. How do tangible technologies support students’ multimodal communication and peer interaction in learning mathematics? Our findings show that the tangible app facilitated students’ multimodal communication and interaction compared to working only with paper and pencil. This study can benefit researchers interested in tangible technologies in educational contexts, particularly for student communication and interaction. It will assist educational technologists in developing technological solutions for classroom communication and social interaction. It will also encourage practitioners to use emerging technologies in their classrooms to support students’ multimodal communication and collaborative interactions. 2. Theoretical Background 2.1. Learning through Social Interaction Vygotsky [ 19 ] saw learning from the socio-constructivist perspective as a collaborative knowledge-building process in which learning takes place through social interaction. According to his work, each learner functions at a particular level independently and has the potential to attain an upper level of learning capacity under teacher guidance, in collaboration with more advanced peers, or with the support of concrete tools or technologyenhanced learning environments. The difference between these two levels is known as the zone of proximal development (ZPD) [ 19 ]. According to Piaget’s [ 20 ] perspective, an individual learner is more likely to learn by interacting with a peer who is viewed as a reciprocal partner and can provide a conflicting perspective on how to solve a problem. When the conflicting perspective creates an optimal mismatch with the learner’s current level of understanding, growth to the upper level of learning capacity is likely to occur [21,22] (cf., [12]). A collaborative knowledge-building process enables support in a timely manner within a learner’s ZPD [ 19 ]. In mathematics classrooms, this process can take place through discussions in pairs, in small groups, or with the whole class. Classroom discussions enable students to explain, argue, and justify, for example, a mathematical concept, which Multimodal Technol. Interact. 2023,7, 6 3 of 17 can help students develop their understanding of that concept (e.g., [ 12 , 23 , 24 ]). From a cognitive perspective, explaining their own mathematical thinking to peers verbally or through pictures, mathematical symbols, or concrete tools [ 25 ] helps students remember that information and relate it to prior information in their memory [26]. We can see classroom discussions as a scaffolding process (see [ 27 ]) in which a more capable peer or teacher helps a student solve a problem, carry out a task, or achieve a goal that would otherwise be beyond their capacity (cf., learning within the ZPD). When learning mathematics through social interaction, students can help one another develop their procedural skills, conceptual understanding, metacognitive strategies, and mathematical practices [ 12 ]. To achieve collaborative knowledge building, students need to communicate well [28]. Different methods of communication can facilitate classroom discussions. 2.2. Multimodal Communication in Mathematics Classrooms Mathematics education has traditionally focused on the use of mathematical symbols to show procedures for arriving at a solution without any accompanying text or drawings. During mathematics lessons, teachers typically explain how to solve difficult math problems and teach new content through speech, mathematical symbols, and/or drawings. In turn, students are mostly passive listeners of the teacher’s instruction and silent mathematics exercise workers. The role of multimodal (i.e., different semiotic systems) communication in mathematics classrooms has recently gained more attention (e.g., [ 10 , 29 – 31 ]). For example, the Finnish National Core Curriculum for Basic Education [ 32 ] has encouraged primary students to communicate their conclusions and solutions to teachers and peers using spoken and written language, drawings, concrete tools, and information and communication technology. Previous studies (e.g., [ 10 , 33 ]) have indicated that multimodal communication in the classroom can help students learn mathematics. Four semiotic systems or languages used for communication in mathematics classrooms can be distinguished: natural (spoken and written), mathematical symbolic (numbers and symbols), pictorial (e.g., pictures and graphs), and tactile functional (e.g., concrete tool manipulation) [ 25 ]. Tactile functional language can be expanded to body language, which includes any communication through physical behaviours, such as facial expressions, gestures, touch, motion, and full-body interaction. The process of using these languages to express mathematical thinking is called languaging [ 34 ], which can be seen as multimodal communication and contributes to mathematics classrooms in three ways [ 10 ]. First, languaging helps students organise their own mathematical thinking and develop better understanding. Second, it provides other students with different views on the subject being discussed, which helps them further develop their own thinking. Third, it assists teachers in evaluating students’ understanding, which can be used for further support and lesson planning. 2.3. Tangible Technologies for Communication and Interaction in the Classroom Tangible technologies have increasingly been piloted in different educational domains and at different levels. Studies to date have shown that these emerging technologies can benefit educational contexts by adding physical actions to computer-based learning activities [ 9 ], concretising abstract to-be-learned content through multimodal mappings [ 9 , 35 ], scaffolding learning and therefore encouraging independent exploration [ 5 , 7 ], increasing engagement and enjoyment [ 5 , 7 ], facilitating multimodal communication [ 4 , 5 ], and promoting collaborative interaction [5,7,9,35]. Unlike screen-based technologies, tangible technologies provide various affordances for face-to-face communication and collaborative interaction [ 2 , 5 ]. First, when gathering around a co-located tangible, students can see one another and each other’s actions [ 9 , 36 ], which encourages verbal and non-verbal (e.g., facial expressions, gestures, and full-body interactions) communication [ 4 , 5 ]. Co-located tangibles also promote students’ collective exploration and collaborative knowledge building by making each student’s input and Multimodal Technol. Interact. 2023,7, 6 4 of 17 the consequent digital output visible to everyone [5]. Second, tangibles facilitate students’ collaborative knowledge construction through shared representations of the task [ 5 , 36 ]. Third, multiple students can simultaneously complete a task by manipulating shared resources, such as multiple physical objects [ 6 , 35 ], which encourages everyone to engage in the group activity [ 5 , 7 ]. Finally, working with tangibles leads to parallel actions in which one student’s interface operation intervenes in others’ current or planned interactions with the interface [ 5 ]. Consequently, students need to pay attention to others’ actions, negotiate with their groupmates (e.g., by discussing or exchanging objects), and synchronise their own actions, all of which promote their collaborative interaction [ 5 ]. To date, tangibles have been used to facilitate collaborative interaction in four different contexts: exploration, problem solving, skill and knowledge development, and communication [2,7]. 3. Materials and Methods 3.1. The Tangible App for Equation Solving The first author, together with a team of computer science students and their supervisor, developed a prototype of the tangible app for primary students to learn linear equation solving (see [ 16 ] for the detailed design, development, and implementation of the app). The design was informed by the literature on to-be-learned content, pedagogy (including learning through peer interaction), and educational technologies as well as our investigation of existing educational solutions and empirical studies [ 3 ]. One design consideration of the app was based on our findings that physical learning tools better promote peer interaction than their digital counterparts. The tangible app is composed of a tablet app and two types of physical objects: X-boxes designed to represent unknowns and widely used base-10 blocks representing constants. The placement and removal of physical objects on a digital scale on a tablet screen is detected using image recognition via an external web camera connected to the tablet. The app is divided into two levels: level 1 for equation solving by substituting values for the unknown (Figure 1) and level 2 for equation solving by performing the same operation on both sides of the equation (Figure 2). Each level has eight equations to be modelled and solved. Multimodal Technol. Interact. 2023, 7, x FOR PEER REVIEW 4 of 18 Unlike screen-based technologies, tangible technologies provide various affordances for face-to-face communication and collaborative interaction [2,5]. First, when gathering around a co-located tangible, students can see one another and each other’s actions [9,36], which encourages verbal and non-verbal (e.g., facial expressions, gestures, and full-body interactions) communication [4,5]. Co-located tangibles also promote students’ collective exploration and collaborative knowledge building by making each student’s input and the consequent digital output visible to everyone [5]. Second, tangibles facilitate students’ collaborative knowledge construction through shared representations of the task [5,36]. Third, multiple students can simultaneously complete a task by manipulating shared resources, such as multiple physical objects [6,35], which encourages everyone to engage in the group activity [5,7]. Finally, working with tangibles leads to parallel actions in which one student’s interface operation intervenes in others’ current or planned interactions with the interface [5]. Consequently, students need to pay attention to others’ actions, negotiate with their groupmates (e.g., by discussing or exchanging objects), and synchronise their own actions, all of which promote their collaborative interaction [5]. To date, tangibles have been used to facilitate collaborative interaction in four different contexts: exploration, problem solving, skill and knowledge development, and communication [2,7]. 3. Materials and Methods 3.1. The Tangible App for Equation Solving The first author, together with a team of computer science students and their supervisor, developed a prototype of the tangible app for primary students to learn linear equation solving (see [16] for the detailed design, development, and implementation of the app). The design was informed by the literature on to-be-learned content, pedagogy (including learning through peer interaction), and educational technologies as well as our investigation of existing educational solutions and empirical studies [3]. One design consideration of the app was based on our findings that physical learning tools better promote peer interaction than their digital counterparts. The tangible app is composed of a tablet app and two types of physical objects: Xboxes designed to represent unknowns and widely used base-10 blocks representing constants. The placement and removal of physical objects on a digital scale on a tablet screen is detected using image recognition via an external web camera connected to the tablet. The app is divided into two levels: level 1 for equation solving by substituting values for the unknown (Figure 1) and level 2 for equation solving by performing the same operation on both sides of the equation (Figure 2). Each level has eight equations to be modelled and solved. Figure 1. How to complete the level 1 exercise x + 1 = 4. (a) Equation modelling. (b) Equation solving by substituting values for the unknown. (c) The digital scale is balanced when the equation is solved (from [16] (p. 10) CC BY-NC). Figure 1. How to complete the level 1 exercise x+ 1 = 4. ( a ) Equation modelling. ( b ) Equation solving by substituting values for the unknown. ( c ) The digital scale is balanced when the equation is solved (from [16] (p. 10) CC BY-NC). Multimodal Technol. Interact. 2023, 7, x FOR PEER REVIEW 5 of 18 Figure 2. How to complete the level 2 exercise x + 2 = 6. (a) Equation modelling. (b) Equation solving by performing the same operation on both sides of the equation. (c) The digital scale is balanced when the equation is solved (from [16] (p. 10) CC BY-NC). Two sets of instructional materials (teacher guides and student worksheets) were also designed to accompany both levels of the app. The teacher guides provide key mathematical concepts that students need to understand to master equation solving, whereas the student worksheets contain the same eight equations as in the app. To complete the worksheets, students are asked to first translate equations, which are presented as a picture, mathematical sentence, or word problem, into two other representations and then solve the equations. 3.2. Participants The equitable school system in Finland [37] made it possible to use convenience sampling in this study, despite a small sample size. Altogether, 12 fourth graders (six girls and six boys, ages 10–11) and 12 fifth graders (four girls and eight boys, ages 11–12) from primary schools in Southern Finland and their class teachers (one female and one male with 10 and 11 years of teaching experience, respectively) willingly participated in class interventions. Students were recruited based on their legal guardians’ informed consent and their mathematics performance throughout their school years. The students in both grades had mixed attainment levels (high, medium, and low) in mathematics and received no formal instruction in equation solving prior to the study. Both teachers sometimes used physical and digital teaching materials in their mathematics classrooms. They had some experience in teaching linear equations but not with teaching materials resembling our tangible app. 3.3. Research Design and Procedures The 45 min class interventions took place during school hours in real classroom settings: fourth-grade interventions took place in their own classroom and fifth-grade interventions took place in a small school classroom. Each intervention was divided into two parts: whole-class instruction and a pair work session. During the first 10–15 min, the fourth-grade teacher (T1) taught equation solving to own students (St1–12) by substituting values of the unknown and the fifth-grade teacher (T2) taught equation solving to own students (St13–24) by performing the same operation on both sides of an equation using the provided teacher guide. Then, as shown in Table 1, each teacher assigned their own students equally into two groups (learning with paper and pencil or the tangible app) and each group into three pairs: highand medium-attaining (A and B), mediumand medium-attaining (B and B), and mediumand low-attaining (B and C). Students with heterogeneous attainments were paired together to create opportunities for them to collaboratively construct knowledge within their ZPD [38]. Additionally, the teachers took the students’ ability to work well together into account when pairing them to ensure peer interaction [38,39]. Figure 2. How to complete the level 2 exercise x+ 2 = 6. ( a ) Equation modelling. ( b ) Equation solving by performing the same operation on both sides of the equation. ( c ) The digital scale is balanced when the equation is solved (from [16] (p. 10) CC BY-NC). Multimodal Technol. Interact. 2023,7, 6 5 of 17 Two sets of instructional materials (teacher guides and student worksheets) were also designed to accompany both levels of the app. The teacher guides provide key mathematical concepts that students need to understand to master equation solving, whereas the student worksheets contain the same eight equations as in the app. To complete the worksheets, students are asked to first translate equations, which are presented as a picture, mathematical sentence, or word problem, into two other representations and then solve the equations. 3.2. Participants The equitable school system in Finland [ 37 ] made it possible to use convenience sampling in this study, despite a small sample size. Altogether, 12 fourth graders (six girls and six boys, ages 10–11) and 12 fifth graders (four girls and eight boys, ages 11–12) from primary schools in Southern Finland and their class teachers (one female and one male with 10 and 11 years of teaching experience, respectively) willingly participated in class interventions. Students were recruited based on their legal guardians’ informed consent and their mathematics performance throughout their school years. The students in both grades had mixed attainment levels (high, medium, and low) in mathematics and received no formal instruction in equation solving prior to the study. Both teachers sometimes used physical and digital teaching materials in their mathematics classrooms. They had some experience in teaching linear equations but not with teaching materials resembling our tangible app. 3.3. Research Design and Procedures The 45 min class interventions took place during school hours in real classroom settings: fourth-grade interventions took place in their own classroom and fifth-grade interventions took place in a small school classroom. Each intervention was divided into two parts: whole-class instruction and a pair work session. During the first 10–15 min , the fourth-grade teacher (T 1 ) taught equation solving to own students (St 1–12 ) by substituting values of the unknown and the fifth-grade teacher (T 2 ) taught equation solving to own students (St 13–24 ) by performing the same operation on both sides of an equation using the provided teacher guide. Then, as shown in Table 1, each teacher assigned their own students equally into two groups (learning with paper and pencil or the tangible app) and each group into three pairs: highand medium-attaining (A and B), mediumand medium-attaining (B and B), and mediumand low-attaining (B and C). Students with heterogeneous attainments were paired together to create opportunities for them to collaboratively construct knowledge within their ZPD [ 38 ]. Additionally, the teachers took the students’ ability to work well together into account when pairing them to ensure peer interaction [38,39]. Table 1. Participating student (N= 24) pairs by grade, mathematics attainment level, and pair work condition. Grade Pair Work Condition Paper-and-Pencil Pair Tangible-App Pair St1(A) and St2(B) St3(A) and St4(B) 4 St5(B) and St6(B) St7(B) and St8(B) St9(B) and St10 (C) St11 (B) and St12 (C) St13 (A) and St14 (B) St15 (A) and St16 (B) 5 St17 (B) and St18 (B) St19 (B) and St20 (B) St21 (B) and St22 (C) St23 (B) and St24 (C) Note. Each student is denoted with a student number (attainment level). St n , n = 1–24. (A), (B), or (C) = high-, medium-, or low-attaining, respectively. After the teacher-led instruction, the students worked in pairs under their teachers’ supervision for 30–35 min. They had to complete eight exercises on the worksheet by Multimodal Technol. Interact. 2023,7, 6 6 of 17 modelling equations represented through one of three representations into two others and then solving each equation. The paper-and-pencil students were only provided with an individual worksheet, while the tangible-app students received an individual worksheet and a shared tangible app. After the interventions, each teacher participated in a face-to-face semi-structured interview regarding their experiences and opinions about the interventions. 3.4. Data Collection and Analysis Each pair was video-recorded with two cameras (one from the side and one from the back) to ensure well-captured data. In total, 530 min of video data from 12 pair work sessions were recorded. All videos were watched and transcribed by the first author. Due to the background noise in the classrooms and no use of additional microphones, the audio quality of most videos was too poor for transcribing students’ dialogues. Therefore, we decided to transcribe only their on-task peer communication topics (modelling and solving equations) and related non-verbal actions. The teacher interviews were audio-recorded and transcribed. Our analysis focused on how each pair work condition promoted students’ multimodal communication and interaction with peers to learn linear equations. We relied primarily on video data and accompanying transcriptions. We also used teacher interview data to compare and contrast our findings. The video analysis consisted of two parts: (1) the frequencies of students’ on-task peer communication (see [ 16 ] for more details) and (2) their peer interaction characteristics. For the first part, the video transcription was first analysed using a qualitative deductive content analysis [ 40 ] to identify students’ on-task peer communication directions (one-way or two-way) and modalities (spoken natural language, body language, or natural language and body language) according to a categorisation matrix [ 16 ], which was built based on a theoretical framework. After that, individual actions were combined into a communication episode, which is a unit of complete actions for one specific objective. For example, one communication episode of modelling an equation might consist of a pair’s discussion about how to model an equation and their whole process of modelling the equation together. Then, all discovered communication episodes were quantified (i.e., counted) and categorised into specific communication directions and modalities for descriptive statistical analysis. Students’ off-task peer communication (e.g., small talk or how to operate the app) and any communication with the teacher were not part of this analysis. A Pearson’s chi-squared test was used to statistically investigate the relationships between pair work conditions and peer communication directions and modalities. For the second part, the video transcription was analysed using a qualitative inductive content analysis (through open coding, creating categories, and abstraction [ 40 ]) to identify certain patterns of how the students in each condition interacted with their groupmates. 4. Results To investigate how tangible technologies support students’ peer interaction when learning linear equations in pairs, we compared the students’ pair work sessions using our tangible app to the sessions that did not. Next, we provide descriptive statistics of students’ peer communication, their peer interaction analysis, and teachers’ observations of their students’ pair work. 4.1. Descriptive Statistics Altogether, 287 episodes of on-task peer communication were observed during 12 pair work sessions (6 sessions for each condition). Most of the communication occurred during the tangible-app sessions (70.4%). In completing the eight exercises, the average peer communication of the tangible-app students (34 episodes/pair, SD = 13.9) was more than twice that of the paper-and-pencil students (14 episodes/pair, SD = 7.7). Table 2shows the frequencies and percentages of students’ on-task communication episodes that occurred during each pair work condition regarding communication directions and modalities. Multimodal Technol. Interact. 2023,7, 6 7 of 17 A chi-squared test showed that there were associations between pair work conditions and peer communication directions (X2 (1, N = 287) = 23.15, p< 0.001) and modalities (X2 (2, N = 287) = 17.13, p< 0.001). Table 2. Frequencies and percentages of on-task peer communication episodes (N= 287) regarding directions and modalities by pair work condition. Pair Work Condition Paper-and-Pencil Tangible App Communication n%n% One-Way Communication Spoken natural language 40 13.9 55 19.2 Body language 7 2.4 5 1.7 Spoken natural language and body language 4 1.4 0 0 Two-Way Communication Spoken natural language 22 7.7 42 14.6 Body language 0 0 11 3.8 Spoken natural language and body language 12 4.2 89 31.0 Total 85 29.6 202 70.4 We further examined two-way communication (sending and receiving information) episodes occurring during each pair work condition to indicate students’ peer interactions. Tangible-app students (70.1% of their communication) interacted with their groupmate much more than paper-and-pencil students (40.0% of their communication). Paper-andpencil students mostly interacted with each other through spoken natural language (64.7% of their two-way communication), while tangible-app students did so through spoken natural language and body language (62.7% of their two-way communication), for example, manipulating the app and simultaneously talking about their own actions. Sometimes, students in the tangible-app condition also interacted with each other only by manipulating the app to complete an exercise without talking to their groupmates. 4.2. Analysis of Peer Interaction The findings from the qualitative analysis of the video data were consistent with the quantitative analysis. Peer interaction was typically observed during tangible-app sessions, while silent and individual work was usually observed during paper-and-pencil sessions. In the following subsections, we describe how the students in each condition worked in pairs, with accompanying examples of video transcription, which were translated into English. 4.2.1. Paper-and-Pencil Condition Without the teacher’s presence, paper-and-pencil students rarely interacted with their groupmates. They usually worked quietly on their individual worksheets at different paces. The following example, where fifth-grade mediumand low-attaining students modelled and solved an equation represented as a picture, illustrates typical pair work of students in this condition (Figure 3). Students St21 (B) and St22 (C) completing exercise 5 (1:00 min): T2: leaves after Exercise 4. St21 (B): silently models the given picture by writing an equation (a mathematical sentence representing the picture) on her own worksheet. St22 (C): silently models the given picture by writing an equation (a mathematical sentence representing the picture) on her own worksheet. St21 (B): silently solves the equation by writing mathematical symbols on her own worksheet. St22 (C): silently solves the equation by writing mathematical symbols on her own worksheet. Multimodal Technol. Interact. 2023,7, 6 8 of 17 Multimodal Technol. Interact. 2023, 7, x FOR PEER REVIEW 8 of 18 worked in pairs, with accompanying examples of video transcription, which were translated into English. 4.2.1. Paper-and-Pencil Condition Without the teacher’s presence, paper-and-pencil students rarely interacted with their groupmates. They usually worked quietly on their individual worksheets at different paces. The following example, where fifth-grade mediumand low-attaining students modelled and solved an equation represented as a picture, illustrates typical pair work of students in this condition (Figure 3). Students St21 (B) and St22 (C) completing exercise 5 (1:00 min): T2: leaves after Exercise 4. St21 (B): silently models the given picture by writing an equation (a mathematical sentence representing the picture) on her own worksheet. St22 (C): silently models the given picture by writing an equation (a mathematical sentence representing the picture) on her own worksheet. St21 (B): silently solves the equation by writing mathematical symbols on her own worksheet. St22 (C): silently solves the equation by writing mathematical symbols on her own worksheet. Figure 3. No peer interaction. students St21 (B) and St22 (C), in the paper-and-pencil group, working on their own worksheets quietly and separately. Students in this condition occasionally sought help from their groupmates by asking or looking at what they had written on their worksheets. However, they seldom had their groupmate’s attention. The teacher’s encouragement to work together appeared to promote peer scaffolding among paper-and-pencil students. With the encouragement of teachers, higher-attaining students verbally advised their groupmates, who then followed the given advice. In the following example, fourth-grade mediumand low-attaining students modelled and solved an equation represented as mathematical symbols, first independently and then with the teacher’s guidance. Students St9 (B) and St10 (C) completing exercise 1 (5:15 min): St9 (B): silently models the given equation by drawing pictures on his own worksheet. St10 (C): silently reads his own worksheet , then says something to St9 (B) while pointing at his groupmate’s worksheet (Figure 4a). St9 (B): does not react to what St10 (C) did and continues solving the equation by writing mathematical symbols on his own worksheet. T1: comes to explain to the students how to model the given equation by drawing. St10 (C): talks to T1 about drawing the equation. St9 (B): finishes exercise 1 on his own worksheet and starts completing exercise 2. Figure 3. No peer interaction. students St 21 (B) and St 22 (C), in the paper-and-pencil group, working on their own worksheets quietly and separately. Students in this condition occasionally sought help from their groupmates by asking or looking at what they had written on their worksheets. However, they seldom had their groupmate’s attention. The teacher’s encouragement to work together appeared to promote peer scaffolding among paper-and-pencil students. With the encouragement of teachers, higher-attaining students verbally advised their groupmates, who then followed the given advice. In the following example, fourth-grade mediumand low-attaining students modelled and solved an equation represented as mathematical symbols, first independently and then with the teacher’s guidance. Students St9(B) and St10 (C) completing exercise 1 (5:15 min): St9(B): silently models the given equation by drawing pictures on his own worksheet. St10 (C): silently reads his own worksheet, then says something to St 9 (B) while pointing at his groupmate’s worksheet (Figure 4a). St9(B): does not react to what St10 (C) did and continues solving the equation by writing mathematical symbols on his own worksheet. T1: comes to explain to the students how to model the given equation by drawing. St10 (C): talks to T1about drawing the equation. St9(B): finishes exercise 1 on his own worksheet and starts completing exercise 2. T1: asks St9(B) to explain to St10 (C) how he drew exercise 1. St9(B): replies to T 1 , then explains to St 10 (C) how to draw the equation while pointing at St 10 (C)’s worksheet (Figure 4b). St10 (C): listens to St9(B), then draws the equation on his own worksheet. St9(B): continues working on exercise 2. St10 (C): silently solves exercise 1 by writing mathematical symbols on his own worksheet. St9(B): finishes exercise 2 on his own worksheet and starts completing exercise 3 without waiting for St10 (C). Multimodal Technol. Interact. 2023, 7, x FOR PEER REVIEW 9 of 18 T1: asks St9 (B) to explain to St10 (C) how he drew exercise 1. St9 (B): replies to T1, then explains to St10 (C) how to draw the equation while pointing at St10 (C)’s worksheet (Figure 4b). St10 (C): listens to St9 (B), then draws the equation on his own worksheet. St9 (B): continues working on exercise 2. St10 (C): silently solves exercise 1 by writing mathematical symbols on his own worksheet. St9 (B): finishes exercise 2 on his own worksheet and starts completing exercise 3 without waiting for St10 (C). Figure 4. Peer communication and scaffolding. (a) St10 (C) (on the right) asking for help from St9 (B) (on the left) and pointing at his groupmate’s worksheet but receiving no reaction from St9 (B). (b) Later, St9 (B) explaining to St10 (C) by speaking and pointing at his groupmate’s worksheet. Sometimes, even the teacher’s encouragement to work together could not make paper-and-pencil students interact with each other. Instead of responding to the teacher’s encouragement, students sometimes continued working separately. A temporary lack of concentration, such as looking at what other pairs were doing, was occasionally observed. In the following example, fourth-grade highand medium-attaining students separately and silently worked on an equation represented as mathematical symbols, even though their teacher had encouraged them to talk to each other. At some point, the high-attaining student lost his concentration and turned to look at what another pair was doing. Students St1 (A) and St2 (B) completing exercise 2 (1:10 min): St1 (A): silently models the given equation by drawing on his own worksheet. St2 (B): silently models the given equation by drawing on her own worksheet. T1: comes and asks St1 (A) and St2 (B) to talk to each other about how to solve the equation. St1 (A): does not respond to the teacher’s reques t and starts to solve the equation by writing mathematical symbols on his own worksheet. St2 (B): does not respond to the teacher re q uest and starts to solve the equation by writing Finnish texts and mathematical symbols on her own worksheet. St1 (A): Finishes exercise 2 on his own worksheet and starts completing exercise 3 without waiting for St2 (B). Stops solving his worksheet and turns to look at what another pair is doing (Figure 5) , then turns back to his own worksheet. Figure 4. Peer communication and scaffolding. ( a ) St 10 (C) (on the right) asking for help from St 9 (B) (on the left) and pointing at his groupmate’s worksheet but receiving no reaction from St 9 (B). (b) Later, St9(B) explaining to St10 (C) by speaking and pointing at his groupmate’s worksheet. Multimodal Technol. Interact. 2023,7, 6 15 of 17 Fourth, the tangible app provided physical objects, which allowed multiple students to use them simultaneously to model and solve equations [ 6 , 35 ]. Our findings support a study [ 41 ] that found that shared input resources balanced students’ participation in collaborative activities by encouraging every student to take part [ 5 , 7 ]. When one student tried to take over the entire operation of the app, the other student was still usually able to manipulate objects close to them. Finally, corresponding to the explanation in [ 5 ] about working with tangibles and parallel actions, students needed to monitor how their groupmates manipulated physical objects so they could adjust their own actions accordingly. For example, both students grasped three base-10 blocks from the desk and intended to add them to the X-box to balance the digital scale. However, one student was faster and added two blocks to the X-box. The other student noticed that, so they only added one block. To conclude, this study not only contributes to research but also to practice. It advances knowledge of how tangible technologies can facilitate students’ multimodal communication and peer interaction. It assists educational technologists in developing emerging technological solutions for classrooms. This study also has practical implications. It encourages practitioners to employ tangible technologies in their classrooms to promote students’ multimodal communication and collaborative interaction, as encouraged by, for example, the Finnish Core Curriculum for Basic Education [ 32 ]. Tangible technologies can also be used to support physical actions in computer-based learning activities, the understanding of abstract contents, scaffolding learning and independent exploration, and learning engagement and enjoyment. The physical and digital attributes of tangible technologies can play an important role in mathematics classrooms, where students, particularly in higher grades, may perceive the use of traditional concrete tools as childish [ 48 ]. While the physicality of tangible technologies concretises abstract mathematical concepts, digitality can engage students who are in favour of digital technology in learning mathematics with tangible technologies. In addition, teacher education and professional development should prepare preand in-service teachers for the pedagogical incorporation of technologies in their mathematics classrooms. 5.3. Limitations and Future Research The current study has some limitations. First, the small convenience sample size of participants makes it difficult to generalise the research findings. Moreover, the 45 min class intervention was relatively short. Larger sample sizes, possibly randomised, and a longer period of intervention could be helpful in the future to promote research validity. Second, researcher triangulation could not be met since the data were analysed by only one researcher. In future studies, more researchers should be involved in data analysis to increase research reliability. Third, the design of this research poses the question of whether the research findings were partly influenced by the fact that the tangible-app students had something (a tablet) to share, but the paper-and-pencil students did not. Future studies should employ two control groups, one sharing a worksheet and one sharing a worksheet and a non-digital learning tool, such as a physical balance scale. Finally, to further explore the potentials of tangible technologies for mathematics education, future research could investigate their benefits for other mathematics contents and different educational levels. Author Contributions: Conceptualisation, methodology, investigation, formalanalysis, data curation, writing—original draft, and visualisation, D.L.; supervision and writing—original draft and review, J.J. and P.P. All authors have read and agreed to the published version of the manuscript. Funding: This research received no external funding. Institutional Review Board Statement: Not applicable. Informed Consent Statement: Prior to the study, informed consent was obtained from the teachers and legal guardians of all students involved. Data Availability Statement: All of the data is contained within the article. Multimodal Technol. Interact. 2023,7, 6 16 of 17 Acknowledgments: The authors would like to acknowledge the contributions of Fouzia Khan, Juho Korkala, Roni Perälä, Niko Sainio, and Krishna Bagale for software development under the supervision of Pekka Mäkiaho; Tapio Lehtonen for the UI; and Kari Kouhia for the prototyping. The authors appreciate the teachers and students who participated in the research. Conflicts of Interest: The authors declare no conflict of interest. References 1. Karno, D.; Hatcher, B. Building Computer Supported Collaborative Learning Environments in Early Childhood Classrooms. Educ. Technol. Res. Dev. 2020,68, 249–267. [CrossRef] 2. Li, Y.; Kothiyal, A.; Weber, T.; Rossmy, B.; Mayer, S.; Hussmann, H. Designing Tangible as an Orchestration Tool for Collaborative Activities. Multimodal Technol. Interact. 2022,6, 30. [CrossRef] 3. Lehtonen, D. ‘Now I Get It!’ Developing a Real-World Design Solution for Understanding Equation-Solving Concepts. Ph.D. Thesis, Tampere University, Tampere, Finland, 2022. Available online: https://urn.fi/URN:ISBN:978-952-03-2250-2 (accessed on 1 December 2022). 4. Salvador, G.; Pérez, D.; Ortega, M.; Soto, E.; Alcañiz, M.; Contero, M. Evaluation of an Augmented Reality Enhanced Tabletop System as a Collaborative Learning Tool: A Case Study on Mathematics at the Primary School. In Proceedings of the 33rd Annual Conference of the European Association for Computer Graphics (Eurographics 2012), Cagliari, Italy, 13–18 May 2012; pp. 9–16. 5. Price, S. Tangibles: Technologies and Interaction for Learning. In The SAGE Handbook of Digital Technology Research; Price, S., Jewitt, C., Brown, B., Eds.; SAGE: Los Angeles, CA, USA, 2015; pp. 307–325. [CrossRef] 6. Ishii, H.; Ullmer, B. Tangible User Interfaces. In The Human-Computer Interaction Handbook Fundamentals, Evolving Technologies, and Emerging Applications, 3rd ed.; Jacko, J.A., Ed.; Taylor & Francis: Boca Raton, FL, USA, 2012; pp. 465–490, ISBN 9781439829431. 7. Li, Y.; Liang, M.; Preissing, J.; Bachl, N.; Dutoit, M.M.; Weber, T.; Mayer, S.; Hussmann, H. A Meta-Analysis of Tangible Learning Studies from the TEI Conference. In Proceedings of the 16th International Conference on Tangible, Embedded, and Embodied Interaction (TEI ‘22), New York, NY, USA, 13–16 February 2022; pp. 1–17. [CrossRef] 8. Markova, M.S.; Wilson, S.M.; Stumpf, S. Tangible User Interfaces for Learning. Int. J. Technol. Enhanced Learn. 2012 ,4, 139–155. [CrossRef] 9. Schneider, B.; Jermann, P.; Zufferey, G.; Dillenbourg, P. Benefits of a Tangible Interface for Collaborative Learning and Interaction. IEEE Trans. Learn. Technol. 2011,4, 222–232. [CrossRef] 10. Joutsenlahti, J.; Kulju, P. Multimodal Languaging as a Pedagogical Model: A Case Study of the Concept of Division in School Mathematics. Educ. Sci. 2017,7, 9. [CrossRef] 11. Bozkurt, G. Social Constructivism: Does It Succeed in Reconciling Individual Cognition with Social Teaching and Learning Practices in Mathematics? J. Educ. Pract. 2017,8, 210–218. 12. Moschkovich, J.N. Scaffolding Student Participation in Mathematical Practices. ZDM 2015,47, 1067–1078. [CrossRef] 13. Bray, A.; Tangney, B. Technology Usage in Mathematics Education Research–A Systematic Review of Recent Trends. Comput. Educ. 2017,114, 255–273. [CrossRef] 14. Genlott, A.A.; Grönlund, Å. Closing the Gaps–Improving Literacy and Mathematics by ICT-Enhanced Collaboration. Comput. Educ. 2016,99, 68–80. [CrossRef] 15. Russo, J.; Bragg, L.A.; Russo, T. How Primary Teachers Use Games to Support Their Teaching of Mathematics. Int. Electron. J. Elem. Educ. 2021,13, 407–419. [CrossRef] 16. Lehtonen, D.; Machado, L.; Joutsenlahti, J.; Päivi, P. The Potentials of Tangible Technologies for Learning Linear Equations. Multimodal Technol. Interact. 2020,4, 77. [CrossRef] 17. Perkkilä, P.; Joutsenlahti, J.; Sarenius, V. Peruskoulun Matematiikan Oppikirjat Osana Oppimateriaalitutkimusta [Elementary School Mathematics Textbooks as Part of Research on Learning Materials]. In Matematiikan Opetus Ja Oppiminen [Mathematics Teaching and Learning]; Joutsenlahti, J., Silfverberg, H., Räsänen, P., Eds.; Niilo Mäki Institute: Jyväskylä, Finland, 2018; pp. 344–367 , ISBN 978-951-39-7584-5. 18. Van den Heuvel-Panhuizen, M.; Wijers, M. Mathematics Standards and Curricula in the Netherlands. ZDM Math. Educ. 2005 ,37, 287–307. [CrossRef] 19. Vygotsky, L.S. Mind and Society: The Development of Higher Psychological Processes; Harvard University Press: Cambridge, MA, USA, 1978. 20. Piaget, J. Science of Education and the Psychology of the Child; Penguin: New York, NY, USA, 1979. 21. Cannella, G.S. Learning through Social Interaction: Shared Cognitive Experience, Negotiation Strategies, and Joint Concept Construction for Young Children. Early Child. Res. Q. 1993,8, 427–444. [CrossRef] 22. Forman, E.; Cordle, J.; Carr, N.; Gregorius, T. Expertise and the Co-construction of Meaning in Collaborative Problem Solving. In Proceedings of the Paper presented at the Meeting of the Jean Piaget Society, Philadelphia, PA, USA, May 1991. 23. Hiebert, J.; Carpenter, T.P. Learning and Teaching with Understanding. In Handbook of Research on Mathematics Teaching and Learning; Grouns, D.A., Ed.; Macmillan: New York, NY, USA, 1992; pp. 65–92. 24. Pape, S.J.; Tchoshanov, M. The Role of Representation(s) in Developing Mathematical Understanding. Theory Into Pract. 2001 ,40, 118–127. [CrossRef] Multimodal Technol. Interact. 2023,7, 6 17 of 17 25. Perkkilä, P.; Joutsenlahti, J. Academic Literacy Supporting Sustainability for Mathematics Education-A Case: Collaborative Working as a Meaning Making for “2/3”? In Transitioning to Quality Education; Jeronen, E., Ed.; MDPI: Basel, Switzerland, 2021; pp. 163–188. [CrossRef] 26. Slavin, R.E. Co-operative Learning: What Makes Group-work Work? Educ. Res. Innov. 2010,7, 161–178. [CrossRef] 27. Wood, D.J.; Bruner, J.S.; Ross, G. The Role of Tutoring in Problem Solving. J. Child Psychol. Psychiatry 1976 ,17, 89–100. [CrossRef] 28. Szewkis, E.; Nussbaum, M.; Rosen, T.; Abalos, J.; Denardin, F.; Caballero, D.; Tagle, A.; Alcoholado, C. Collaboration within Large Groups in the Classroom. Int. J. Comput.-Support. Collab. Learn. 2011,6, 561–575. [CrossRef] 29. Morgan, C.; Planas, N.; Schütte, M. Developing a Perspective on Multiplicity in the Study of Language in Mathematics Classrooms. In Classroom Research on Mathematics and Language: Seeing Learners and Teachers Differently, 1st ed.; Planas, N., Morgan, C., Schütte, M., Eds.; Routledge: London, UK, 2021; pp. 3–21. 30. O’Halloran, K.L. The Language of Learning Mathematics: A Multimodal Perspective. J. Math. Behav. 2015 ,40, 63–74. [CrossRef] 31. Tang, K.S.; Ho, C.; Putra, G.B.S. Developing Multimodal Communication Competencies: A Case of Disciplinary Literacy Focus in Singapore. In Using Multimodal Representations to Support Learning in the Science Classroom; Hand, B., McDermott, M., Prain, V., Eds.; Springer: New York, NY, USA, 2016; pp. 135–158. [CrossRef] 32. Finnish National Agency for Education. National Core Curriculum for Basic Education 2014; Finnish National Agency for Education: Helsinki, Finland, 2016; ISBN 9789521362590. 33. Alfaro Viquez, H.; Joutsenlahti, J. Promoting Learning with Understanding: Introducing Languaging Exercises in Calculus Course for Engineering Students at the University Level. LUMAT 2020,8, 229–251. [CrossRef] 34. Joutsenlahti, J. Kielentäminen Matematiikan Opiskelussa [Languaging in Mathematics Learning]. In Opettaja, Asiantuntijuus ja Yhteiskunta. Ainedidaktinen Symposium 2003 [Teacher, Expertise, and Society, Proceedings of the Subject Didactics Symposium 2003], Turku, Finland, 7 February 2003; Virta, A., Marttila, O., Eds.; University of Turku: Turku, Finland, 2003; pp. 188–196. 35. O’Malley, C.; Fraser, D.S. Literature Review in Learning with Tangible Technologies; Report 12; Futurelab: Bristol, UK, 2004; pp. 1–48, ISBN 0-9548594-2-1. 36. Dillenbourg, P.; Evans, M. Interactive Tabletops in Education. Int. J. Comput.-Support. Collab. Learn. 2011,6, 491–514. [CrossRef] 37. Organisation for Economic Cooperation and Development (OECD). Education Policy Outlook: Finland. Available online: www.oecd.org/education/policy-outlook/country-profile-Finland-2020.pdf (accessed on 28 December 2022). 38. Tam, M. Constructivism, Instructional Design, and Technology: Implications for Transforming Distance Learning. Educ. Technol. Soc. 2000,3, 50–60. 39. Slavin, R.E. Research on Cooperative Learning and Achievement: What We Know, What We Need to Know. Contemp. Educ. Psychol. 1996,21, 43–69. [CrossRef] 40. Elo, S.; Kyngäs, H. The Qualitative Content Analysis Process. J. Adv. Nurs. 2008,62, 107–115. [CrossRef] [PubMed] 41. Starcic, A.I.; Cotic, M.; Zajc, M. Design-based Research on the Use of a Tangible User Interface for Geometry Teaching in an Inclusive Classroom. Br. J. Educ. Technol. 2013,44, 729–744. [CrossRef] 42. Zamorano Urrutia, F.J.; Cortés Loyola, C.; Herrera Marín, M. A Tangible User Interface to Facilitate Learning of Trigonometry. Int. J. Emerg. Technol. Learn. 2019,14, 152–164. [CrossRef] 43. Meier, A.; Spada, H.; Rummel, N. A Rating Scheme for Assessing the Quality of Computer-supported Collaboration Processes. Int. J. Comput.-Support. Collab. Learn. 2007,2, 63–86. [CrossRef] 44. Lehtonen, D.; Joutsenlahti, J. The Benefits of Using Manipulatives in Classrooms for Equation Concepts Understanding. In Changing Subjects, Changing Pedagogies: Diversities in School and Education; Pyyry, N., Tainio, L., Juuti, K., Vasquez, R., Paananen, M., Eds.; Finnish Research Association for Subject Didactics: Helsinki, Finland, 2017; pp. 164–185, ISBN 978-952-5993-23-3. 45. Hengeveld, B.; Hummels, C.; van Balkom, H.; Voort, R.; de Moor, J. Wrapping up LinguaBytes, for Now. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (TEI ’13), Barcelona, Spain, 10–13 February 2013; pp. 237–244. [CrossRef] 46. Kilpatrick, J.; Swafford, J.; Findell, B. Adding It Up: Helping Children Learn Mathematics; National Academy Press: Washington, DC, USA, 2001; ISBN 0-309-50524-0. 47. Laine, A.; Ahtee, M.; Näveri, L.; Pehkonen, E.; Hannula, M. Teachers’ Influence on the Quality of Pupils’ Written Explanations– Third-Graders Solving a Simplified Arithmagon Task During a Mathematics Lesson. LUMAT 2018,6, 87–104. [CrossRef] 48. Marshall, L.; Swan, P. Exploring the Use of Mathematics Manipulative Materials: Is It What We Think It Is? In Sustainability in Higher Education: Directions for Change, Proceedings of EDU-COM 2008 International Conference, Khon Kaen, Thailand, 19–21 November 2008; Renner, J., Cross, J., McCormack, L., Eds.; Edith Cowan University: Perth, Australia, 2008; pp. 338–350. Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.