Experimenting with the structured assessment dialogue in physics and mathematics classes

This is an electronic reprint of the original article. This reprint may differ from the original in pagination and typographic detail. Author(s): Title: Year: Version: Please cite the original version: All material supplied via JYX is protected by copyright and other intellectual property rights, and duplication or sale of all or part of any of the repository collections is not permitted, except that material may be duplicated by you for your research use or educational purposes in electronic or print form. You must obtain permission for any other use. Electronic or print copies may not be offered, whether for sale or otherwise to anyone who is not an authorised user. Experimenting with the structured assessment dialogue in physics and mathematics classes Nieminen, Pasi; Hähkiöniemi, Markus; Bruun, Jesper; Viiri, Jouni Nieminen, P., Hähkiöniemi, M., Bruun, J., & Viiri, J. (2017). Experimenting with the structured assessment dialogue in physics and mathematics classes. In Proceedings of the annual FMSERA symposium 2016 (pp. 66-75). Finnish Mathematics and Science Education Research Association (FMSERA). Proceedings of the FMSERA annual symposium. https://journal.fi/fmsera/article/view/60944/27039 2017 Nieminen et al. Proceedings of the annual FMSERA symposium 2016 66 EXPERIMENTING WITH STRUCTURED ASSESSMENT DIALOGUE IN PHYSICS AND MATHEMATICS CLASSES Pasi Nieminen1, Markus Hähkiöniemi1, Jesper Bruun2 and Jouni Viiri1 1University of Jyväskylä, 2University of Copenhagen ABSTRACT This study deals with a formal, pre-planned teacher–student discussion, called structured assessment dialogue (SAD) that assesses students’ inquiry competences by combining formative and summative assessments. Six SAD sessions (four lower-secondary physics and two upper-secondary mathematics) were implemented, video recorded and analysed using theory-based categories and networks. The results showed that students usually gave lower-order answers even though the teacher asked higher-order questions. In the lower secondary, the teacher struggled to engage students by using what students themselves had said. In the upper secondary, the teacher focused on addressing a particular idea, which was difficult for the student. Despite the challenges, SAD could be a useful method in Finnish schools if teachers and students acclimate to it. INTRODUCTION Classroom assessment tends to evaluate students’ achievements for grading or to promote their learning by means of continual feedback about progress and difficulties concerning the learning process. In educational literature (e.g., Harlen & Qualter, 2014), these two aims generally are known as summative and formative assessments, respectively. In a good assessment, the decent criteria to attain the learning goals increase the validity of the summative assessment and help communicate the students’ learning progress for formative purposes. Student learning can be assessed by using various data sources about their learning, such as tests and different student products. Likewise, it can be assessed by means of classroom dialogue, which has been studied widely and recognised as central to successful learning (Alexander, 2006; Mortimer & Scott, 2003). In classroom dialogue, students can express and share their thinking and ideas, and the teacher has the opportunity to explore these. There are studies about teachers’ formative practices during unplanned spontaneous assessment conversations (e.g., Nieminen, Hähkiöniemi, Leskinen & Viiri, 2016; Ruiz-Primo & Furtak, 2006). During Nieminen et al. Proceedings of the annual FMSERA symposium 2016 67 these conversations, teachers need, in real time, to elicit information from students’ understanding and make decisions about the amount and manner of guiding that will help students in their learning. Here, the teacher’s discursive actions, such as asking questions and maintaining discussion, play a central role (Chin, 2007; Dysthe, 1996; Sahin & Kulm, 2008). Although there are many studies about classroom dialogue, most research has been done in authentic classroom settings during whole class or teacher–student discussions. In this study, we focused on a novel method called structured classroom dialogue (SAD). Here, the teacher–student discussions were pre-planned in order to assess how learning criteria had been attained after a certain inquiry activity. Thus, the teacher needed to plan main questions beforehand according to the criteria but still was required also to perform spontaneous discursive actions, such as follow-up questions. SAD can be used for formative and summative purposes and can be studied from different perspectives, but this paper focuses only on the discursive actions of the teacher and students. However, as SAD is a novel method, we present it generally. The SAD was conducted three times in three classes by one physics and one mathematics teacher. Video–audio data was analysed in order to answer the following research question: How did the teachers and students use different discursive actions during the SAD in relation to the learning criteria? METHODS Description of SAD SAD takes place after students have undertaken an inquiry activity. Before starting the inquiry activity, the teacher explains to the students the task and the activity-specified learning criteria. The criteria help students understand what they are expected to perform, and they help the teacher assess the students’ work. Table 1 shows the learning criteria for a physics inquiry activity about “Interaction of a magnet and a coil”. After the inquiry, the SAD starts as a ritualised dialogue between the teacher and one student (the focus-student dialogue), then moves on to a peer-feedback phase, in which the focus student holds a discussion with a group of feedback students (5–7 persons). During these interactions, the rest of the class observes and reflects on their own understanding. The duration of the focus-student dialogue is about 5 minutes. For this discussion, the teacher has prepared questions that help map the students’ attainment of the learning criteria in the inquiry activity (see example in Table 1). The teacher should pose questions that follow up on students’ responses and utterances, and Nieminen et al. Proceedings of the annual FMSERA symposium 2016 68 the teacher should value the students’ contributions as well as challenge the students’ thinking and reasoning. The teacher also may add new information and clarify conflicts in students’ arguments and reasoning. It is also important to make room for and promote student reflection. Table 1. Learning criteria and the pre-planned questions for the inquiry task “Interaction of a magnet and a coil“ Level Criterion Pre-planned questions I Student is able to describe how the interaction between a magnet and a coil is observed. Did you observe that the needle of the ampere meter moved? What did you do then? Did the poles of the permanent magnet have an effect ono the magnitude of the electric current? II Student is able to describe which variables have an effect on the magnitude of interaction. How did the changing of the magnets (different strength) affect the magnitude of the electric current? How did the changing of the coils (different amounts of loops of wire) affect the magnitude of the electric current? III Student is able to explain which variables have an effect on the generation of electric current in a coil. Could you explain which variables have an effect on the generation of an electric current in a coil? During the focus-student dialogue, feedback students take notes (retain or on paper) to assess the level of the dialogue with respect to the learning criteria given in the beginning of the inquiry. After this dialogue, feedback students hold discussions with the focus student. The duration of this feedback dialogue is also about 5 minutes. The teacher’s role here is to support students when they give feedback to ensure that the dialogue is relevant and productive. Ideally, the teacher’s interventions should diminish from session to session. The feedback session starts with a discussion about how the things which were said in the focus-student dialogue are related to the learning criteria. To help justify their suggestions, the criteria are explicitly presented during this part by handing them out on paper, for example. After this, students are asked to discuss what should be added to the focus-student dialogue in order to better meet the criteria for learning. The discussions are followed by a 5-minute reflection session, during which students fill the student self-reflection form. Data gathering The SAD was implemented in two lower-secondary physics classes (n = 2 x 16) by the same teacher and in one upper-secondary mathematics class (n = 27). Both Nieminen et al. Proceedings of the annual FMSERA symposium 2016 69 teachers were experienced. In all the classes, SADs were conducted three times, but the first time was a practice without the data collection. Before a SAD session, students worked about 45 minutes in an inquiry task (electromagnetism or geometry) that had specific learning criteria. For the focus student dialogue, the teachers tended to select a student who had adequate social and subject competences. Our research design also included questionnaires for students and teachers (Dolin et al., in press), but we do not deal with that data here due to the limited length of the paper. Analysis of discursive actions Dialogues were coded by 5-second intervals. Coding concerned the discursive actions, related criteria and gestures (see Fig. 1). The codes are based on Quistgaard’s (2017) interpretation of Dysthe’s (1996) framework (Table 2). Figure 1. Example of the spreadsheet used for coding of the focus-student dialogue Table 2. Codes for teachers’ discursive actions (Dysthe, 1996; Quistgaard, 2017) Code Description Uptake Incorporating students’ responses into the next question, thus getting the students to reflect further about what they said, and integrating the answer into the dialogue. Focus Meant as an opposite to uptake. Focus can be seen as an emphasis on the set teaching goals, where uptake can go out on a tangent. Precise valuing Analogous to high valuing, but might not be strictly positive. The point is that it is precise and puts value to what is said. Lower-order question Lower order covers the knowledge and comprehension steps (Bloom's taxonomy; Krathwohl, 2002). Higher-order question Higher order covers application, analysis, synthesis and evaluation (Bloom's taxonomy; Krathwohl, 2002). Nieminen et al. Proceedings of the annual FMSERA symposium 2016 70 This paper focuses on discursive actions, so the codes for teacher’s actions are used in the results later on (Fig. 2 and 3). Four students’ discursive actions, lowerorder or higher-order answers and lower-order or higher-order statements, also are related to Bloom’s taxonomy (Krathwohl, 2002). The coded data are used to create networks of interrelated codes that could be used to investigate the structure and characteristics of the dialogue. RESULTS Focus student dialogue The four networks depicting the dialogue for the lower secondary are similar to each other in terms of which actions are prevalent. Likewise, the two networks for the upper secondary are also very similar. However, the lower-secondary and upper-secondary dialogues show differences. Dolin et al. (in press) perform a detailed analysis of the networks in this study in relation to other dialogues. Here, we discuss representative networks for the lower secondary and upper secondary in relation to the analyses performed above. The two networks are shown in Figures 2 and 3. Both networks show a low level of student engagement; the student is silent for the majority of the dialogue. Besides this, the two dialogues show different patterns in the lowerand upper-secondary levels. The lower-secondary dialogue is dominated by a lower-order answer to a lower-order question pattern. The lower-order question/lower-order answer pattern does not dominate the upper- -secondary dialogue. Here, the teacher seems to focus on making the student address a particular point, which involves asking higher-order questions in terms of Bloom’s taxonomy. This is seen as the large blue circle and surrounding smaller circles in Figure 3. However, the teacher’s efforts do not lead to higherorder answers but to students’ confusion, silence or lower-order answers. For example, in the following excerpt, the main task is to express and justify a claim about the relation between two angles. One of the angles was between two lines tangent to a circle. The other angle was a corresponding central angle in the circle. The student had just stated that the angle between the radius and tangent was 90°. First, the teacher asked a higher-order question, and the student gave a lower-order answer. At last, the teacher summarised what they have just said. Teacher: You have drawn these tangents so that they cut here in that point (shows the intersection point). If they would cut here, for example, (shows another point further from the circle) so that they would not go so steep, would these angles here still be 90°? (Points to the angle between radius and tangent) Student: I don’t know. Probably not. Teacher: You are not sure about that? Student: I am not quite sure. Nieminen et al. Proceedings of the annual FMSERA symposium 2016 71 Teacher: Okay, so you are not quite sure if this is true generally. Like, whether this is true only in this situation or true in general. Figure 2. The network for the fourth SAD in the lower secondary. The left part of the network presents the students’ contributions. The right part of the network presents the teacher’s contributions. The student and teacher nodes in the same position indicate that they were related. The colour of the node indicates the discursive action. The text on the nodes represents the learning criterion addressed in the discursive action, if the action is not related to any criterion (silence) or if it is outside of any criterion (talk concerns something else); NoSC = no student criterion; NoTC = no teacher criterion. The size of the nodes represents the time spent on a particular discursive action. Links indicate that one action followed another at one time during the dialogue. Nieminen et al. Proceedings of the annual FMSERA symposium 2016 72 Another key difference between the two dialogues is a different focus on the teacher using what the student has said previously in trying to shape the dialogue. This action is represented by the uptake code. The lower-secondary teacher seems to focus more on including the previous statements of the student than does the upper-secondary teacher. For example, in the following excerpt, the teacher asks about the student’s observations, the student gives short straight answers, and the teacher uses the short answers to ask follow-up questions (uptake). Teacher: How about if you changed the coil? Did the number of rounds affect the matter (magnitude of electric current)? Student: Yes. Teacher: How did it affect it? Student: Erm... when the coil was smaller the current was a little smaller. Teacher: What if the coil was bigger? More rounds? Student: It was bigger. Finally, the upper-secondary teacher more often used summarising, which means that the teacher repeats or sums up what has been said by the student without correcting it. In this dialogue, summarising seems to be tightly connected to precise valuing, which aims at assigning value on something specific that the student said. This may not be strictly positive (but often is), but it is not an explicit correction. Interestingly, in the lower-secondary network, summarising is not at all dominant and precise valuing is absent. In many ways, the lower-secondary network seems to show a teacher who tries to involve the student by using what the student is saying, but this is not reflected in the student’s participation. Dolin et al. (in press) characterises these kinds of dialogues as struggles, because the teacher tries to engage the student, but the student resists. The upper-secondary network seems to show a teacher who really wants to engage with a particular issue. However, the issue seems beyond the students’ abilities in the particular context. Dolin et al. (in press) characterises dialogues of this sort as difficult content. Nieminen et al. Proceedings of the annual FMSERA symposium 2016 73 Figure 3. The network for the first SAD in upper secondary is shown in the same way as in Figure 2 Peer feedback dialogue Generally speaking, the dialogue between the focus student and the peers was poor in the lower-secondary school because students did not talk without the teacher’s questioning. The feedback group was not able to evaluate the focusstudent dialogue well, and they could not add more details or understandings in relation to the focus-student dialogue. Also, in the upper-secondary school, the feedback dialogue did not progress without the teacher’s questioning, but the teacher did not need to press as much as in the lower-secondary school. The feedback group was able to evaluate the dialogue better and add new information than in the lower secondary.