Triggering, developing and internalising teamworking skills in neuro-typical and neuro-atypical students with a computer orchestrated group learning environment: a multi case study

50th Annual Conference in September 2022 499 Manish Malik and Julie-Ann SimeTriggering and internalising teamworking skills in neuro-typical and neuro-atypical students with a computer orchestrated group learning environment: A multi case study - 10.5821/conference-9788412322262.1163 doi:10.5821/conference-9788412322262.1163 TRIGGERING, DEVELOPING AND INTERNALISING TEAMWORKING SKILLS IN NEURO-TYPICAL AND NEURO-ATYPICAL STUDENTS WITH A COMPUTER ORCHESTRATED GROUP LEARNING ENVIRONMENT: A MULTI CASE STUDY M. Malik1 University of Portsmouth Portsmouth, U.K. J. A. Sime Lancaster University Lancaster, U.K. Conference Key Areas: Diversity and inclusiveness and E-learning Keywords: Neurologically atypical, ASD, ADHD, Collaborative learning, Computer supported collaboration (CSCL) at computer ABSTRACT Project-based learning and flipped classroom approaches are often used for developing team working skills in graduates. However, many engineering schools face efficiency and effectiveness challenges when it comes to facilitating students in these settings. For neuro-atypical (NAT) students, such as those with Attention Deficit Hyperactivity Disorder (ADHD) or Autism, support for developing teamworking skills can be limited. Even neuro-typical (NT) students find teamwork challenging and can benefit from an intervention that supports development of such skills. Self, Co and Shared regulation skills are considered important for effective team working. Regulation is a multi-staged process, which includes goal setting, planning, doing, monitoring and evaluating own and a team's work. Research on use of computer scripts to successfully orchestrate the multiple stages at a shared level shows only partial success. Many Computer Supported and Collaborative Learning studies cite over-scripting as a common criticism related to orchestration of shared regulation and team work. This work investigates "How computer orchestration scripts affect the triggering and internalisation of Self, Co and Social regulation skills in NT and NAT students when using a Computer Orchestrated Group Learning Environment (COGLE)?". COGLE was used with first year neurotypical and neuro-atypical engineering students to study its impact on triggering existing and/or internalising new regulation scripts in team working. Qualitative data from two literal replication cases were analysed. This work shows how different types of scripts in COGLE helped trigger, develop and internalise regulation skills and highlights areas where more work is needed. 1 Corresponding Author , M. Malik, [email protected] 50th Annual Conference in September 2022 500 1 INTRODUCTION 1.1 Increase in collaborative approaches within Engineering Education Collaboration skills are considered important within most professions. In engineering, team working skills are one of many crucial skills for the success of any real-world open-ended project. Higher engineering education curricula, in the UK and elsewhere, have seen recent changes in response to the calls from industry and professional bodies to increase focus on problem solving and team working and the use of collaborative approaches to develop these graduate skills [1-3,8-9]. Learning in groups within engineering has been shown to have greater achievement, with a moderate effect size (d=0.25), than learning individually [4]. As a result, collaborative approaches such as Flipped Classroom (FC) and Project Based Learning (PjBL) have gained popularity over the years within Engineering Education [5-6]. In FC, students are expected to prepare the content themselves and take part within collaborative active learning activities in class [5, 7]. In PjBL, students work collaboratively in a group, on a defined real-world project and are expected to learn what is needed to deliver the project. The popularity of such approaches has meant that there has been a shift in the expected role of the academics from being a ‘sage on the stage’ disseminating information to a more supporting ‘guide on the side’ role [6]. There are, however, many challenges that come with this shift in expectations and practice. 1.2 Challenges collaborative approaches bring to Engineering Education Without theoretical knowledge that underpins the change in their role and the skills needed by staff in facilitating groups, staff often face challenges when adopting PjBL or FC approach [6-7]. Not just staff, the students also find it hard to develop and use interpersonal skills if they have not had and do not have access to any formal team skills training [6-7]. Furthermore, PjBL is resource intensive and requires more staff to facilitate teams and to put in extra effort in guiding students. This is particularly important where students are from multicultural backgrounds [6]. As PjBL needs more resources from schools or departments, it can lead to a reduction in institutional support for such approaches over time [6]. This puts any benefits of the PjBL approach, where realisable, at risk. Likewise, students find it time consuming and hard to prepare on their own in the FC approach. Due to limited student preparation, staff find it tricky to carry out their planned collaborative activities in a way that is fair and benefits the entire class [10]. In practice, staff often find themselves performing both the roles of teacher and facilitator, especially when students complain about not being able to learn or work on their own on open-ended real-world problems without being taught first and a similar criticism is faced by flipped classroom approach [6,10]. 1.3 Context and Rationale More and more NAT are entering higher education as diagnosis rates are on the rise. NAT is an umbrella term that includes (ADHD and Autism Spectrum Disorder 50th Annual Conference in September 2022 501 (ASD) students. Researchers have claimed that NAT students are more likely to study engineering and technology courses [23]. As a result, it is very likely that most engineering cohorts will have a mix of NT and NAT students learning together. Formally diagnosed, or not, these students face socio-communication challenges that may affect them in realising their full potential [23]. Social interactions and communication are crucial for success within PjBL and FC activities, making it harder for NAT students to succeed. In addition, reasonable adjustments are needed by law and teaching NT and NAT students alongside each other can pose problems [24]. In practice, the demands of PjBL and FC can put most students under pressure, including NT, diagnosed NAT and those that remain undiagnosed [25]. Therefore, the challenges faced in social interactions and communication within group work can limit the success of FC and PjBL for NT as well as NAT students. Both FC and Project based learning are used within the School of Energy and Electronic Engineering at the University of Portsmouth. Here, teams of students may need to collaboratively design, build and test say a communication system as part of a PjBL project or collaboratively discuss design challenges related to electronic subsystems within FC activities. Disability data from Higher Education Statistics Agency show 0.9% (male), 0.2% (female) & 3.6% (other) current UK domiciled students have ASD or other social communication disorder [24]. Research suggests females can be better at masking social behaviours common in NAT and are not being diagnosed as a result [23]-[24]. As NAT students are more likely to study engineering [23], supporting the already small % of female students entering engineering degrees becomes even more important. In the last three years alone there has been an increase in students with specific learning disabilities (including ADHD) and ASD to 15.7% and 1.2% respectively within the school. This increasing trend in NAT students joining our courses, combined with potential benefits to NT students from an intervention that can cater to both NT and NAT students learning together, provides the rationale for this study. COGLE may provide an approach to teaching that is inclusive for students with socio-communication challenges and enhancing their learning and team working experience. 2 METHODOLOGY 2.1 The intervention: Computer Orchestrated Group Learning Environment COGLE was developed within the School of Energy and Electronic Engineering at the University of Portsmouth. It orchestrates small face-to-face groups of students watching together short videos followed by orchestrating them practice related questions till they all achieve group-wide mastery, i.e. they can all show that they understand the content by answering questions. During each session students are paired several times to carry out peer instruction with those who get the questions wrong. COGLE runs within a web browser with videos hosted anywhere on the web, for example YouTube®. The questions are carefully designed to encourage discussions between students. To achieve group-wide mastery all students in a team 50th Annual Conference in September 2022 502 have to get 10 questions correct in a row. Anyone making a mistake resets the count to zero and the team must continue. COGLE plays a remedial video based on the most prevalent mistake in the group. Working iteratively to build understanding, in a constructivist way, towards a shared goal of GWM is seen as crucial in triggering and achieving SRL, CoRL, and SSRL and team effectiveness. Triggering and internalising regulation, particularly SSRL skills, through orchestrating a simple goal of achieving GWM has not been studied before. 2.2 Underpinning pedagogical theories Zimmerman’s SRL theory and its extension into the social in the form of CoRL and more recently SSRL, explain how shared goals, planning, monitoring and reflection can make individuals become more effective in regulating collaboration at different levels [11]. The repeated orchestration through textual scripts in COGLE has the potential to aid the triggering, development and internalisation of regulation skills, as predicted by Script Guidance Theory [25]. This theory posits that a target behaviour, such as SSRL, can be modelled in individuals by scripting it repeatedly, causing internalisation of an appropriated version of the external script [25]-[26]. Scripts can orchestrate and model collaborative behaviour and are therefore used to “Enhance the probability that knowledge generative interactions such as conflict resolution, explanation or mutual regulation occur during the collaboration process” [21, p. 1]. Another theory that underpins the success of scripting regulation skills is the SelfDetermination Theory of motivation [27]. It posits that learners become selfmotivated in a learning environment that makes them: more self-confident by making them more competent; more part of a team by enhancing their relatedness to their teammates; but also, more autonomous as they take control of the tasks themselves and internalise or remember the triggered scripts. SSRL is seen as the highest level of regulation of collaboration [11]. 2.3 Research site, participants and research design Two Electronic Engineering modules, one at level 3 (first year of foundation degree) and one at level 4 (first year of undergraduate degree), were chosen where FC and PjBL activities were being used respectively. This formed the basis for students from level 3 to use COGLE to prepare for FC activity and level 4 students to prepare for PjBL task. The module co-ordinator confirmed observing similar challenges as noted above in both the modules. The two modules also benefit from covering the same basic material in the early parts of the course to ensure students are introduced to basics of electronics before moving on to advanced topics in level 4 module. In order to investigate the potential of COGLE in supporting NT and NAT students together within a real-world setting participants were invited from these two modules to join the study. 50th Annual Conference in September 2022 503 The study was approved by the University’s research ethics committee and students from the two cohorts were invited, in particular those with ASD and ADHD, and give informed consent for taking part in this study. This was the only way we were allowed to recruit students as access to entire cohort was not permitted by the ethics committee. A total of 19 students joined the study, 9 from level 3 foundation course in Engineering and Technology and 10 from the BEng Electronic Engineering course. Three foundation students did not complete and left the study mid-way as one of their teammates, an international student, left without giving any explanation leaving a group of 2 who did not wish to continue. The remaining 6 level 3 students were put in teams of 3 each randomly and were put into the FC case-study. One level 3 student self-declared as autistic based on a diagnosis in their immediate family. Low diagnosis rates are a common issue and self-identification in this way can be helpful. The remaining 10 level 4 students joined the PjBL case-study and were put in random teams of 3 or 4 students. In level 4 one student was diagnosed with Autism (comorbid with ADHD) and one student had ADHD confirmed via consent by the disability support unit in the university. 16 students completed the two case studies. Students on the FC case-study learned together for 4, two-hour sessions in their designated teams prior to attempting a two-hour FC collaborative design challenge in the same teams. Students on the PjBL case-study, did the same 4 two hour sessions and in addition 3 further two hour sessions (7 sessions in total) to master the content needed for the PjBL task in their designated teams. The FC task involved designing a bass and treble filter circuit to be used as input to a head-phone amplifier in a twohour Student Orchestrated Working Together (SOWT) session. The PjBL task was more complex where the students were required to design both the filter circuits as well as the headphone amplifier within a two-hour SOWT session. The content used in the FC case-study was also used in the PjBL case-study in addition to more content on advanced topics as needed for the PjBL project. The two case-studies thus formed part of a literal replication multi-case study design used here [28]. The use of orchestrated group-wide mastery in two different case-studies should generate robust evidence advocating for using such an approach with NT and NAT students within Engineering Education settings to enhance their knowledge and internalise team working scripts used within COGLE. 2.4 Success and challenges in scripting regulation skills Self-Regulated Learning (SRL) involves planning, monitoring and reflecting on your own actions to achieve individual goals against a set standard [11], [12]. Scripting of self and co-regulation of learning (CoRL), which involves regulating actions of some other teammates, has helped learners improve their group-awareness, sociometacognitive and socio-emotional regulation skills [12]-[17]. For developing SRL and CoRL skills scripting information sharing, argumentation and negotiations has been effective, as defined in the collaborative framework by [18]. Very few studies, however, have investigated the transfer of these scripted skills into un-scripted 50th Annual Conference in September 2022 504 environments. Transfer has been more successful where adaptable scripts are used [19] or where students practice the scripted interactions multiple times [16] and where there was support to help students in challenging situations [16]. SRL, CoRL and SSRL skills are considered as key to team effectiveness [11]. In most studies CoRL by others is shown to help develop SRL and self-efficacy [20]-[22]. Dependencies of later stages of regulation on earlier stages in SSRL and the diversity in learner knowledge and skills, makes it very hard for Socially Shared Regulation of Learning (SSRL) to be successful when each of these stages is scripted separately [13]-[14] and scripting SSRL may easily be perceived by learners as over-scripting [20]. Although some success in triggering, developing and internalising Self and Co regulation has been reported, successful scripting of SSRL in teams has been limited [13]-[14], [16] 2.5 Methodology for evaluation COGLE plays a role is the development of team working skills of these students. After each session the students completed a Daily Events Survey (DES) with free text space to answer the following questions. 1. Describe what did and did not go well, when prompted by the system, to teach or explain to other student(s) a concept. a. Also, describe how the system made you or others in your team contributed to improving or worsening the situations you described as “did" or "did not go well” above respectively. 2. Describe what did and did not go well, when prompted by the system, to learn from other student(s), a concept. a. Also, describe how the system made you or others in your team contributed to improving or worsening the situations you described as “did" or "did not go well” above respectively. In addition, qualitative data was collected through an hour-long interview for NT students and up to two hours for NAT students. The interviews were transcribed verbatim by a professional transcriber. This paper presents some key themes from the qualitative data from the DES that was analysed using grounded theory inspired thematic analysis [29]. A full analysis from all sources will be presented in a subsequent research article. 3 RESULTS 3.1 FC Case The daily events survey was completed by all 6 students after each of the 4 sessions and the SOWT session. Then using grounded theory inspired thematic analysis, the themes shown in Table 1, emerged in the majority of the responses from a majority of the sessions. 50th Annual Conference in September 2022 505 Table 1. FC case: Themes from the DES Theme/ Sub-theme Description Sample quotes Orchestrates GWM and CoRL GWM script paired teammates who taught each other. “[Pairing] was a bit weird as explaining a topic [at first but it]… went well. [Pairing] allowed us to discuss the topic.” Teammates can takeover pairing. GWM linked penalty caused positive frustration, that triggered overtaking COGLE’s pairing. “We all helped to explain and understand concepts [on our own] when it was required [to achieve GWM].” Co/SSRL practiced and internalised A simple goal of GWM made them switch between working under their shared scripts and COGLE scripts. “I think [today] it was more of a team effort rather than the system that helped us progress this time.” Shared goal and working together Aimed to finish the SOWT task by working together. “We were very quickly able to come up with suitable numbers and circuits between us, …we did struggle… however we worked well as a team through the issue” Team effectiveness and satisfaction Felt teammates worked together “We combined all our ideas together to get it all right.” Themes in Table 1, show that the repeated computer orchestration of pairing, although initially awkward, soon enabled the externalisation of knowledge between teammates in order to achieve the GWM goal. The pairing script at first triggered CoRL in teammates, and later the students took over the pairing themselves, showing early internalisation of CoRL. The GWM script triggered shared working, monitoring and reviewing as teammates wanted to complete mastery without having to do many more questions. There was no need to script these phases with separate scripts. Students were able to develop and internalise both CoRL and SSRL skills within just 4 sessions. The scripts were appropriated, internalised and also used in the SOWT sessions even when the COGLE scripting was removed thus showing transfer. The teams were able to complete the set task in the FC without any help from any staff effectively and efficiently. They were also satisfied with their solutions. The presence of a NAT student in a team of NT students did not cause any issues to either any NT or NAT student. Thus, showing the inclusiveness of COGLE. 3.2 PjBL case Likewise, the daily events survey was completed by all 10 students after each of the 7 sessions and the SOWT. Then using grounded theory inspired thematic analysis, similar themes emerged in the majority of the responses from a majority of the sessions. 50th Annual Conference in September 2022 506 Themes in Table 2, show that the repeated computer orchestration of pairing, helped externalisation of knowledge needed to achieve the GWM goal. Here too, CoRL was triggered by script at first and later taken over by the students, showing early internalisation of CoRL. The GWM script triggered shared working, monitoring and reviewing as teammates wanted to complete mastery without having to do many more questions. There was no need to script these phases with separate scripts. Students were able to develop and internalise both CoRL and SSRL skills within just 4 sessions. The scripts were appropriated as they were internalised and were used in the SOWT sessions even when the COGLE scripting was removed thus showing transfer. The teams were able to complete the set task in the PjBL without any help from any staff effectively and efficiently. They were also satisfied with their solutions. The presence of a NAT student in a team of NT students did not cause any issues to either any NT or NAT student. Thus, showing the inclusiveness of COGLE. Table 2. PjBL case: Themes from DES Theme/ Sub-theme Description Sample quotes Orchestrates GWM and CoRL GWM script paired teammates who taught each other. “The system prompted [collaboration] at correct times which made the session go well.” Teammates can takeover pairing. GWM linked penalty caused positive frustration, that triggered overtaking COGLE’s pairing. “Sometimes it was difficult for other students to explain a topic [when picked by the system] but we would work together to help explain that topic” Co/SSRL practiced and internalized As simple goal of GWM made them switch between working under their shared scripts and COGLE scripts. “If the [chosen] explainer wasn't explaining well, then the other person would chip into try and help, usually it worked.” Shared goal and working together Aimed to finish the SOWT task by working together. “We managed to get a lot done… we worked efficiently and as team. Choosing resistor values cause a bit of a debate but we overcame this by agreeing to one value” Team effectiveness and satisfaction Felt teammates worked together “We produced a reasonable design, and was mostly happy with the outcome”. 4 Discussion and implications for engineering educators The two cases provide repeating results adding strength to the claim that COGLE’s simple GWM goal alongside flexibility in the pairing script is able to support a mix of NT and NAT students where both shared and practiced regulation and team working skills before demonstrating effective teamworking when orchestration was removed altogether. The flexibility to rely on scripts during learning sessions as and when needed or to takeover regulation themselves, helped students practice and gain confidence in regulation skills. It also allowed those with existing skills or those becoming more confident to takeover regulation from COGLE and practice it themselves. By externalising their scripts, students shared their skills with the others in the team. The frustration due to the penalty in the GWM script, whereby the count leading up to the target of answering 10 consecutive questions was reset if even one 50th Annual Conference in September 2022 507 of the teammates made a mistake, helped trigger this takeover and the shared monitoring, working and reviewing took place between teammates. It also meant that when a shared approach was successful, students would use it in future too. COGLE encouraged being supportive of each other but also supported learners when needed. Regulation changed hands between COGLE and teammates and back until the teammates were ready themselves. Due to changes in teaching approaches during the pandemic, many staff now use chunks of video and quizzes as normal teaching material. Although orchestrating the use of these videos and quizzes still present a challenge. As higher education settles into its new normal post pandemic, orchestration systems such as COGLE offer new ways in managing classroom interactions and activities, in addition to promoting the triggering, development and internalisation of regulation skills crucial to team working. 5 SUMMARY AND ACKNOWLEDGEMENTS This work presented the evaluation of Self, Co and Shared regulation and team working skills in collaborative settings such as FC and PjBL after using a novel group learning environment, COGLE. It shows how both NT and NAT students were motivated to come together as a team within a computer orchestrated environment which used a GWM script to prepare them for collaborative activities in a SOWT session later on. They completed the FC and PjBL SOWT activity that followed successfully without being orchestrated by any computer scripts, indicating successful transfer of Self, Co and Shared regulation skills. Furthermore, COGLE was able to support NAT students in overcoming socio-communication challenges in an inclusive manner whilst learning alongside NT students. The use of COGLE frees staff from having to teach instead of focussing on guiding students during PjBL and FC activities, making it more efficient and effective. The post pandemic classroom needs more innovative solutions like COGLE to successfully transform engineering education from being more lecture based to more interactive, skills focussed and student centric. COGLE prepares students before they set foot on collaborative tasks that require both, team working skills and mastery of technical knowledge, the two most important pillars in engineering education. The two cases show the transferability of the intervention design for mixed groups of NT and NAT students in terms of triggering, developing and internalising regulation skill and effective team working skills. This work benefited from the dean’s fund for supporting research within the school. The authors are grateful for the continued support for this project as we investigate further collaborative learning within engineering education with bigger cohort size in order to help with the transferability of the results further.