Effectiveness of a Collaborative Platform for the Mastery of Competencies in the Distance Learning Modality during COVID-19

sustainability Article Effectiveness of a Collaborative Platform for the Mastery of Competencies in the Distance Learning Modality during COVID-19 Sara Domínguez-Lloria 1, Sara Fernández-Aguayo 2, José-Antonio Marín-Marín3 and Myriam Alvariñas-Villaverde 4,*   Citation: Domínguez-Lloria, S.; Fernández-Aguayo, S.; Marín-Marín, J.-A.; Alvariñas-Villaverde, M. Effectiveness of a Collaborative Platform for the Mastery of Competencies in the Distance Learning Modality during COVID-19. Sustainability 2021,13, 5854. https://doi.org/10.3390/su13115854 Academic Editor: David González-Gómez Received: 22 April 2021 Accepted: 21 May 2021 Published: 23 May 2021 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). 1Department of Applied Didactics, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain; [email protected] 2Department of Didactics and School Organization, University of Vigo, 36310 Pontevedra, Spain; [email protected] 3Department of Didactics and School Organization, University of Granada, 18071 Granada, Spain; [email protected] 4Department of Special Didactics, University of Vigo, 36310 Vigo, Spain *Correspondence: [email protected] Abstract: The need to adapt to new learning scenarios due to the impact of COVID-19 on our education system is undeniable. This fact means that we must adapt teaching to non-face-to-face scenarios in order to adapt the teaching processes without the students losing the acquisition of competences. The aim of this study is to evaluate the effectiveness of a digital application for the development of competences of primary education undergraduate students in a non-face-to-face teaching context derived from the pandemic through a service-learning project. To achieve this objective, a specific programme was designed using a collaborative platform. Numerous instruments were used to assess the acquisition of basic, specific, general and transversal competences. On a quantitative level, the Teamworks Skills Questionnaire and an evaluation rubric were used, and on a qualitative level, a student field diary, a survey to measure satisfaction and an observational record were used. The main results show that after the implementation of the use of the platform, both the level of competence of the students and their satisfaction with the project were very high and that the impact on their training was very positive. Among the main conclusions, we highlight that the use of this type of platform allows students to acquire competences in non-classroom contexts. On the other hand, we conclude that training in the use of digital tools in initial teacher training is necessary for good professional performance. On the other hand, this experience allowed future teachers to realise the need to master new technologies in order to be able to adapt to the new needs that different educational scenarios pose in the information society. Keywords: teaching skills; effectiveness; distance learning; COVID-19; collaborative platform; ITC; teacher training; service learning 1. Introduction The global pandemic caused by COVID-19 has changed the strategies that have been traditionally used in teaching-learning processes towards online or hybrid teaching formats in universities around the world. The need to facilitate students’ mastery of the competences described in their respective university degrees has forced teaching staff to use tools that guarantee the acquisition of these competences, which in turn facilitate employability and enable them to respond to the needs they will encounter in the workplace [1,2]. The development of competences in university students is one of the basic objectives of the university, since it is here that highly qualified professionals are trained to solve the problems that society poses to them [ 3 ]. López-Ruiz [ 4 ] states that students’ competencybased learning facilitates the development of both professional and personal skills. Sustainability 2021,13, 5854. https://doi.org/10.3390/su13115854 https://www.mdpi.com/journal/sustainability Sustainability 2021,13, 5854 2 of 14 In the case of teacher training, it is necessary to respond to the new situation by enabling students to acquire competences by increasing their motivation and responsibility in the teaching-learning process. Implementing projects that enable the mastery of competences makes it necessary to create innovative environments that favour the employability of graduates and, therefore, the socioeconomic development of a country [ 5 ]. Villa and Poblete [ 6 ] affirm that training must bring together aspects that are in line with today’s reality: flexibility, mobility and transferability. The need to train in competences puts the student in a position to cooperate, communicate, make decisions, create and innovate. These competences make the teacher a versatile and flexible professional with the capacity for continuous adaptation to the socio-occupational environment [ 7 ]. Along the same lines, García-Peñalvo et al. [ 8 ] affirm that a more participative and collaborative methodological context is necessary to guarantee the acquisition of transversal competences and that these are not affected by non-presence. In addition, the need for adequate digital competence for educators has been critical to avoid disruptions in the teaching-learning processes resulting from the COVID-19 pandemic [9]. Numerous studies [ 10 , 11 ] have addressed this demand for prospective teachers in faculties of education, as well as the demand for active teachers through professional development programmes. COVID-19 has arguably emphasised the importance of teacher professional development for online and blended learning [12]. Therefore, the adaptation to the use of new technologies in the current context is undeniable and a methodological change is needed on the part of the teacher to incorporate the use of new technologies in the classroom [13]. The authors of [ 14 ] state that digital transformation implies changes that allows for new operational and educational models that transform the operations, strategic directions and value of institutions. Janssen and Bodemer [ 15 ] assert that ICTs favour collaborative knowledge construction, where students shape robust processes of meaningful learning through the exchange of information and agreements among peers. Furthermore, we must bear in mind that learning through mobile devices facilitates the construction of knowledge, the resolution of learning problems and the development of diverse skills or abilities in an autonomous and ubiquitous way [ 16 , 17 ], affirming that technological and digital development has become a medium that influences the construction and interpretation of the reality that surrounds us. Apps are intuitive, useful and simple resources for learning. Villalonga and Marta-Lazo [ 18 ] praise applications for mobile devices that facilitate tools, activities and sources of information that are used in everyday life for the purpose of learning. This is what they call “learning”, and they frame these apps in what they call a personal learning environment (PLE). For this reason, we consider, along with the authors of [ 19 ], that a change in the way of teaching that adapts to the new reality is fundamental, and specifically, in order to develop the possibility of establishing digital competence in students, the incorporation of mobile devices in classroom activities is necessary. When technology is combined with an active methodology such as service learning (SL), the advantages for students tend to be greater. SL enables the dual purpose of university education: training for work and training for active citizenship [ 20 ]. Students learn and develop active participation in providing services and solving community problems [21]. We believe that this methodology allows teachers to optimise students’ academic performance, improve their social competences and civic skills, promote their freedom to organise their studies and their responsibility for them and reinforce the social dimension of the university [22–25]. SL brings students, teachers and the community together, so that all become teaching resources and problem solvers. It enhances academic and real-world learning, instilling in students a sense of civic engagement and responsibility [26]. Numerous research studies support the findings that active participation in servicelearning projects clearly enhances learning acquisition in a meaningful and lasting way [27–30]. Sustainability 2021,13, 5854 3 of 14 Bearing these premises in mind, the aim of this study is to evaluate the effectiveness of a digital application for the development of competences, in a context of distance learning due to the pandemic derived from COVID-19, through a service-learning project. Problem Statement Competences linked to the mastery of skills in real environments are vital for the employability of future university students [ 31 ]. As a consequence of the situation resulting from the COVID-19 pandemic, a large part of university students has seen their teaching activity transferred to the non-classroom context. This context is mainly detrimental to the mastery of transversal competences such as adaptability, teamwork or reflective/critical and creative problem solving, which are essential for the training of future teachers. In response to this situation, an experience has been proposed to achieve the mastery of competences with students from the University of Santiago de Compostela in the bachelor’s degree in primary education with a mention in music education. The main objective of the experience is to find out the degree of development of the competences of the subject “Processes and didactic projects in music education” through the use of the digital application MIRO. For this purpose, a research question was established: RQ1: Does a collaborative application enable students to achieve adequate mastery of competences? 2. Materials and Methods The problem under study was approached from an ethnographically oriented case study methodology, from a sociocritical perspective, using mixed procedures. In recent years, the case study has increasingly been used as a mixed method approach, combining both quantitative and qualitative procedures [32,33]. The specific instruments used in the research are described in the programme evaluation. 2.1. Description of the MIRO App For the selection of the application, the characteristics and needs of the programmed activities were taken into account. After a research process, it was decided to select the MIRO application as it had all the necessary characteristics to carry out the project in a fully telematic way, incorporating numerous functionalities. We also considered the importance of students acquiring both digital and collaborative skills that would allow them to join a new normality that will never be the same as in the pre-pandemic era in the classroom, incorporating tools that would allow them to work virtually. Among the most outstanding features, we can comment that this application allowed teachers to moderate unlimited groups. It also made it possible to keep a very detailed record of time and student interaction. The development of the boards made it possible to observe the process of content creation in a step-by-step manner, not one focused on the final product. As a contribution to the students, we observed that it had a very interactive and intuitive interface and a series of built-in tools such as the possibility of adding all kinds of external resources, voting and communication systems within the application itself, such as videoconferencing and chat to carry out their interactions. 2.2. Participants This project was implemented in the first semester of the academic year 2020/21. The participants of the project were 21 students with an average age of 24 years. All students study at the Faculty of Education Sciences at the University of Santiago de Compostela. Overall, 76.2% of the participants were women while 23.8% were men. The collaborative work was carried out for a service-learning project in the subject “Processes and projects in music education”. This project was carried out continuously since 2018, and because of the COVID-19 pandemic, it needed to be restructured to adapt to the needs of social distancing. Sustainability 2021,13, 5854 4 of 14 2.3. Instruments The Teamworks Skills Questionnaire (TSQ) was used to assess the acquisition of transversal and specific skills. This questionnaire has six factors for measuring teamwork skills habits: coordination, decision making, leadership, interpersonal skills, adaptability and communication [ 34 ]. This questionnaire is a tool for indirect measurement of transversal teamwork competencies and has already been used in various contexts [ 34 , 35 ]. Additionally, for the evaluation of competence acquisition, an ad hoc evaluation rubric was implemented that allowed us to assess specific, basic, general and transversal competences. The rubric is made up of 16 items that are related to the competences and each item has 4 achievement indicators for its assessment. Specifically, four items assess the specific competences, five items focus on the assessment of the general competences, four items were developed for the assessment of the basic competences and the remaining three items assess the transversal competences. To determine student satisfaction, an ad hoc survey was carried out to find out their opinion about the experience, perceptions and suggestions [ 36 ], consisting of 16 questions. The first two questions asked the gender and age of the participants, and the remaining questions were divided into six Likert-type questions (1–5), five yes/no questions and three open questions. The acquisition of general and transversal competences was assessed by means of an individual field diary kept by the students. In this diary, each student recorded his/her work process, the difficulties encountered, the way of resolving them, proposals for improvement and a personal assessment of his/her work process. In addition, the teacher recorded the general assessment of the activity in the observational register according to different parameters such as compliance with deadlines, the number of critical incidents occurring during the process and the quality of the work carried out. All of this made it possible to monitor the acquisition of the competences that were to be acquired. Table 1shows the relationship between the instruments used, their objective and the proportion they represented in the students’ marks. Table 1. Data collection instruments. Instrument Objective Weighting Evaluation Rubric To assess the acquisition of specific, basic, general and transversal competences. 60% Student field diary To assess the acquisition of general and transversal competences. 25% Observational register Recording critical incidents. 0% Teamworks Skills Questionnaire Determine the degree of improvement in the acquisition of specific and transversal competences of students. 15% Satisfaction Survey To find out the level of student satisfaction with the programme. 0% 2.4. Procedure Before data collection, the research process was explained to the students, and they all signed the informed consent form. The ethical standards recognised by the Declaration of Helsinki (Hong Kong revision, September 1989) and in accordance with current Spanish legal regulations governing research were followed in the development of this research. Data analysis was performed using the SPSS 25.0 statistical package (IBM Corp., Armonk, NY, USA) and Microsoft Excel 2019. To determine whether there are differences between the results before and after the programme, the Wilcoxon non-parametric test was used given the size of the sample. To establish the different categories of analysis, the recommendations of Krippendorf [ 37 ] were followed, starting with the determination of the units of analysis, the reduction of the data, verification of the representativeness and analysis of the data to Sustainability 2021,13, 5854 5 of 14 establish common patterns among the results provided by all the instruments. Therefore, the categories were designed a posteriori on the basis of the results found. The research was carried out in four stages, which are described below: Stage 1. In the first week, a multidisciplinary meeting was held in order to create the working groups made up of students. A total of 7 working groups were organised, each consisting of 3 members. The aim was for all the students to interact in a multidisciplinary way, producing music didactic videos in collaboration with the orchestra of the University of Santiago de Compostela. The aim of the videos was to produce music education didactic material in an open access audio-visual format for the educational community. All the videos included a document of complementary activities that could be downloaded to complete the educational experience. All students were presented with the evaluation rubric available on the virtual campus, which determined the detailed grading. This rubric accounted for 60% of the final mark. Stage 2. In the second week, they were provided with a list of composers, the videos recorded by the university orchestra and the didactic contents to be developed. The different groups had the possibility of selecting two of them. In order to carry out the project, a planning schedule was established with a timetable showing the scope of the work, the dates on which it was to be delivered, the collaborative tool to be used and the teaching schedule prior to delivery. The timetable, phases and activities are detailed in Figure 1. Sustainability 2021, 13, x FOR PEER REVIEW 7 of 15 Figure 1. Timeline, tasks and phases of work. Training in the use and handling of the MIRO tool was carried out through video tutorials made ad hoc, which were available to students on the virtual campus of the subject. Each phase was presented with a video tutorial. The activities were designed in such a way that it is not possible to move on to the next phase without elaborating on the previous one. A collaborative panel was developed in the same way for all groups and phase 1 was developed so that they could see the possibilities of the platform. For the following phases, only the task was provided, and it was the responsibility of each group to elaborate the panel for its development. Each phase requires a higher level of collaboration from the team in order to be carried out correctly. In phase 1 the work is individual; each member of the group does the same search for information and puts it on the board. From this activity onwards, the degree of collaboration gradually increases. The activities in each phase are described in Table 3 below. Table 3. Programme. Stage 1: Research and Fieldwork Description Activity 1: Research. Each student has to provide information in the corresponding panel on four of the following statements according to the piece to be analysed: What do I know before I start? Composer Family of instruments Major mode—minor mode Type of rhythm Basso continuo Concerto Grosso Figure 1. Timeline, tasks and phases of work. Stage 3. Between the third and twelfth week, the collaborative work tasks began, the development of which was supported by the collaborative platform MIRO, to facilitate the sharing of information, the elaboration of didactic content and decision-making skills. Stage 4. In the thirteenth week, the marks were published and the group was given the option of checking the errors they obtained in each section of the evaluation rubric available from the beginning of the activity. Finally, in the final week, the oral and public presentation of the project took place. For this, the students had between 10 and 15 min for their presentation. Sustainability 2021,13, 5854 6 of 14 2.5. Programme The main objective of the research experience was the development of the competences of the subject “Teaching processes and projects in music education” through the use of the digital application MIRO. In addition, specific, general and basic competences of the teaching degree were included, as can be seen in Table 2. Table 2. Relationship between objectives and competences. Type of Competence Competences Objective Specific S53. Develop and assess curriculum content using appropriate teaching resources and promote the corresponding competences in learners. S54. Understand the principles that contribute to cultural, personal and social education through the arts. S55. Know the school curriculum of arts education, in its plastic, audio-visual and musical aspects. S56. Acquire resources to promote lifelong participation in musical and plastic activities inside and outside school. Implement curricular content in the area of musical expression didactics. General G2. Design, plan and evaluate teaching and learning processes, both individually and in collaboration with other teachers and professionals at the centre. G5. Promote coexistence in the classroom and outside it, solve problems of discipline and contribute to the peaceful resolution of conflicts. Encourage and value effort, perseverance and personal discipline in students. G11. Know and apply information and communication technologies in the classroom. Selectively discern audio-visual information that contributes to learning, civic education and cultural wealth. Design and implement service-learning projects as future teachers. Basic B.4 Students are able to convey information, ideas, problems and solutions to both specialist and non-specialist audiences. B.5 Students develop those learning skills necessary to undertake further study with a high degree of autonomy. Encourage student autonomy in problem solving, as well as collaborative work. Transversal T.1 Teamwork and autonomy in problem solving. T.2 Handling educational and communication technologies. T.3 Instrumental knowledge of information and communication technologies. Encourage multidisciplinary work with the relation of contents from other areas. In order to achieve these goals, different activities common to the whole subject were carried out with the aim of developing a social project through the service-learning methodology and the use of different technologies such as the collaborative platform MIRO. The programme was developed in four phases. Figure 1shows what was worked on in each phase and the timeframe. Training in the use and handling of the MIRO tool was carried out through video tutorials made ad hoc, which were available to students on the virtual campus of the subject. Each phase was presented with a video tutorial. The activities were designed in such a way that it is not possible to move on to the next phase without elaborating on the previous one. A collaborative panel was developed in the same way for all groups and phase 1 was developed so that they could see the possibilities of the platform. For the following phases, only the task was provided, and it was the responsibility of each group to elaborate the panel for its development. Each phase requires a higher level of collaboration from the team in order to be carried out correctly. In phase 1 the work is individual; each member of the group does the same search for information and puts it on the board. From this activity onwards, the degree of collaboration gradually increases. The activities in each phase are described in Table 3below. Sustainability 2021,13, 5854 7 of 14 Table 3. Programme. Stage 1: Research and Fieldwork Description Activity 1: Research. Each student has to provide information in the corresponding panel on four of the following statements according to the piece to be analysed: What do I know before I start? Composer Family of instruments Major mode—minor mode Type of rhythm Basso continuo Concerto Grosso Activity 2: Organise: Each student organises all the information collected by all the members of the group on the corresponding panel. Timing 1 week Degree Collaboration None. They contain information, but each student works individually. Stage 2: Brainstorming and Selection of Ideas Description In this activity they choose the didactic resource they are going to elaborate. To do so, they have to create different panels to reflect the following tasks. All of them have to be developed in the collaborative app: Sharing/brainstorming to choose what to do as a didactic re-course. Selection process and tools used to do it. Selected idea. An area where to put new information about the piece if the need arises during the process. Timing 1 week Degree Collaboration It increases. Through the app they have to brainstorm ideas and choose one. It is mandatory that everyone participates to the same extent. Stage 3: Designing Creative Learning Resources Description In this stage, the didactic resource is designed and the work schedule is drawn up. Each group prepares the necessary panels to reflect the necessary tasks in the general panel. The minimum tasks to be reflected are: Design of the didactic resource. Work schedule for the design: times and workload of each one. Timing 1 week Degree Collaboration Augment. The team has to be able to agree on different ideas and complex processes through the tools offered by the collaborative app (chat, voting, comments, etc.). Stage 4: Creation of Creative Learning Resources Description In the last stage, the group elaborates the didactic resource designed together with the orchestra. In the collaborative panel they should reflect, using the most appropriate panels, on the process of creating the resources: how it has been done, difficulties, solutions, changes, etc. It is important to point out that this is an online resource, so tasks that need to be printed out should be avoided. Timing 2 weeks Degree Collaboration Augment. The group has to solve problems, make decisions and elaborate the didactic resource in a collaborative way without a division of tasks. 2.6. Evaluation Techniques and Instruments The evaluation carried out was structured according to [ 38 ] and was divided into initial, process and final stages. Sustainability 2021,13, 5854 8 of 14 The initial evaluation allowed us to know the initial state of the group and was carried out before the implementation of the programme. The process evaluation was carried out during the programme and the information gathered allowed us to make decisions during the process. The final evaluation was carried out at the end of the programme and allowed us to know if the objectives of the programme were achieved. Table 4details the instruments used in each type of evaluation. Table 4. Types and instruments of assessment. Evaluation Instrument Who Does It Initial Teamworks Skills Questionnaire Students Process Field diary Students Observational Register Teacher Final Evaluation Rubric Teacher Teamworks Skills Questionnaire Students Satisfaction Survey Students In addition, three types of monitoring activities were carried out to monitor each group’s programme and recorded in the field diary: 1. Presentation of the activities of each phase within the stipulated time. 2. In the interactive classes via videoconference, students could consult any doubts, problems and incidents they considered appropriate. The teacher advised them and gave them the necessary feedback so that they could continue to develop the project and consolidate their learning. 3. Weekly classes by videoconference or blended learning. On a weekly basis, in the interactive classroom classes, the difficulties of group work, joint organisation and the preparation and presentation of the final project were worked on. 3. Results The main results are grouped into categories of analysis below for ease of interpretation. 3.1. Level of Mastery of Competences The data obtained reveal an improvement in the mastery of the competences, obtaining an average mark of 8.83 out of 10. This mark was obtained from the average of three instruments describe in Table 5. In the evaluation rubric, the average mark for the group was 8.5 (10), while the averages for the project resolution rubric and the student’s field diary both reached 9 (10). Table 5. Average, minimum and maximum marks showing the level of improvement in competences. Instruments Average Mark Min. Mark Max. Mark Evaluation Rubric 8.5 6.5 9.5 Student field diary 9 8.5 9.5 Total 8.75 (Notable) The level of competences acquired by the students was very high in two tests: the evaluation rubric and the field diary. The tests carried out to measure these competences were the evaluation rubric with an average mark of x = 8.5 (min 6.5 and max: 9.5) and the students’ field diary with an average mark of x = 9 (min 8.5 and max: 9.5). In order to check the effectiveness of the use of the MIRO application for the mastery of transversal competences, we carried out an analysis of related samples. As we can see, although there were no significant differences, the means in the post-test for all the competences assessed were higher. In fact, the minimum score was higher in all skills except in personal skills related to listening, respect and tolerance, empathy, leadership and problem solving (Table 6). However, only decision making was significant ( p-value = 0.002 ). Sustainability 2021,13, 5854 9 of 14 The reason why the difference is not significant may be because the sample was very low and the level of competence of the students was already very high before starting the experience. Table 6. Wilcoxon test, N= 21. Competences Average Pre Average Post Min Pre Min Post Max Pre Max Post Sig Adaptability 15.22 15.88 9.00 13 20 20 0.726 Coordination 14.11 15.33 9.00 12 20 20 0.514 Decisions 18.00 21.73 10.00 16 23 24 0.002 Leadership 17.11 19.44 9.00 13 26 27 0.342 Skills 22.55 21.77 18.00 16 24 24 0.588 Communication 23.44 24.55 14.00 20 28 28 0.674 3.2. Student Satisfaction The results obtained in the satisfaction survey are described below, grouped according to the type of answer. In total, 100% of the students stated that the project allowed them to work collaboratively and that the app facilitated group collaboration and improved their digital competence. Furthermore, the whole group said that after this project they realised the need and importance of training in learning and knowledge technology (LKT) tools, both on a personal level and as part of teacher training. Satisfaction with the programme was very high. With regard to the Likert-type responses, six questions were asked with answers ranging from 1 to 5, with 1 being negative or very little and 5 being positive or very much. Figure 2shows the data obtained. The overall assessment of the project, as can be seen in question 1, was positive for 100% of the students. As for the information and explanations received to be able to carry out the project, 78% of the participants indicated very positive, and the remaining 22% positive, as can be seen in question 2. In question 3, the app used for collaborative development was rated as very positive by 78% and positive by 22%, while the remaining 22% were in between. Question 5 shows that 78% of the participants felt very good and comfortable doing the project and the remaining 22% said that they felt good. Finally, question 6 shows that 55.5% of the students say that the app facilitated collaborative work, while 44.5% of the sample gave a neutral answer.