Digital rubric-based assessment of oral presentation competence with technological resources for preservice teachers

21 ESTUDIOS SOBRE EDUCACIÓN / 2022 Digital rubric-based assessment of oral presentation competence with technological resources for preservice teachers Evaluación basada en rúbricas digitales de la competencia de presentación oral con recursos tecnológicos para profesorado en formación ANA BELÉN PÉREZ-TORREGROSA Departamento de Didáctica y Organización Escolar Facultad de Ciencias de la Educación Universidad de Málaga Campus de Teatinos, 29010, Málaga (España) [email protected] https://orcid.org/0000-0002-3428-6348 MARÍA JESÚS GALLEGO-ARRUFAT Departamento de Didáctica y Organización Escolar Facultad de Ciencias de la Educación Universidad de Granada Campus Universitario de Cartuja, 18071, Granada (España) [email protected] https://orcid.org/0000-0002-2296-5431 MANUEL CEBRIÁN-DE-LA-SERNA Departamento de Didáctica y Organización Escolar Facultad de Ciencias de la Educación Universidad de Málaga Campus de Teatinos, 29010, Málaga (España) [email protected] https://orcid.org/0000-0002-0246-7398 ISSN: 1578-7001 / DOI: 10.15581/004.43.009 Resumen: Este estudio está dirigido a la e-evaluación de la competencia de presentación oral utilizando recursos tecnológicos en un modelo activo que combina aprendizaje basado en proyectos y aula invertida. Este estudio aplica el uso de una rúbrica digital para la evaluación de la competencia en presentación oral en diferentes situaciones de evaluación activa y progresiva de 99 futuros docentes en las que la participación es opcional u obligatoria. Los resultados muestran que la rúbrica digital empleada en varios momentos es una metodología y una tecnología que facilita el proceso de retroalimentación y diálogo entre docentes y estudiantes sobre los criterios de evaluación. Los resultados apoyan futuras decisiones de diseño metodológico de evaluación formativa apropiadas en entornos de aprendizaje online. Palabras clave: Rúbrica, Competencia de Presentación Oral, Evaluación, Futuros Docentes. PÉREZ-TORREGROSA / GALLEGO-ARRUFAT / CEBRIÁN-DE-LA-SERNA 22 ESTUDIOS SOBRE EDUCACIÓN / 2022 Abstract: This study focuses on e-assessment of oral presentation competence using technology resources in a model that combines project-based learning and fl ipped learning. This study uses a digital rubric to assess oral presentation competence in different situations of progressive assessment for 99 preservice teachers, situations in which participation was either optional or compulsory. Findings show that the digital rubric used at various times is a methodology and a technology that facilitates the feedback process and dialogue between teachers and students about the assessment criteria. The results support future decisions for methodological design of formative assessment appropriate to online learning environments. Keywords: Rubric, Oral presentation competence, Assessment, Preservice teachers. INTRODUCTION D eveloping oral presentation competence is an essential goal of higher education, as this competence is crucial for future personal and professional success (Dunbar et al., 2006; Morreale et al., 2017; van Ginkel et al., 2015). Oral presentation competence is recognized as a central professional competence (Campbell et al., 2001). Studying oral presentation competence is important because it helps to develop and improve skills that are necessary for many aspects of professional careers. The concept of oral presentation competence is grounded in the interrelation of the cognitive, behavioral, and affective domains (Bower et al., 2011; De Grez et al., 2009b; van Ginkel et al., 2017). These three domains are brought to bear on speaking in public in order to inform, express oneself, and connect with others in a reformulation of “competence-based teacher training” (and its goal of training in specifi c teaching skills) (Mulder, 2014). Oral presentation competence includes appropriate content, design, and organization of the presentation, as well as presentation skills (Ritchie, 2016). These must be developed in conjunction with digital competence (Redecker, 2017). We cannot separate progress in acquiring and improving oral and digital skills because the two together form the core of the teaching profession. The teaching profession places great emphasis on the importance of oral presentation skills, although few studies in education research focus on improving preservice teachers’ oral presentation competence by assessing it. Technology development has clearly changed both the learning environment and emphasis on use of support resources, linking oral presentation and digital competences more closely, while the Covid-19 pandemic has made it necessary to use these competences in online environments. Peer and teacher assessment The expansion of learning for undergraduates has generated studies of continuous and progressive assessment with digital rubrics in engineering, medicine, DIGITAL RUBRIC-BASED ASSESSMENT OF ORAL PRESENTATION COMPETENCE 23 ESTUDIOS SOBRE EDUCACIÓN / 2022 anthropology, education, and mathematics (among other subjects) (Cebrián-dela-Serna and Bergman 2014; Crawford et al., 2020; Haanstad, 2020; Houston and Thompson, 2017; Zheng et al., 2019). Such assessment is especially important in the preservice training of preservice teachers, due to the profession’s exercise of the double role of assessor and assessed and their evolution (Pathak and Le Vasan, 2015), as well as the importance of oral presentation competence (Mulder, 2014; Pathak and Le Vasan, 2015; van Ginkel et al., 2017). The digital rubric has a specifi c methodology for competence-based assessment and is a tool for such assessment that facilitates the feedback process and dialogue on the assessment criteria, converts continuous assessment into formative (Houston and Thompson, 2017; Nordrum et al., 2013). The rubric enables clear orientation for the academic tasks, for the teaching and assessment of oral presentations on academic projects and simulated professional scenarios (De Grez et al., 2012; Galván-Sánchez et al., 2017). This study focuses on developing oral presentation competence, which is key in the early preservice teaching profession and throughout life. The development occurs through intervention using a strategy of continuous and progressive assessment by teachers and peers. A meta-analysis by Falchikov and Goldfi nch (2000) discovered that the highest correlations between scores given by classmates and by the teacher occurred when global judgements based on well-defi ned criteria rather than individual dimensions scoring criteria were used. Although peer scores do not differ considerably from teacher scores (Mulder, 2014). Given the wide use of peer-assessment, especially in higher education, the relative precision of scoring by classmates vs. the instructor is an important concern for both educators and researchers. This concern has grown with the increase in peer-assessment in online environments. Li et al. (2016) provide a synthesis of research fi ndings on peer-assessment since 1999, when computer-assisted peer-assessment began to be used. Adopting a hierarchical linear modeling approach, they found that the average estimated Pearson correlation between scores by classmates and instructors was .63, a moderately strong value. This correlation is signifi cantly higher when peer-assessment is optional and not compulsory (among other factors), when it is not anonymous, when peer-assessment provides both scores and qualitative comments (not just scores), and when peer assessors participate in development of the scoring criteria. Assessment drives the activities in which the students participate. Because these activities support students’ learning, careful design of a combined assessment strategy (including peer-assessment) guarantees that students are involved in the associated learning resources (Meyers and Nulty 2009). PÉREZ-TORREGROSA / GALLEGO-ARRUFAT / CEBRIÁN-DE-LA-SERNA 24 ESTUDIOS SOBRE EDUCACIÓN / 2022 Rubric for assessment The digital rubric provides the opportunity for formative assessment of students’ oral presentation, as it is an instrument that explains the criteria and expectations. It also facilitates feedback processes (van Ginkel et al., 2017). In training reviewers to use the rubric, time should be dedicated to watching the presentations and then discussing how to assess them for each competence. Such preparation promotes greater consistency in individual scoring among assessors (Dunbar et al., 2006). The rubrics can thus be a tool that infl uences efforts to convert continuous assessment into formative. Digital rubrics are used to assess oral presentation competence in numerous fi elds –engineering, the sciences, conservation-restoration and design, among others (Delgado and Fonseca-Mora, 2010; Menéndez-Varela and Gregori-Giralt, 2016; Pathak and Le Vasan, 2015). It is important that students provide feedback information with the rubric –not just scores but also comments. In this way, rubrics can be a tool for articulating feedback information (Nordrum et al., 2013). Progressive assessment A single session of peer-assessment with a rubric is not enough to generalize about any improvement in this competence. The review by Zheng et al. (2019) showed that most studies grounded in “Activity theory” implement only one round of peerassessment and do not reward assessors (for example, points in the subject ratings or extra points). When attempts have been made to integrate formative assessment for learning with summative assessment of learning, formulas such as repetition of the measures have been proposed (Houston and Thompson, 2017). This study is based on three main characteristics of assessment: – Continuity, which guarantees more objectivity in the assessment, since repeating assessments enables contrast of the results throughout the formative process and monitoring learning in oral competence. – Co-participation, which is grounded in involvement of the students themselves in self-assessment and assessment of their peers using the same criteria with which the teacher will assess them. This procedure makes students aware of their real formative level, enabling them to assume full responsibility for directing their own learning. It also increases the level of reliability (De Grez et al., 2009a; De Grez et al., 2012; Magin and Helmore, 2001; Skovholt et al., 2019). – Progressive assessment, fueled by the grades that can be assigned, according DIGITAL RUBRIC-BASED ASSESSMENT OF ORAL PRESENTATION COMPETENCE 25 ESTUDIOS SOBRE EDUCACIÓN / 2022 to the student him-or herself and provisionally, especially for activities whose purpose is learning and assessment. During progressive assessment, students have the possibility of modifying those grades according to the progress achieved in a negotiation between student and teacher (Madruga et al., 2020). Oral presentation competence is a goal for university education and professional success. It combines the importance to the teaching profession of orality, expressive narration, and dialogue among peers. In this study, it is important to research oral presentation competence, together with progressive assessment. Oral competence is very important for any teacher in any teaching mode because it is their tool for communication and dialogue to build knowledge with the student, being especially important in the domain of distance communication with the use of technology. The study focuses on assessing oral presentation competence when students defend projects publicly and orally that promote participation and the use of quality technological resources (digital competence). Starting from these premises, the goals of this study are: General goal. To understand the relationships between assessments by students and teachers through experimentation using different assessment strategies (two types of participation, “optional/compulsory”, Strategies A and B) with rubrics to encourage the learning of oral presentation competence. Specifi c goals 1. To describe and compare the trends in scores from peer and teacher assessments during progressive assessment of oral presentation competence, depending on who performs the assessment at what time in the learning process. 2. To explore the progressive assessment methodology for oral presentation competence, depending on who performs the assessment (peer and teacher), the time of the assessment (initial, intermediate, and fi nal), and the participation strategy (A and B). 3. To determine the relationship of the general grade obtained in the subject to the rubric score for oral presentation competence depending on who performs the assessment and the time of the assessment, especially for use and quality of the technological resources used. 4. To verify the value of the different strategies, persons, and times of assessment of oral presentation competence for predicting fi nal grades in the subject. PÉREZ-TORREGROSA / GALLEGO-ARRUFAT / CEBRIÁN-DE-LA-SERNA 26 ESTUDIOS SOBRE EDUCACIÓN / 2022 METHOD Participants The sample is a non-probability convenience sample (Cohen et al., 2011) composed of n = 99 undergraduates from the south of Spain. During one semester two natural groups of subjects in Educational Technology (EdTech) prepared oral presentations in stable work teams of 4-5 members (with an occasional exception, minimum 3 and maximum 6 members). The students underwent progressive evaluation (initial, intermediate, and fi nal) by peers and by the teachers using a digital rubric. Setting, tools, and rubric The group (n = 52) that was required to participate in the full assessment process followed Strategy A (compulsory). The criterion for inclusion in the study was having completed the three e-rubric applications (electronic assessment by peers and teacher) progressively and in full, as well as diligently attending. The group (n = 47) not required to perform progressive assessments except for the last presentation, the score on which was decisive for the grade, followed Strategy B. Both groups followed a combined model of project-based learning (PBL) and fl ipped learning. The general methodology of the subjects consisted mainly of an inverted classroom model –Flipped Classroom–, where the materials and resources were previously uploaded to the internet with specifi c activities that were carried out during class time and at the end of the class an evaluation rubric was applied. There was a single project to be developed for the whole course by each team, under the Project Based Learning model, which served as the central axis of the work of each team and student, where in three moments the development of the work was presented in public for monitoring and evaluation by the teacher and peers. The analytic rubric used to assess oral presentation competence consisted of four assessment criteria with indicators and evidence assigned different point values and three or four levels of achievement (Table 1). The rubric is available on https://acortar.link/ud3JX5 DIGITAL RUBRIC-BASED ASSESSMENT OF ORAL PRESENTATION COMPETENCE 27 ESTUDIOS SOBRE EDUCACIÓN / 2022 Table 1. Rubric to assess oral presentation competence ASSESSMENT CRITERIA, INDICATORS AND EVIDENCE ACHIEVEMENT LEVELS 1. To present suffi cient, relevant, well-structured content. 1.1. Quantity and relevance of information. 1. Shifts into a different topic, with conceptual errors. 1.1.1. Presents appropriate, precise information from a scholarly perspective. 2. Focuses on secondary issues, missing some important ones (more than 2 issues ignored). 3. Focuses on the main issues but forgets 1-2 of them. 4. Treats all of the most important elements on the topic well and comprehensively. 1.2. Organization of message. 1. Disorganized message, jumps from one topic to another. Cannot be followed. 1.2.1. Presents an organized message. 2. Lacks clear organization, is diffi cult to follow. 3. Is organized but is not very coherent. 4. Is clearly, coherently organized (not confusing). 2. To convey a message confi dently to an audience. 2.1. Formal qualities of the presentation. 1. Does not manage to fi nish the presentation and does not allow time for comments and questions. Finishes very early. 2.1.1. Adaptation to time and other requirements of the presentation. 2. Adapts presentation to the stipulated time and comments/questions but does not allow enough time for questions. 3. Adapts to the stipulated time and leaves enough time for comments/questions. 2.2. Body language and audience rapport. 1. Is stiff, with nervous uncontrolled gestures. Does not move around. 2.2.1. Moves around in a confi dent, relaxed manner. 2. Is stiff but controls nerves. Does not move around. 3. Is sometimes relaxed. Moves around some. 4. Appears relaxed and moves around freely with grace and composure. 2.3. Oral expression. 1. Unintelligible. Volume of voice is very low, speaks in a monotone. Poor intonation. 2.3.1. Voice is always clear and has good intonation. 2. Is hard to understand. Low volume of voice with poor intonation and uneven tone. 3. Fairly intelligible. Normal volume of voice and/or good intonation. 4. Easy to understand. Normal volume of voice, speaks clearly with good intonation. [CONTINÚA EN LA PÁGINA SIGUIENTE] PÉREZ-TORREGROSA / GALLEGO-ARRUFAT / CEBRIÁN-DE-LA-SERNA 28 ESTUDIOS SOBRE EDUCACIÓN / 2022 ASSESSMENT CRITERIA, INDICATORS AND EVIDENCE ACHIEVEMENT LEVELS 3. To adapt to the audience (general public or tribunal) by establishing fl uid interaction. 3.1. Motivates and captures interest of the audience or tribunal.1. Does not awaken interest or encourage participation. 3.1.1. Encourages interest and participation of the audience or tribunal. 2. Awakens some interest but does not encourage participation. 3. Awakens some interest and sometimes encourages participation. 4. Awakens interest, encourages participation, and gets audience to participate. 3.2. Management and control of audience or tribunal. 1. Answers shift into what the speaker knows, with digressions. Does not know how to react to unexpected situations. 3.2.1. Handling of questions and answers, comments from the audience, and unexpected situations. 2. Shows uncertainty in answering questions. Does not know how to react to unexpected situations. 3. Answers questions well but takes a long time. Hesitates and stutters when faced with unexpected situations. 4. Answers questions correctly and credibly. Handles unexpected situations well. 4. To be able to use high-quality technology resources integrated into the message. 4.1. Degree of assurance in handling resources. 1. Reads the whole talk from written notes and hardly makes eye contact with audience. 4.1.1. Uses technological resources for presentations to support speech and message. 2. Often reads from notes or slides and sometimes makes eye contact with audience. 3. Depends on slides or transparencies as well as written notes, using the former as a closed script and establishing frequent contact with audience. 4. Slides or transparencies are integral support for the presentation and complement it. 4.2. Quality of technological resources. 1. Uses (many/few) slides with a lot of text and overloaded outlines. 4.2.1. Prepares high-quality technological resources. 2. Uses (many/few) slides that are neither attractive nor clear. 3. Uses an appropriate number of slides, which are attractive but unclear and inappropriate. 4. Uses a good number of slides, which are attractive, clear, and appropriate, as well as other multimedia resources. Table 1. Rubric to assess oral presentation competence DIGITAL RUBRIC-BASED ASSESSMENT OF ORAL PRESENTATION COMPETENCE 29 ESTUDIOS SOBRE EDUCACIÓN / 2022 The rubric was thus fl exible. The assessment criteria were: C1. To present suffi cient, relevant, well-structured content (hereafter Content). C2. To convey a message confi dently to an audience (hereafter Message). C3. To adapt to the audience (general public or tribunal) by establishing fl uid interaction (hereafter Adapting). C4. To be able to use high-quality technology resources integrated into the message (hereafter Resources). Design and validation of rubric To collect information on the preservice teachers’ progress in developing oral presentation competence, an analytic rubric was created on the platform Corubric. com that teacher and students access for assessment and feedback. The rubric was designed and validated during the 2017-2018 academic year by expert teachers and researchers in a frame project on electronic assessment of competences from eight universities in fi ve countries. The validation process has been developed in a previous study and the rubric’s reliability was calculated using Cronbach’s alpha (α), the value 0.934 (Fernández-Medina et al., 2021). We have obtained a value of α = 0.80 in this study. Measurements and procedure The procedure is an adaptation of the 3P model developed by Biggs (2003) (Figure 1), which stresses the process of e-assessment in project-based learning (PBL) + fl ipped learning. Following the team-based learning model, the participants were distributed into 29 teams to work on an educational technology project throughout the semester, with three assessment times (initial, intermediate, and fi nal product). The project represented a variable proportion of the total grade in the subject according to the universities involved (10% UGR and 20% UMA). After the presentation of the project, students and teachers had some time to assess each team using the platform. In the initial and intermediate evaluations, students could indicate which indicators they did or did not wish to include. PÉREZ-TORREGROSA / GALLEGO-ARRUFAT / CEBRIÁN-DE-LA-SERNA 36 ESTUDIOS SOBRE EDUCACIÓN / 2022 Table 4. Coeffi cients of the regression models (dependent variable subject rating), taking total assessments of peer and teacher assessments as predictors (Compulsory Group) Initial B ẞp 95% CI Total Peer -.011 -.068 .650 -.057-.036 Total Teacher -.006 -.166 .270 -.017-.005 Intermediate B ẞp 95% CI Total Peer .001 .006 .967 -.050-.052 Total Teacher .016 .400 .013 .004-.029 Final B ẞp 95% CI Total Peer -.027 -.134 .271 -.075-.022 Total Teacher .054 .621 <.001 .033-.075 B: Slope; ẞ: Standardized Slope; p: Critical level of signifi cance; 95% CI: 95% Confi dence Interval for B. DISCUSSION AND CONCLUSIONS This study has confi rmed the value of design, validation, and application of a rubric for oral presentation competence (Jönsson and Panadero, 2017; Menéndez-Varela and Gregori-Giralt, 2016; van Ginkel et al., 2015; van Ginkel et al., 2017). More specifi cally, the rubric integrates oral and digital competences, as it assesses the capability of using and having good command of technological resources (van Ginkel et al., 2017, p. 486), capabilities included in the assessment criterion “to be able to use high-quality technology resources integrated into the message” (Resources). Two indicators (4.1. and 4.2.) were used because it is the form of technological communication par excellence that supports oral discourse. Because evidence of students’ involvement, commitment, and motivation is important in the practice of learning, we analyzed the impact of an innovative intervention that stressed continuity, co-participation, and progressive assessment in the acquisition of oral presentation competence (De Grez et al., 2009a). The association of this potential involvement with both optional and compulsory participation in the assessment made it a signifi cant variable in peer-assessment. Although the research on feedback in the peer-assessment of oral presentation began as relatively limited (Topping, 1998), it has since increased (Li et al., 2016; Zheng et al., 2019) but has not been researched by itself. Studies rarely state whether participation in peer-assessment was compulsory or optional, although one could expect this variable to impact acceptability and reliability (Topping, 1998, p. 268). DIGITAL RUBRIC-BASED ASSESSMENT OF ORAL PRESENTATION COMPETENCE 37 ESTUDIOS SOBRE EDUCACIÓN / 2022 In our study, the results of the total oral presentation competence (peer and teacher assessment) show signifi cant differences in the assessments based on type of participation (compulsory or optional) in the assessment process. When participation is compulsory (Strategy A), mean scores are higher (Co>Op) in all assessments (initial, intermediate and fi nal). The same trend appears when participation is optional (Strategy B), except only the score obtained for teacher on the initial assessment (Co>Op). Therefore, it seems that the nature of participation, despite being an important methodological element, is not a determining factor in the total scores of the rubric for peer and teacher assessment. Researching the effect of applying digital rubrics with repeated measurements (often three measurements) supports decoupling assessment with digital rubrics from summative assessment (Houston and Thompson, 2017). We conclude that the progression brings successive applications somewhat close to formative assessment, although these assessments are more closely connected when feedback is provided (Madruga et al., 2020). The initial, intermediate, and fi nal assessments with feedback by peers and teachers are an important step in the progress toward formative assessment. In De Grez et al. (2009b) and Menéndez-Varela and Gregori-Giralt (2016), achieving the learning outcomes and obtaining evidence of learning progress demonstrate the validity of the inferences drawn from the assessment system. Peer-assessment seems suffi ciently reliable and valid in a wide variety of applications, although the literature considers score reliability rather than more detailed formative assessment. The 3P model illustrates the infl uence of active e-assessment of oral presentation competence on the learning outcomes, expressed in the form of fi nal grades in the Educational Technology subjects in this study. We verify that the results for general grade in the subject are related to the score on the oral presentation competence rubric. When we analyze peer-assessment, we confi rm no signifi cant relation between the score of the rubric and subject rating, but in the teacher assessment there is. This score in the e-rubric (particularly at the intermediate and fi nal moments) is statistically signifi cant, so contributes to grade in the subject. Based on the assumption of a solid relationship between oral and digital competence when assessing oral presentation competence, assessment criterion Resources on the digital rubric focuses on quality of resources and mastery of technology for the oral presentation. Our study reveals a correlation not previously studied in the literature consulted: the students who obtain better scores on use and quality of technology resources as support for speech and messages obtain higher scores on oral presentation competence. This correlation is signifi cant at all three evaluation times in teacher assessment. It is lower in peer-assessment and PÉREZ-TORREGROSA / GALLEGO-ARRUFAT / CEBRIÁN-DE-LA-SERNA 38 ESTUDIOS SOBRE EDUCACIÓN / 2022 depends on the time of the assessment. These fi ndings are important for preservice teachers’ practice and will be a determining factor in their scores on oral presentation competence, since it is essential that they use high-quality resources to support their oral presentation. Our fi ndings are similar to the results of other studies on precision of peer-assessment grades compared to teacher assessment grades (De Grez et al., 2012; Li et al., 2016; Magin and Helmore, 2001). With the increasing use of digital platforms –even greater now due to the Covid-19 pandemic– this is an important concern for both educators and researchers. Assessments by students using rubrics differ from the teachers’ assessments, but successive applications of progressive assessment produce closer scores as the number of assessments performed with the digital rubric increases. The meta-analysis by Li et al. (2016) shows that this correlation is signifi cantly greater when peer-assessment is optional instead of compulsory, among other factors. In our study, the total score for oral presentation competence in initial assessment of ICC shows no agreement on the total score for oral presentation competence and signifi cant agreement in the intermediate and fi nal assessment. It is worth stressing that criterion Resources, which combines evidence from the rubric on quality and use of technological resources, shows signifi cant agreement between peer and teacher assessment starting with the fi rst assessment (initial assessment). In conclusion, this study obtains signifi cant interaction effects between assessors for all assessments, indicating that changes in assessment at different moments in time depend on the assessor. Finally, we encounter some challenges in assessing oral presentation competence, such as challenges associated with performing this task in an in-person vs. online environment. On the one hand, criteria Message and Adapting merit differentiated analysis as a whole and in each of these environments. On the other, it is necessary to explore formulas to improve participation in assessment in blended learning environments with combinations of virtual and in-person oral presentation. Future studies must further develop assessment of improvement in the quality of the projects as a whole (not only of the oral presentation but of the project), underscoring the need to combine oral and digital competence through feedback, promoting assessment of the projects to improve learning. It is important for stakeholders in higher education to recognize the need to give students a voice in improving competencies that are especially useful for employment (Campbell et al., 2001; Romero-García et al., 2020). The study confi rms an effective methodology, with a platform and methodology that allows teachers to evaluate the impact of their digital rubrics as more fl exible, more observable and instantaneous, thus facilitating better decision making accordingly than other studies presented. DIGITAL RUBRIC-BASED ASSESSMENT OF ORAL PRESENTATION COMPETENCE 39 ESTUDIOS SOBRE EDUCACIÓN / 2022 However, it remains to extend the study in the future compared to control groups where the rubric methodology is not used. Fecha de recepción del original: 8 de marzo 2022 Fecha de aceptación de la versión defi nitiva: 25 de mayo 2022 REFERENCES Biggs, J. B. (2003). Teaching for quality learning at university: What the student does. Open University Press. Bower, M., Cavanagh, M., Moloney, R., and Dao, M. (2011). Developing communication competence using an online video refl ection system: Pre-service teachers’ experiences. Asia-Pacifi c Journal of Teacher Education, 39(4), 311-326. https://doi.org/10.1080/1359866X.2011.614685 Campbell, K. S., Mothersbaugh, D. L., Brammer, C., and Taylor, T. (2001). Peer versus self assessment of oral business presentation performance. Business Communication Quarterly, 64(3), 23-40. https://doi.org/10.1177/108056 990106400303 Cebrián-de-la-Serna, M., and Bergman, M. E. (2014). Formative Assessment with eRubrics: an Approach to the State of the Art. REDU. Revista de Docencia Universitaria, 12(1). 15-29. Cohen, L. M., Mannion, L. L., and Morrison, K. (2011). Research methods in education. New York. Crawford, A. R., Johnson, E. S., Zheng, Y. Z., and Moylan, L. A. (2020). Developing an understanding procedures observation rubric for mathematics intervention teachers. School Science and Mathematics, 120(3), 153-164. https://doi. org/10.1111/ssm.12393 Creswell, J. W., and Creswell, J. D. (2018). Research design: qualitative, quantitative, and mixed methods approaches. SAGE. De Grez, L., Valcke, M., and Roozen, I. (2009a). The impact of an innovative instructional intervention on the acquisition of oral presentation skills in higher education. Computers and Education, 53(1), 112-120. https://doi.org/10.1016/j. compedu.2009.01.005 De Grez, L., Valcke, M., and Roozen, I. (2009b). The impact of goal orientation, self-refl ection and personal characteristics on the acquisition of oral presentation skills. European Journal of Psychology of Education, 24(3), 293-306. https:// doi.org/10.1007/BF03174762 De Grez, L., Valcke, M., and Roozen, I. (2012). How effective are self-and peer PÉREZ-TORREGROSA / GALLEGO-ARRUFAT / CEBRIÁN-DE-LA-SERNA 40 ESTUDIOS SOBRE EDUCACIÓN / 2022 assessment of oral presentation skills compared with teachers’ assessments? Active Learning in Higher Education, 13(2), 129-142. https://doi.org/10.1177/ 1469787412441284 Delgado, M. A., and Fonseca-Mora, M. C. (2010). The use of co-operative work and rubrics to develop competences. Education for Chemical Engineers, 5(3), e33-e39. https://doi.org/10.1016/j.ece.2010.05.002 Dunbar, N. E., Brooks, C. F., and Kubicka-Miller, T. (2006). Oral communication skills in higher education: Using a performance-based evaluation rubric to assess communication skills. Innovative Higher Education, 31(2), 115-128. https://doi.org/10.1007/s10755-006-9012-x Falchikov, N., and Goldfi nch, J. (2000). Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Review of Educational Research, 70(3), 287-322. https://doi.org/10.3102/00346543070003287 Fernández-Medina, C., Luque-Guerrero, C., Ruiz-Rey, F., Rivera-Rogel, D., Andrade Vargas, L., and Cebrián-de-la-Serna, M. (2021). Assessment oral competence with digital rubrics for the Ibero-American Knowledge Space. Pixel-Bit. Revista de Medios y Educación, 62, 71-106. https://doi.org/10.12795/ pixelbit.83050. Galván-Sánchez, I., Verano-Tacoronte, D., González-Betancor, S. M., Fernández-Monroy, M., and Bolívar-Cruz, A. (2017). Assessing oral presentation skills in Electrical Engineering: Developing a valid and reliable rubric. International Journal of Electrical Engineering Education, 54(1), 17-34. https://doi. org/10.1177/0020720916659501 Haanstad, E. J. (2020). Collaborative Ethnographic Assessment: An Anthropological Rubric for a Community Ecosystem. Teaching Anthropology, 9(2), 1-8. https://doi.org/10.22582/ta.v9i2.528 Houston, D., and Thompson, J. N. (2017). Blending Formative and Summative Assessment in a Capstone Subject:‘It’s not your tools, it’s how you use them’. Journal of University Teaching and Learning Practice, 14(3), 2. Jönsson, A., and Panadero, E. (2017). The use and design of rubrics to support assessment for learning. In D. Carless, S.M. Bridges, C.K.Y. Chan and R. Glofcheski R (Eds.), Scaling up assessment for learning in higher education (pp. 99-111). Springer. Li, H., Xiong, Y., Zang, X., L. Kornhaber, M., Lyu, Y., Chung, K. S., and K. Suen, H. (2016). Peer assessment in the digital age: A meta-analysis comparing peer and teacher ratings. Assessment and Evaluation in Higher Education, 41(2), 245264. https://doi.org/10.1080/02602938.2014.999746 DIGITAL RUBRIC-BASED ASSESSMENT OF ORAL PRESENTATION COMPETENCE 41 ESTUDIOS SOBRE EDUCACIÓN / 2022 Madruga, T., Thiam, S., Vasquez, A., Kothari, R., and Krauss, G. G. (2020). Reviewer perspective impact on design review feedback. The International Journal of Engineering Education, 36(2), 675-686. Magin, D., and Helmore, P. (2001). Peer and teacher assessments of oral presentation skills: how reliable are they? Studies in Higher Education, 26(3), 287-298. https://doi.org/10.1080/03075070120076264 Menéndez-Varela, J. L., and Gregori-Giralt, E. (2016). The contribution of rubrics to the validity of performance assessment: a study of the conservationrestoration and design undergraduate degrees. Assessment and Evaluation in Higher Education, 41(2), 228-244. https://doi.org/10.1080/02602938.2014.99 8169 Meyers, N. M., and Nulty, D. D. (2009). How to use (fi ve) curriculum design principles to align authentic learning environments, assessment, students’ approaches to thinking and learning outcomes. Assessment and Evaluation in Higher Education, 34(5), 565-577. https://doi.org/10.1080/02602930802226502 Morreale, S. P., Valenzano, J. M., and Bauer, J. A. (2017). Why communication education is important: A third study on the centrality of the discipline’s content and pedagogy. Communication Education, 66(4), 402-422. https://doi.org/ 10.1080/03634523.2016.1265136 Montalvão, D., and Baker, T. (2015). Correlating peer and tutor assessment on a lowstakes engineering assignment. International Journal of Mechanical Engineering Education, 43(3), 168-179. https://doi.org/10.1177/0306419015603008 Mulder, M. (2014). Conceptions of professional competence. In S. Billett, C. Harteis, and H. Gruber (Eds.), International handbook of research in professional and practice-based learning (pp. 107-137). Springer. Nordrum, L., Evans, K., and Gustafsson, M. (2013). Comparing student learning experiences of in-text commentary and rubric-articulated feedback: strategies for formative assessment. Assessment and Evaluation in Higher Education, 38(8), 919-940. https://doi.org/10.1080/02602938.2012.758229 Pathak, A., and Le Vasan, M. (2015). Developing Oral Presentation Competence in Professional Contexts: A Design-Based Collaborative Approach. International Journal of Evaluation and Research in Education, 4(4), 179-184. http:// dx.doi.org/10.11591/ijere.v4i4.4509 Redecker, C. (2017). European framework for the digital competence of educators: DigCompEdu. Report for the European Union no. JRC107466. Joint Research Centre. Romero-García, C., Buzón-García, O., Sacristán-San-Cristóbal, M., and NavarroAsencio, E. (2020). Evaluation of a Program for the Improvement of Learning PÉREZ-TORREGROSA / GALLEGO-ARRUFAT / CEBRIÁN-DE-LA-SERNA 42 ESTUDIOS SOBRE EDUCACIÓN / 2022 and Digital Competence in Future Teachers Utilizing Active Methodologies. Estudios sobre Educación, 39, 179-205. https://doi.org/10.15581/004.39.179-205 Skovholt, K., Nordenström, E., and Stokoe, E. (2019). Evaluative conduct in teacher–student supervision: When students assess their own performance. Linguistics and Education, 50, 46-55. https://doi.org/10.1016/j.linged.2019.03.001 van Ginkel, S., Gulikers, J., Biemans, H., and Mulder, M. (2015). Towards a set of design principles for developing oral presentation competence: A synthesis of research in higher education. Educational Research Review, 14, 62-80. https:// doi.org/10.1016/j.edurev.2015.02.002 van Ginkel, S., Gulikers, J., Biemans, H., and Mulder, M. (2017). The impact of the feedback source on developing oral presentation competence. Studies in Higher Education, 42(9), 1671-1685. https://doi.org/10.1080/03075079.2015. 1117064 van Ginkel, S., Laurentzen, R., Mulder, M., Mononen, A., Kyttä, J., and Kortelainen, M. J. (2017). Assessing oral presentation performance: Designing a rubric and testing its validity with an expert group. Journal of Applied Research in Higher Education, 9(3), 474-486. https://doi.org/10.1108/JARHE-02-2016-0012 Zheng, L., Chen, N. S., Cui, P., and Zhang, X. (2019). A Systematic Review of Technology-Supported Peer Assessment Research: An Activity Theory Approach. International Review of Research in Open and Distributed Learning, 20(5), 168-191. https://doi.org/10.19173/irrodl.v20i5.4333