From language teaching to language assessment with the help of technological resources: higher education students and oral production

ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 93 FROM LANGUAGE TEACHING TO LANGUAGE ASSESSMENT WITH THE HELP OF TECHNOLOGICAL RESOURCES: HIGHER EDUCATION STUDENTS AND ORAL PRODUCTION DE LA ENSEÑANZA A LA EVALUACIÓN DE LENGUAS CON LA AYUDA DE RECURSOS TECNOLÓGICOS: LOS ESTUDIANTES DE EDUCACIÓN SUPERIOR Y LA PRODUCCIÓN ORAL Inmaculada Senra Silva Universidad Nacional de Educación a Distancia, España [email protected] Abstract In this paper I present the results of a study carried out in order to assess the oral production skills of distance learning students enrolled in a university degree, and their experience with digital resources to produce their assessment tasks. Both the practice of speaking in a foreign language and the assessment of oral production skills constitute an important endeavor that often meets methodological and technological difficulties in traditional face-to-face learning environments, but more particularly in distance learning ones. After analyzing students’ needs and peculiarities coming from a distance learning context, some research has been conducted in order to test and propose an activity where the students individually design and produce a video in which they explain a topic of their choice as a continuous evaluation task. They then receive the teacher's feedback by means of a rubric which allows them to improve their learning at From language teaching to language assessment with the help of … ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 94 the same time that it enables teachers to obtain valuable information about students’ progress as regards oral productive skills. The results of the study show how students can improve their oral production skills with self-study and how the preparation of this assessment activity has a positive effect on their language proficiency as well as on their digital competence. Keywords: oral skills, language learning in distance learning, continuous assessment, English as a Foreign Language. Resumen En este artículo se presentan los resultados de un estudio realizado con el fin de evaluar las habilidades de producción oral de estudiantes de educación a distancia matriculados en una carrera universitaria, así como su experiencia con los recursos digitales para producir las tareas de evaluación. La práctica de comunicación oral en una lengua extranjera así como la evaluación de las habilidades de producción oral suponen un reto importante que a menudo encuentra dificultades metodológicas y tecnológicas en los entornos tradicionales de aprendizaje presencial, pero particularmente en los de aprendizaje a distancia. Tras analizar las necesidades y peculiaridades de los estudiantes provenientes de un contexto de educación a distancia, se ha llevado a cabo una investigación para probar y proponer una actividad donde los estudiantes individualmente diseñan y producen un video en el que explican un tema de su elección como tarea de evaluación continua. Seguidamente reciben la retroalimentación del docente a través de una rúbrica que les permite mejorar su aprendizaje al mismo tiempo que permite a los profesores obtener información valiosa sobre su progreso en cuanto a habilidades productivas orales se refiere. Los Inmaculada Senra Silva ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 95 resultados del estudio muestran cómo los estudiantes pueden mejorar sus habilidades de producción oral con el autoaprendizaje y cómo la preparación de esta actividad de evaluación tiene un efecto positivo en su dominio global del idioma, así como en su competencia digital. Palabras clave: clave: habilidades orales, aprendizaje de idiomas a distancia, evaluación continua, inglés como lengua extranjera. 1. Introduction The development and assessment of oral skills always entails a considerable challenge in English as a Foreign Language (EFL) as compared to other language skills like reading, writing or listening. In the context of distance education, the use of Information and Communication Technology (ICT) plays an important role for language development as new technological solutions are constantly needed not only to enhance the development of oral skills in a foreign language and strengthen the presence of oral activities, but also to assess students’ oral skills improvement at the end of the semester. As a matter of fact, ample research analyzes the use of technology in the assessment of EFL oral skills, a field which also moves quickly given the continuous development of new software which can be used (Blake, 2016; Giglio, 2019; Kissau, Davin and Wang, 2019; Marcum, 2017; Namaziandost and Nasri, 2019; Senra Silva, 2017; Wulandari, 2019; Živković, 2014). The teaching teams of the Inglés Instrumental III-VI courses in the English Studies Degree at the Universidad Nacional de Educación a Distancia (UNED), including teaching staff and teachertutors, have run an experience to assess and find out the best ways of incorporating students’ oral presentations as part of their mid-term activities to be assessed by tutors in regular courses, which will From language teaching to language assessment with the help of … ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 96 eventually be part of their final grade. In so doing, students had the chance to develop both technological and oral skills in the foreign language. This presentation introduces the results of a study carried out with a group of students who had to a) familiarize themselves with some free software that would enable them to prepare the activity, b) plan and record an oral activity to be sent to their tutors for assessment, and c) answer a post-activity online questionnaire about their views and experience. The results of this study are expected to help redirect the use of mid-term activities by our students at the (UNED) and, above all, to strengthen the presence of oral activities in distance learning contexts. Although this study was carried out to enhance the development of oral skills in a foreign language, some pedagogical implications are drawn for the teaching/learning of other subjects in distance education contexts. 2. Distance learning in higher education and the development of oral production skills A constant raise of distance education programs at the university level has been experienced in the last few years, a phenomenon which has been accelerated due to the recent pandemic situation appearing in 2020 in which higher education institutions have made extra efforts to develop and consolidate an academic offer which meets the needs of society. Students at this educational level have also experienced a structural change in the case of traditional face-to-face teaching institutions and an enhancement of the resources for teaching/learning in distance learning contexts. UNED is the largest public university in Spain and the main institution for distance and blended learning in the country. It has been teaching in the last 50 years following a hybrid teaching methodology in which students combine online learning with Inmaculada Senra Silva ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 97 attendance to the 60 associate centers which are spread all over the country and also abroad and where students can find libraries, tutoring and other learning resources. UNED has as its main goal to provide higher and continuous education opportunities to all through a distance education system. For over 20 years now, a degree in English Studies has been offered at UNED to train prospective English language teachers and professionals within the field of English Philology. The development of proficiency in English is core to these students who will eventually become professionals who will use the English language on a daily basis and who, therefore, need to develop an advanced competence in all receptive and productive skills in English. In this sense, the Degree in English Studies includes a set of subjects dedicated to taking students from a level of A2 to C1 in English proficiency in the span of four years. The teaching staff of these subjects have worked widely on the development of teaching resources in order to develop students’ writing (Chacón Beltrán, 2017; Senra Silva, 2010; and oral skills (Senra Silva, 2017) but no actions have been undertaken to evaluate students’ oral productive skills. As can be expected, the development of productive oral skills entails an additional challenge which can be overcome nowadays with the use of new technological resources and opportunities. For instance, at present students can meet online to talk in English and practice their oral communication skills with other students and they can even interact with a tutor. Regardless to say that the opportunities to access real or graded input in the language have now increased due to the use of the Internet. Nowadays teaching materials include a whole range of activities which students can undertake not only through From language teaching to language assessment with the help of … ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 98 traditional printed or recorded materials, but also through virtual courses that give the chance of more interaction, unlimited practice and a sense of being part of a community which students in a distance learning environment could not experience a few years ago. Teaching staff in distance education contexts have access to unlimited opportunities for innovation due to the introduction of ever improving teaching/learning resources. So, helping students develop their speaking skills by providing opportunities for oral communication and designing innovative technical solutions for self or collaborative development is not an issue. A more complicated endeavor which teachers need to face, however, is the assessment of those oral skills in such a distance context, and it seems that a reliance on technological developments can be a solution (Senra Silva, 2010). Assessment of oral production in various ways have proved to be beneficial in higher education (Miskam and Saidalvi, 2020; Sinclair, 2014, 2016). Among others, they have been shown to build self-confidence and self-awareness. They can also support the development of other skills such as organizing and summarizing ideas. Despite all these benefits, oral forms of assessment have been underrepresented in distance learning (Sinclair, 2016) where the focus has been on written skills. Therefore, new ways of oral assessment should be developed since higher education institutions must prepare their students for communicating not only in an academic environment but also for a future workplace. In June 2022 the European Commission made public the European Strategy for Universities which states as one of the mains Inmaculada Senra Silva ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 99 aims to develop skills and competences and promote innovation for the digital transition: “the digital transition gives universities a fundamental role in equipping students and researchers with the digital skills and competences needed in the new reality and in the promotion of innovation and new technologies.” (European Commission 2022, 12). In the framework of the 2022 European Year of Youth , the European Commission has stated that it owns “young people the chance to acquire excellent and relevant skills and competences. Offering each young European a real chance to upskill and reskill, helps lay the foundation for a stronger, more prosperous and resilient Europe.” (European Commission 2022, 16). In this sense, universities have the pressure of training their students in digital competence. In this digital era the role of new technologies in language learning is of upmost importance, but particularly in distance language learning. At UNED tutoring lessons (both face-to-face in the various associate centers and online via web conferences) take place each week, but attendance is not compulsory. Students sometimes cannot physically attend tutoring lessons because of work, physical or mental health issues, caring commitments, etc. Therefore, it is important to attempt to innovate in learning teaching and assessment, restructuring language learning subject, and empowering students in the use of ICT, moving from pure assessment to learning. The purpose of the present study was to assess the usefulness of a free software used to record an oral activity to be sent by students to their UNED tutors for assessment. From language teaching to language assessment with the help of … ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 100 The research questions intended to be answered with this research were the following: 1. Can innovative approaches to the assessment of oral production skills in higher education students improve their oral productive skills and performance? 2. Can innovative approaches to the assessment of oral production skills in higher education students improve assessment and its efficacy? 3. Do students’ attitudes towards assessment change by introducing innovative oral language assessment procedures? 4. What do students think about being evaluated through a performance task instead of a face-to-face interview? 3. Methodology 3.1. Pilot study A pilot study was carried out to assess the feasibility of the course innovation. 9 students (8 students enrolled in the Degree in English Studies and 1 in the English Applied Linguistics Master at the UNED), a teacher-tutor, and two professors who were members of the teaching team of the four instrumental subjects of the Degree in English Studies ( Inglés Instrumental III, IV, V and VI ) participated in this pilot study. All of them had been enrolled at the UNED for several years (between 2 and 11), so they had valuable experience about long distance learning. Instructions were carefully prepared on how to perform the task. Explanations were provided in the virtual course of the different subjects. Students were asked to design and make a Inmaculada Senra Silva ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 101 Power Point presentation (5-10 slides) about a topic of their choosing and with a duration of about 5 minutes, record it with a specific software programme (Screencast-O-Matic) and send it to the tutor via de virtual course. The software programme designed at Birmingham City University, includes an easy-to-use screen recorder and has a free online version. It allows to capture any area of your screen with the option to add narration from your microphone and video from your webcam and save the recording for sharing or playback. Subsequent personalized feedback was created according to a rubric which included four items, namely, pronunciation, use of vocabulary, grammatical accuracy and oral fluency with quality definitions at four levels of achievement (poor, fair, good, excellent). Students were asked to answer one questionnaire in their first language, namely Spanish, prior to the completion of the task (see appendix 1) and another one after they had completed it explaining their experience throughout the process (see appendix 2). The purpose of the first questionnaire was to find out some information about the students’ academic background as well as their preferences with regards to the type of continuous evaluation ( Prueba de Evaluación Continua (PEC)). The second questionnaire was designed to elicit information about the students’ experience with the software and their perceptions and preferences regarding its use. In relation to the first questionnaire, students were asked to provide their views on the evaluation system at the UNED consisting of a PEC plus an onsite written exam. There was a mixed response. Some students mentioned that it was a very satisfying method and a good option since doing the PEC forces From language teaching to language assessment with the help of … ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 108 remain calm during the oral presentation, and to be able to learn from my mistakes”. Other benefits mentioned by some students were: “By seeing myself speaking on a recording, I can spot the mistakes I make, which is helpful in correcting those mistakes.” [Student2] “I had to do it about 11 times until I got the hang of it and that's what has helped me to have more fluency, by repeating it.” [Student10] “Actually, if you invest time, with the recording you can improve, revising and correcting. Unfortunately, there is not always the time to do so.” [Student20] “It has helped me to be able to organize ideas clearly and coherently when planning my presentation. Also to lose a little the fear of expressing myself in English, at the same time that it has forced me to enrich the speech, since when you see yourself recorded you realize that many times you use the same vocabulary all the time. This has made me interested in searching for synonyms or other ways of expressing what I want to say using other words; forcing myself to repeat the recording so I could improve it each time. That is why I think it has been a further step towards my learning English.” [Student15] “It has allowed me to see myself as a listener of my own presentation, which has been important for selfcorrecting.” [Student32] For the sake of the research, students were asked if this way of assessing through a recorded oral presentation would be useful in content subjects. In this sense, 62,1 % believed it would, whereas 37,9% maintained it would not. Some students pointed out the fact Inmaculada Senra Silva ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 109 that in most of the subjects what they do is to practice writing with almost no oral practice, which is fundamental when learning English. With regards to the questionnaire distributed after the students had completed and handed in their self-recording task with Screencast-O-Matic, 66 participants were selected as valid informants having completed the whole questionnaire. Out of this sample, it turned out that 86,4% of the informants found it “easy” or “very easy” to find Screencast-O-Matic in internet, and 83,3% found it “easy” or “very easy” to download and install the software in their computers and 80,3% found it “easy” or “very easy” to learn how to use the software. Another important skill for the design of this project was that of the students’ familiarity with Power Point presentations as they would be assisted by a presentation in their recording. 69,7% of the students found it “easy” or “very easy” to prepare their presentation with Power Point. This figure was somehow surprising as a higher number of students would have been expected to be familiarized with Power Point, as it is a software that has been in the market for a number of years and seems to be so common in the academic context. The good news was that students participating in this study would become acquainted with that commonly used software after their participation in this project. Saving and sending the recording to the teacher-tutor was not a problem either as 77,3% of the students found it “easy” or “very easy” to do. When asked whether the design of this task and making use of Screencast-O-Matic and Power Point had helped them to improve their oral production skills, 62,1% of the students answered affirmatively whereas 10,6% thought they had not improved their From language teaching to language assessment with the help of … ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 110 oral production skills and 27,3% did not answer, what suggests they were not sure about it, as can be seen in Figure 3. Figure 3: Students’ perception of self-improvement in oral production skill in English after having completed the task with Screencast-O-Matic As for the qualitative explanation to their answers, some students that acknowledged an improvement in their oral production skills claimed that the chance of repeating and rehearsing gave them the chance to improve as it was stated in some of the comments: “Because it involves practice and when you are not used to speaking with a purpose, it is always good to record yourself on video to learn to refine your expression, style, eliminate unnecessary tag lines and thus improve for future presentations.” [Student 4] “Because you have to speak clearly to be understood well in the video. We must not only be careful with our pronunciation, but also with the intonation.” [Student6] “The possibility of being able to see yourself after the recording allows you to observe your own mistakes, something to complement with the teacher's assessment. Yes No N/A Inmaculada Senra Silva ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 111 In the interview format, this evaluation of the work itself cannot be carried out.” [Student1] “Because by recording yourself and being able to repeat the work as many times as you want, in the end you are improving your oral skills.” [Student36] “It is a medium in which you get used to speaking and can help you gain fluency.” [Student7] Some students liked the idea of not having to perform the task face to face in front of a teacher as it was a less stressful situation: “By being able to do various tests without the pressure of being evaluated live, I think you can see yourself and identify errors and points for improvement that, in another situation, your nerves would not leave you. The recording is done in a more relaxed way than if it were live and I think you can do better, without added pressure.” [Student54] “Because you can repeat as many times as you need, you see your gestures and hear yourself, and you seek to improve with each repetition. And since you can do it from home, it is very good, since the nerves do not influence when the examiner is not present. I would put two PEC, at least, in the English Instrumentals, for those who want to practice more and raise the grade more.” [Student31] “Because it will let you redo as many times as you want and you can send the video that you feel most comfortable with.” [Student6] “I can record the activity until I get it right and listen to myself to smooth out the flaws.” [Student8] From language teaching to language assessment with the help of … ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 112 The students who thought that they had not improved made some of the following comments, mainly questioning the effect these tasks would have on their oral skills due to the short duration of the resulting recording: “Because a recording of 6 minutes and another of 9 are not enough to improve my oral English. Another thing is that these recordings are useful to teachers when evaluating students, seeing the progress they are making or determining the level they have.” [Student7] “Because it is a half hour exercise; You may appreciate my oral skills, but you won't improve it in that time.” [Student28] “This task requires a lot of effort and time, since you have to prepare the presentation, do the "Power Point", rehearse, record, and repeat the recording several times. However, I prefer the positive part, because that effort, as almost always, pays off in learning and improving the language, which in the end is what it is about.” [Student16] Those who did not answer or were not sure claimed the following, mainly mentioning that they did not like being recorded and that the use of this technology should not substitute the presence of a teacher: “I guess until I have the answer of my mistakes, I won't know what to improve. It is the first time I make a video like this or record myself.” [Student9] “It is a good tool to send a video but without a doubt, I prefer to do this in person. I don't like recording myself.” [Student12] Inmaculada Senra Silva ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 113 “It allows me to evaluate myself. But it's not enough. It does not replace a teacher specialized in teaching public speaking.” [Student22] In relation to the question of whether students would be willing to incorporate Screencast-O-Matic on a regular basis as a standard form of assessing oral production skills and a continuous evaluation format which would comprise 20% of the final grade, 84,4% of the participants answered positively whereas 15,6% answered negatively, as can be seen in Figure 4. Figure 4: Students’ opinion on whether they would like to start using Screencast-O-Matic on a regular basis for oral production assessment in the subject. Here follows a selection of students’ comments in favor of using Screencast-O-Matic: “The record is objective and can be reviewed by both parties and even by someone outside in case of doubt or discrepancy; It also requires adequate preparation at the level expected.” [Student3] From language teaching to language assessment with the help of … ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 114 “Because it is a very practical way to improve English at the oral level. Unlike a video conference, the student is alone. There is no teacher or other student to ask you questions to continue the conversation. Therefore, in addition to improvising, you must have a sufficient level to make a short speech without interruptions.” [Student6] “Because Screencast-O-Matic allows us to present information in an engaging way. A way that grabs student's attention. You learn and at the same time you enjoy the learning.” [Student8] “Because new programs such as screencasts have become a fairly good component of the programs of distance learning, because they supply a sense of link between the teaching staff and the students, without compromising the capability of accessing materials of online learning, at any time and place that may be convenient.” [Student10] “The PECS of the English subjects are oral live with a tutor, it can be in person, or through skype or similar means, but they required to reconcile schedules with the tutor, to have everything prepared at a certain time. This form gives you much more freedom, you can do it at the time you want, according to your own schedules and work and personal circumstances, something that, for a distance career, not face-to-face, seems to me very valuable and a point in favor.” [Student20] “Because it is an autonomous tool that allows the student to record themselves when it suits them and send it when they want, as long as it is within the allotted period of time.” [Student23] Inmaculada Senra Silva ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 115 “I think that the results can be better when doing the test without stress and it also gives more flexibility because everyone does the recording when they can.” [Student50] “Because I believe that while it brings you closer to your tutor, you can carry out your PEC when and where you think appropriate, in peace of mind.” [Student54] “Because it is a facility for distance students.” [Student55] “The "nerves" factor is eliminated.” [Student60] “Because if you are not happy with your recording you can repeat it as many times as necessary and thus perfect your presentation.” [Student62] “It allows you to self-correct.” [Student64] “It has been quite a discovery for me. In addition, you also continue practicing the writings to be able to record later. It has helped me to listen to myself, see myself and search for and improve pronunciations that were not correct. Very positive for me. I hope they continue with them.” [Student65] Among the advantages, students put an emphasis on (a) flexibility, which allows them to do it whenever suits them, without the need of having a face-to-face meeting with their tutor. Another aspect that is highlighted is (b) the absence of stress for the student, since they can do it from home and repeat it as many times as needed until they get a satisfactory result. They also mention (c) the suitability to the teaching/learning model they have chosen, namely, distance learning, and finally, (d) it seems to be a model that is trustworthy as, in case of discrepancy, it is recorded and it is easy to get back and check any aspect regarding the final grade. And, finally, From language teaching to language assessment with the help of … ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 116 and probably more importantly than all (e) it allows students to learn and improve their oral production skills and self-correct. And here are some reasons for not using this software as part of the assessment of the subject. “Because it is easy to use by following the instructions in the "PDF" document that has been sent to us. Although I prefer to take the test in person, facing the tutor or teacher, because it is more real and more demanding and of course I am going to make more mistakes, but even so, due to the aforementioned, I think it is better.” [Student8] “It is necessary to interact with teachers as much as possible.” [Student11] “Maybe not all of them. There are people who are bad at technology, and others who get nervous having to record themselves.” [Student12] Finally, students were given the chance to provide any comments or suggestions as regards the continuous assessment or the use of new ways of assessing long distance learning students. “I find it very useful and a way to seek objectivity and transparency in the evaluation process.” [Student3] “I like this system, although video conferencing is also a very interesting option and there is much more interaction with the tutors.” [Student5] “I think that promoting the use of new technologies in the PEC can be very useful for all of us who wish to teach English in the future. Powerpoint presentations accompanied by videos can facilitate the teaching of English, and for this reason, it is positive that future Inmaculada Senra Silva ELIA 22, 2022, pp. 93-123 DOI: http://dx.doi.org/10.12795/elia.2022.i22.04 117 teachers begin to become familiar with these programs from the beginning of the degree.” [Student29] “I consider new programs like screencasts have become a fairly good component of the programs of distance learning, because they supply a sense of link between the teaching staff and the students, without compromising the capability of accessing materials of on-line learning, at any time and place that is convenient.” [Student33] “From my point of view, screencast is a good technique to evaluate students, because students feel in their comfort zone and they give the best of them. Many times, nerves betray us. So it is when you feel confident you act naturally.” [Student47] “Anything that involves improving the distance teaching and assessment system is progress. We will have to adapt to the changes that cost us the most and the others will be assimilated quickly.” [Student48] 5. Conclusion The students and teachers involved in the study expressed their satisfaction with the activity. The main goal was to assess students’ oral production. In order to do this, we designed an activity where students produced a video, and later received the teachers’ feedback with an assessment rubric. Due to the preponderance of the teaching of written communication skills in distant learning settings, teachers must provide opportunities to develop and practice students’ oral communication skills (Barrett and Liu, 2016). There is also a need for diversity in oral assessment methods which could provide opportunities for practicing digital technology at the same time that