Translation tutors’ assessment perceptions and practices in Spain: results from the EACT project

HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X Translation tutors’ assessment perceptions and practices in Spain: results from the EACT project Percepciones y prácticas de evaluación del profesorado de Traducción en España: resultados del proyecto EACT ELSA HUERTAS BARROS University of Westminster. School of Humanities. 309 Regent Street, London, W1B 2UW, UK / Reino Unido. Dirección de correo electrónico: e.hu[email protected] ORCID: https://orcid.org/0000-0002-1470-3845 LARA DOMÍNGUEZ ARAÚJO Universidade de Vigo. Departamento de Tradución e Lingüística. Facultade de Filoloxía e Tradución. As Lagoas, Marcosende. 36310 Vigo, Galicia, España. Dirección de correo electrónico: [email protected] ORCID: ORCID: https://orcid.org/0000-0003-1257-838X Recibido/Received: 1/1/2024. Aceptado/Accepted: 10/9/2024. Cómo citar / How to cite: Huertas Barros, Elsa and Lara Domínguez Araújo, «Translation tutors’ assessment perceptions and practices in Spain: results from the EACT Project», Hermēneus. Revista de Traducción e Interpretación, 26 (2024): pp. 253-282. DOI: https://doi.org/10.24197/her.26.2024.253-282 Artículo de acceso abierto distribuido bajo una Licencia Creative Commons Atribución 4.0 Internacional (CC-BY 4.0). / Open access article under a Creative Commons Attribution 4.0 International License (CC-BY 4.0). Abstract: This paper presents findings from a survey-based case study conducted as a part of the research project “Evaluation in the Acquisition of Translation Competence” (EACT) on assessment practices in practical translation modules in undergraduate translation programmes in Spain. The survey included both quantitative and qualitative questions and was completed by ninety-seven translation tutors from five Spanish universities. The purpose of the study was to investigate tutors’ assessment perceptions and practices in undergraduate translation modules to provide an overview of the current situation in Spain and detect existing problems and gaps that require attention. The results show that assessment is a very important aspect of tutor’s teaching and practice. Tutors attach practically the same importance to the translation process as the end product. Most respondents use different types of tasks depending on the competence(s) being assessed and the level of the module. The assessment procedures are in most cases based on professional practice. Source texts are mainly selected on the basis of authenticity, relevance for the competence being assessed, and an appropriate level of difficulty. Tutors use both grading 254 Elsa Huertas Barros and Lara Domínguez Araújo HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X scales and rubrics, with the former being the most common assessment instrument. Most tutors revamp the assessment tasks regularly, although assessment criteria are not revised as frequently. Overall, tutors seem to be adapting to new realities and embedding new translation approaches into their teaching practice but have mixed feelings about their level of satisfaction with assessment methods. Keywords: Assessment; assessment practices; translator competence; translator education; survey. Resumen: En este artículo se presentan los resultados de un estudio de caso realizado en el marco del proyecto de investigación «Evaluación en la Adquisición de la Competencia Traductora» (EACT) sobre las prácticas de evaluación en asignaturas prácticas de traducción de los grados de traducción impartidos en España. A partir de un cuestionario con preguntas cuantitativas y cualitativas, completado por un total de noventa y siete docentes de cinco universidades españolas, se ofrece una panorámica de la situación actual sobre las percepciones y prácticas de evaluación en las asignaturas de traducción en España, se detectan problemas y se proponen áreas de mejora. Los resultados indican que el profesorado otorga una gran importancia a la evaluación en su práctica docente y en general le atribuye prácticamente la misma importancia al proceso que al producto de traducción. La mayoría utiliza distintas tareas según la competencia evaluada y el nivel de la asignatura. Los procedimientos de evaluación utilizados suelen basarse en la práctica profesional y los textos de partida suelen seleccionarse especialmente por su autenticidad, relevancia para la competencia evaluada y adecuación al nivel de dificultad requerido. Como instrumentos de evaluación emplean baremos y rúbricas, siendo los primeros los más comunes. Asimismo, la mayoría actualiza con frecuencia las tareas de evaluación empleadas, si bien los criterios de evaluación no se revisan con tanta asiduidad. En general, el profesorado parece adaptarse a las nuevas realidades e incorpora nuevos enfoques de traducción a su práctica docente aunque cabe destacar que la satisfacción con los métodos de evaluación empleados no es total. Palabras clave: Competencia traductora; encuesta; evaluación; formación de traductores; prácticas de evaluación . Summary: Introduction; 1. Empirical studies on the assessment of translation competence; 2. Methodology and scope, 2.1. Sample, 2.2. Study design and data collection; 3. Results and discussion: tutors’ perceptions of assessment practices in practical translation modules in undergraduate translation programmes in Spain, 3.1. Importance granted to assessment and assessment of the translation product vs. process, 3.2. Formative and blind assessment, 3.3. Factors influencing assessment: assessment tasks and directionality, 3.4. Assessment procedures and text selection, 3.5. Assessment tools, 3.6. Changes to and adaptation of assessment criteria and tasks, 3.7. Satisfaction; 4. Conclusions; Funding information; Acknowledgments; References. Sumario: Introducción; 1. Estudios empíricos sobre la evaluación de la competencia traductora; 2. Metodología y ámbito de estudio, 2.1. Muestra, 2.2. Diseño del estudio y recogida de datos; 3. Discusión de resultados: percepciones del profesorado con respecto a las prácticas de evaluación en las asignaturas prácticas de traducción impartidas en los grados en Traducción en España, 3.1. Importancia otorgada a la evaluación y evaluación del producto de traducción vs evaluación del proceso de traducción, 3.2. Evaluación formativa y evaluación ciega, 3.3. Factores que influyen en la evaluación: tareas de evaluación y direccionalidad, 3.4. Procedimientos de evaluación y selección de textos, 3.5. Instrumentos de evaluación, 3.6. Cambios y adaptaciones de los criterios y tareas de evaluación, 3.7. Nivel de satisfacción; 4. Conclusiones; Información sobre financiación; Agradecimientos; Referencias. Translation tutors’ assessment perceptions and practices in Spain 255 HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X INTRODUCTION With the emergence of some of the most influential models of translation competence (e.g., Kelly, 2002, 2007; PACTE, 2003, 2017; EMT, 2022), the past two decades have witnessed an increasing number of studies centring on the evaluation of translation competence. While most of these proposals focus on summative assessment and evaluation of the product –with an emphasis on error analysis (e.g., House, 1977, 2014; Nord, 1992; Pym, 1992)– some of the latest research attaches increasing importance to formative assessment and suggests ways of assessing the translation process more systematically. In an attempt to facilitate this task, some recent proposals (e.g., NACT1 and EFFORT2 projects) have established competence levels in the acquisition of translation competence, including a proposal of a European Framework for Translation. Adab and Schäffner’s (2000) contribution to research on translator competence assessment addresses the evaluation of the product in terms of expected levels of translation competence. Huertas Barros and Vine (2016, 2018, 2019a) examine the translation product from the perspective of tutors’ perception of summative assessment practices in language-specific translation modules. Other proposals focus on the decision-making process. For instance, Way (2008) presents a systematic approach based on translator competence self-assessment as well as a Project Management Sheet (2009) to analyse translation problems and find effective tools and reliable resources to solve them. Gaballo (2009) suggests a systemicfunctional model of translator competence that can serve as a tool to assess translator competence acquisition and the translation process. Way (2014) proposes a decision-making framework for legal translation that improves students’ confidence and performance. Massey and Ehrensberger-Dow (2012, 2013) explore the assessment of translation processes in laboratorybased and workplace projects to complement traditional assessment. More recently, Huertas Barros and Vine (2019b) suggest ways of increasing 1 Establishing Competence Levels in the Acquisition of Translation Competence in Written Translation / Nivelación de competencias en la adquisición de la competencia traductora (traducción escrita). Research project funded by the Spanish Ministry of the Economy and Competitiveness (reference: FFI2013-42522-P). Further information available at: https://ddd.uab.cat/record/273075. 2 Towards a European Framework of Reference for Translation. Further information available at: https://www.effortproject.eu/. 256 Elsa Huertas Barros and Lara Domínguez Araújo HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X assessment literacy for students to fully understand the purpose and processes of assessment. Over the past few years, there has also been progress with regard to assessment instruments and tasks, with alternative proposals made for assessing students’ translation competence instead of the traditional translation task. These include diagnostic questionnaires, questionnaires on translation problems, translation commentaries or annotations, reflective diaries, recordings of the translation process, peer and selfassessment and portfolios, among others (e.g., Adab, 2000; Martínez and Hurtado Albir, 2001; Orozco and Hurtado Albir, 2002; Kelly, 2005; Colina, 2008; Galán-Mañas and Hurtado Albir, 2015; Hurtado Albir, 2007, 2015a, 2015b; Hurtado and Olalla-Soler, 2016; Li, 2018). A range of rubrics has also been proposed lately for assessing students’ work both formatively and summatively in a more manageable way (e.g., Angelelli and Jacobson, 2009; Presas, 2012; Hurtado Albir, 2015; Pavani, 2016; Galán-Mañas 2016, 2018; Hurtado Albir and Pavani, 2018; Way, 2021). While it is clear that translation assessment remains a core element of the teaching and learning process, there is a recognition that “individual impressionistic assessments need to be made more valid through the introduction of more objective assessment criteria” (Huertas Barros and Vine, 2016, p. 29). In this paper, we present our findings from a survey-based study on tutors’ assessment perceptions and practices in the practical translation modules of five undergraduate translation programmes in Spain. The study is framed within the research project “Evaluation in the Acquisition of Translation Competence” (EACT)”3 and has two main aims: (1) to investigate and gain a clearer understanding of the current situation of assessment in undergraduate translation degrees in Spain; and (2) to propose a range of assessment criteria and procedures for a more systematic evaluation of students’ level of translation competence. EACT is the continuation of the NACT research project led by the PACTE group, which established performance levels in translation competence acquisition (PACTE, 2018; Hurtado Albir and Rodríguez Inés, 2022). More specifically, the EACT project focused on translation levels A1, A2, and B1 (that is, introduction to translation and introduction to the fields of specialised translation) from the students’ first foreign language (German, 3 Further information available at: https://webs.uab.cat/eact/en/. Also, see the “Funding information” section. Translation tutors’ assessment perceptions and practices in Spain 257 HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X French, or English) into their mother tongue (Catalan, Galician, Spanish, or Basque). The project also covered translation between two first languages when applicable (Spanish-Basque, Spanish-Catalan, and Spanish-Galician). The EACT project involved nineteen researchers from five Spanish universities representing a range of languages and geographical areas (Universitat Autònoma de Barcelona (UAB); Universidad de Granada (UGR); Universitat Jaume I (UJI); Universidad del País Vasco/Euskal Herriko Unibertsitatea (UPV/EHU); and Universidade de Vigo (UVIGO), as well as the University of Westminster in the UK. The present study aims to investigate translation tutors’ assessment perceptions and practices to gain a clearer understanding of the strenghts and weaknesses of current assessment practices in undergraduate translation modules in Spain4. 1. EMPIRICAL STUDIES ON THE ASSESSMENT OF TRANSLATION COMPETENCE Despite an increasing number of research studies on assessment of translation competence over the past decade, empirical research on this topic remains scarce, both at undergraduate and postgraduate level. Of the existing empirical studies, the following deserve particular consideration. Conde Ruano’s research (2009) evaluated the process and product of translations with a focus on Spain, Mexico, and the United Kingdom. The study was carried out with eighty-eight participants (i.e., forty potential translation recipients, eleven professional translators, twenty-seven undergraduate translation students, and ten translation tutors), who were surveyed during the academic year 2004-2005, and included Spanish and English as main languages. The main findings relating to assessment of the texts included the following: 1) textual factors seemed to have a significant impact on assessors’ marking behaviour in the assessment of translations. There was a tendency for all assessors to be influenced by textual factors; 2) translation recipients were more inexperienced, insecure, and superficial in their assessment of translation. Professional translators were used to being assessed themselves and adopted a more professional attitude when marking. Undergraduate students tended to be more meticulous and homogeneous when assessing the translation, and 4 See also further studies conducted as part of the EACT project, see Gay Punzano and Hurtado Albir (forthcoming). 258 Elsa Huertas Barros and Lara Domínguez Araújo HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X translation tutors focused on training aspects and the learning process and demonstrated more experience in marking a high volume of work; and 3) the duration and sequencing of the task had an impact on the assessors’ marking behaviour. In other words, there was a tendency for assessors to make fewer corrections and leave fewer comments on texts as the marking process progressed. They needed a period of time to familiarise themselves with the task, and the final part of a translation did not influence their overall opinion of the translation as much as the previous parts. In an empirical study on conference interpreting, Domínguez Araújo (2015) examines tutors’ and students’ perceptions and practices in three postgraduate courses in Spain and Portugal, with the aim of improving the learning process and formative assessment. The empirical study involved an interview and survey of eleven conference interpreting tutors and focus groups, in addition to a survey of twenty-seven students, and class observation. The study centred on formative assessment and the main findings were as follows: 1) tutors and students alike considered assessment to be a core element for fostering student metacognition, selfregulation, and autonomy, but. the disparity of assessment criteria and excessive assessment were some drawbacks; 2) regarding perceptions of assessment, participants felt students were able to learn how to manage feedback and highlighted the importance of non-verbal communication and first impressions. They agreed on the importance of tutors sharing the learning objectives with students; 3) according to participants, effective assessment should include strengths, weaknesses, and strategies to overcome said weaknesses, and according to students, it should be inclusive, ipsative, and objective. Tutors also highlighted the importance of establishing priorities and fostering students’ self-regulation and motivation; and 4) regarding assessment in the classroom and other forms of assessment, there was agreement on the benefits of self-assessment and peer-assessment, but, according to tutors, there was also a need for more clarity, explicitation, and consistency of assessment criteria. Pavani’s work (2016) focused on summative assessment at undergraduate level in translation training. The author carried out an empirical study involving a survey of sixteen translation tutors in Italy and Spain. The findings suggested that most translation tutors used a translation task of one or more texts as the most common form of assessment. Other tasks included: translation revisions; sight translation; terminology-related tasks; documentary research; invoicing; creation of a translation memory; terminology logs; glossaries; translation Translation tutors’ assessment perceptions and practices in Spain 259 HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X commentaries; translation comparison; and analysis of translation problems. Most tutors used a grading scale based on translation errors to assess a translation. The most important contribution of Pavani’s work is the proposal of a multidimensional system for summative assessment supported by an empirical study to assess its effectiveness. Garde (2016) provided a state-of-the-art of assessment in translation informed by the results of semi-directed interviews and a survey completed by tutors and students (389 participants) from eight Canadian universities. The results showed that the assessment methods currently in use have not changed substantially in the past few years despite the significant changes to the translation profession during this period. The author concluded that most tutors see assessment as an individual act that focuses on the product (and not the process) and does not account for progression in the learning process. For their part, students felt stressed about assessment, hence the need for tutors to coordinate and put measures in place to mitigate this. The author advocates for a deep reflection on teaching and learning approaches so that they can accommodate the shifts undergone by the translation industry over the past few years. Huertas Barros and Vine (2016, 2018, 2019a, 2019b) have carried out a number of empirical studies on assessment practices in undergraduate and postgraduate translation courses in the UK. The first stage focused on a case study of translation tutors’ (2016) and students’ (2019a) perceptions of assessment practices in an MA Translation and Interpreting (T&I) course at the University of Westminster. A total of fifteen translation tutors and fifty-three postgraduate translation students were surveyed as part of the first stage. The results showed differing levels of trust and satisfaction with the assessment instruments used and brought to the fore the need to refine assessment instruments, methods and practices to make them more valid and reliable for students and tutors. In the second stage of their research, Huertas Barros and Vine (2018) carried out documentary research and a UK-wide survey on assessment practices in MA T&I programmes involving a total of twenty-seven universities. The survey focused on universities’ learning outcomes, assessment instruments and the assessment criteria used in languagespecific practical translation modules. The survey results revealed that models of translator competence had become widely adopted by universities, as well as there being a gradual shift from a focus on the product to a focus on the translation process. Nevertheless, the results also 260 Elsa Huertas Barros and Lara Domínguez Araújo HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X showed the need to include or strengthen assessment literacy and ensure reliability of professional judgment. In order to mitigate such issues, in their most recent work (2019a, 2019b), the authors suggested informed proposals in which assessment literacy can be increased or effectively integrated into translation course design by constructing assessment standards in communities (i.e., among students, tutors, and the industry). Huertas Barros and Vine’s proposal (2019b) was supported by an empirical study on students’ perceptions of assessment criteria and their experience of negotiating such standards in communities of practice. The results of the survey indicated that the collaborative creation of assessment criteria as part of the case study provided tutors and students with a clearer understanding of what was being assessed, the criteria used, and how they were applied. Haro Soler’s work (2018) centred on self-confidence and its role in translator training. The author conducted an empirical study, involving both a survey and focus groups, with thirty-nine undergraduate students from the University of Granada in Spain. The study analysed the impact of assessment practices on students’ self-confidence in a specialised translation module (Spanish-English). The study concluded that teamwork and group presentations of work carried out outside the classroom generally have a positive impact on students’ self-confidence. In a similar vein, Haro Soler’s study (2018) revealed that the use of rubrics, assessment criteria, constructive feedback, and self-assessment instruments and systems –such as the Achilles’ Heel record sheet (Way, 2008)– also had a significant influence on students’ self-confidence. 2. METHODOLOGY AND SCOPE The data and results presented in this paper come from the analysis of a survey conducted in 2021 with translation tutors from five universities in Spain (UPV/EHU, UVIGO, UJI, UAB, and UGR). The study involves a mixed-methods approach combining both quantitative and qualitative data collection and analysis, with the former taking precedence in the study. In this section, we describe the population and sample of the study as well as the design of the instrument used for data collection. Translation tutors’ assessment perceptions and practices in Spain 261 HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X 2.1. Sample The study involved ninety-seven tutors teaching on translation modules in undergraduate translation programmes in the five universities mentioned above. The sample included tutors teaching from or into any of the four official languages in Spain (Spanish, Catalan, Galician, and Basque) in any year of the universities’ four-year translation degrees. It encompassed universities from monolingual and bilingual areas, with varied experience in translation teaching and a diverse student body. This offers a representative panorama of the diverse public-university environment in Spain. Following a thorough analysis of the information available in the translation curricula of the aforementioned universities, a potential sample of 163 translation tutors was identified. In terms of participation by university, 25% of the sample comprised tutors from UAB; 25% from UVIGO; 20.7% from UGR; 15.2% from UPV/EHU; and 14.1% from UJI. Comparing the total population in each university, the university with the largest representation was UPV/EHU (amounting to 82.35% of the potential tutors taking part in the survey), followed by UVIGO (71.88%), and UJI (61.9%). UAB and UGR had the lowest representation rate, with 47.92% and 43.18% of tutors completing the survey respectively. A total of 11.3% of respondents were involved in the EACT project. Of the ninety-seven tutors comprising the sample, 69.1% were women and 30.9% were men, with varied experience in translator training and an average age of forty-eight years (standard deviation of 10.5 years). More specifically, 35.1% of the participants in the sample had more than 20 years’ experience; 33% had between 11- and 20-years’ experience; and 31.9% had fewer than 11 years’ experience in teaching translation. In terms of their native languages, 56.3% of participants had a single mother tongue; 38.5% were bilingual; and 5.2% had more than two mother tongues. Spanish was the most widely represented mother tongue (69.8%), followed by Catalan (22.9%) and Galician (19.8%). English (8.3%), Basque (7.3%), German (6.3%), and French (5.2%) were also represented, as well as Amazigh, Chinese, Georgian, Dutch, Portuguese, Russian, and Arabic, the latter to a lesser extent (11.5% collectively).5 As for their 5 These tutors taught in a variety of language pairs depending on the translation languages offered at their universities. The language combinations ranged from English-Spanish (the most common pair) to Russian-Basque or Chinese-Spanish/Catalan (the least 268 Elsa Huertas Barros and Lara Domínguez Araújo HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X selected the option “Other cases”. These included: text length (suited to the allocated time); students’ interest in the topic; text variety; translation problems students have dealt with as part of the module; and the source text not being available in the target language in question. These findings show tutors’ awareness of the importance of text selection, as underlined by Conde Ruano’s study (2009), which suggested that textual factors – such as the degree of specialisation or text length – have an impact on assessment. The results also reveal a learning-centred approach in line with the importance tutors attach to the process when assessing translation competence, as well as a will to build bridges between academia and professional practice, which also shows in tutors’ use of assessment procedures based on professional practice (see table 2). 3.5. Assessment tools As for assessment tools, the use of grading scales for correction purposes is widespread among the tutors (69.5% use them regularly and 21.1% use them occasionally). This mirrors Pavani’s findings in Italy and Spain (2016). As for the type of scale used, the vast majority of tutors use grading scales with assigned numerical values (80.2%), which may suggest an interest in and desire to achieve more objectivity and transparency in the assessment process. On the other hand, the fact that 86% of tutors reward students for correct solutions in their grading scales indicates this traditional grading system has evolved from an error-based correction to a more student-centred, competence-based approach. This may be a reflection of tutors’ greater awareness of the impact that assessment practices have on students’ self-confidence (as suggested by Haro Soler, 2018). Regarding authorship of the grading scales, 72.9% of tutors use their own grading scale. The remaining tutors employ grading scales from scholars such as Hurtado Albir (1999, cited by five respondents); the Petra-E framework (one respondent) and ATA (American Translators Association, one respondent); or grading scales agreed by several tutors teaching on the same module (two respondents). More than half of the tutors (55.1%) use different grading scales depending on the module, while 40.7% use the same grading scale in all the modules they teach on (the remaining 9.3% only teach on one translation module). In contrast, rubrics – which are a more recently introduced tool in translator education – are less widely used than grading scales. That said, 69.1% of tutors claim to use them regularly (39.2%) or occasionally Translation tutors’ assessment perceptions and practices in Spain 269 HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X (29.9%). In such cases, tutors use different rubrics depending on the task being assessed (73.1%) and half of them combine qualitative and quantitative rubrics. According to the findings, tutors seem to value rubrics very highly (with a mean above 3.5 and a median of 4), as they feel rubrics allow for greater transparency and accuracy as well as helping them to identify those areas in which students need to improve. Other benefits highlighted by tutors regarding the use of rubrics include objectivity (M = 3.46; Mdn = 4) and motivation for the students (M = 3.30; Mdn = 3). Rubrics were also mentioned in the final open-ended question in the final section for further consideration. Three respondents consider rubrics particularly useful in formative assessment, especially for identifying areas showing room for improvement or for highlighting good solutions implemented by students, while two other participants feel rubrics help simplify assessment. A further respondent feels rubrics make discussions on assessment between tutors and students easier, while another underlines the transparency they provide in the marking process. 3.6. Changes to and adaptation of assessment criteria and tasks A high percentage of tutors (80.4%) have changed the assessment tasks they normally use in the translation classroom over the past five years, and 58.7% have changed the assessment criteria they use over the same period. This finding suggests that translation tutors are aware of the importance of refreshing and refining assessment practices on a regular basis. When looking at both assessment tasks and criteria more closely (see table 3), the data obtained show that assessment criteria tend to be more stable than assessment tasks, with the latter being refined more frequently. In both cases, such changes have mainly been implemented to reflect more accurately the way professional translators work (68.5% in the case of assessment tasks and 70.4% in the case of assessment criteria respectively). This is in line with their previous responses on assessment procedures and text selection. On other occasions, such changes have been implemented to adapt to new approaches to teaching translation (58.9% and 70.4% respectively) and to make the most of new software/resources available at the tutors’ universities (42.5% and 35.2% respectively). Two participants who selected the option “Other reasons” highlighted that they have changed the assessment tasks they use to refine them based on their professional experience. Two other participants claimed that they have implemented such changes to better adapt to current trends in text 270 Elsa Huertas Barros and Lara Domínguez Araújo HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X selection. Other reasons behind changes to assessment tasks included optimising the time participants spend on assessment; adapting to the university context and students’ level of language competence; allowing for assessment of other competences; and offering students a wider range of translation tasks (one participant per response). As for changes to assessment criteria, the three tutors who selected the option “Other reasons” highlighted that they have done so to refine these criteria based on their professional experience. Other reasons included standardising assessment criteria in line with those used by other colleagues; reflecting students’ level of translation competence; and considering students’ progress. Changes in the last 5 years Assessment tasks Assessment criteria N = 97 (n = 92; DK/DA = 5) N = 97 (n = 92; DK/DA = 5) Yes 80.4 % 58.7 % No 19.6 % 41.3 % Reason for the change N = 97 (n = 73; n. a. = 23; DK/DA = 1) N = 97 (n = 54; n. a. = 43) To better reflect how professional translators work 68.5 % 70.4 % To adapt to new approaches to teaching translation 58.9 % 70.4 % To take advantage of changes in the software / resources available in my university 42.5 % 35.2 % To adapt to changes in student numbers 35.6 % 33.3 % Other reasons 12.3 % 11.1 % Table 3. Changes in assessment procedures in the last 5 years In terms of changes as a result of COVID-19, 56.5% of respondents said that they did not change their assessment practices after the sudden switch to online teaching caused by the pandemic. The tutors who did change their assessment practices (43.5%) did so mostly by changing the percentage assigned to each task (67.5%) and/or by changing the allocated time for its Translation tutors’ assessment perceptions and practices in Spain 271 HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X completion (55%), while some tutors increased (20%) or reduced (20%) the number of assessment tasks. This may be attributed to a lack of time for adapting assessments to online teaching, a lack of flexibility from their universities, or the psychological need to pretend to continue with “business as usual”. Moreover, in the open-ended option for this question, respondents mentioned participation in the discussion fora available on their module’s online platform as a new assessed task; a reduction in text length and time to complete assessed tasks; and a complete shift towards continuous assessment. All these options were selected by one participant. Occasionally, such adaptations and refinements have been implemented due to external factors and circumstances (e.g., COVID-19). 3.7. Satisfaction Responses show that tutors were highly satisfied with the assessment they carried out (M = 3.27; Mdn = 3), although this satisfaction was not absolute. Tutors’ satisfaction with their assessment practices is mainly attributed to four factors: using formative assessment (i.e., assessment that does not count towards the final mark) provides students with feedback (72.7%); using translation grading scales and rubrics provides a basis for assessing all students equally (69.3%); self-assessment and peerassessment encourage students to reflect on what they have learned (65.9%); and rubrics let students know what to expect and guarantee greater transparency (61.4%). A total of 10.2% of tutors selected the response “Other reasons”. Three respondents mentioned the fact that the assessment system is fair and clear (with two of them referring specifically to the use of grading scales); two respondents highlighted the importance of feedback as part of the learning process and formative assessment; and another tutor claimed that the subjectivity and uncertainty that may be associated with assessment prepares students for the profession. Finally, one tutor mentioned the combination of product and process assessment. With regard to dissatisfaction, respondents who were not satisfied with their assessment largely attributed this to two main factors: the subjectivity generally inherent to any assessment (71.4%) and the assignment of a numerical mark (71.4%). These results confirm the need, brought to the fore by Huertas Barros and Vine’s studies (2016, 2018, 272 Elsa Huertas Barros and Lara Domínguez Araújo HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X 2019a, 2019b), to refine assessment instruments, methods, and practices to make them more valid and reliable for students and tutors. Other reasons for tutors’ dissatisfaction with the assessment they carried out included: (1) the lack of standardisation of the assessment procedures and criteria used by different tutors at the same institution; (2) the difficulty involved in making the assessment process transparent; (3) the difficulty involved in tailoring assessment owing to overly large groups; (4) the heterogeneity of translation students (including different levels of language competence, cultural competence, and technological competence); (5) the complex nature of translation, in that there can be various suitable solutions; and (6) the lack of teacher training in assessment. All these options were selected by 57% of tutors who were not satisfied with their assessment. Further reasons for dissatisfaction relate to the conditions in which translation teaching takes place and the difficulty of implementing a learning-centred approach when the academic settings or external circumstances are unsuited to this. The other reasons for dissatisfaction, selected by 28.6% of the respondents, included: (1) the lack of importance attributed to formative assessment (i.e., assessment that does not count towards final grades); (2) the difficulty involved in using varied, extensive assessment tasks owing to overly large groups; (3) the challenges posed by the use of new translation resources stemming from technological development (such as use of translation memories and machine translation); and (4) the difficulty involved in balancing professional requirements and pedagogical objectives. These results reveal the need for improvement highlighted in previous studies, such as a need for clearer, more coherent, and more explicit assessment criteria (Domínguez Araújo, 2015) and a need to embed assessment literacy into course design to ensure the reliability of professional judgment (Huertas Barros and Vine, 2018, 2019a, 2019b). 4. CONCLUSIONS The findings of our study show that respondents generally consider assessment to be a very important aspect of their teaching activity and practice. With regard to the assessment of students’ level of translation competence, the results suggest that tutors attach practically the same importance to the translation process and the end product. Respondents perceive both elements as essential, particularly with regard to the Translation tutors’ assessment perceptions and practices in Spain 273 HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X competence(s) being assessed, the year of the undergraduate degree, and in the case of specialised and general translation. It is worth mentioning that respondents consider the translation process to be more important in the case of specialised translation than in the case of general translation. In order to consolidate the translation process, tutors use formative activities to identify students’ progress, their strengths and weaknesses, and the main difficulties they encounter during the translation process. As far as assessment tasks are concerned, most respondents use different types of tasks depending on the competence(s) being assessed and the level of the module. In terms of blind assessment, the percentage of tutors who apply this method is very similar to the percentage of tutors who do not. Furthermore, the assessment procedures used by most tutors are based on procedures used in professional practice, especially when it comes to the competences being evaluated, the year of the degree, and specialised translation. The results show that when tutors teach on various modules, their assessment practices differ according to language directionality but not the particular language pair(s) involved. With regard to the texts used in assessed tasks, these are selected on the basis of authenticity, relevance for the competence being assessed, and an appropriate level of difficulty. Further criteria for text selection mentioned by tutors included text length, potential interest for students, genre variety, the inclusion of translation problems dealt with during the module, and the translation not being available in the target language. As for assessment tools, grading scales and rubrics are both used by participants. Grading scales are by far the most common assessment instrument, while the use of rubrics is not as widespread among tutors. Grading scales, which are normally designed by tutors themselves, generally contain numeric values assigned to criteria and take into consideration good solutions proposed by students. Rubrics, on the other hand, are highly valued by tutors as they feel they provide equity, objectivity, transparency, and accuracy when identifying areas for further improvement. The results show that tutors use both quantitative and qualitative rubrics, as well as dedicated rubrics for each assessed task. Nevertheless, this appreciation for rubrics does not seem to translate into widespread use, highlighting the fact that tutors’ willingness to use rubrics does not always translate into their practical application. This may also be attributed to the different teaching styles and approaches encompassed in the sample. This finding highlights the need to embed assessment literacy into module design more effectively to ensure assessment in general – and 274 Elsa Huertas Barros and Lara Domínguez Araújo HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X professional judgment more specifically – is more valid and reliable (as shown in Huertas Barros and Vine’s studies (2018, 2019a, 2019b). In terms of reviewing assessment practices, the findings show that most tutors have revised and revamped the assessment tasks they use in their onsite translation classes over the past five years. While this suggests tutors consider assessment to be an important aspect of the learning and teaching process, a considerably lower percentage of tutors (approximately half of the respondents) have reviewed or revamped their assessment criteria over the same period of time. The main reasons for which assessment practices have been reviewed regularly are changes in the software and resources available at universities and the need to incorporate new pedagogical approaches to translation teaching. On other occasions, such changes are associated with external factors. For instance, 43.5% of tutors changed their assessment practices due to the sudden switch to online teaching caused by COVID-19. Tutors’ predisposition to enhance assessment practices on a regular basis reflects their interest and willingness to keep adapting to new realities and embedding new translation approaches into their teaching practice. This, in turn, highlights the need to regularly update and revamp assessment procedures, including assessment instruments, criteria, and tasks. In this vein, the findings show tutors have mixed feelings about assessment methods. Causes for satisfaction include the positive impact that feedback has on students’ learning process; the greater equity, transparency, and clearer expectations provided by grading scales and rubrics; and the benefits associated with self-reflection and peerassessment. Tutors’ degree of satisfaction with their assessment practices may also be attributed to their awareness of assessment and the widespread use of tailored assessment instruments. The latter seem to help them identify students’ strengths and weaknesses more easily. On the other hand, the main causes of tutors’ dissatisfaction with their assessment practices are subjectivity and the difficulty of ensuring transparency. This brings to the fore the need to refine assessment instruments, methods, and practices so that both students and tutors deem them to be more valid and reliable, as well as a need to recognise that assessment lacks precision (Huertas Barros and Vine, 2016, 2019a, 2019b). Overall, these findings show that tutors have a fairly positive perception of their assessment practices. A variety of assessment tasks are being used in translation modules and tutors are willing to review and Translation tutors’ assessment perceptions and practices in Spain 275 HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X enhance these regularly. Professional practice also seems to have an influence on text selection, the design of assessment procedures, and changes to assessment criteria. This may be due to tutors’ involvement in the professional world. The findings also suggest a process-centred approach to assessment, with a moderate shift from a focus on the product to a focus on the translation process, as well as the introduction of professional practices into the teaching sphere. This study sheds light on the current assessment practices used in translation training centres in Spain and identifies their strengths and weaknesses, which can lead in turn to future improvement. New developments and enhancements in these areas will, consequently, result in higher quality teaching and satisfaction with assessment practices amongst both tutors and students. While the aforementioned trends in relation to tutors’ assessment practices and perceptions are fairly positive, such trends cannot be generalised across the entire tutor population in Spain, given that the sample was selected on a voluntary basis rather than being random. This opens up the possibility of vast and diverse avenues for future research. For instance, it would be worth considering future qualitative studies of a similar nature that would complement the results obtained in this study. There is also a need for similar studies that specifically address specialised translation courses (e.g., technical, scientific, medical, legal translation) and interpreting, as well as the use of AI tools in assessment. FUNDING INFORMATION Research project funded by the Spanish Ministry of Science and Innovation, under the name Evaluation in the Acquisition of Translation Competence (“Evaluación en la adquisición de la competencia traductora”, EACT). Reference: PGC 2018- 094275-B-I00). Further information available at: https://webs.uab.cat/eact/en/ 276 Elsa Huertas Barros and Lara Domínguez Araújo HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X ACKNOWLEDGMENTS We sincerely thank our dear colleagues Rosa Agost, Tomás Conde Ruano, Lourdes Gay Punzano, María del Mar Haro- Soler, Amparo Hurtado Albir, Christian Olalla-Soler, and Catherine Way, who are co-authors of the questionnaire. We also thank the external experts for their valuable advice during the questionnaire design and piloting stages. Finally, we would like to thank all the translation tutors who took the time to complete the survey. REFERENCES Adab, Beberly (2000). Evaluating Translation Competence. In Christina Schäffner and Beberly Adab (Eds.), Developing translation competence (pp. 215-228). John Benjamins. https://doi.org/10.10 75/btl.38 Angelelli, Claudia, and Jacobson, Holly E. (Eds.) (2009). Testing and Assessment in Translation and Interpreting Studies. John Benjamins. https://doi.org/10.1075/ata.xiv Colina, Sonia (2008). Translation quality evaluation: Some empirical evidence for a functionalist approach. The Translator 14 (1), 97-134. https://doi.org/10.1080/13556509.2008.10799251 Conde Ruano, José Tomás (2009). Proceso y resultado de la evaluación de traducciones [Doctoral thesis]. Universidad de Granada. http://hdl.handle.net/10481/2309 Domínguez Araujo, Lara (2015). La evaluación para el aprendizaje de la interpretación de conferencias. [Doctoral thesis]. Universitat Autònoma de Barcelona. http://hdl.handle.net/10803/317965 Domínguez Araújo, Lara (2016). La importancia de la evaluación formativa para los docentes de interpretación de conferencias. Translation tutors’ assessment perceptions and practices in Spain 277 HERMĒNEUS, 26 (2024): págs. 253-282 ISSN: 2530-609X Polissema. Revista de Letras do ISCAP, 16, 25-52. https://doi.org/ 10.34630/polissema.v0i16.2958 EMT Expert Group (2022). European Master’s in Translation. EMT Competence Framework. Brussels: European Commission. Retrieved on 12/28/2023 from https://commission.europa.eu/news/up datedversion-emt-competence-framework-now-available-2022-10-2 1_en Gaballo, Viviana (2009). The Umbilical Cord Cut: English as Taught in Translator Training Programs. In Viviana Gaballo (Ed.), English in Translation Studies: Methodological Perspectives (pp. 41-64). Edizioni Università di Macerata. Galán-Mañas, Anabel (2016). Learning Portfolio in translator training: the tool of choice for competence development and assessment. The Interpreter and Translator Trainer 10 (2), 161-182. https://doi.org/ 10.1080/1750399X.2015.1103108 Galán-Mañas, Anabel (2018). Professional portfolio in translator training: professional competence development and assessment. The Interpreter and Translator Trainer 13 (1), 44-63. https://doi.org/ 10.1080/1750399X.2018.1541295 Galán-Mañas, Anabel, and Amparo Hurtado Albir (2015). Competence Assessment Procedures in Translator Training. The Interpreter and Translator Trainer 9 (1), 63-82. https://doi.org/10.1080/1750399 X.2015.1010358 Garde, Philippe (2016). L’évaluation en didactique de la traduction : un état des lieux. Didactic assessment in translation: a comprehensive update. Jostrans, 26, 20-49. https://jostrans.org/issue26/art_gardy.pdf Gay Punzano, Lourdes (forthcoming), and Hurtado Albir, Amparo (2024) Assessment procedures in translation degree programmes in Spain. Results of the EACT project. Meta, 69 (2). Haro-Soler, María del Mar (2018). Las creencias de autoeficacia del estudiantado de Traducción: una radiografía de su desarrollo.