Evaluating the design of digital tools for the transition to an e‑continuous assessment in higher education

Vol.:(0123456789) Journal of Computing in Higher Education https://doi.org/10.1007/s12528-023-09381-2 1 3 Evaluating thedesign ofdigital tools forthetransition toane‑continuous assessment inhigher education JoséI.Castillo‑Manzano1 · MercedesCastro‑Nuño1 · LourdesLópez‑Valpuesta1 · MaríaTeresaSanz‑Díaz2 · RocíoYñiguez2 Accepted: 26 July 2023 © The Author(s) 2023 Abstract Evaluation is a crucial part of the teaching and learning process in any higher education institution and one that has gone through a deep change. This has been particularly true since the Bologna Declaration (http:// www. ehea. info/ pageminis terialconfe rencebolog na1999, 1999) ushered in the European higher education area, with the subsequent major rise in the employment of continuous assessment methods focused on student participation. This article analyses the impact on academic performance of e-continuous assessment based on e-tests on a virtual platform as a previous step towards the substitution of the traditional evaluation system, based on a final exam, with a continuous evaluation system, prescribed as an alternative preferred by the regulations of multiple Spanish universities. Microeconometric models have been applied to a database of 250 first-year students on the Business Administration and Management course at the University of Seville (Spain). Our findings show that e-tests could prevent the risk of students dropping out and could also provide a credible predictor of students’ academic marks in the theoretical contents of the subject, but not in those of a practical or applied nature. Based on the results of this evaluation, an e-continuous assessment has been developed in the subject, which has become the majority option for students, with 90% participation, while also increasing pass rates. Moreover, the positive effect of a computing environment does not appear to be limited to the classroom, but also extends to students’ home environments. This teaching experience shows that the swift feedback that e-tools provides, especially in especially in environments of large class size such as in the class evaluated, could support instructors’ personal tutoring of students’ progress and promote a greater implementation of e-continuous assessment in Spanish higher education. Keywords e-test· Continuous assessment· Academic performance· Higher education· Microeconometric models· Bologna Declaration Extended author information available on the last page of the article J.I.Castillo-Manzano et al. 1 3 Introduction The Bologna Declaration of 1999, which ushered in the European Higher Education Area (EHEA), proposed, among other questions, a change of the teaching–learning model and, to this end, introduced Continuous Assessment (hereinafter, CA) in European universities (Sanz-Pérez, 2019). The academic literature has highlighted the advantages of CA from the perspective not only of students, since it enhances their motivation for learning through feedback on their knowledge (Day etal., 2018), but also of instructors, as it encourages them to obtain feedback from students on their learning (Myllymäki, 2013). A number of publications have also focused on the impact of continuous assessment on student engagement (Cole & Spence, 2012), perception-satisfaction (Carrillo-Peña & Pérez, 2012) and dropping out of the subject (García etal., 2014), with its impact on examination marks being one of the most controversial effects (Carrillo-Peña & Pérez, 2012; Cole & Spence, 2012; Gallardo & Montolio, 2011; García etal., 2014; González etal., 2015). However, one of the main barriers to the application of effective CA has always been class size (Broadbent etal., 2018) since, when there is a high number of students in the class, instructor feedback to students can become a highly repetitive and timeconsuming process, making delivery of ‘timely’ feedback very difficult (McCarthy, 2017). Furthermore, CA seems to work better in small classes since in large groups teachers and instructors may be unable to give the individualised attention that the system requires for every student to develop the prescribed skills (González-Campos etal., 2018). Despite its recent introduction in Europe at the end of the twentieth century, the use of CA in higher education is not a new idea. End-of-semester exams have been supplemented or replaced by several CA systems in Australia, the UK, and New Zealand over the last 40years (Richardson, 2015). Universities in the USA have also used CA for decades. In fact, at Harvard University, since 2010, a final exam has only been held by special permission as a supplementary tool to CA (Harvard Magazine, 2010). In the case of Europe, although the Declaration of Bologna established the year 2010 as the common horizon for the implementation of the EHEA, the implementation of CA as an element of the new teaching and learning paradigm was not carried out at the same rate across the many different countries. For example, the use of CA at Danish universities was allowed in 2016 (Bjælde etal., 2017). In the case of Spain, where this paper is framed, CA has constituted one of the central elements upon which the methodology of the new university model has been built (Quiroga etal., 2014), although the process This adaptation process has occurred progressively since 2010, as each university has introduced it into their University student statutes, whilst always considering that the evaluation of academic performance should converge towards a system that contemplates CA (Ministerio de Educación, 2010). All things considered, this paper analyses the transition towards a CA system in the subject Principles of Economics, which is taught in the first year of the 1 3 Evaluating thedesign ofdigital tools forthetransition to… degree in Business Administration and Management at the University of Seville (Spain) with a high number of students enrolled, and specifically evaluates the contribution made by certain digital tools to optimise the operation of the CA system. In accordance with the Spanish University regulation that came into force in the 2010–11 academic year and that required the implementation of the EHEA, the faculty of this subject began the gradual process of transition from traditional evaluation, based exclusively on an exam at the end of the course, into a system in which the CA was the priority choice by the student, thus following the mandate of the regulations on evaluation of the University of Seville, which, like the rest of the universities in its environment, considers CA to be the priority evaluation system today. In an initial phase of this transition process towards the implementation of CA, students were encouraged to choose to take the subject through CA, a conventional system, based on written activities carried out throughout the semester, under supervision and tutoring, involving advanced readings, assignments and communication exercises (as recommended by Bridges etal., 2022). The faculty found that this CA alternative was not very successful, and that over 95% of the students continued to take the final exam. This result obliged the teaching staff of the subject to open a period of reflection in search of other CA options that would enjoy a greater degree of acceptance by the students, in which the debate between Clark and Kozma on educational technology was settled in the 1990s (Clark, 1994; Kozma, 1994). The advantages of using digital tools such as digital platforms (Tormos etal., 2014), mobile web-enabled technologies (Castillo-Manzano etal., 2017;Jahnke & Liebscher, 2020) and interactive response systems (Castillo-Manzano etal., 2016) were analysed to mitigate the problems of increasing class sizes (Girma & Darza, 2020) and the demands on instructor time and resources (Wilson etal., 2011). These digital tools became especially relevant during the global COVID-19 pandemic since many educational institutions were forced to switch to e-learning (Maatuk etal., 2021), thereby generating different expectations regarding the possibility of returning to traditional methods in the post-pandemic scenario (Moore & Piety, 2022). As a result of this reflection, during the 2016–2017 and 2017–2018 academic years, pilot experiences were carried out based on taking digital multiple-choice tests (e-tests) throughout the semester (Playfoot etal., 2022), along with a final exam. The results of these pilot tests, the results of which are included in this paper, have been decisive in designing a CA system that, complying with the normative evaluation guidelines of the University of Seville, would optimise both student learning and the work of the teaching staff, who teach in more than one class group per semester (each class group has an average of 70 students). In this way, in the 2020–2021 academic year, the implementation of an e-Continuous Assessment (hereinafter, e-CA), was carried out, which is currently followed by 90% of the students1 and in which class activities are included, carried out through the virtual platform, combined with the completion of multiple-choice tests 1 Students who do not opt for this e-CA option or fail to pass it, have the option of taking a final exam. J.I.Castillo-Manzano et al. 1 3 in electronic format (e-tests). This approach follows the thesis of Kozma (1994) in that technology, as an attribute of learning media, directly influences such media. In addition to contributing to greater student participation, the implementation of this e-CA system has made it possible to improve academic performance rates. Bearing in mind the theoretical framework provided by Fig.1 below, this paper collects the experience accumulated during the implementation of the e-tests, to promote the transition and consolidate a system of e-CA in higher education to enhance student learning and improve academic outcomes. It should not be forgotten that several scholars, such as Holmes (2018), Lopez-Tocon (2021), and Zorio-Grima and Merello (2020), have stated the pertinence of contributing new evidence in this regard. A vast amount of evidence shows that new technologies and tools have opened up new possibilities for teaching and learning paradigms, in general, due to their potentially beneficial characteristics for educational change. However, a better understanding is needed of the role played by digital technologies in support of the evaluation processes introduced by the EHEA. In short, the goal of this paper can be given in terms of the following research questions: • Do e-tests lead to similar results in terms of academic performance to those of the traditional final examination? • Are e-tests an appropriate option to assess both theoretical and practical knowledge of an Economics subject? • Can e-tests become an effective tool for the design of an e-CA system that meets the demands and expectations of students? • Are e-tests useful, even if the student does not pass them, by reducing the probability of the student dropping out of the subject? Fig. 1 Research conceptual and theoretical framework Source: prepared by the authors 1 3 Evaluating thedesign ofdigital tools forthetransition to… • Are there other socio-demographic or academic factors that can influence the student’s academic performance and that must be taken into account to isolate the effect of the e-tests carried out? Research design Participants The target population of this study is a sample of first-year students studying for the Business Administration and Management Degree at the University of Seville, enrolled in the subject of Principles of Economics, in two successive academic years: 2016–2017 (106 students) and 2017–2018 (149 students). Students were allowed to change groups if they did not want to take part in this research and all the remaining students were asked for their consent to take part and to complete an initial questionnaire regarding their profile (see the Appendix). The subject analysed, Principles of Economics, is a core class at the University of Seville and thus compulsory. Its content is mainly theoretical. Two strategies were adopted to minimise the ‘Hawthorne effect’, which is the potential bias that can occur when the rise in the interest of students and instructors on the application of a new strategy of teaching innovation generates a boost to performance (Bartsch, 2013). Firstly, the objective of our research was not explained to the students in advance. Secondly, the research phases were distributed independently; instructors of the groups that carried out e-tests did not design the tests, develop the database, apply the econometric model, nor interpret the results. e‑test implementation The students that formed part of this research took three e-tests corresponding to the three programme blocks (Fundamentals of Economics and Economic Thought, Macroeconomics, Microeconomics). They chose to take an e-test for each block due to the thematic unit of the subjects taught in each part and to the organisation of teaching in a subject of 6 ECTS (60h of classes), where 8 topics are taught that include theoretical content and practical exercises. All three blocks were structured in the same way: 10 multiple choice questions with three possible answers each, only one of which was correct (Rodríguez 2005). e-tests were carried out in the first 15min of class once instructors had previously taught the content of the programme blocks. As mentioned in the introduction, during these two courses in which the capacity of the e-tests to evaluate the knowledge acquired by the students and become a true e-CA system was being tested, the students were obliged to sit a final written examination to pass the course, regardless of the results achieved in the tests. This J.I.Castillo-Manzano et al. 1 3 exam consisted of three theory questions (theory paper) plus a practical question (practical paper) that required them to solve a mathematical problem. The ethical appropriateness of our research design was endorsed externally by the study being financed through a competitive call made by the University of Seville with evaluation by anonymous reviewers (Project: 21105, Support for Teaching Coordination and Innovation Programme). Methodology Two microeconometric models were used depending on the particular objective. Namely, both probit and bivariate probit models were used to test the formative efficiency of e-tests (see Table3) taking into account students’ demographic, socioeconomic and economic attributes included in Table1. A probit model (rather than a logit) was utilised to analyse the first outcome variable (the likelihood of sitting the examination) since it maximises the log pseudo-likelihood, whereas a bivariate probit model was applied for the other two outcome variables (the likelihood of passing the theory and practical papers), since it is a model category specially designed for cases where two questions with very closely linked binary answers need to be answered. In this case, there should be a relatively strong a priori correlation between the two, as the factors that affect whether the student passes the theory and practical examination papers can be expected to be similar. Variables In order to isolate the effect that carrying out the e-tests has had on student performance, which is the objective of our paper, a set of variables related to students’ individual profiles has also been included in the microeconometric models given that, according to the previous academic literature, they can influence academic results. This data was taken, as aforementioned in Sect.”Participants”, from a questionnaire (see the Appendix) filled out by students at the beginning of the course and has been classified into the following three groups: (a) Demographic and personal information: gender (Covarrubias etal., 2018); age (Dumford & Miller, 2018); and student’s vocation to study management at university (Salanova Soria etal., 2005). (b) Socio-economic information: worker (Covarrubias etal., 2018); family responsibilities (El Massah & Fadly, 2017); parents (Beattie etal., 2018); people living in the family household (Millea etal., 2018); non-local (Millea etal., 2018); wages (Covarrubias etal., 2018); and computers (Zou, 2013). (c) Academic information: Second session (Lara etal., 2009); Erasmus (Beattie etal., 2018); university access (Ballard & Johnson, 2005); first year at university (Herzog, 2018); grant (Glocker, 2011); number of final examinations sat and failed (Cappellari etal., 2012); and highest year (Beattie etal., 2018). 1 3 Evaluating thedesign ofdigital tools forthetransition to… Table 1 Variables and descriptive statistics Variable Description No. observations (only dummies = 1) Mean Std. Dev. Dependent variables Final examination 1 if student has sat the final examination; 0 otherwise 223 0.875 0.332 Theory pass 1 if student has passed the final theory exam paper; 0 otherwise 94 0.369 0.483 Practical pass 1 if student has passed the final practical exam paper; 0 otherwise 98 0.384 0.487 Explanatory variables e-test information Number of e-tests Number of e-tests completed – 2.404 0.741 e-tests passed 1 if student has passed the e-tests; 0 otherwise 137 0.537 0.500 Demographic and personal information Gender 1 if female; 0 if male 95 0.373 0.484 Age Student’s age – 18.533 1.425 Vocation Student’s self-assessment on a scale of 1 (lowest) to 5 (highest) of their vocation for studying on the management course – 3.815 0.907 Socio-economic information Worker 1 if student has a paid job; 0 otherwise 23 0.091 0.288 Family responsibilities 1 if student has economically dependent family members; 0 otherwise 10 0.040 0.196 Parents Student’s personal assessment of whether their decision to attend university is due to at least one of the parents having attended university: scale of 1 (lowest) to 5 (highest) – 2.302 1.289 People in household Number of people living in the same household as student – 4.008 0.919 Non-local 1 if the province of Seville was not student’s place of residence before attending university; 0 otherwise 148 0.583 0.494 Wages Number of people with paid jobs in student’s family household – 1.480 0.786 Computers Number of computers in student’s family household – 2.838 1.512 Academic information Second session 1 if student has attended lessons during the afternoon/evening session (from 4 to 8pm); 0 if student has attended lessons during the morning session (from 9.30 am to 1.30pm) 80 0.314 0.465 J.I.Castillo-Manzano et al. 1 3 Table 1 (continued) Variable Description No. observations (only dummies = 1) Mean Std. Dev. Erasmus 1 if student is a foreigner on the Erasmus programme; 0 otherwise 2 0.008 0.089 University access 1 if student is attending university after graduating from high school; 0 otherwise, including adult education for those over 25years of age 179 0.702 0.458 First year at university 1 if student’s first year at university; 0 if repeating the year 220 0.863 0.345 Grant 1 if student has applied for a public scholarship for university studies that requires passing the subjects; 0 otherwise 82 0.324 0.469 Final examinations sat Number of times student has completed and failed the Principles of Economics course, plus 1 – 1.099 0.359 Highest year Highest academic year of the degree course in which student is enrolled (1–4) – 1.095 0.344 Cluster variable Academic year 1 if student is enrolled in the first academic year analysed; 2 if student is enrolled in the second academic year analysed – 1.584 0.494 1 3 Evaluating thedesign ofdigital tools forthetransition to… Correlation coefficients between the control variables were extremely low, which precluded any autocorrelation problems. The correlation matrix is available from the authors upon request. Finally, a variable cluster was included to correct any heteroscedasticity problems due to any difference in the level of difficulty between the two academic years analysed, or any other differences, such as the number of holidays in the academic calendar. Table1 presents definitions and descriptive statistics of the dependent variables, explanatory variables, and the cluster variable. Table 2 Results of probit and bivariate probit estimations In brackets in the coefficient column, standard errors robust to heteroscedasticity by two clusters defined by the variable academic year One, two, and three asterisks indicate that the coefficient is statistically and significantly different from zero at 10%, 5%, and 1%, respectively. 1% indicates the greatest significance, while 10% is a weak significance level Explanatory variables Probit estimation Bivariate probit estimation Dependent variable Final examination Theory pass Practical pass Number of e-tests 0.620 (0.185)*** 0.242 (0.092)*** 0.295 (0.247) e-tests passed − 0.041 (0.436) 0.380 (0.011)*** − 0.297 (0.197) Gender − 0.166 (0.089)* − 0.238 (0.119)** 0.066 (0.142) Age − 0.086 (0.069) 0.016 (0.036) 0.178 (0.071)** Vocation 0.158 (0.096)* 0.089 (0.001)*** 0.084 (0.021)*** Worker 0.913 (0.017)*** 0.245 (0.312) 0.362 (0.103)*** Family responsibilities 0.055 (0.131) − 0.441 (0.286) − 1.252 (0.419)*** Parents − 0.158 (0.096) − 0.049 (0.052) − 0.022 (0.049) People in household 0.002 (0.069) 0.0205 (0.030) 0.123 (0.135) Non-local 0.390 (0.004)*** − 0.118 (0.157) 0.168 (0.082)** Wages − 0.166 (0.100)* − 0.049 (0.272) − 0.094 (0.100) Computers 0.201 (0.039)*** 0.109 (0.059)* 0.102 (0.052)** Second session 0.636 (0.023)*** − 0.235 (0.049)*** − 0.263 (0.142)* Erasmus − 2.560 (1.144)** 0.072 (1.674) 0.069 (1.504) University access − 0.013 (0.043) 0.102 (0.195) 0.149 (0.539) First year at university 0.629 (0.017)*** 0.149 (0.163) 0.419 (0.324) Grant − 0.511 (0.421) 0.495 (0.019)*** − 0.041 (0.189) Final examinations sat 0.621 (0.204)*** 0.924 (0.030)*** 0.499 (0.109)*** Highest year 0.930 (0.325)*** − 0.594 (0.000)*** − 0.985 (0.429)** Constant − 1.372 (0.379)*** − 2.471 (1.433)* − 5.058 (2.978)* No. Observations 243 Log pseudo-likelihood − 6.716 − 278.718 Pseudo R2 0.244 – Wald test of Rho = 0 (p value) – − 2.584 (0.108) J.I.Castillo-Manzano et al. 1 3 Acknowledgements The authors would like to express their gratitude to the University of Seville for its support through the university’s 2nd and 3rd Own Teaching Plan Aid Program, 2016 and 2017 Calls. Funding Funding for open access publishing: Universidad de Sevilla/CBUA. This study was funded by University of Seville (2nd and 3rd Own Teaching Plan Aid Program). Authors would alsowish to acknowledge the funding provided bythe Departamento de Análisis Económico y Economía Política de la Universidad de Sevilla (Department of Economic Analysis and Political Economy, at the University of Seville). Declarations Conflict of interest The authors have no conflicts of interest to declare that are relevant to the content of this article. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. References Abio, G., Alcañiz, M., Gómez-Puig, M., Rubert, G., Serrano, M., Stoyanova, A., & Vilalta-Bufí, M. (2019). Retaking a course in economics: Innovative teaching strategies to improve academic performance in groups of low-performing students. Innovations in Education and Teaching International, 56(2), 206–216. Alauddin, M., & Khan, A. (2010). Does performance in progressive assessment influence the outcome in final examination? An Australian experience. Educational Assessment, Evaluation and Accountability, 22, 293–305. Ballard, C., & Johnson, M. (2005). Gender, expectations, and grades in introductory microeconomics at a US university. Feminist Economics, 11, 95–122. Bartsch, R. A. (2013). Designing SoTL studies—part I: Validity. New Directions for Teaching and Learning, 136, 17–33. Beattie, G., Laliberté, J. W., & Oreopoulos, P. (2018). Thrivers and divers: Using non-academic measures to predict college success and failure. Economics of Education Review, 62, 170–182. Bjælde, O. E., Jørgensen, T. H., & Lindberg, A. B. (2017). Continuous assessment in higher education in Denmark. Early experiences from two science courses. Dansk Universitetspædagogisk Tidsskrift, 12(23), 1. Bologna Declaration. (1999). Retrieved December 29, 2022 from, http:// www. ehea. info/ pageminis terialconfe rencebolog na1999 Bridges, P., Cooper, A., Evanson, P., Haines, C., Jenkins, D., Scurry, D., Woolf, H., & Yorke, M. (2002). Coursework marks high, examination marks low: Discuss. Assessment and Evaluation in Higher Education, 27(1), 35–48. Broadbent, J., Panadero, E., & Boud, D. (2018). Implementing summative assessment with a formative flavour: A case study in a large class. Assessment and Evaluation in Higher Education, 43, 307–322. Cappellari, L., Lucifora, C., & Pozzoli, D. (2012). Determinants of grades in maths for students in economics. Education Economics, 20, 1–17. Carrillo-Peña, M. T., & Pérez, J. (2012). Continuous assessment improved academic achievement and satisfaction of psychology students in Spain. Teaching of Psychology, 39, 45–47. 1 3 Evaluating thedesign ofdigital tools forthetransition to… Castillo-Manzano, J. I., Castro-Nuño, M., Lopez-Valpuesta, L., Sanz Díaz, M. T., & Yñiguez, R. (2016). Measuring the effect of ARS on academic performance: A global meta-analysis. Computers and Education, 96, 109–121. Castillo-Manzano, J.I., Castro-Nuño, M., López-Valpuesta, L., Sanz-Díaz, M.T., & Yñiguez, R. (2017). To take or not to take the laptop or tablet to classes, that is the question. Computers in Human Behavior, 68, 326–333. https:// doi. org/ 10. 1016/j. chb. 2016. 11. 017 Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21–29. Cole, J. S., & Spence, S. W. T. (2012). Using continuous assessment to promote student engagement in a large class. European Journal of Engineering Education, 37, 508–525. Covarrubias, R., Gallimore, R., & Okagaki, L. (2018). I know that I should be here: Lessons learned from the first-year performance of Borderline University applicants. Journal of College Student Retention: Research, Theory and Practice, 20, 92–115. Day, I. N., van Blankenstein, F. M., Westenberg, P. M., & Admiraal, W. F. (2018). Explaining individual student success using continuous assessment types and student characteristics. Higher Education Research and Development, 37(5), 937–951. Dumford, A. D., & Miller, A. L. (2018). Online learning in higher education: Exploring advantages and disadvantages for engagement. Journal of Computing in Higher Education, 30, 452–465. El Massah, S., & Fadly, D. (2017). Predictors of academic performance for finance students: Women at higher education in the UAE. International Journal of Educational Management, 31, 854–864. Gallardo, E. G., & Montolio, D. (2011). Is there a relation between continuous evaluation and students’ results? E-Public: E-Journal on Teaching Public Economy, 8, 63–79. García, A. S., García-Álvarez, M. T., & Blanca Moreno, B. (2014). Analysis of assessment opportunities of learning spaces: On-line versus face to face methodologies. Computers in Human Behavior, 30, 372–377. Girma, F., & Darza, D. (2020). Assessing the implementation of continuous assessment: The case of Arbaminch University in natural science departments. Asian Journal of Advances in Research, 3(1), 167–179. Glocker, D. (2011). The effect of student aid on the duration of study. Economics of Education Review, 30(1), 177–190. González, M. O., Jareño, F., & López, R. (2015). Impact of students’ behavior on continuous assessment in higher education. Innovation: the European Journal of Social Science Research, 28, 498–507. González-Campos, G., Castañeda-Vázquez, C., & Campos-Mesa, M. (2018). Continuous assessment and interactive response systems in higher education. Journal of Human Sport and Exercise, 13, 667–681. Herzog, S. (2018). Financial aid and college persistence: Do student loans help or hurt? Research in Higher Education, 59(3), 273–301. Holmes, N. (2018). Engaging with assessment: Increasing student engagement through continuous assessment. Active Learning in Higher Education, 19(1), 23–34. Hsieh, T. L. (2019). A preliminary study of multiple college admission criteria in Taiwan: The relationship among motivation, standardized tests, high school achievements, and college success. Higher Education Research and Development, 38(4), 762–779. Jahnke, I., & Liebscher, J. (2020). Three types of integrated course designs for using mobile technologies to support creativity in higher education. Computers and Education, 146, 103782. Kara, O., Bagheri, F., & Tolin, T. (2009). Factors affecting students’ grades in principles of economics. American Journal of Business Education, 2, 25–34. Kozma, R. B. (1994). A reply: Media and methods. Educational Technology Research and Development, 42(3), 11–14. Lara, A., Aguiar, M. E., Cerpa, G., & Núñez, H. (2009). Relaciones docente-alumno y rendimiento académico: Un caso del Centro Universitario de Ciencias Exactas e Ingenierías de la Universidad de Guadalajara. Sinéctica, 33, 1–15. López-Tocón, I. (2021). Moodle quizzes as a continuous assessment in higher education: An exploratory approach in physical chemistry. Education Science, 11, 500. Maatuk, A. M., Elberkawi, E. K., Aljawarneh, S., Rashaideh, H., & Alharbi, H. (2021). The COVID19 pandemic and E-learning: Challenges and opportunities from the perspective of students and instructors. Journal of Computing in Higher Education, in Press. https:// doi. org/ 10. 1007/ s1252802109274-2 J.I.Castillo-Manzano et al. 1 3 Harvard Magazine. (2010). John Harvard’s Journal: Bye-bye, Blue Books? http:// harva rdmag azine. com/ 2010/ 07/ byebyebluebooks (Dic 29, 2022). Marcerano, O. D., & Navarro, M. L. (2007). Success in university: A quantilic approximation. Revista De Economía Aplicada, 15(44), 5–39. McCarthy, J. (2017). Enhancing feedback in higher education: Students’ attitudes towards online and in-class formative assessment feedback models. Active Learning in Higher Education, 18, 127–141. Millea, M., Wills, R., Elder, A., & Molina, D. (2018). What matters in college student success? Determinants of college retention and graduation rates. Education, 138(4), 309–322. Ministerio de Educación. (2010). Real Decreto 1791/2010, de 30 de diciembre, por el que se aprueba el Estatuto del Estudiante Universitario. BOE núm. 318, de 31/12/2010. Moore, S. L., & Piety, P. J. (2022). Online learning ecosystems: Comprehensive planning and support for distance learners. Distance Education, 43(2), 179–203. Myllymäki, S. (2013). Incorporation of continuous student assessment into lectures in engineering education. European Journal of Engineering Education, 38, 385–393. Playfoot, D., Wilkinson, L. L., & Mead, J. (2022). Is continuous assessment inclusive? An analysis of factors influencing student grades. Assessment and Evaluation in Higher Education. https:// doi. org/ 10. 1080/ 02602 938. 2022. 21508 34 Quiroga, V., García, E. M., Juy, J. P., Martínez, E. M., & Sánchez, L. M. (2014). La evaluación continua en el Grado de Trabajo Social de la Universidad de Barcelona. Cuadernos De Trabajo Social, 27(2), 353–363. Richardson, J. T. E. (2015). Coursework versus examinations in end-of-module assessment: A literature review. Assessment and Evaluation in Higher Education, 40(3), 439–455. Rodríguez, M. C. (2005). Three options are optimal for multiple-choice items: A meta-analysis of 80 years of research. Educational Measurement: Issues and Practice, 24(2), 3–13. Salanova Soria, M., Martínez Martínez, I. M., Bresó Esteve, E., Llorens Gumbau, S., & Grau Gumbau, R. (2005). Bienestar psicológico en estudiantes universitarios: Facilitadores y obstaculizadores del desempeño académico. Anales De Psicología, 21, 170–180. Sanz-Pérez, E. S. (2019). Students’ performance and perceptions on continuous assessment. Redefining a chemical engineering subject in the European higher education area. Education for Chemical Engineers, 28, 13–24. Swope, K. J., & Schmitt, P. M. (2006). The performance of economics graduates over the entire curriculum: The determinants of success. The Journal of Economic Education, 37, 387–394. Tormos, B. V., Climent, H., Olmeda-González, P. C., & Arnau-Martínez, F. J. (2014). Use of new methodologies for students assessment in large groups in engineering education. Multidisciplinary Journal for Education, Social and Technological Sciences, 1(1), 121–152. Whalen, D., Saunders, K., & Shelley, M. (2009). Leveraging what we know to enhance short-term and long-term retention of university students. Journal of College Student Retention: Research, Theory and Practice, 11, 407–430. Wilson, K., Boyd, C., Chen, L., & Jamal, S. (2011). Improving student performance in a first-year geography course: Examining the importance of computer-assisted formative assessment. Computers and Education, 57, 1493–1500. Zorio-Grima, A., & Merello, P. (2020). Class-attendance and online-tests results: Reflections for continuous assessment. Journal of Teaching in International Business, 31(1), 75–97. Zou, B. (2013). Teachers’ support in using computers for developing students’ listening and speaking skills in pre-sessional English courses. Computer Assisted Language Learning, 26, 83–99. Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. José I. Castillo‑Manzano is a full professor in the Faculty of Economics at the University of Seville. His research interests include policy evaluation and educational technology. He has published more than 70 papers in journals with JCR impact factor. 1 3 Evaluating thedesign ofdigital tools forthetransition to… Authors and Affiliations JoséI.Castillo‑Manzano1 · MercedesCastro‑Nuño1 · LourdesLópez‑Valpuesta1 · MaríaTeresaSanz‑Díaz2 · RocíoYñiguez2 * José I. Castillo-Manzano [email protected] Mercedes Castro-Nuño [email protected] Lourdes López-Valpuesta [email protected] María Teresa Sanz-Díaz [email protected] Rocío Yñiguez o[email protected] 1 Applied Economics & Management Research Group, Facultad de Ciencias Económicas y Empresariales, Universidad de Sevilla, Avda. Ramón y Cajal, 1, 41018Seville, Spain 2 Teoría Económica y Economía Política Research Group, Universidad Sevilla, Avda. Ramón y Cajal 1, 41018Sevilla, Spain Mercedes Castro‑Nuño is associate professor in the Faculty of Economics at the University of Seville. Her research interests include public policies evaluation, higher education, educational technologies and meta-analysis studies technology. She has published more than 40 papers in journals with JCR impact factor. Lourdes López‑Valpuesta is a full professor in the Faculty of Economics at the University of Seville. She has published more than 40 papers in journals with JCR impact factor, mainly in the fields of policy evaluation, teaching innovation and educational technologies research. Teresa Sanz‑Díaz is associate professor in the Faculty of Economics at the University of Seville. Among other areas, part of her research has been focused on ICT in higher education, educational technology and data envelopment analysis. She has published 12 papers in journals with JCR impact. Rocio Yñiguez is associate professor in the Faculty of Economics at the University of Seville. She has published 21 papers in journals with JCR impact (most of them Q1). Her research is mainly focused on the fields of teaching and learning innovation, Education and gender, education policy, quality in education and modern technologies and virtual learning.