When Perception Is Not Reality: Misalignment Between Self-Reported Metacognitive Ability and Academic Performance in University Students

Research Paper Recommended citation: Isotalus, H. K., Avdic, D., Beck, R., Bullock, S., & Joshi, A. (2025). When Perception Is Not Reality: Misalignment Between Self-Reported Metacognitive Ability and Academic Performance in University Students. In Kangaslampi, R., Langie, G., Järvinen, H.-M., & Nagy, B. (Eds.), SEFI 53rd Annual Conference. European Society for Engineering Education (SEFI), Tampere, Finland. DOI: 10.5281/zenodo.17631615. This Conference Paper is brought to you for open access by the 53rd Annual Conference of the European Society for Engineering Education (SEFI) at Tampere University in Tampere, Finland. This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License. WHEN PERCEPTION IS NOT REALITY: MISALIGNMENT BETWEEN SELF-REPORTED METACOGNITIVE ABILITY AND ACADEMIC PERFORMANCE IN UNIVERSITY STUDENTS H.K., Isotalusa, 1 , D., Avdicb, R.A., Beckc, S., Bullockd, A. Joshie a University of Bristol, Bristol, UK, 0000-0002-3393-9263 b University of Bristol, Bristol, UK, 0000-0002-7982-2917 c University of Bristol, Bristol, UK, 0009-0000-6352-9858 d University of Bristol, Bristol, UK, 0000-0001-9086-1055 e University of Bristol, Bristol, UK, 0000-0001-9717-0724 Conference Key Areas: Engineering skills, professional skills, and transversal skills, Improving higher engineering education through researching engineering education, Keywords: metacognition, self-regulation, academic performance, lifelong learning ABSTRACT Metacognitive skills – awareness and regulation of one’s own cognitive processes – are widely theorised to play a role in academic success. This study investigates the relationship between metacognitive qualities and academic attainment in Engineering and Data Science students at a UK university. A total of 132 students across four degree programmes completed a survey consisting of adapted versions of the Self Reflection and Insight Scale (SRIS), the Metacognitive Awareness Inventory (MAI), a Subjective Social Comparison of Academic Performance (SSCAP) measure, and demographic questions. Contrary to theoretical expectations and prior findings, no associations were observed between academic grades and either SRIS or MAI outcomes. Bayesian post-hoc analyses were conducted to further study the relationship between grades and Knowledge of Cognition and Regulation of Cognition, as measured by MAI subscales. We found substantial evidence in favour of the null hypotheses (BF01 = 7.9; BF01 = 9.0). However, SSCAP scores were strongly associated with academic performance (ρ = 0.510), and moderately with MAI sub domains (ρ = 0.335, ρ = 0.340, all ps<0.001). T To our best knowledge, these findings 1 Corresponding Author H.K. Isotalus [email protected] are first to show evidence for no association between academic achievement and measures of metacognitive knowledge and regulation. Further, these findings suggest that while self-reported metacognitive skills may not directly predict academic grades, students' subjective academic self-concept may serve as a more sensitive indicator of academic performance. Our findings further complicate the narrative between metacognition and academic attainment. 1 INTRODUCTION The rapid acceleration of new knowledge and techniques in data science and engineering leads to knowledge requiring updating almost as soon as it has been learnt. Inevitably, engineers and data scientists must engage in lifelong learning for the durations of their careers. However, high amounts of invested time and engagement with learning do not always lead to better outcomes. Instead, success in lifelong learning hinges on knowing how to evaluate ones’ own ability and to update knowledge effectively. Metacognition refers to the awareness and management of one’s own cognition, or “thinking about thinking” (Flavell, 1979). These skills can be broken into metacognitive knowledge and regulation (Brown, 1987; Schraw & Moshman, 1995). Knowledge of cognition refers to understanding our own cognitive processes, such as which concepts we have understood and which require further study, knowing how to learn, and knowing about different learning strategies. Metacognitive regulation on the other hand refers to the management and actions taken to regulate learning, such as making a study plan (Schraw et al, 1995; Stanton et al, 2015). Poor metacognitive regulation skills may lead to an otherwise capable student to laboriously engage in study without making progress. Reflecting on their learning processes, engineering and data science students can identify areas of struggle, recognise effective learning strategies, and adjust their approaches to improve outcomes to meet individual needs and requirements of the task at hand. Therefore, self-reflection and ability to gain insight into ones’ own thoughts and behaviours are core components of metacognitive ability (Schraw & Dennison, 1994). Another core part of metacognition in students aspiring to achieve high grades or perform well in the workplace is understanding what is expected of them. This requires not only establishing what the requirements of specific tasks and ones’ own capabilities are but also understanding where we are positioned compared to peers. Even the strongest performers have flaws, but understanding social positioning allows them to plan their study and careers accordingly. Therefore, one can feasibly expect that metacognitive skills and subjective social comparison are associated positively with academic performance. However, while many studies have found such relationships (Wang et al, 1990), not all have replicated these findings (Sperling et al, 2004). To this end, we sought to understand the relationship between metacognitive skills and academic performance in engineering and data science students. Metacognitive skills may be particularly important in engineering and data science education, where rapid advances in technology necessitate the need to continuously adapt to changes, often at speed. Although metacognitive skill training is not always explicitly narrated in the curriculum, many engineering and data science programmes incorporate such skill development in form of self-reflection, project management, problem-based learning, and planning (Dennis & Sommerville, 2023; Ruijten et al, 2025; Ugulino & Pires, 2021). The current study aims to establish which measures of metacognition and social comparison are most associated with academic performance in engineering and data science students at the University of Bristol. We hypothesised that Self-reported Knowledge and Regulation of metacognition questionnaire scores and a subjective social comparison measure of academic performance will be positively correlated with student grades. 2 METHODOLOGY 2.1 Participants and Data Collection All University of Bristol students enrolled on either MSc Data Science, MSc Digital Health, or BEng or MEng Aerospace Engineering in the academic year 2024/2025 were eligible to participate in this study. Invitations were sent to students’ University affiliated e-mail addresses and/or presented on a slide at the start of a teaching session. All survey data were collected in March 2025. Participants were not paid for their participation, but they could opt-in for a raffle to win one of twenty £10 vouchers. We aimed to recruit as many participants as possible from the above named degree programmes. A power calculation with α = 0.05, β = 0.20 and r = 0.30 indicated a minimum sample size of 85 (Browner et al, 2022), where the coefficient was estimated based on a similar study (Young & Fry, 2008). The study was approved by Faculty of Engineering Research Ethics Board, University of Bristol, Bristol, UK (Ref: 21928). 2.2 Materials The survey was delivered via Microsoft Forms, and consisted of an Information Sheet, Consent Form, Demographic Questionnaire, a Subjective Social Comparison of Academic Performance (SSCAP), the Self Reflection and Insight Scale (SRISadapted; Grant et al, 2002), and the Metacognitive Awareness Inventory (MAIadapted; Schraw et al, 1994). The SSCAP was adapted from the MacArthur Ladder Scale of Subjective Social Status (Adler et al, 2000) to measure how students felt they compared to their peers academically, on a 10point scale (Fig 1). This is a non-validated measure developed by the research team. The original MacArthur Ladder Scale of Subjective Social Status is a widely used single-item measure of perceived social status relative to peers’. SRIS and MAI are validated, widely used, self-report assessments of metacognition. SRIS consists of three distinct dimensions of self-reflection and insight: Need for Self-Regulation (n-SRIS), Engagement in Self-Regulation (e-SRIS), and Insight (i-SRIS). Fig. 1. The adapted ladder scale used to measure Subjective Social Comparison of Academic Performance (SSCAP). N-SRIS measures one’s perception of value of monitoring and adjusting cognitive strategies, e-SRIS measures engagement in such regulation, and i-SRIS measures the depth of understanding of one’s own cognitive processes and behaviours. MAI on the other hand measures the two dimensions of metacognition; knowledge of cognition (MAI-KC) and regulation of cognition (MAI-RC). Both MAI and SRIS show good external and internal validity (Schraw et al, 1994; Banner et al, 2024). Adaptations to all questionnaires were made to allow students to opt out of scale questions they did not want to answer without opting out of the entire survey. To measure academic attainment, we took the mean grade across all taught modules from the same academic year the questionnaires were filled, that had been completed and that had a mark assigned prior to survey completion. All data cleaning and questionnaire scoring was done using Python 3.11.9. Inferential analyses were ran on JASP v0.19.3 (2024). 2.3 Procedure and analysis SRIS and MAI were scored as per published guidelines. Where appropriate, missing values were imputed using median substitution, provided that the overall response count met the retention threshold described below. As questionnaire scales were discrete, all inferential analyses were performed using Spearman's rank correlation coefficients. The Benjamini & Hochberg (1995) family-wise false discovery rate correction was used to correct for multiplicity, with corrected α-threshold of 0.05. 3 RESULTS 3.1 Demographics A total of 147 students filled the questionnaire. Participants who filled the entire survey in under 150 seconds (n = 8), who had 10 or more missing answers across the 72 SRIS and MAI questions (n = 3), and Table 1. Sample Demographics and Descriptive Summaries Across Obtained Self-Report and Objective Measures Demographics N (%) Gender Man 83 (62.9) Woman 47 (35.6) Other 2 (1.5) Age Range 18-20 28 (21.2) 21-23 63 (47.7) 24-26 31 (23.5) 27+ 10 (7.6) Ethnicity Asian / British Asian 87 (65.9) White 36 (27.3) Other / Not Stated 9 (6.8) Country of Origin India 43 (32.6) UK 33 (25) China 32 (24.2) Other 24 (18.2) Programme of Study MSc. Data Science 70 (53) MEng. Aerospace Engineering 37 (28) BEng. Aerospace Engineering 14 (10.6) MSc. Digital Health 11 (8.3) Level Postgraduate (taught) 93 (70.5) Undergraduate 39 (29.6) Descriptives Mean (SD) Self-Report Measures* SSCAP 6.5 (1.7) n-SRIS 28.0 (4.7) e-SRIS 26.6 (4.7) i-SRIS 32.7 (6.8) MAI-KC 13.3 (2.9) MAI-RC 26.1 (6.2) Objective Measures Average Student Grades 63.2 (11.3) * SSCAP = Subjective Social Comparison of Academic Performance; n-SRIS = Need for Regulation; e-SRIS = Engagement in SelfRegulation; i-SRIS = Insight scores from the SRIS; MAI-KC = Knowledge of Cognition subscale from MAI; MAI-RC = Regulation of Cognition subscale score from MAI. those who had no academic grade data available (n = 4) were excluded from analyses, resulting in a final sample of 132 (Table 1). 3.2 Correlations Unlike predicted, grade average was not correlated with any of the SRIS or MAI subscales. However, there was a substantial positive association between grade average and SSCAP performance (Fig 2). In other words, students who reported thinking their academic performance was higher compared to peers were also likely to have higher grades. Full correlation results are given in Table 2 and Fig 3. The SSCAP, but not academic performance, had a moderate positive association with both MAI sub-scales. We also observed a low association between SSCAP and n-SRIS, but this association did not remain significant after correcting for multiplicity. Of the MAI sub-scales, knowledge of cognition was mildly associated with the i-SRIS scale, while Regulation of cognition was mildly associated with n-SRIS scale. Bayesian analyses allow quantifying the relative support for null and alternative hypotheses given the data. In other words, unlike p-value testing, Bayesian analyses do not just test whether there is support to reject the null hypothesis, but they can also quantify the evidence in favour of a null effect. As we found no correlation between MAI sub-scale items and overall average grades, unlike we had hypothesised, we next explored whether the lack of correlation was a real negative finding through post-hoc Bayesian correlation analyses. As we had no prior hypotheses about data distribution given our results were unexpected, we used default priors and set seed to 123 for reproducibility. Table 2. Spearman’s Correlations and p-values Among SRIS, MAI, SSCAP, and Average Grades Spearman’s ρ p-value Average grades n-SRIS e-SRIS i-SRIS MAI-KC MAI-RC n-SRIS -0.067 0.445 e-SRIS -0.016 0.858 0.584* <0.001 i-SRIS 0.112 0.200 -0.095 0.280 0.044 0.616 MAI-KC 0.065 0.456 0.144 0.099 0.029 0.742 0.282* 0.001 MAI-RC -0.053 0.548 0.259* 0.003 0.065 0.457 0.053 0.543 0.633* <0.001 SSCAP 0.510* <0.001 0.214 0.014 0.140 0.110 0.170 0.051 0.340* <0.001 0.335* <0.001 Abbreviations as defined in Table 1. All reported p-values are uncorrected. * denotes where result remained statistically significant (p<0.05) following multiplicity correction The Bayesian correlation analysis confirmed that there were no correlations between grade averages and MAI-KC, with a BF10 = 0.13 (BF01 = 7.9) indicating substantial evidence in favour of the null hypothesis, with a 95% credible interval (CI) of [-0.123, 0.215] for the coefficient. Findings between grade averages and MAI-RC were similar, with a BF10 = 0.11 (BF01 = 9.0; 95%CI [-0.187, 0.151]) indicating substantial evidence in favour of the null. Therefore, we not just found no evidence of a relationship between mean academic attainment and MAI scores, but we found evidence to support that there is no relationship between academic attainment and MAI subscales in this study. Fig. 2. Average grades against subjective social comparison of academic performance. Rank correlations were used and the fitted line is shown for illustration purposes only. Fig. 3. Heatmap of Spearman’s rank correlations displaying relationships between average grades and all questionnaire measures. Abbreviations as defined in Table 1. 4 DISCUSSION AND CONCLUSIONS We found a strong correlation between Subjective Social Comparison of Academic Performance (SSCAP) and academic achievement in engineering and data science students at a UK university. However, unlike expected, we did not observe a relationship between academic achievement and other measures of metacognitive ability. This finding contradicts our hypothesis, as we specifically predicted academic achievement to be positively associated with each MAI sub-scale. To our knowledge our study is the first to provide evidence against a relationship between either MAI subscale measure and academic success. We also found that the two MAI subscales were intercorrelated, suggesting that knowledge and regulation of cognition are associated constructs. This finding replicates many previous studies (Schraw et al, 1994; Sperling et al, 2004; Young et al, 2008). Although the lack of a direct association between metacognitive skills and academic achievement in this study diverges from theoretical expectations, it is not entirely unprecedented. Using MAI sub-scales others, such as Schraw and Dennison (1994), have shown that knowledge, but not regulation, of cognition, is positively correlated with academic test performance. However, other studies have failed to replicate this relationship and reported mixed results. For instance, Sperling et al. (2004) investigated the relationship between the MAI and SAT scores and found no correlation between verbal SAT scores and either MAI subscale. On the contrary, they found a negative relationship between SAT math scores and both MAI subscales. Together these findings complicate the narrative between the impact of metacognitive skills on academic success. It is important to note that metacognitive skills, particularly knowledge and regulation of cognition, are complex and context dependent. Some studies suggest that variability in metacognitive skills may have a more pronounced relationship with academic outcomes in younger students or those with less developed metacognitive skills (Dignath & Büttner, 2008; Veenman, 2013). In the context of engineering education, for example, where problem-solving and critical thinking are central, the relationship between metacognitive skills and academic performance may be influenced by additional factors such as prior domain knowledge and intrinsic motivation, which were not directly accounted for in this study. Others have explored the potential for metacognitive skills to be trained and developed through targeted interventions (Dignath & Büttner, 2008; Hacker et al, 2009; Karaoglan et al, 2022; Schaeffner et al, 2021; Turan et al, 2009). Metacognitive training programs, particularly those focusing on self-regulation strategies, reflection, and goal-setting, have been shown to improve academic performance in various educational contexts (Brown, 1978; Flavell 1979; Perry et al, 2019). Although some of these interventions are employed in higher education (Rujiten et al, 2025), majority of these findings are from school-aged children. It appears that metacognitive skills are associated with overall cognitive ability in developing children (Dignath & Büttner, 2008), highlighting the need to better understand the utility and generalisability of these training programmes in adults. It is also possible that variability in metacognitive ability is not sufficiently large in university students in our context to lead to meaningful variability in academic attainment. Most students enrolled on these programmes are highly cognisant and academically capable, with majority of the sample studying at a postgraduate level, and are therefore likely to already have high metacognitive attainment. We did not observe many low grades in our dataset. Therefore, the current data would not capture the kind of variability in cognitive or metacognitive ability that is present in the society at large, or changes in academic ability and metacognitive skill over time, limiting the generalisability of these findings. It may be that those students who perform worse early on in their studies, but then go on to improve their academic performance, benefit from being metacognitive. Future research emphasis should be on capturing the longer term variability in both metacognitive performance and academic ability. Highlighted in Finn and Tauber’s 2015 review article, several converging lines of research suggest that students’ metacognitive self-assessments are rarely unblemished insights into how they monitor learning, but rather often exposed to subjective cues, such as processing fluency, or how cognitively effortless or laborious one finds the task (Reber & Greifeneder, 2017). Fluency-based experiences and epistemological beliefs interact to create illusions of knowing, where students feel confident in their understanding despite poor recall or comprehension (Kruger & Dunning, 1999; Finn et al, 2015). This phenomenon may find salience in structured, high-achieving academic environments should students equate ease of learning with depth of learning. As such, self-reported metacognitive ability - especially when captured at a single time point - may reflect students’ prior success with fluency-driven strategies rather than genuine strategic control or metacognitive accuracy. How well students though they were performing academically compared to their peers was also associated with both metacognitive skill and regulation. This suggests that regardless of academic attainment, engineering and data science students who subjectively place themselves as high achievers compared to their peers are more likely to have better metacognitive skills. This is somewhat surprising, as few participants in this study placed themselves as average or below on the social comparison ladder (i.e. 5 or below). Instead, no student ranked themselves on the bottom step of the ladder, and even students who had received grades in the fail-range (below 50 or 40, depending on the course) were likely to rank themselves as above-average academic performers within their cohort. Overconfidence among below-average performers has been previously reported using different methods (e.g. Kruger et al, 1999). However, the ladder social comparison score was strongly associated with actual academic performance. Although we found an association between grades and subjective impression of academic performance relative to peers’, this association may not be causal. This finding could reflect that university students have a good understanding of how they are likely doing academically in comparison to peers, from socialisation and class discussions. Further research into this area is required to understand what may explain this relationship and whether this can lend support to interventions that may guide academic development. An interesting further line of enquiry would be to look at those students who are more accurate at placing themselves on this scale compared to those who stumble on the ladder in relation to their academic performance. The lack of a relationship between metacognitive skills and academic attainment challenges existing theories, yet highlights the importance of context, measurement methods, and individual differences. Many higher education providers are using metacognitive skills training in engineering education to boost student outcomes. It is unclear how impactful these interventions are in enhancing students’ longer term trajectories. Our findings suggest further need to understand the impact of embedding social comparison and group-work into the curricula to enhance metacognitive skills, as this may be a stronger indicator of academic performance than knowledge or regulation of cognition. Further, students often report enjoying formal metacognitive training when it is embedded in the curriculum and they understand its value (Ugulino et al, 2021). The timeliness of these unexpected findings underscores the critical feedback loop between university faculty and students. Without explicit guidance on the value of challenge, failure, and cognitive effort, both parties may conflate fluency as a proxy for understanding, inadvertently reinforcing ineffective habits.