From Logs to Insights: How Engineering Technology Students' Self-Reflection Levels Evolve Over Time

Research Paper Recommended citation: Tuyaerts, S., De Laet, T., Van den Broeck, L., & Langie, G. (2025). From Logs to Insights: How Engineering Technology Students’ Self-Reflection Levels Evolve Over Time. In Kangaslampi, R., Langie, G., Järvinen, H.-M., & Nagy, B. (Eds.), SEFI 53rd Annual Conference. European Society for Engineering Education (SEFI), Tampere, Finland. DOI: 10.5281/zenodo.17631903. This Conference Paper is brought to you for open access by the 53rd Annual Conference of the European Society for Engineering Education (SEFI) at Tampere University in Tampere, Finland. This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License. FROM LOGS TO INSIGHTS: HOW ENGINEERING TECHNOLOGY STUDENTS’ SELF-REFLECTION LEVELS EVOLVE OVER TIME S. Tuyaerts a,1, T. De Laet b, L. Van den Broeck c, G. Langie d a KU Leuven, ETHER, LESEC, Leuven, Belgium, 0000-0001-5107-7524 b KU Leuven, LESEC, Leuven, Belgium, 0000-0003-0624-3305 c KU Leuven, ETHER, LESEC, Leuven, Belgium, 0000-0002-6276-7501 d KU Leuven, ETHER, LESEC, Leuven, Belgium, 0000-0002-9061-6727 Conference Key Areas: Continuing education and life-long learning in engineering, Engineering skills, professional skills, and transversal skills Keywords: self-reflection, self-regulation, reflective log, engineering, higher education ABSTRACT Engineering students should be prepared for lifelong learning after graduation, as keeping up with the quick pace of technological advancements is part of a competent engineer’s job. In order to be effective lifelong learners, students should hone their self-regulation and self-reflection skills. This study investigates whether engineering students improve in self-reflection over time, as measured by reflective logs zooming in on the students’ learning process during the study period leading up to the first semester examinations. A cohort of 80 Flemish Engineering Technology students submitted their reflective logs twice, both in 2024 and 2025. Their answers were coded based on a self-reflection level matrix developed by the authors and the resulting reflection levels were analyzed by means of a paired Wilcoxon signed-rank test. The participating students exhibit a small to medium increase in self-reflection level from 2024 to 2025, which may be natural growth or attributed to the intervention pilot they participate in. The results are related to existing literature on reflective logs as both assessment and learning tools, potentially influencing each other, as well as self-reflection’s positive influence on self-regulation and lifelong learning. 1 Corresponding Author S. Tuyaerts [email protected]. 1 INTRODUCTION In today’s fast-evolving society, it is beyond dispute that lifelong learning (LLL) has become of paramount importance and a desirable skill in the workforce (Engineers Ireland, 2019; FEANI (EMC), 2020; KIVI Engineering Society, 2021). Professional engineers need to be able to keep up with the current high pace of technological developments, shifting professional responsibilities and keep their knowledge up to date (Knapper and Cropley, 2000; Guest, 2006; Martinez-Mediano and Lord, 2012; Cruz et al., 2020). Self-regulated learning (SRL) is recognized as a core competency for LLL (Van den Broeck et al., 2024; Kirby et al., 2010; Alt et al., 2022; Tuyaerts et al., 2025) and may even serve as a proxy for it in educational contexts (Stefanou et al., 2012). Various SRL models exist, each with a unique focus (Panadero, 2017). Pintrich’s four-phase model specifies different areas for regulation, leading to a combination of phases and regulation areas that cover many SRL processes. It emphasizes the role of motivation in SRL (Pintrich, 2000; Panadero, 2017), in contrast to Efklides’s twolevel model, which emphasizes the interplay between affect, motivation, and metacognition through their interactions across macro and micro levels (Efklides, 2011; Panadero, 2017). Zimmerman defines SRL as “the self-directive process by which learners transform their mental abilities into academic skills” (Zimmerman, 2002). SRL is considered an abstraction of the learning process and its continuous improvement. In this model, learners cycle through three phases: the forethought phase, during which a learner sets goals for themselves or comes up with a plan of action for the learning activity, the performance phase, during which more traditional learning activities take place as well as other strategies, like self-monitoring or helpseeking, and the self-reflection phase, making the learner reflect on their learning process and potential improvements to it (Zimmerman and Moylan, 2009). Self-reflection involves the inspection and evaluation of one’s thoughts, feelings, and behavior (Grant et al., 2002), and usually operates through re-examination and evaluation of experiences, beliefs, and knowledge (Kember et al., 2008). At its highest level, it may lead to new understanding and appreciations (Boud et al., 2013), as well as changes in unconscious beliefs, effectively transforming an individual’s belief structure (Kember et al., 2008). Self-reflection plays a critical role in SRL (Lyons and Zelazo, 2011) and it is intuitively easy to see why: one needs to be able to reflect on their learning process in order to determine what could be improved Zimmerman (2002). In contrast, practicing aspects of SRL, such as selfmonitoring, may also improve self-reflection quality as this may lead to higher awareness and insight in one’s thoughts, feelings and behaviors (Grant et al., 2002). Self-reflection and SRL are thus closely related concepts. Unfortunately, they are not always easy to measure as, by nature, they cannot directly be observed (Williams et al., 2019). SRL is a complex construct involving several cognitive, motivational, affective, and behavioral variables (Zeidner, 2019). Self-reflection may operate on various levels, ranging from a complete lack of reflection to a transformation in perspective (Kember et al., 2008). Even though existing work on the assessment of self-reflection levels exists, it remains challenging to measure (Lundgren and Poell, 2016), for it is possible on the one hand that students do engage in self-reflection, yet fail at expressing that they do so, and on the other hand that students only think they engage in self-reflection, while in reality they don’t really know how to do so (Kreber, 2005). This paper will investigate Engineering Technology students’ self-reflection levels over time, as measured by means of reflective logs on their learning process. As the students in our sample partake in two different interventions, we also wish to investigate whether this influences their growth in self-reflection. More specifically, the following research questions will be addressed: RQ1: To what extent do Engineering Technology students’ self-reflection levels improve from 2024 to 2025? RQ2: To what extent does the type of intervention in which the students participate affect their growth in self-reflection? 2 METHODOLOGY 2.1 Context This study uses data collected from Engineering Technology students at KU Leuven’s Ghent Campus, where two longitudinal interventions are being piloted as part of a larger-scale project on SRL. Students in the sample were enrolled in either Group A, receiving training in both study skills and SRL, or in Group B, receiving training in study skills only. Both interventions span the whole bachelor program (six semesters) and cover self-reflection. Consequently, we hypothesize that both groups’ self-reflection levels will increase over time (RQ1), but to a greater extent in Group A, as self-reflection is a core component of SRL. The first reflective log was collected after one semester of the intervention (February 2024), and the second log after three semesters (February 2025). Intervention A: The intervention including SRL aims to first build awareness on the existence of a student’s learning process and make the students realize they have agency over this process, then show how SRL may manifest in different learning contexts, to finally motivate students to keep using their SRL skills after the end of the intervention. The competency of SRL is trained through theory and practical applications related to the study strategies also employed in the control group intervention. Intervention B: This intervention also aims to improve students’ learning process, but does not cover SRL. Students learn about study skills such as planning, goal setting, various learning strategies, feedback literacy, and self-reflection, through both theory and practical assignments. This intervention covers some study skills in greater detail than the intervention on SRL to balance the time investment expected from participating students. The collection and use of this data for research purposes was approved by the SMEC in the file G-2023-6988-R4(AMD). 2.2 Data Collection This study collected qualitative event-measure data in the format of reflective logs. Students were asked to submit their answers to a log prompt as part of the intervention program they were participating in, using the university’s online learning platform. The logs aim to capture aspects of SRL, but as only one intervention group received explicit instruction on SRL, the prompt was worded in such a way that it would elicit this from students without specific terminology. The authors refined the prompt over several iterations and deliberately included a specific learning context for increased validity (Zimmerman and Moylan, 2009; Lundgren and Poell, 2016). The final prompt, as formulated below, was offered in Dutch as it is the Engineering Technology program’s language and the mother tongue of most of the students enrolled in it. Answers to this reflective log prompt were collected from the same cohort of students at the start of the Spring semester in both 2024 and 2025. As part of the intervention program, a reflective log on the context of study activities during the Spring semester, as opposed to the examination period, was also collected in 2024 based on a different prompt. As answers to this other log prompt had yet to be collected in 2025, the 2024 responses were omitted for this study’s analysis. Think back to a specific moment in time that occurred during the last study and examination period that made you put in effort in order to manage your behavior, thoughts, or feelings (in relation to studying). This can either be a positive or a negative experience when you reflect back on it. 1. What happened? 2. How did you tackle the situation, what steps did you take? 3. If you were to find yourself in a similar situation in the future, would you take it on the same way or differently? 2.3 Data Analysis Apart from blank submissions, the collected logs were coded for self-reflection level by the first author based on a collaboratively developed coding matrix, drawn up by all authors. The authors started out by taking inspiration from existing works (Lundgren and Poell, 2016; Learman et al., 2008; Koole et al., 2012; O’Sullivan et al., 2010; Driessen et al., 2007). The four-category scheme by Kember et al. was used as a starting point as its target audience are students and the protocol is suitable for research measuring reflection levels as an outcome (Kember, 1999; Kember et al., 2008). Over the span of several months, the authors refined the reflection level matrix until they were satisfied with the definitions and achieved an interrater agreement of 68%. The overall reflection levels are a synthesis of the reflection level in terms of, on the one hand, contextual elements and external circumstances, and the student’s own role and experience, on the other hand. These final reflection levels can be summarized as:  No Reflection (Level 0): The student shows no self-reflection. They provide little to no context to the experience they are reflecting on, and show little to no introspection. The numerical equivalent of zero was chosen to reflect an absence of reflection depth.  Superficial Reflection (Level 1): The student attempts to reflect on their learning practice, yet their account remains largely descriptive in nature, both in terms of contextual elements and their own role or experience.  Reflection (Level 2): The student puts forward some elements, be they contextual or personal, which are relevant to the experience. In addition, the student examines these elements in an attempt to seek understanding, or provides a superficial insight into their experience.  Critical Reflection (Level 3): The student provides elements, be they contextual or personal, that are linked to the main event in a meaningful way. The student is able to explain why these elements are important, and/or why they gained a particular insight from this experience. As a consequence, it should be clear to the reader why the student chose to reflect on this particular event and why it is important to them. Due to a lack of entries meeting the requirements for the highest reflection level, the two highest levels were merged into one (corresponding to Reflection (Level 2)) for this analysis. The coded reflection levels were analyzed for growth and group differences by means of Wilcoxon signed-rank tests. When a significant effect was found, Cohen’s D was calculated to gauge the effect size of it. 3 RESULTS Table 1 summarizes the reflection levels of the reflective logs collected, grouped per year. In the end, 80 students submitted a non-blank reflective log at both time points (NA = 38, NB = 42). Table 1. Distribution of self-reflection levels in reflective logs, grouped per year and intervention program. Percentages and numbers sum to 100% and the total across rows. No Refl. (Level 0) Sup. Refl. (Level 1) Reflection (Level 2-3) Total 2024 (all) 18% (n = 14) 59% (n = 47) 24% (n = 19) n = 80 Group A 18% (n = 7) 63% (n = 24) 18% (n = 7) n = 38 Group B 17% (n = 7) 55% (n = 23) 29% (n = 12) n = 42 2025 (all) 6% (n = 5) 59% (n = 47) 35% (n = 28) n = 80 Group A 11% (n = 4) 61% (n = 23) 29% (n = 11) n = 38 Group B 2% (n = 1) 57% (n = 24) 40% (n = 17) n = 42 Total 12% (n = 19) 59% (n = 94) 29% (n = 47) n = 160 Students’ reflection levels mainly fell into the middle category of Superficial Reflection (Level 1). Examples of logs exhibiting low and high self-reflection quality are given below by students #1 and #2, respectively. Some students reported on negative experiences, like student #1 did. In some cases the student details how their reaction to what happened resolved the problem, while in other cases the student failed to turn things around. By contrast, some students reported on solely positive experiences, in which case learning may just as well have taken place, as illustrated by student #2’s story. “There were specific times when I did not feel like studying. I took a little break and went for a run.” (Student #1, coded as No Reflection (Level 0)) “Both during the study and examination periods I had a good feeling. Sometimes I dared not say this to others, in case I would end up getting poor grades after all. I had a structure during the study period, every day I went to the learning center, you see, and studied there together with friends. Thanks to this, I mainly have positive experiences from this difficult period. I had an especially hard time after the holidays due to sleeping less and the sudden approach of the examinations. But I was aware that this is normal and that the only thing I could do was continue. I have learned that I need other people around me while I’m studying and that I also need structure. Like for example a fixed spot and fixed study blocks. This gave me peace, you see. During the study period I had trouble gauging how the examinations would go. Personally I knew I had done everything I could and so could not feel sorry for myself. Especially not what I did during the study period, moreso what I did over the course of the semester. Now I know I have to process the learning content after class. During the examination period I could also turn the page well, even if an exam did not go that well, and start studying for another course.” (Student #2, coded as Reflection (Level 2)) The students’ reflection levels did not differ significantly between groups A and B, as indicated by Wilcoxon signed-rank tests over the 2024 data (W = 641.5, p = .460) and those collected in 2025 (W = 583, p = .140). By consequence, we reject the hypothesis to RQ2 that the intervention the students partake in has a significant effect on their growth in self-reflection and will answer RQ1 based on the combined pool of students. Fig. 1. D istribution of self - reflection levels per year. Left: 2024. Right: 2025. Figure 1 summarizes the sample’s self-reflection levels per year. Students exhibit a small to medium growth (d = 0.346) in self-reflection levels from 2024 (M = 1.08) to 2025 (M = 1.32), as indicated by a paired Wilcoxon signed-rank test (V = 87.5, p = .005). This confirms the hypothesis to RQ1 that students grow in self-reflection level over time. A growth in self-reflection is illustrated by the logs submitted by student #3. “1) I had a hard time studying during a particular period. 2) I tried to motivate myself in various ways and thought that in hindsight, after the examination period, I would prefer to have continued learning as opposed to having to do a retake. 3) I would take it on exactly the same way.” (Student #3, log submitted in 2024) “1) I experienced a time during the study period in which I had trouble concentrating. There was not much time left before my first exam and I still had to cover a lot of ground. I noticed that I clutched to social media often. 2) To handle the situation, I consciously distanced myself from social media and asked myself why it is that I have trouble concentrating. I think this happened because of the urge to study everything to the minute detail, what made making progress hard and therefore distracted me. Because of this I studied in a different way, namely by first studying the things of which I thought they were important. After I did this for the whole syllabus, I had an easier time gauging what truly were details and what not. 3) In the future I would utilize the same method, but from the start. In addition, I would perhaps ask others what they would do.” (Student #3, log submitted in 2025) 4 DISCUSSION AND CONCLUSIONS This study sought to explore to what extent Engineering Technology students’ selfreflection levels improved over the course of one academic year (RQ1) and what effect the ongoing intervention pilots had on this growth (RQ2). Based on the results discussed in Section 3, we conclude that the students exhibit a small to medium growth in self-reflection level from 2024 to 2025 (RQ1), yet this growth does not appear to be significantly affected by the type of intervention the students participate in (RQ2). 4.1 Reflective Logs as Assessment and Learning Tools Reflective logs are used both as an assessment tool and as a learning opportunity for students (Moon, 2006; Yeoh, 2017; Buzza et al., 2013). They are used to investigate competencies such as self-reflection, SRL, and feedback literacy, among others (Coppens et al., 2023; Buzza et al., 2013). Other studies employing this data format tend to find that students submit logs that are of relatively superficial reflection depth, often only providing a descriptive account. This happens both in the field of engineering education (Coppens et al., 2023) and beyond, such as in science or business education (Lázaro et al., 2022; Yeoh, 2017). It is the highest levels of selfreflection, often referred to as transformative reflection or critical reflection, that show up rarely (Buzza et al., 2013). Coppens et al. (2023) used reflective logs to measure a similar engineering student population’s reflection level, as well as their aptitude for feedback literacy. Their coding scheme is also based on the work by Kember et al. (2008), but differs in their inclusion of feedback literacy assignment and discretization criteria of the levels. They found that the large majority of students showed little depth in their reflections and as a consequence, 87% of their collected logs were coded as their lowest reflection level (Coppens et al., 2023). Looking at how their reflection levels are defined, their lowest level corresponds to a combination of the two lowest levels in this study. When we combine the number of logs in this study’s No Reflection (Level 0) (12%) and Superficial Reflection (Level 1) (59%), we observe a somewhat more similar distribution with 71% of students’ logs classified there. In our student population, we observe more students attempting to reach an understanding of their experience or more advanced levels of self-reflection, i.e. a reflection level of at least Reflection (Level 2) (24%), compared to the students of Coppens et al. (2023) (13%). Despite those being of a student population similar to ours, these differences may be explained by either the incorporation of feedback literacy aspects assessment in Coppens et al.’ coding scheme, as opposed to solely self-reflection criteria in our study, the different log prompt or context of the reflection exercise, or the ongoing intervention that may improve our students’ SRL and self-reflection skills. 4.2 Self-Reflection as a Core Component of Self-Regulation Self-reflection can be considered a core component of SRL (Zimmerman and Moylan, 2009). It is by consequence no surprise that improvements in self-reflection capacity may enhance a learner’s SRL as well. Lyons and Zelazo found that selfreflection aptitude leads to more conscious awareness of one’s thoughts and actions, in turn helping to calibrate monitoring processes. These motivate appropriate performance adjustments and thus lead to improved SRL capacity (Lyons and Zelazo, 2011). Reflection practice may strengthen the interplay between monitoring and control regions, helping to automate the regulation process (Lyons and Zelazo, 2011). Ryan also argues that making reflections more conscious may facilitate decision-making, and in turn, even LLL (Ryan, 2015). We observe such developments in the reflective logs we collected, as illustrated by the growth exhibited by student #3 in Section 3. In 2024, this student reports in a rather vague manner on how they tried to motivate themselves to persevere in their study efforts. Very little concrete information on their thoughts and control adjustments are given, making it difficult to detect to what extent self-regulatory actions have taken place. In 2025, however, the same student narrates on how they consciously reflected on what made it difficult for them to concentrate. Thanks to the insight that came from this reflection, the student knew how to adjust their behavior to improve their concentration efforts and learning productivity. It is worth noting that the completion of a measurement like a reflective log may trigger learning, thus affecting results. By expressing their own observation of their learning process, students may become more conscious of their experiences (Wallin and Adawi, 2018). This may in turn affect behavior, as through reflection the students may evaluate and self-assess their efforts (Fabriz et al., 2014). Based on their self-evaluation, they may decide to adapt their behavior in future learning endeavors, regulating their learning behavior by informing future learning processes (Alt et al., 2022). The writing of a reflective log, whether it be utilized as a measurement or as a learning opportunity, may trigger developments in selfreflection competency as well as in SRL. This can be considered good news as it requires little effort from teachers and students, can be adapted to course content if desired, and can be incorporated into many different classroom contexts. 4.3 Study Limitations This data has been collected from students participating in a pilot investigating the effects of longitudinal interventions on SRL and study strategies. The project is currently halfway complete and the reflective logs will be collected a third time next academic year (2026). As these students are offered an intervention that may enhance their self-reflection skills, it remains to be seen (1) whether the results of this study can be replicated with a sample of students who do not participate in such intervention programs, and (2) what the effects of the two interventions on selfreflection development are, if any. As of now, it appears there is no evidence for a differing effect of the intervention type on self-reflection growth. This taps into the limitations of the paired Wilcoxon signed-ranks test employed to answer RQ2, as the sample studied is not truly random. As it is not representative for the whole engineering student population, affecting generalizability. The test may also suffer from reduced statistical power due to many tied observations, as there were only possible three values for the students’ reflection level. 4.4 Conclusion This study employed reflective logs to assess Flemish Engineering Technology students’ level of self-reflection. The context for reflection was the students’ learning process during the first semester examination period and data was collected twice: in February of 2024 and in February of 2025. A paired Wilcoxon signed-rank test detected a small to medium increase in self-reflection level from 2024 to 2025. As a pilot intervention is currently underway for the targeted students, it remains to be seen how their self-reflection levels will continue to evolve.