Evolving ChatGPT Usage in Engineering Education: Insights from a Mobile Robot Control Course

Practice Paper Recommended citation: Torta, E., Sonawane, A., van Beek, F., Sexton, M., & de Vos, K. (2025). Evolving ChatGPT Usage in Engineering Education: Insights from a Mobile Robot Control Course. In Kangaslampi, R., Langie, G., Järvinen, H.-M., & Nagy, B. (Eds.), SEFI 53rd Annual Conference. European Society for Engineering Education (SEFI), Tampere, Finland. DOI: 10.5281/zenodo.17631891. This Conference Paper is brought to you for open access by the 53rd Annual Conference of the European Society for Engineering Education (SEFI) at Tampere University in Tampere, Finland. This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License. 1 EVOLVING CHATGPT USAGE IN ENGINEERING EDUCATION: INSIGHTS FROM A MOBILE ROBOT CONTROL COURSE Elena Torta a, Apoorva Sonawane b, Femke van Beek c, Koen de Vosd, Melissa Sextone a Eindhoven University of Technology, Eindhoven, NL, 0000-0001-9198-1374 b Eindhoven University of Technology, Eindhoven, NL, 0009-0003-8512-7042 c Eindhoven University of Technology, Eindhoven, NL, 0000-0003-4741-6196 d Eindhoven University of Technology, Eindhoven, NL , 0009-0007-2701-089X e Vrije Universiteit Amsterdam, Amsterdam, NL, 0000-0001-9005-988X Conference Key Areas: Digital tools and AI in engineering education Keywords: Generative AI in Education, Robotics ABSTRACT The rise of generative AI tools such as ChatGPT presents both opportunities and challenges for engineering education. This study explores how master's students in a mobile robot control course used ChatGPT across two consecutive years (2023– 2024). The course combines foundational lectures with programming tasks and handson experiments. Using surveys and semi-structured interviews, we examine (1) how students used ChatGPT, (2) how usage varied by learning objective, and (3) how critically students interpreted its responses. Results show that students frequently used ChatGPT for coding support and conceptual clarification—particularly for challenging topics like particle filters—while relying on it less for system integration or report writing. ChatGPT was often used as a “sparring partner” to troubleshoot coding issues, refine ideas, and understand theoretical concepts. Students consistently demonstrated a critical approach to its responses, verifying outputs using simulators, their own knowledge, or external sources. These findings suggest that ChatGPT can support students in their learning process when tasks are well-defined and when validation tools like simulators are available. However, its value is limited for open-ended or integrative assignments requiring creativity or system-level thinking. We recommend providing students with clear guidance on the responsible use of generative AI and highlight ways these tools can be thoughtfully embedded into engineering curricula. 2 1 INTRODUCTION ChatGPT, a series of highly advanced generative AI models, was first released in November 2022. Within two months, the tool reached 100 million active users per month, making it the fastest-growing consumer application in history (Hu, 2023). Even though it is a rather young tool, its power with respect to providing personalized learning experiences is evident (Limna et al., 2022; Qadir, 2023). Recent research has shown how generative tools can influence a student’s learning experience (Zhou et al., 2024). Research on students’ usage of ChatGPT for academic purposes indicates that it is commonly used for tasks such as writing – ranging from idea generation to editing – and as a 'private tutor' to aid in understanding various concepts (Onal, Kulavuz-Onal, & Childers, 2025; Klimova & Luz de Campos, 2024). In addition to these uses, students also turn to tools like ChatGPT for assistance with programming and debugging common issues (Baig & Yadegaridehkordi, 2024). The ease of use of these tools makes them appealing to students and may enhance their self-regulation (Zhou et al., 2024). For example, such models have the potential to provide tailored learning material based on the profile of the student interacting with the tool (Limna et al., 2022; Qadir, 2023). This is particularly relevant for creating a learning experience tailored to students' needs and interests—a key characteristic of challenge-based learning. Such personalized learning experiences lead to stronger student engagement, promoting (deep) learning (Robledo-Rella et al., 2025). However, mainstream generative AI tools are not without limitations. One of the main limitations is that they are prone to writing plausible-sounding but incorrect answers, potentially misleading users (Hua et al., 2024). Additionally, research indicates that when generative AI tools are used to provide direct solutions, they offer little benefit, or even harm, to novice learners. These tools are only effective when users already have foundational knowledge of a topic (Prather et al., 2025), making them less supportive of students' educational development. Given the rapid evolution of tools like ChatGPT, it is crucial to recognize both their potential benefits and limitations. As a first step in integrating these tools into (higher) education, it is essential to understand how students are actually using them in specific courses. Starting from the assumption that prohibiting generative AI tools in specific courses is neither feasible nor beneficial and recognizing that an increasing number of students will use these tools, this paper examines how students used ChatGPT in a master'slevel robotics course that covers foundational subjects such as motion planning and localization with a substantial programming component. To examine this, we took a longitudinal approach studying two consecutive editions of the same course to help rule out the “novelty effect” of such tools. This study combines student surveys and student interviews to provide complementary perspectives. By examining students' real use of ChatGPT, we aim to gain insights that could inform potential course adaptations (if necessary) and provide actionable guidance for students on the responsible use of AI tools in education. 1.1 Research Questions The study was conducted to answer the following research questions: 3 RQ1: In what ways is ChatGPT used in a typical master’s-level robotics course? RQ2: How does students’ use of ChatGPT differ in order to meet different types of learning objectives? RQ3: Do students interpret the answers provided by the tool with a critical mindset? 2 CONTEXT AND PRACTICAL WORK 2.1 Course Description The Mobile Robot Control course is a master's-level course offered by Eindhoven University of Technology (TU/e) in the final quarter of the first year of the Master’s program. While the majority of students typically come from the Mechanical Engineering or Systems and Control programs, the course is open as an elective to all TU/e master’s students. As a result, the student cohort has diverse academic backgrounds, though most have a strong foundation in mechanical engineering or systems and control. The course is structured as a workshop-style class. In the first part of the course, students receive introductory lectures on the fundamental methods to enable mobile robot navigation, i.e. local and global path planning and localization. They are provided with C++ code snippets to help them implement the methods introduced during the lectures. In the second part of the course, students are required to integrate the individual components (i.e. local and global path planning and localization) in a closed-loop control algorithm for a mobile robot. They have a specific lecture on software design related to this activity. To experiment with the methods, students are provided with a simulation environment (Fig. 1, Left) as well as testing time with real hardware (Fig. 1, Right). The software needed to connect their system implementation, and the simulated or real hardware is provided to them. The course’s learning objectives are reported in Table 1. There are three summative assessment moments. The students are required to describe how they decided to implement each single component in the first version of a written report to be delivered in the middle part of the course. Next, they are requested to integrate each single component in a working solution to enable mobile robot navigation (LO5, LO6). They show their integrated solution to all other teams during a dedicated experimentation session referred to as “the final challenge” that happens at the end of the course. They then write an extensive report about their design of the single components and their integration with a reflection on their performance during the final challenge (all LOs). Students reported using generative AI for report writing, further clarifying methods or topics, and assisting with coding or debugging. These uses could impact the achievement of key learning objectives and significantly influence course assessments. Figure 1: (Left) The simulation environment provided to students of Mobile Robot Control. (Right) The physical robot deployed during the final challenge 4 Table 1. Learning objectives of Mobile Robot Control LO Description 1 Describe the problems of mobile robot navigation 2 Describe with your own words and develop a global path planning algorithm, such as A* 3 Describe with your own words and develop a local path planner for obstacle avoidance. 4 Describe with your own words and develop a localization algorithm, such as a particle filter. 5 Design an architecture that integrates different algorithms to enable a mobile robot to fulfill a given use-case 6 Validate your system architecture on a physical robot 7 Use common tools in the robotics industry 2.2 Methodology We have systematically analyzed how ChatGPT has been used by students to achieve the course learning objectives (LOs 2--6). The analysis was based on a mixedmethods approach. The analysis was performed at the end of two consecutive editions of the course, the first ending in June 2023 and the second ending in June 2024. Results are reported separately for each year as well as in aggregated form. Survey A survey was distributed to all students who successfully completed the Mobile Robot Control course immediately following the release of grades in the academic years 2023 and 2024. The questions were structured as binary yes/no items (Table 2). Table 2. ChatGPT survey ChatGPT usage (yes/no questions) Have you used ChatGPT during 4SC020 Mobile Robot Control? Have you used ChatGPT to develop (part of) the software? Have you used ChatGPT to gather background knowledge to understand the subject? Have you used ChatGPT to improve your report? Interviews We conducted individual semi-structured interviews with five students for the first edition of the course (2023) and four students for the second edition (2024). Students were selected based on their availability and each represented a different group. They received no monetary compensation. Interviews followed an interview guide. Due to space constraints, we report the guide as an online appendix in the accompanying online repository1. During the interviews, the questions focused on how students used Chat GPT during the course , grouping the questions into coding, knowledge gathering, and report writing. Each question was asked per learning objective (i.e. LOs 2-6; see Table 1). Verbatim transcripts of the interviews were automatically generated using a recording software. The audio recordings were deleted after the transcripts were generated. Analysis Code analysis was performed on students’ interviews. Initially, one course teacher, an educational advisor, and an independent teacher, all affiliated with the university’s Mechanical Engineering department, collaboratively developed a preliminary coding 5 scheme (deductive step) by reviewing the first round of student interviews and based on their own experience, as the intention was to explore the ways in which students report using ChatGPT. Following the second round of interviews, the initial coding scheme was refined, and additional codes were added based both on the new data and the categories derived from the interview questions (inductive step). Once the final coding scheme was agreed upon, it was used to analyze both rounds of student interviews using the qualitative analysis software ATLAS.ti. The final coding is reported as an online appendix in the accompanying online repository1. The same table also reports the relationship between each code and the research questions. 3 RESULTS AND INSIGHTS 3.1 Survey The survey was sent to all students who passed the course after receiving their grades (approaximately 50 students per edition). Ten students responded in both 2023 and 2024, representing about 20% of students that attended the course. In 2023, nine students reported using ChatGPT, while in 2024, seven students did. Notably, in 2023, students primarily used ChatGPT for coding rather than knowledge gathering, whereas in 2024, this trend was reversed (Fig. 3). However, in both years, students consistently reported using ChatGPT the least for improving their reports (Fig. 3). Fig. 3. In what ways students used ChatGPT in 2023 and 2024. 3.2 RQ1: General Use of the Tool When answering the first research question, three themes emerged for the student usage of ChatGPT: sparring partner, coding, and report writing. ChatGPT as Sparring Partner Over the two years, interviewed students consistently reported using ChatGPT as an interactive “sparring partner”. For example, a student from 2023 shares, “So you can ask ‘what can we change?’ And then he says, ‘Ohh you can maybe change this parameter,’ and you can have a look at that.” Another student from 2024 comments, “ But there were like a couple things where I I wasn't sure how to implement stuff. And I did ask ChatGPT”. Students engaged in an interactive dialogue with the tool. One student from 2023 explains, “uh, yes, it was mostly like a dialogue interaction. Yeah, just like it felt like more like a person. Say ‘OK, no, this is wrong. How do you do it the other way?’ And it started responding accordingly.” 1https://gitlab.tue.nl/et_projects/papers/2025sefi 6 These interactions could involve either diving deeper into concepts or theory. For examples, a student from 2024 shares, “So I also asked about that. Like to like figure out how I could do that weighing, make it work.” Sometimes there were back and forth interactions between the student and ChatGPT on how to implement a certain method. One student from 2023 explains, “so then I just really went back and forth with it to say ‘OK, I've tried this, it doesn't work out’. So I'm putting this, these interesting results and then back and forth till I could get to the to the right answer. Um, that it really helps. In the end it was still trial and error, but yeah.” Explaining theory, more in a “frontal lecture” manner was also consistently mentioned as a preferred way to interact with the tool over the two years. One student from 2023 shares, “…for example and then it just writes in text and how the particle filter works and what parameters you can tune”, and another student from 2024 “ So to get a little bit more understanding of the terminology or for the exercises, more for the theory part.” ChatGPT for Coding Across the two years, students reported that they used ChatGPT for all coding tasks, especially for debugging. A student from 2023 shares, “Then I went to ChatGPT saying that OK, this is the error which I'm getting. Then it get told me ‘OK, you can do try the these options and check whether it's working or not.” Another student from 2024 explains, “I mostly also used it as I said for debugging the problems that I found on the way with the path planner.” Students also reported using ChatGPT to clarify code snippets provided to them. For example, a student from 2023 explains, “So I copy paste them, put them in the chatGPT and either told it to ‘just complete it’ as I thought it should do it, or ask it to explain it in a more layman's.” Another student from 2024 comments, “There is not that much explanation to it just.  Check it, like choose the part of the code that was changed.  Ask it to give me a short transcript or explanation” . Some students admitted to using ChatGPT to generate from scratch, typically for small, well-defined functions. A student from 2023 explains, “So I would then just ask ‘complete this function’ or ‘write this function where the correct pointers' to yeah to actually get the correct output.” Another student from 2024 shares, “I was not that accustomed to C++, just typing what I wanted much easier”. Contrary to the researcher’s expectations, students did not use ChatGPT to translate code from one language that could be more familiar to them, e.g., python to the target language of the course, i.e., C++. Surprisingly, less students from 2024 declared they used the tool for coding, which was consistent with the survey results in Fig. 3. This could be a trend, or only an effect of the limited sampling of the interviews over the entire course population. ChatGPT for Report Writing Among the three categories, students reported using the tool least for writing the report. They were very explicit about it, as one student from 2023 states, “and for reporting I didn't use ChatGPT.” Another student from 2024 claimed, “I don't think I 7 used it for the report at all.” This was consistent between the two years and the survey results (Fig. 3). When students did use the tool for writing, they primarly acknowledged using ChatGPT for basic grammatical improvements. As one student from 2024 explains, “Yeah, maybe some spelling errors, but not more than that.” Additionally, they used it to improve the “academic tone” of their writing. A student from 2023 shares, “…I write something myself the and then I let ChatGPT improve it, make it sound a bit more academical…” 3.3 RQ2: Dependence on Learning Objectives The usage of ChatGPT varied across learning objectives and for different reasons. However, over the two years, students consistently used ChatGPT for LO4, which relates to the understanding and development of a localization algorithm, the most. A student from 2023 explains, “For example, the particle filter and it also beginning quite vague [how] it works. So then if you don't understand it, you copy paste a part of it.” Another student from 2024 shares, “ cally More of the concept [of the particle filter], basi ” Another student . because like the codes was really, you know, framed, basically .”I definitely used it [ChatGPT] for some parts [of LO4]simply states, “from 2024 LO4 was challenging for the students for different reasons. The theoretical background includes concepts such as conditional probabilities which students do not normally encounter in the typical mechanical engineering or systems and control curricula. The implementation of the algorithm is also not straightforward. Students found ChatGPT useful for both the coding and the knowledge gathering part. Yet the learning objective did not require students to create something that is new or largely deviate from the body of knowledge that is already available and on which ChatGPT was trained. Students also consistently declared that they used ChatGPT for the integration of learning objective LO5 and LO6 the least. The learning objectives require the students to devise and implement a way to integrate all the algorithms that they developed into a functioning navigation system for a mobile robot. The learning objective requires creativity and solutions that are specific to what the students developed. Because of the open nature of the assignment, ChatGPT was used the least. When used, it is for practical matters, like understanding how building multiple source file works as a 2024 student explains, “.lists I did need some help with the make file the Cmake ” 3.4 RQ3: Attitude Towards ChatGPT Answers Students consistently reported that they reflected on the answers that ChatGPT provides. Depending on the type of request, they had different strategies to fact-check the answers. They questioned the validity of ChatGPT’s answers based on their own understanding, such as one student from 2023 explains, “… because when it comes to concept, the logic didn't fall in place. So need to say ‘OK this is wrong’. It seems to be wrong.” Another student from 2024 comments, “If you take into account that the answer's probably aren't always correct, but it can at least how I see it is it.”. After the contemplation, students engaged in an interactive dialogue with the tool when possible checking the legitimacy of the answers. For this course, a simulator was 8 provided to them which they used to check the validity of the answers. A student from 2023 explains, “… from the course itself, we had a couple of tests to run and then there would still just out the wrong results.” Another student from 2023 shares, “after implementing something it didn't go the right way. So we went back and asked, ‘OK, maybe this is something wrong is happening’ Then I gave an explanation. OK, ‘this might be the reason.” Students also fact-checked the answers provided by ChatGPT by additionally querying conventional search engines as one student from 2024 shares, “So I was just asking it like what method I could use and then just go online, ”.double check the method There were mixed opinions regarding the need for a specific lecture on ChatGPT within the context of the course - ss well as what the content of that lecture should be. With respect to the interactive usage and prompting students are not convinced that a lecture can provide more training than what students can already gain by practice and checking online resources. However, raising awareness about the accepatble use of the tool within the course elicited more agreement, as one student from 2024 puts it ““Not entirely, but maybe just a tip that not to trust ChatGPT. That's it.” . 4 CONCLUSIONS AND IMPLICATIONS In this section we analyse the results with respect to the three research questions drawing guidelines for course adaptations. With respect to RQ1, the cumulative survey responses over the two years show a consistent usage for knowledge gathering (44%), coding (39%) and significantly less for writing the report (19%). This is in alignment with the qualitative responses to the interviews. Prior literature however, indicates predominant use also for report writing (Onal, Kulavuz-Onal, & Childers, 2025). Different reasons could explain the observation. First, the observed difference can simply be true for this specific course. Second, sampling. The students that answered the questionnaire and participated in the interview did not fully reflect all the student population of the course. Students that used ChatGPT more heavily for report writing may have just not participated in our study. A third hypothesis could be that there is a sort of stigma associated to explicitly admitting the use of ChatGPT for report writing. To rule out the third possible reason, we advise to explicitely stating in what way ChatGPT can be used for report writing at the beginning of the course (e.g., enhancing the academic tone is allowed but no writing from scratch). We believe that this action can foster a higher level of transparency between the students and the lecturers. With respect to RQ2, the different types of learning objectives suggest different types of use of the tool. From the qualitative analysis of the students’ interviews we can argue that when the task is well-defined ChatGPT can provide good guidance also in the learning process due to its interactivity (i.e. usage as sparring partner). The guidance is directly linked to the availability of tools to test and further reason about the responses that are provided stimulating short loops of critical thinking. In the case of this course, the simulator provided to students served this purpose. Particularly for robotics course we argue for the need to provide students with good simulation software to allow them to critically evaluate the answers provided by ChatGPT in an easier and faster ways to stimulate critical thinking grounded on the evidence provided by the simulator. To foster a higher level of critical thinking, the simulator could be enhanced with more advanced data visualizations such as paths and trajecory information obtained from simulation episodes.