Shaping Online Student Success: The Power of Scenario-Based Learning on Understanding and Confidence

Practice Paper Recommended citation: Malone, M., Mullan, C., & Farrell, F. (2025). Shaping Online Student Success: The Power of Scenario-Based Learning on Understanding and Confidence. In Kangaslampi, R., Langie, G., Järvinen, H.-M., & Nagy, B. (Eds.), SEFI 53rd Annual Conference. European Society for Engineering Education (SEFI), Tampere, Finland. DOI: 10.5281/zenodo.17631429. This Conference Paper is brought to you for open access by the 53rd Annual Conference of the European Society for Engineering Education (SEFI) at Tampere University in Tampere, Finland. This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License. Shaping Online Student Success: The Power of Scenario-Based Learning on Understanding and Confidence M Malone a,1, C Mullan b, F Farrell c, a ATU, Sligo, Ireland, ORCID 0009-0006-6965-0222 b ATU, Sligo, Ireland, ORCID 0009-0003-0789-9063 c ATU, Sligo, Ireland, ORCID 0009-0006-4256-3145 Conference Key Areas: Digital tools and AI in engineering education and Open and online education for engineers Keywords: Scenario Based Learning, Digital Transformation, Online Learning, STEM Education, Problem-Solving. ABSTRACT The global demand for online learning is rapidly increasing, with learner participation project to exceed 1 billion learners by 2028. In Ireland, 40% of workers were engaged in online courses in 2024. While online education offers flexibility; challenges like reduced collaboration and the growing use of AI-assisted assignments persist. Educators are increasingly adopting active learning approaches to enhance engagement and learning outcomes. Active learning is one of five key student-centric learning elements which significantly influence student satisfaction with online courses. Scenario-based learning (SBL) is emerging as an effective active learning method, particularly in engineering education, where the industry frequently calls for stronger integration of problem-solving skills. This study evaluated an interactive online SBL scenario using the virtual platform ThingLink, where students applied the 5-Whys technique in an industrial lab setting. Engaging with virtual stakeholders, they identified root causes receiving instant feedback. A survey showed 84% found the platform engaging, 84% found it easy to use, and 81% saw strong theory-to-practice connections. The results highlight SBL’s potential to enhance online education and effectively support digital learning transformation. 1 Corresponding Author M Malone mary.malon[email protected] 1 INTRODUCTION To meet Industry 4.0 challenges, engineering graduates must develop key skills, with problem-solving being fundamental (Achmad et al., 2025). Fleming et al., (2024) confirm its priority through job market and salary analysis, aligning with de Campos et al., (2020) who highlight problem-solving as central to engineers' ability to address technological and societal challenges. Active learning methods are becoming key in today’s classrooms to address this skill, shifting from students passively listening to actively getting involved and participating in their learning journey. One such method is Scenario-Based Learning (SBL), a strategy where learners are immersed in realistic and practical situations requiring them to apply theoretical knowledge to solve problems. SBL is grounded in constructivist learning theories, particularly those proposed by Vygotsky (1978) and Dewey (1938), which suggest that knowledge is constructed through experience and social interaction (Orhan, 2024). This strategy has been shown to not only enhance the retention of information but also supports the development of critical thinking and problem-solving skills (Angafor et al., 2023; Aslan & Arabacı, 2023). By engaging students in realistic scenarios, SBL aims to enhance both conceptual understanding and skill development, making it a highly effective tool for fostering critical thinking, problem-solving, and decision-making skills (Ahmed, 2019; Battista, 2017). Chernikova et al., (2020) reported that SBL is particularly effective in fields requiring complex decision-making and collaborative problem-solving, such as medicine and engineering. Additionally, Mehall, (2022) compared in-class and eLearning versions of SBL in business education and found both methods to be equally effective in developing communication and decision-making skills. Teaching methods such as SBL enhance educational quality by fostering engagement, interactive learning, and inclusivity (Kulal et al., 2024), with students favouring elearning for its flexibility and personalised experience (Vacalares et al., 2024). Linking SBL with the digital environment allows access to self-paced learning and accessibility to a large number of students. However, several challenges exist in implementing SBL effectively. Mehall, (2022) noted that eLearning SBL may reduce student-instructor and peer interactions, potentially affecting engagement and learning outcomes. Chernikova et al., (2020) also pointed out that novice learners may struggle with SBL if not provided with appropriate scaffolding, as cognitive overload can hinder the learning process. These findings indicate that while SBL is a valuable pedagogical approach; shown to have significant effects on cognitive learning, skills development, and practical application; its effectiveness depends on careful instructional design and the availability of sufficient support mechanisms (Mamakli et al., 2023). This study examined the perceptions of online students’ experience in using SBL in applying a well-known root-cause technique called the 5-Whys.Specifically, addressing two core questions: 1. Do students feel that their understanding and confidence in applying the 5Whys technique has improved after participating in the virtual scenario? 2. How do students perceive the ease of use, engagement, and enjoyment aspects of the virtual platform when learning and practicing the 5-Whys technique? 2 CONTEXT AND PRACTICAL WORK 2.1 Online Scenario Design and Development Atlantic Technological University (ATU) operates across nine campuses in the west and northwest of Ireland offering a wide range of multi-disciplinary courses. ATU has been a leader in digital education, since initiating its online learning programs in 2002, with 67% (10,173) currently enrolled in online learning reflecting its dedication to accessible and industry-relevant education. 50% of the engineering students study online. The aim of the scenario was to mimic the realities of applying the 5-Whys problem solving technique in an industrial environment. Using extensive industry experience the researchers were able to develop various scenarios that would be suitable for an industrial setting. The final selected scenario was developed to align with the learning objectives of two online modules at ATU(Introduction to Lean Sigma Quality NFQ Level 7 and Quality Management NFQ Level 8). The staff involved in the delivery of these courses were also involved to ensure the learning outcomes of the modules were aligned with the scenario activity. The initial concept was outlined in a storyboard to define the need, requirements, and framework for the scenario. ThingLink was selected as the technology platform due to its suitability for scenario-based learning and its availability within ATU. Its use was also justified by the fact that ThingLink, a Web 2.0 tool, emerged as a key support resource during the COVID-19 pandemic, enabling the creation of interactive and engaging learning materials (Rodríguez & Pulido-Montes, 2022). Once the scenario was built in ThingLink it underwent multiple rounds of testing and was peerreviewed by experienced faculty members with expertise in both industrial applications of the 5-Whys technique and its instruction. 2.2 Student Experience and Evaluation of the Scenario On accepting the invitation to explore the interactive tool, students entered the platform and were greeted first with a short introductory video to recap on the 5Whys technique. Following this they were immediately presented with a realistic problem set in a pharmaceutical laboratory, where samples had to be discarded. Using the ThingLink platform students could explore various virtual locations to gather information and apply the 5-Whys tool (See Fig. 1). The interactive scenario simulated a real-world investigation process, encouraging learners to analyse operational data and identify a root cause. No additional instructions on how to use the platform were provided. As they progressed through the scenario and identified their final course of action, they received immediate feedback on their proposed solution and guidance on further exploratory options. This approach is supported by Bardach et al., (2021) research that demonstrated having both feedback and reflection embedded in their online scenario enhanced the ‘self-efficacy and classroom readiness’ for student teacher training. It also addresses the challenge for online learners with fewer student-faculty interactions (Shu et al., 2023). Fig. 1. An example of the various virtual locations in the ThingLink Scenario On completion of the scenario the students were then provided the opportunity to rate their experience using five-point Likert scale statements and open-ended questions to capture detailed perspectives. The survey was self-developed guided by the works of Mio et al., (2019). It consisted of some profiling multiple choice questions along with their self-assessed rating of their current theoretical and practical experience in using the 5-Whys technique. 3 RESULTS AND INSIGHTS The response rate for this survey was 28% aligning with Fosnacht et al., (2017) who found that response rates of 20-25% provide fairly confident estimates in higher education surveys. For ease of analysis and presentation, responses for Strongly Disagree and Disagree were grouped together, as were Strongly Agree and Agree. The open-ended questions were analysed in detail manually to understand the relevant perspectives from the participants. Cronbach's alpha assessed the internal consistency of the Likert scale questions which yielded a highly reliable score of 0.9548 (Creswell & Creswell, 2023). 3.1 Participant Overview Demographically, there was a relatively even spread across experience levels from early career professionals to experienced professionals reducing the risk of potential bias toward any single career stage. Participants represented a diverse array of industries, such as Technology, Manufacturing and Retail with Healthcare and Life Sciences being the most prevalent at 38%. Additionally, there was variation in professional roles within each industry. The largest share of participants held positions in Leadership and Management (34%), followed closely by those in Engineering and Technical roles (31%). 3.2 Participants perspective before using the software The survey asked participants to reflect on their understanding and confidence levels of the 5-Whys before accessing the interactive scenario. 56% of participants had an average level of understanding of the 5-Why theory before accessing the platform, which could be attributed to their previous experience within the traditional online classes covering the theory and their professional background.This prior knowledge serves as an essential form of scaffolding, helping to support learners as they navigate the interactive scenario (Chernikova et al., 2020). Participants who demonstrated a high level of understanding generally reported being ‘reasonable’ to ‘very confident’ in their ability to apply the 5-Whys technique. Most of these participants had 6 years plus professional experience. This suggests that knowledge and familiarity with the method directly contributes to self-assurance in using it, reinforcing previous literature on experiential learning and expertise development (Chernikova et al., 2020). 3.3 Students’ engagement and enjoyment of the scenario The data in Fig. 2 shows an overwhelmingly positive response with 84% of the participants found the scenario was an enjoying and engaging way to support their learning. For example, P16 noted ‘it certainly was an interesting way to learn’ and P23 stating they ‘loved being able to explore all the options’. Participants also valued the connection to real-world scenarios. For example, P8 noted that it was ‘a very real-life scenario’. Some of these participants expressed a desire for a more complex scenario, noting that their work environments are inherently more intricate. For example, P10, who strongly agreed with this statement, noted that while ‘overall it was very realistic’ in their real-life experience it did not consider some elements of a highly regulated industry.Those who disagreed or felt neutral about the enjoyment and engagement of the scenario did not provide additional commentary to explain their reasoning. Fig. 2. Survey results: Levels of student engagement and enjoyment of the scenario 78% of students agreed or strongly agreed that throughout the scenario they had to reflect on what they knew. This suggests that the participants were actively engaged throughout the activity. According to Kolb's, (2014) experiential learning theory, reflection is a crucial step in transforming experience into meaningful learning. The scenario-based approach appears to have facilitated this process, reinforcing previous findings that interactive and problem-based learning environments enhance critical thinking and knowledge retention (Boud et al., 2013). 9% 9% 16% 6% 6% 6% 84% 84% 78% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100% The scenario is an engaging way of supporting the learning of 5Whys I enjoyed the scenario because it had a real‐life component. In order to proceed through the scenario, I had to reflect on what I knew. Students’ engagement and enjoyment of the scenario Strongly Disagree + Disagree Neither Agree nor Disagree Agree + Strongly Agree 3.4 Student’s perception of their skill development Ability to Apply in Professional Environment: The data in Fig. 3 revealed that most participants found the activity helped to improve their learning and skill development of the 5-Whys tool. Of those participants that had only some or no understanding of the method before using the scenario, 83% reported that it helped them see connections between theory and real-world application. Over 50% of participants found that the scenario gave them confidence to apply the 5-Whys in their professional environment and that their ability to analyse problems improved as a result of completing the scenario. The 13% who were not confident in their ability to apply the method before using the scenario, agreed that the scenario has given them confidence to apply the 5-Whys in their professional environment after. Fig. 3. Survey Results: Level of student’s perception of their skill development Improving Problem Solving Skills: The majority of participants (72%) found the scenario helpful in improving their problem-solving skills. P13 noted ‘it was very helpful showing the steps of problem solving and showed the use, effectiveness and value of the 5-Whys technique.’ This aligns with research suggesting that SBL enhances cognitive skill development by simulating real-world problem-solving processes (Angafor et al., 2023; Chernikova et al., 2020). Additionally, 56% of participants agreed or strongly agreed that the scenario improved their ability to analyse problems. An encouraging 81% of students found that the scenario helped them see connections between the theory and its real-world application. P30 detailed it was a ‘good system to use to aid in understanding’ and P18 noted ‘I did find it beneficial as when I did go through it, I recalled many things in the course I had learnt’. This finding reinforces the importance of experiential and SBL in bridging the gap between abstract concepts and practical use (Ahmed, 2019; Battista, 2017). By engaging learners in authentic problem-solving contexts, the scenario likely facilitated knowledge transfer, making it easier for participants to recognise how the 5-Whys technique applies in professional settings. Confirmed also by the positive response to the previous statement around their confidence in applying it professionally. 13% 16% 13% 16% 6% 13% 31% 28% 81% 72% 56% 56% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100% The scenario helped me to see connections between theory and its real-world application in… I found the use of the scenario a helpful way of improving my problem‐solving skills. My ability to analyse problems and situations improved as a consequence of doing the… This scenario has given me the confidence to apply it in my professional environment. Student’s perception of their skill development Strongly Disagree + Disagree Neither Agree nor Disagree Agree + Strongly Agree A pattern emerged among participants who selected ‘Neither Agree nor Disagree’ for statements regarding the tool's usefulness in improving problem-solving skills, ability to analyse problems, and confidence in professional application. These participants had average to high levels of understanding and were already confident in applying the 5-Whys before using the tool. This neutrality suggests that while they did not find the tool ineffective, they also did not perceive any significant additional benefit. Similarly, the participants that ‘strongly disagreed or disagreed’ with statements in Fig. 5 all had over six years’ experience and had already indicated a reasonable confidence level in applying the 5-Whys along with a solid understanding. This suggests that while the tool may successfully introduce the 5-Whys to less experienced users, it might lack depth and complexity for more experienced users, emphasising the importance of thoughtful instructional design as discussed by Mamakli et al., (2023). 3.5 Student’s evaluation of the software and scenario Fig. 4 outlines that the vast majority of participants found the software easy to use (84%) and navigate (72%). P36 found that it was ‘easily understandable and applicable to my experience’ and P26 highlighted that ‘the video is excellent, and the virtual experience is very good’. P13 noted the scenario was ‘straight forward, easy to use and excellent presentation’. Of those who disagreed with the statements in Fig. 4 they noted challenges in navigating through the scenario and completing it smoothly. For example, P25 found it ‘a little clunky’ and some participants wanted more instruction on how to use the platform. Some participants had compatibility issues using a Mac. Participants noted some recommendations and technical improvements for this scenario for future iterations. Improvements included adding industry-specific scenarios, more complex scenarios for deeper learning, providing clearer instructions on using the platform and addressing issues with forward and backward navigation. Fig. 4. Survey Results: Level of student’s evaluation of the software and scenario 3.6 Overall Student Experience The overall student experience was rated at an average of 8/10, indicating a generally very positive reception. The SBL tool was well received in terms of engagement and connecting theory to practice. With participants noting it was ‘an 16% 13% 13% 3% 72% 84% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% The scenario was clear to navigate Thinglink software was easy to use Student’s evaluation of the software and scenario Strongly Disagree + Disagree Neither Agree nor Disagree Agree + Strongly Agree interesting way to use learn a new diagnostic skill’ (P20), it was a ‘good system to aid in understanding a range of root causes for an issue’ (P30) and that ‘the example was easily understandable and applicable to my experience’ (P36). However, some experienced users found it of less value. Refining the tool based on these insights can enhance its effectiveness and broaden its utility across different levels of expertise and professional sectors. 4 CONCLUSIONS AND IMPLICATIONS By embedding SBL with traditional online learning, this study demonstrates how technology can bridge the gap between theory and practice, reinforcing active learning principles and the need of integrating STEM teaching within real-life contexts (Altan et al., 2019). While prior studies (Chernikova et al., 2020; Mehall, 2022) have shown the value of SBL in education, this study offers a distinct contribution by applying SBL to an authentic problem in a fully online format. By integrating interactive virtual environments (ThingLink) with the industry-relevant 5Whys tool, it aligns closely with engineering quality and lean practices. It also extends existing models by examining learner perceptions across experience levels, highlighting how scaffolding and realism influence perceived skill transfer (Mamakli et al., 2023). The greater benefit reported by novice learners reinforces SoTL principles around differentiated instruction and the need for adaptable scenario complexity in online engineering education (Potter & Kustra, 2011). This adaptability can be achieved by involving more experienced students in scenario development in which they would also benefit from peer-to-peer learning in online settings (Balta et al., 2017; Jacobs et al., 2023; Sadykova, 2014). Additionally given the importance placed by employers on effective problem-solving skills Fleming et al., (2024) note SBL could also serve as an effective recruitment selection tool. A key innovation of this work is SBL’s potential as an assessment tool, promoting academic integrity in an AI-driven world. By enabling authentic, practical assessments, SBL mitigates risks of plagiarism and AI misuse. Additionally, embedded feedback mechanisms provide immediate, constructive input, enhancing learning as recommended by the work of Bardach et al., (2021). Furthermore, ThingLink and similar platforms offer scalable, cost-effective learning solutions. By integrating Universal Design for Learning (UDL) principles into the scenario, they enhance inclusivity, catering to diverse student needs (CAST Inc., 2024). Additionally, SBL can function as an Open Educational Resource (OER), promoting equitable access to high-quality, interactive learning for underserved and remote learners. Overall, this research provides a foundation for further exploration into the application of SBL in both teaching and assessment. It could foster a more interactive, skill-based, and integrity-focused education, aligning with the accreditation standards of professional engineering bodies like those in the Washington Accord (Vinothkumar, 2019). 5 ACKNOWLEDGEMENTS We thank the Advance Centre at ATU Sligo for supporting the virtual scenario's development. Special thanks to Yvonne Sarsfield, Instructional Designer, for her expertise in building it with ThingLink. We are also grateful to the students who shared their insights and experiences.