CLearinghouse of Engineering Education Resources (CLEER): Bridging Research and Practice

Invited Workshop Recommended citation: Kranjc Horvat, A., Groll, J., & Tormey, R. (2025). CLearinghouse of Engineering Education Resources (CLEER): Bridging Research and Practice. In Kangaslampi, R., Langie, G., Järvinen, H.-M., & Nagy, B. (Eds.), SEFI 53rd Annual Conference. European Society for Engineering Education (SEFI), Tampere, Finland. DOI: 10.5281/zenodo.17632086. This Conference Paper is brought to you for open access by the 53rd Annual Conference of the European Society for Engineering Education (SEFI) at Tampere University in Tampere, Finland. This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License. CLEARINGHOUSE OF ENGINEERING EDUCATION RESOURCES (CLEER): BRIDGING RESEARCH AND PRACTICE A Kranjc Horvat a, 1 , J Groll b, R Tormey c a EPFL, Lausanne, Switzerland, 0000-0002-2584-4715 b EPFL, Lausanne, Switzerland c EPFL, Lausanne, Switzerland, 0000-0003-2502-9451 Conference Key Areas: Improving higher engineering education through researching engineering education, Digital tools and AI in engineering education Keywords: engineering education research, AI tools, teacher education ABSTRACT One of the primary objectives of engineering education research is to improve teaching practices. However, many research findings, even those published in top journals, never find their way into practice. Two major reasons for this gap between research and practice are time constraints and the overwhelming volume of available resources. The CLearinghouse of Engineering Education Resources (CLEER) addresses these challenges by offering engineering educators, early-career researchers, and pedagogical advisors a tool that saves time and helps them locate high-quality, relevant materials more easily. By curating resources, CLEER reduces the number of studies users need to search through to find only high-quality studies, while a search in natural language increases the relevance of search results. In this workshop, we took a critical look at CLEER, explored practical ways it can be used by educators, and used it to answer questions with high-quality engineering education resources. 1 Corresponding Author A Kranjc Horvat [email protected] 1 BACKGROUND AND RATIONALE The gap between research and practice in education, including engineering education, is a longstanding issue (Anwer & Reiss, 2023; Vanderlinde & Van Braak, 2010). Indeed, many engineering educators are not even aware of the field of engineering education research until they start innovating on their own (Mirabelli et al., 2023). However, when they enter the field they are often met with several barriers to meaningfully interacting with the existing research, including time constraints, motivation, complex language, and conceptual difficulties in understanding educational research methods (Borrego, 2007; Cain, 2016; Dart et al., 2021; Gore & Gitlin, 2004; Hemsley-Brown & Sharp, 2003; Koski et al., 2023; Vanderlinde & Van Braak, 2010). These barriers are further exacerbated by an overwhelming number of available resources of mixed quality (La Velle & Flores, 2018) that are published in unfamiliar journals. Even highly motivated educators can thus easily become frustrated when trying to connect research to practice, especially as most research they find would be about education rather than for educational practice (Hordern, 2018). While tackling all these barriers at once is an impossible task, smaller steps can be taken to bridge the gap between educational research and practice. For example, critically assessing literature to find high-quality publications is a very timeconsuming task. By providing educators with a curated database of relevant resources that can be accessed through an intuitive search tool, we can reduce the time they spend searching for literature and give them more time to engage meaningfully with the content. 1.1 CLEER: CLearinghouse of Engineering Education Resources CLEER (CLearinghouse of Engineering Education Resources) is a curated, searchable database of engineering education resources that is designed to streamline the process of discovering and evaluating relevant literature. The tool was developed at EPFL in Switzerland with the support of EuroTeQ Engineering University 2 . CLEAR primarily serves early-career researchers exploring the field of engineering education research, experienced engineering educators seeking to deepen their practice, and pedagogical advisors supporting pedagogical development. By offering an intuitive search interface, CLEER enables users to efficiently identify relevant publications, thereby reducing the time and effort required to find high-quality resources. Curation protocol The engineering education resources are curated using criteria developed by nine experts of the EuroTeQ expert team on engineering education research through a three-round Delphi study. The experts identified criteria for high-quality research that classify research papers on a three-level scale: accepted, accepted with disclaimers, and rejected. The criteria for rejection and disclaimers are presented in Table 1. 2 EuroTeQ Engineering University is an alliance of technical universities across Europe that aims to shape the future of European engineering education through international collaboration. In addition, experts identified five engineering education journals that they considered to be of sufficiently high general quality for automatic inclusion of all papers. These journals are Journal of Engineering Education (JEE), European Journal of Engineering Education (EJEE), Australasian Journal of Engineering Education (AJEE), Engineering Studies, and Transactions on Education. In addition to the greenlit materials, CLEER is filled with publications from other selected journals and conferences. The full list of journals and conferences included in the database is available on the CLEER website. Full-text versions of these published papers are fed into the automated LLM-based curation process twice to increase the reliability of the curation. Both during training and as needed, experts are also included in the loop. Based on the results of the curation, papers are either rejected or entered the database with or without disclaimers. Search tool The CLEER search tool allows users to search the database in three modes: keyword search, semantic search, and hybrid search. Users can therefore search the database using either direct keywords (keyword search) or natural language (semantic and hybrid search). More specifically, semantic search is a search method that focuses on understanding the context of the question rather than merely matching keywords. As such, semantic search and hybrid search help users who might not be that familiar with the specific vocabulary of engineering education research to search the database more efficiently. The user interface of CLEER was developed based on a user study specifically conducted for CLEER in collaboration with students from Worcester Polytechnic University. Several interviews and focus groups with representatives of three identified target user groups resulted in defining several leading concepts. First, users want their search results to be presented transparently. The information that tells them why individual results were ranked as more or less relevant to the search query must therefore be presented in the results. Second, the results of the curations should be presented in the results sections, and the results need to have denoted disclaimers and any other notes that might need to be associated with the papers. Lastly, the user interface needs to provide users with smart filters that facilitate their curation of the search results. The CLEER curation and search protocols combine expert knowledge, user studies, and modern LLM technology to bring engineering education research and practice one step closer. Table 1. An overview of CLEER criteria for rejection and disclaimers. If a resource does not meet the criteria in bold (when applicable), the resource is automatically rejected. If a resource does not meet the criteria in italic (when applicable), it is accepted with a relevant disclaimer. Please note that certain criteria may not apply to all types of resources, which are individually addressed in the curation tool. Category Description (A high-quality resource …) Literature review Must include a literature review with at least five references. Should include a theoretical framework. Should critically assess the literature. Methodology Must include a methodology section with methodology that is consistent with the research question and adequate analytical methods. Should sufficiently describe the methodology, including a full description of measures and instruments. Should include full context description (i.e., participants, course name …). Should clearly describe the educational practice. Results Must represent data clearly with no cherry-picking. Must present data of sufficient quality. Discussion Must include a discussion that connects results to the research question and acknowledges the limitations of the selected methodology. Must not cherry-pick. Should not overgeneralise. Should connect to existing literature. Should include implications. Overall Must clearly state research question/hypothesis. Must acknowledge ethical considerations. Should address limitations. 2 WORKSHOP OBJECTIVES This workshop aimed to introduce participants to the CLEER database and encourage them to explore ways to utilise CLEER to support teachers in their educational research. 2.1 Target audience This session was designed for engineering educators, pedagogical advisors, and engineering education researchers who are passionate about closing the gap between research and practice and equipping teachers with high-quality engineering education resources. 2.2 Learning outcomes By the end of the workshop, the participants were able to: - Effectively use CLEER as a tool for scoping engineering education literature. - Reflect on their literature research strategies. - Identify characteristics of high-quality research. Additionally, the participants’ inputs in this workshop actively contributed to further developments of the CLEER curation and search tool. 3 WORKSHOP DESIGN 3.1 Time plan The detailed time plan presented in Table 2 was adhered to quite closely. All users had sufficient time to explore CLEER within the context of the Archimedean Oath. 3.2 Interactivity In this workshop, participants worked in small groups (2-3 participants each) to explore the CLEER tool. First, the participants reflected on their use of literature search tools and the issues they might encounter. Second, participants individually reflected on what makes an academic paper be considered “high-quality”. Third, each participant had the opportunity to explore CLEER in the context of the Archimedean Oath both individually and in a group. Last, the reflections and feedback were done in a group setting to allow everyone the time to provide their opinions and contribute to future developments of CLEER. 4 WORKSHOP RESULTS The outcomes of the CLEER workshop were multifold. In the first interactive activity, participants reflected on their use of literature search tools, discussing both how they had learned to conduct literature research and how they now teach it to new generations of learners. Participants reported primarily relying on classical tools such as Web of Science, Scopus, and Google Scholar, while increasingly supplementing these with generative AI–based tools, including Research Rabbit and ChatGPT. Through discussion, participants identified common pitfalls of these tools, particularly the lack of transparency in search algorithms and unclear inclusion criteria. These reflections resonate strongly with the three main pillars of CLEER: transparency in curation, transparency in search relevance sorting, and ethical use of generative AI throughout the tool. The second interactive activity focused on defining what constitutes a high-quality educational research resource. Participants agreed that such a resource should be peer-reviewed and include (1) a comprehensive literature review with relevant references, (2) a clearly articulated research question, (3) a suitable and welldescribed methodology, and (4) a coherent data analysis. These criteria align closely with those identified in the earlier Delphi study, thereby reinforcing the robustness and relevance of the CLEER selection framework. Table 2. Time plan for the workshop. Run time Activity Notes 5 min Introduction - Overview of the workshop 7 min Literature search practices - Think-pair-share - Personal experiences in literature search and identification of common problems 3 min CLEER introduction - Description of main pillars of CLEER 10 min Criteria for high-quality resources - Post-it storm - Identification of criteria 5 min CLEER curation principles - Deep dive into CLEER curation principles and ethical practices 3 min CLEER vs other tools - Comparing CLEER to other literature search tools 2 min Archimedean Oath: introduction - Presentation of the Archimedean Oath case study 20 min CLEER and Archimedean Oath - Individual exploration of CLEER in the context of the Archimedean Oath - Group discussions on how CLEER changes the workflow 5 min Reflection and feedback - Reflection on the use of CLEER - Feedback on the usability and usefulness of the tool - Suggestions for further improvements Finally, the group exploration of CLEER led to several valuable suggestions for future improvement, particularly regarding user interface design and the expansion of source coverage to include additional types of educational resources. The feedback received from the workshop participants will be strongly considered in the future updates of the tool. Overall, participants expressed strong support for CLEER as a tool they would recommend to engineering educators, PhD students, and pedagogical advisors. We warmly thank all workshop participants for their contributions and the stimulating discussions at the CLEER stand. Readers interested in exploring CLEER further are invited to visit cleer.euroteq.eu. REFERENCES Anwer, M., & Reiss, M. (2023). Linking research and practice in education: The views of expert researchers in the field. Journal of Education for Teaching, 49(2), 326–340. https://doi.org/10.1080/02607476.2022.2055447 Borrego, M. (2007). Conceptual Difficulties Experienced by Trained Engineers Learning Educational Research Methods. Journal of Engineering Education, 96(2), 91–102. https://doi.org/10.1002/j.2168-9830.2007.tb00920.x Cain, T. (2016). Research utilisation and the struggle for the teacher’s soul: A narrative review. European Journal of Teacher Education, 39(5), 616–629. https://doi.org/10.1080/02619768.2016.1252912 Cruz, S., London, J., & Lightner, T. (2020). Measuring Links Between Awareness and Implementation of Engineering Education Research in Practice. 2020 ASEE Virtual Annual Conference Content Access Proceedings, 34962. https://doi.org/10.18260/1-2--34962 Dart, S., Trad, S., & Blackmore, K. (2021). Navigating the path from technical engineering to engineering education research: A conceptual model of the transition process. European Journal of Engineering Education, 46(6), 1076– 1091. https://doi.org/10.1080/03043797.2021.1992609 Gore, J. M., & Gitlin, A. D. (2004). [RE]Visioning the academic–teacher divide: Power and knowledge in the educational community. Teachers and Teaching, 10(1), 35–58. https://doi.org/10.1080/13540600320000170918 Hemsley-Brown, J., & Sharp, C. (2003). The Use of Research to Improve Professional Practice: A systematic review of the literature. Oxford Review of Education, 29(4), 449–471. https://doi.org/10.1080/0305498032000153025 Koski, A., Puustinen, M., Säntti, J., & Salminen, J. (2023). Exploring the researchbased approach of academic classroom teachers in Finland. Teachers and Teaching, 1–17. https://doi.org/10.1080/13540602.2023.2208034 La Velle, L., & Flores, M. A. (2018). Perspectives on evidence-based knowledge for teachers: Acquisition, mobilisation and utilisation. Journal of Education for Teaching, 44(5), 524–538. https://doi.org/10.1080/02607476.2018.1516345 Mirabelli, J. F., Barlow, A. J., Sanders, J. L., Ko, E., Jensen, K., & Cross, K. J. (2023). Mid-career transitions into engineering education research via structured mentorship opportunities: Barriers and perceptions. Australasian Journal of Engineering Education, 28(1), 59–73. https://doi.org/10.1080/22054952.2023.2217046 Vanderlinde, R., & Van Braak, J. (2010). The gap between educational research and practice: Views of teachers, school leaders, intermediaries and researchers. British Educational Research Journal, 36(2), 299–316. https://doi.org/10.1080/01411920902919257