Research Methods in MSc Theses: A Preliminary Analysis of 2024 IT Theses from a Finnish University

Research Paper Recommended citation: Poranen, T., Zhang, Z., Järvinen, H.-M., & Kilamo, T. (2025). Research Methods in MSc Theses: A Preliminary Analysis of 2024 IT Theses from a Finnish University. In Kangaslampi, R., Langie, G., Järvinen, H.-M., & Nagy, B. (Eds.), SEFI 53rd Annual Conference. European Society for Engineering Education (SEFI), Tampere, Finland. DOI: 10.5281/zenodo.17631899. This Conference Paper is brought to you for open access by the 53rd Annual Conference of the European Society for Engineering Education (SEFI) at Tampere University in Tampere, Finland. This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License. RESEARCH METHODS IN MSC THESES: A PRELIMINARY ANALYSIS OF 2024 IT THESES FROM A FINNISH UNIVERSITY T. Poranen a,1, Z. Zhang b, H.-J. Järvinen c,T. Kilamo d a Tampere University, Tampere, Finland, 0000-0002-4638-0243 b Tampere University, Tampere, Finland, 0000-0002-6205-4210 c Tampere University, Tampere, Finland, 0000-0003-0047-2051 d Tampere University, Tampere, Finland, 0000-0002-9561-1116 Conference Key Areas: Improving higher engineering education through researching engineering education, Curriculum development and emerging curriculum models in engineering Keywords: Master’s theses, information technology, research methods ABSTRACT The crucial role of a master’s thesis in information technology in the degree studies is to demonstrate the student’s ability to apply theoretical knowledge to a practical or research-oriented problem. In our research, we performed an AI-assisted document analysis to 100 information technology master’s theses written in a Finnish university in 2024. We analyzed the research method used, did the thesis include a design or implementation of some kind of artifact (constructive work), and how often the thesis was done for a company or other external sponsor. Based on the document analysis, we identified that 48% of the theses were done for a company, 35% were student’s own topics, 16% were done for a research group, and one was done for an association. The theses were classified into seven categories based on their main research method: constructive, mathematical, case study, systematic review, comparative analysis, action research, and mixed methods. Of these, 52% were constructive works, involving the design or implementation of an artifact. Main methodologies in the constructive works included design science research and engineering design process. The results have practical impact in curriculum development and can support supervision work. 1Corresponding Author T. Poranen Last name [email protected] 1 INTRODUCTION A master's thesis is usually written as the last academic achievement during a student's studies. The purpose of the thesis is to demonstrate student's familiarity in the field of the thesis topic, to show skills for applying scientific knowledge and methods to a relevant scientific or practical problem, and to demonstrate communication and language skills. Some of the theses are theoretically oriented literature reviews, and some rely heavily on reporting constructive work. The topics are provided by companies and research groups, or are self-discovered by the student. Methodologically there is a high variation between the theses. A better understanding of the research methods and artifact types used in these theses is essential for improving supervision practices and the teaching of research methods. Such insights can help supervisors guide students in selecting feasible project scopes and appropriate methods. To get a deeper insight into this, we decided to analyze a set of master's theses. This is done to gain information to support both designing the degree curriculum and supervising the various topics in master's theses. Our research questions are the following: 1. What are the most common research methods and corresponding method references used in information technology (IT) master's theses? 2. How often did the thesis include the design or implementation of some kind artifact (software, piece of software, process model, etc.)? 3. How often was the thesis done for an external company, a customer, or a research group? To answer the research questions, we analyzed 100 information technology master’s theses written in 2024 in a Finnish university. Our research method was document analysis (Bowen, 2009) with AI-assistance (Mahadevkar et al., 2024). Previous studies have explored MSc thesis practices in engineering education, particularly in relation to supervision practices of industrial theses (Jaakkola et al., 2022a; Jaakkola et al., 2022b), industry impact on thesis topics (Järvinen & Mikkonen, 2016), research quality (Haavisto et al. 2023), etc. To the best of our knowledge, this is the first attempt to apply AI-assisted analysis to extract research method selection and project contexts in such studies. The rest of the paper is structured as follows. In Section 2 we give an overview to the role of master’s theses in engineering education and take a look at the previous research. Then, in Section 3, we describe our research method in detail. The results of the document analysis are given in Section 4, and the findings are discussed in Section 5. The same section also concludes the work. 2 METHODOLOGY By Finnish legislation, the advanced studies of the major subject of the degree must include a thesis comprising a minimum of 20 and a maximum of 40 ECTS credits (Finlex 2013/1039, 2013). In Finland, the master’s theses in the technical fields are 30 ECTS. When the size of the master’s degree studies is 120 ECTS, the thesis makes one quarter of the whole degree studies. Degree education should be organized in such a way that the degree can be completed through full-time studies within two academic years. The practical requirements of the thesis are not defined by the governmental decree, but the universities have set their own requirements. Tampere University (2025a) has defined the goals of the master’s thesis as follows:  the student is well familiar with their field and especially the field of their thesis topic  the student possesses the skills necessary for applying scientific knowledge and methods or the knowledge and skills required for independent and demanding artistic work  the student has good communication and language skills that can be applied in their field and other applicable fields. Haavisto et al. (2023) have studied topic areas and some bibliometrics (e.g. number of words and pages, and number of references) of BSc theses. They reported that 14 out of the analyzed 98 theses contained some constructive elements, such as implementation of an algorithm, or designing something that can be considered new, or had a small-scale comparison. The supervision practices of software engineering students within a company context are studied by Jaakkola and others (Jaakkola et al., 2022a; Jaakkola et al., 2022b). They list a set of anti-patterns and actions to rescue the situation. Regarding the industrial thesis in the field of software engineering, the authors write that typically the thesis solves a practical problem in the form of a software, system, or process artifact, or seeks the possibilities or limits of new technology. Järvinen and Mikkonen (2016) studied the industrial impact on topics and types of master’s theses made in 1990-2016. In their dataset, during the academic year 2015-2016, almost 80% of software engineering master’s theses were sponsored by a company, and about 95% were constructive in nature, and only 5% were researchoriented. We assume that most universities have some supportive courses for thesis writing on BSc and MSc levels. For example, Tampere University has a master’s level course Tools for Theses (5 ECTS) that is compulsory to all students graduating from Information Technology. Currently, the course has four parts: Research methods (3 ECTS), Good scientific practice (1 ECTS), Information searching skills (0 ECTS) and Career planning (1 ECTS). A supervision plan and a seminar presentation are a part of the thesis (30 ECTS), and no additional credits are given for them. The research methods part of the Tools for Theses comprises four components, i.e. introduction, qualitative methods, quantitative methods, and applied research. The introductory component introduces the purpose of research methods, the scientific method, research approaches and classifications, ethical considerations, etc. The subsequent components focus on specific approaches: qualitative methods emphasize subjectivity, context, and flexible design, along with techniques like interviews, observation, and document analysis; quantitative methods focus on statistical reasoning, experimental design, and causal inference; applied research addresses practical problem-solving through constructive research, design science, and action research. Supplementary readings are provided to support students in understanding key concepts and applying research methods in practice. Ideally, students are expected to complete the Tools for Theses course before starting MSc thesis work to ensure they can select appropriate methods and techniques for addressing their research questions. 3 RESEARCH METHOD An overview of the research process is shown in Figure 1. In Phase 1, we conducted a search for the Tampere University’s (2025b) thesis repository to find all master's theses published in 2024 in Information Technology. On 24.2.2025, the search resulted in 100 theses. The current number is higher because there is a processing delay before the theses are published in the thesis repository. Fig. 1. Research process. In Phase 2, we performed an AI-assisted document analysis using ChatGPT-4o (2025). We used the paid version of ChatGPT. We disabled the option to use uploaded documents as training material for the model. Each document analysis was initiated in a temporary chat, ensuring that no information from previous chats was used. Each thesis was uploaded with the following prompt: Please analyze the master’s thesis to identify: 1) The research methods applied in the thesis (please provide only the method names). 2) Whether there are references cited for describing the research methods; if yes, list them with the full reference descriptions exactly as shown in the thesis reference list, if no, answer NO. 3) Whether the thesis includes design or implementation artifacts (e.g., software, a piece of software, process models, etc.); if yes, please briefly list them. 4) Whether the work was conducted for an external company, customer, or research group; if yes, provide the name only, otherwise, answer NO. The ChatGPT output to the four questions of the prompt was then placed in a shared spreadsheet for manual verification and classification (Phase 3). Here is a sample ChatGPT’s analysis of a thesis: 1. The research method applied in the thesis: Design Science Research (DSR). 2. Yes, the thesis cites the following reference: Ken Peffers et al. “A design science research methodology for information systems research”. In: Journal of Management Information Systems 24 (Jan. 2007), pp. 45–77 3. The thesis includes several design and implementation artifacts: Smart Rule Engine, Core Engine Implementation following Clean Architecture, Database using PostgreSQL and Docker, REST API developed with Django, Rest Framework, IoT Platform Service, AI Service integrated with GPT-4, Chat Interface designed using Figma, Frontend developed in React. 4. Yes, the work was conducted at GPT-Lab under the supervision of Professor Pekka Abrahamsson. ChatGPT identified the research method as Design Science Research and provided the research method reference cited in the thesis. It then detected that the thesis included a constructive component where several artifacts were designed and implemented. Finally, it reported that the thesis was conducted as part of a university research project. If ChatGPT was unable to answer one or more of the prompt’s questions, or when its responses lacked clarity or contained discrepancies, the thesis was manually reviewed by one of the authors. To ensure the accuracy of data collected through document analysis, a random sample of automated responses was cross-checked against the original documents. This combination of automated and manual analysis was used to enhance the reliability and validity of the document analysis process. Classification and other findings from data are analyzed in the next section 4 ANALYSIS OF THERESULTS The total number of analyzed theses was 100 (68 were written in English, and 32 in Finnish). Next, we go through the analysis results of the research methods, the main method references, the sponsor of the thesis, and the research methods in different sponsor categories. 4.1 Research methods Among the 100 theses, many adopted a combination of qualitative and quantitative research methods. However, for the purposes of this analysis, we categorized each thesis according to its primary research method, i.e. the one most essential to the thesis’s research design and contribution. Each thesis was assigned to one of the following seven categories: Constructive, Mathematical, Case study, Systematic review, Comparative analysis, Action research, and Mixed. The distribution of primary research methods is illustrated in Figure 2. Fig. 2. Primary research methods used in MSc theses. The most frequently applied method was the constructive method, used in 52 theses, over half of the total. This strongly emphasizes artifact development, such as tools, models, frameworks, or proof-of-concept prototypes. These theses typically focus on designing and implementing solutions that are subsequently evaluated through empirical studies, such as experiments or case studies, to assess their effectiveness. Notably, nine of these theses explicitly adopted design science research (DSR) methodologies, and three engineering design process. The mathematical method, applied in 18 theses, encompassed a broad range of techniques, including supervised, unsupervised, and reinforcement learning; statistical classification methods such as Naive Bayes and logistic regression; neural network architectures like LSTMs, transformers, and CNNs; as well as image processing techniques such as Structure-from-Motion and feature matching. These methods were applied in diverse domains, including invoice coding, time-entry classification, image stitching, pose estimation, etc. Case study methodology was applied in 16 theses, typically used to investigate realworld software engineering and IT contexts. These theses explored topics such as technology adoption, user satisfaction, and system performance by collecting qualitative data through interviews, observations, and on-site analysis. Some studies also included quantitative performance measurements. In addition, comparative analysis was emphasized in four theses to evaluate the performance or deployment feasibility of different technologies. Furthermore, there are also three theses integrating qualitative and quantitative methods equally. They were categorized as using mixed methods. There were six theses adopting a systematic review as their primary method. These studies aim to synthesize existing research on specific topics and identify trends or future research directions. Among these, one thesis conducted a multivocal literature review, one used grey literature, and two were systematic literature reviews (SLRs). The remaining two theses applied a systematic review method complemented by interviews or case studies. Additionally, one thesis employed action research. The study involved two cases and two projects in which the student worked directly with project teams to develop a data collection system and iteratively refine practices throughout the project lifecycle. Overall, the results illustrate the dominance of practice-oriented and artifact development research approaches in information technology master’s theses, particularly through constructive and design science methodologies. Table 1 lists main research method references for different research method categories. References of the mathematical research are omitted because of the high variation of different mathematical and machine learning methods. For the constructive research, many theses gave references to DSR and Engineering design process. It is also worth noting that there were 14 theses that did not reference any research method. Table 1. Research methods and main references. Method References Constru ctive Hevner, A., Chatterjee, S., Hevner, A., & Chatterjee, S. (2010). Design science research in information systems. Design Research in Information Systems: Theory and Practice, 9-22. Perjons, E., & Johannesson, P. (2022). An Introduction to Design Science. Springer Nature. Peffers, K., Tuunanen, T., Rothenberger, M. A., & Chatterjee, S. (2007). A design science research methodology for information systems research. Journal of Management Information Systems, 24(3), 45-77. Pasian, B. (2016). The Constructive Research Approach: Problem Solving for Complex Projects. In Designs, Methods and Practices for Research of Project Management (pp. 125-136). Routledge. Kasanen, E., Lukka, K. and Siitonen, A. (1993). The constructive approach in management accounting research. Journal of Management Accounting Research 5 (1993). Dym, C. L., Agogino, A. M., Eris, O., Frey, D. D., & Leifer, L. J. (2005). Engineering design thinking, teaching, and learning. Journal of Engineering Education, 94(1), 103-120. Wind, S. A., Alemdar, M., Lingle, J. A., Moore, R. and Asilkalkan, A. (2019). Exploring student understanding of the engineering design process using distractor analysis. International Journal of STEM Education 6, article 4, 1 - 18. Math. Many publications on various machine learning algorithms. Case study Gillham, B. (2010). Case study research methods. Continuum, London. Runeson, P., Host, M., Rainer, A., & Regnell, B. (2012). Case study research in software engineering: Guidelines and examples. John Wiley & Sons. System atic review Garousi, V., Felderer, M., & Mäntylä, M. V. (2019). Guidelines for including grey literature and conducting multivocal literature reviews in software engineering. Information and Software Technology, 106, 101-121 Kitchenham, B., & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. EBSE Technical Report, EBSE-2007-01. Snyder, H. (2019). Literature review as a research methodology: An overview and guidelines. Journal of Business Research, 104, 333 - 339. Compar ative analysis Coccia, M., & Benati, I. (2018). Comparative studies. Global Encyclopedia of Public Administration, Public Policy, and Governance–section Bureaucracy (edited by Ali Farazmand). Chapter, (1197 - 1). Action research Rapoport, R. N. (1970). Three dilemmas in action research: with special reference to the Tavistock experience. Human relations, 23(6), 499 - 513. Mixed Saunders, M., Lewis, P., & Thornhill, A. (2009). Research methods for business students. Pearson education. Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications 4.2 Thesis sponsor Table 2 summarizes the types of collaborators or sponsors involved in the MSc theses. Nearly half of the theses (48) were conducted in collaboration with a company, which indicates a strong industrial connection in thesis work. Table 2. Thesis sponsor Sponsor Number of theses Company 48 Own topic 35 Research group 16 Association 1 Total 100 There are 16 theses completed in collaboration with research groups in academic and research institutes, with topics driven by research projects. In addition, there are 35 theses with topics self-defined by the student without an external sponsor. These theses reflect individual interests and initiatives by students. Notably, one thesis was completed in cooperation with an association, which is not common in theses emphasizing technologies. Overall, the data indicates that a significant majority (64%) of the theses had an external stakeholder, either industrial or academic, contributing to the topic definition or project execution. 4.3 Research methods across sponsor categories The distribution of research methods across different sponsor categories reveals distinct patterns in method selection depending on the nature of the collaboration. Fig. 3. Distribution of research methods within each sponsor category. As shown in Figure 3, company sponsored theses were dominated by the constructive method (29 out of 48). This reflects a strong emphasis on practical problem-solving and artifact development in industrial contexts. This category also included seven case studies, six theses using mathematical methods, as well as systematic reviews, comparative analyses, and mixed methods. In contrast, selfdefined theses exhibited a more diverse methodological spread, with a notable number applying mathematical methods (10 out of 35), followed by 13 constructive methods and seven case studies. Theses completed in collaboration with research groups leaned toward constructive and case study methods, which aligns with typical