What impacts learning effectiveness of a mobile learning app focused on first-year students?

Johannsen, Florian et al. Article — Published Version What impacts learning effectiveness of a mobile learning app focused on first-year students? Information Systems and e-Business Management Provided in Cooperation with: Springer Nature Suggested Citation: Johannsen, Florian et al. (2023) : What impacts learning effectiveness of a mobile learning app focused on first-year students?, Information Systems and e-Business Management, ISSN 1617-9854, Springer, Berlin, Heidelberg, Vol. 21, Iss. 3, pp. 629-673, https://doi.org/10.1007/s10257-023-00644-0 This Version is available at: https://hdl.handle.net/10419/307576 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/ Vol.:(0123456789) Information Systems and e-Business Management (2023) 21:629–673 https://doi.org/10.1007/s10257-023-00644-0 1 3 ORIGINAL ARTICLE What impacts learning effectiveness ofamobile learning app focused onfirst‑year students? FlorianJohannsen1 · MartinKnipp2· ThomasLoy2· MiladMirbabaie3· NicholasR.J.Möllmann4· JohannesVoshaar2· JochenZimmermann2 Received: 26 July 2022 / Revised: 5 May 2023 / Accepted: 21 June 2023 / Published online: 26 July 2023 © The Author(s) 2023 Abstract In recent years, the application of digital technologies for learning purposes is increasingly discussed as smartphones have become an integral part of students’ everyday life. These technologies are particularly promising in the so-called “transition-in” phase of the student lifecycle when first-year students start to develop a student identity and integrate into the university environment. At that stage, most premature dropouts are observed, presumably due to a lack of self-organization or self-responsibility. Considering this, a mobile app to tackle insufficient student experiences, support learning strategies, and foster self-organization in the “transitionin” phase was developed. The research at hand proposes a generalizable success model for mobile apps with a focus on first-year students, which is based on the IS success model (Delone and McLean in Inf Syst Res 3(1):60–95, 1992) and analyzes those factors that influence student satisfaction with such an app, the intention to reuse the app, and—foremost—students’ learning effectiveness. The results indicate that learning effectiveness is determined both by the perceived user satisfaction and users’ intention to reuse, which are particularly influenced by perceived enjoyment but also system and information quality. Finally, design principles are derived to develop similar mobile solutions. Keywords HCI· Learning management systems· Mobile learning· IS success model· Transition-in phase 1 Introduction For some time now, the European labor market is facing a severe lack of skilled professionals (Peichl etal. 2022). In 2022 alone, 29% of companies in the European Union (EU) reported problems in finding suitable personnel, which is an alltime high considering the development in recent decades (Peichl etal. 2022). The Extended author information available on the last page of the article 630 F.Johannsen et al. 1 3 situation is agitated in Germany, with 50% of enterprises seriously suffering from the shortage of specialists (ifo Institut 2022; Peichl etal. 2022). Consequently, more than 770,000 vacant positions for the entire economy will not be adequately occupied in 2023 (cf. Statista 2023). In this context, a high student dropout rate is seen as a serious problem in meeting the economy’s demand for qualified workers in the upcoming years (cf. Ahlers and Quispe Villalobos 2022; Behr etal. 2021; Heublein 2014). While in Germany, 14.7% of Bachelor students do not finish their studies, this number is even higher in other EU countries like the Netherlands (28.3%) or Italy (34.1%) (Behr etal. 2020; Schnepf 2014). For education policies, such student dropout rates imply not only inefficiently used resources for higher education but also high educational costs for students not achieving the aspired educational goals (Baars and Arnold 2014; Behr etal. 2021). At the same time, dissatisfaction and negative psychological long-term effects are observed for corresponding students, which may paralyze them when searching for alternative pathways to gain a foothold in the labor market (Behr etal. 2021; Ibrahim etal. 2013; Roso-Bas etal. 2016). In terms of time, most student dropouts happen during the first year of studies (Isleib etal. 2019; Neugebauer etal. 2019; Opazo etal. 2021). According to the student lifecycle (Lizzio 2011), which describes the evolution from a prospective to a commencing, continuing, and finally graduating student, first-year students find themselves in the “transition-in” phase. In this phase, self-organized researchbased learning (cf. Huber et al. 2009) or self-responsibility, which are essential for a successful transition from the highly-structured school environment into the university system, are often perceived as challenging (cf. Zehetmeier etal. 2014). Moreover, an inadequate student experience and psychological factors—such as inefficient learning strategies or insufficient intrinsic motivation—are identified as additional reasons for early dropout (cf. Blüthmann etal. 2011; Heinze 2018; Neugebauer etal. 2019). Therefore, “remedial support early in the curriculum” (Baars and Arnold 2014, p. 106) is necessary to reach students who are at risk of dropping out prematurely. Parallel to this, universities also experience a change in students’ way of consuming and processing information, organizing their daily routines, socializing or communicating with one another (Musik and Bogner 2019), which is mainly triggered by technological progress (Cho etal. 2021; Gómez-Galán etal. 2020; Gupta etal. 2021; Youssef etal. 2021). Consequently, the impact of new technologies on students’ learning behaviors or interactions with lecturers is increasingly discussed in higher education (Ronzhina etal. 2021; Sultana 2020). By now, it is widely recognized that digital technologies may support behaviorist, constructivist, collaborative, situated, and informal/lifelong learning (e.g., Criollo-C etal. 2021; Goksu 2021; Gupta etal. 2021). Thus, in the recent past, special attention was given to learning management systems (LMS), which are “web-based software platforms that provide an interactive online learning environment and automate the administration, organization, delivery, and reporting of educational content and learner outcomes” (Turnbull etal. 2020, p. 1). The functionalities of today’s open-source (e.g., Moodle) or proprietary LMS solutions (e.g., Blackboard, WebCT of the University of Columbia) are diverse and range from course management to communication tools and progress tracking abilities amongst 631 1 3 What impacts learning effectiveness ofamobile learning app… others (Al-Sharhan etal. 2020; Koh and Kan 2021). Although several studies have shown a positive effect of LMS usage on students’ learning performance (e.g., Leontyeva 2018; Msomi and Bansilal 2019; Oguguo etal. 2021), there are concerns that the new generation of “information consumers”—who are now entering the university system—will refrain from using LMS if these systems have not been optimized for mobile devices or just serve the provision of course materials (cf. Koh and Kan 2021; Turnbull etal. 2020). As a consequence, higher education gradually focuses on the ubiquity and great acceptance of mobile phones (cf. Al-Bashayreh etal. 2022; Author selfcitation 2; Beatson etal. 2020), which have become an integral part of students’ daily lives to establish and maintain social networks (Criollo-C etal. 2021; Diacopoulos and Crompton 2020; Goksu 2021). The COVID-19 pandemic even accelerated this development, as higher education was challenged to find alternative teaching options and students primarily interacted electronically with fellow students and instructors (e.g., Al-Bashayreh etal. 2022). Several studies show a positive effect of smartphone-based learning on student performance for various courses (e.g., accounting, psychology, etc.; cf. Beatson etal. 2020; DilibertoMacaluso and Hughes 2016; Voshaar etal. 2023). In particular, research outlines the supportive impact of gamification on learning effectiveness (Pechenkina etal. 2017; Voshaar etal. 2023). Against this backdrop, this research addresses the necessity for abovementioned “remedial support” (Baars and Arnold 2014, p. 106) at initial stages of the student lifecycle to prevent first-year students from dropping out early and considers their affinity towards mobile phones equally. So, we focus on designing a mobile app for first-year students who are just about to develop a student identity and integrate into the student world (Lizzio 2011; Matheson 2018; Msomi and Bansilal 2019). We claim that a mobile app may be a suitable solution to tackle insufficient student experiences and support learning strategies as well as selforganization in the “transition-in” phase of the student lifecycle. We built a corresponding mobile app using a Design Science Research (DSR) approach, and the results of a first evaluation at the Universityof Bremen (Germany) encouraged us to pursue the project and develop the app further (cf. Johannsen etal. 2021). Further, in this research, a success model for mobile apps for first-year students in the “transition-in” phase, which is based on the IS success model of Delone and McLean (2003), is proposed and those factors that influence student satisfaction with the app, the intention to reuse the app, and students’ learning effectiveness are analyzed. This prepares the ground for formulating design principles for mobile app development afterwards. Accordingly, we pose the following research questions: • Which factors contribute to the success of a mobile app to support first-year students in the “transition-in” phase in terms of learning effectiveness, user satisfaction, and intention to reuse the app? • What design principles can be derived for a mobile app to support first-year students during the “transition-in” phase? 632 F.Johannsen et al. 1 3 The contributions of this research are threefold: First, a self-developed mobile learning app with the aim to support students in the “transition-in” phase by improving learning strategies and self-organization abilities as well as promoting the perceived student experience is introduced. Thereby, we contribute to the ongoing search for (technological) solutions to prevent early student dropouts. Second, factors positively affecting user satisfaction, intention to reuse the app, and students’ learning effectiveness are identified with the help of a success model for mobile apps and data collected in an introductory accounting course at the Universityof Bremen (Germany). Based on that, mobile app functionalities can be assessed more purposefully regarding their relevance for first-year students, which complements the existing body of knowledge regarding student app design (e.g., Almaiah etal. 2022; Laine and Lindberg 2020). Third, the findings are used to formulate design principles (cf. Gregor and Hevner 2013) for mobile apps to support students in the “transition-in” phase, which are largely missing for apps that focus on this particular stage of the student lifecycle yet. Other institutions may reference these propositions to create beneficial mobile solutions for first-year students who strive to adapt to the university environment. The structure of this paper is as follows: Section 2 provides an overview of mobile apps for higher education, the student lifecycle, and a self-developed mobile learning app to tackle challenges in the “transition-in” phase. Section3 introduces the research model and describes the data collection. Afterwards, the results are presented (Sect.4) and discussed (Sect.5). The paper concludes with a summary and an outlook. 2 Conceptual basics andrelated work 2.1 The use ofmobile apps inhigher education The use of mobile apps in higher education teaching—such as “learning management applications”, “vodcasts and podcasts”, “language learning applications”, “game-based learning applications” or “collaborative learning applications” (Goundar and Kumar 2022)—is discussed lively in the literature (e.g., Beatson etal. 2020; Gupta etal. 2021; Liu and Guo 2017; Ronzhina etal. 2021; Voshaar etal. 2023). In the following, we summarize related work on mobile learning apps in terms of potentials and challenges, technical and organizational issues, associated theories, as well as future developments. This should give readers a better overview of this mature research area. In general, the potential of mobile student apps to support learning effectiveness is well-analyzed for various classroom and course examples (cf. Castek and Beach 2013). For instance, Larkin (2015) evaluates apps to foster the building of mathematical knowledge, while Diliberto-Macaluso and Hughes (2016) show that mobile apps may help psychology students achieve their learning objectives. Hence, apps can help to develop students’ self-regulation and deep thinking abilities or support them in labeling, summarizing, and discovering new knowledge amongst others (cf. Diliberto-Macaluso and Hughes 2016; Larkin 2015). In medical education, 633 1 3 What impacts learning effectiveness ofamobile learning app… the benefits of mobile apps to offer an “enjoyable learning experience” are pointed out by Morris etal. (2016) for a neuroanatomy course. Mohapatra etal. (2015) present an overview of apps that are judged to be beneficial for medical education in general, with a particular focus on their ability to manage information from one or more sources to foster communication and support effective time management. Steel (2012) focuses on language students in particular and discusses the potential of mobile apps for this group, e.g., in terms of vocabulary acquisition. An overview of corresponding apps for language students is given by Gangaiamaran and Pasupathi (2017). In accounting and management, Beatson etal. (2020) and Voshaar etal. (2023) find out that students’ behavioral engagement with the help of mobile apps and gamification elements is positively associated with exam results. Seow and Wong (2016) introduce the so-called “Accounting Challenge (ACE)” app, which helps to keep up students’ motivation in studying accounting through gamification as well. On the contrary, there are challenges of using mobile apps for student education. As Goundar and Kumar (2022) point out, the literature to date has a strong focus on “solution papers”, which introduce fully developed mobile applications that are supposed to improve learning performance. However, a discussion as to what degree singular app functionalities affect students’ cognitive knowledge processing or an explication of the implications for learning theories often come up short (e.g., Damyanov and Tsankov 2018). Along these lines, Mehdipour and Zerehkafi (2013) provide technical as well as social and educational challenges for mobile learning scenarios. These include content security and copyright issues, accessibility and cost barriers for end-users, or the lack of a learning theory for the mobile age in general, to mention just a few (cf. Mehdipour and Zerehkafi 2013). Furthermore, digital technologies may not adequately reproduce the emotional side of interactive learning, so attention should be given to the right balance between digital and human educational interactions (Montiel etal. 2020). A classification scheme for mobile learning challenges according to “management and institutional challenges”, “design challenges”, “technical challenges”, “evaluation challenges”, and “cultural/ social challenges” is introduced by Damyanov and Tsankov (2018). In summary, education institutions need to establish a clear mobile learning policy, offer pedagogical support, consider the hardware capabilities of mobile devices, provide a suitable technical infrastructure, and deal with the cultural differences concerning perceptions and attitudes towards digital technologies (cf. Damyanov and Tsankov 2018). From a technical perspective, the requirements on mobile learning environments, the core functionalities of apps to assure their practicability for educational purposes, and engineering processes for app realization are particularly important. In this context, Zhu etal. (2015) propose a design framework for mobile augmented reality education in healthcare. Further, Clayton and Murphy (2016) analyze mobile apps’ peer-learning and -teaching capabilities for conducting collaborative video design projects. The establishment of a content delivery infrastructure for educational material and suggestions on integrating mobile apps is done by Khaddage etal. (2011). Vázquez-Cano (2014) focuses on the mandatory capabilities of smartphones to support distance learning, while Pechenkina etal. (2017) identify 634 F.Johannsen et al. 1 3 the potential of gamification elements to increase student engagement, retention, and achievement. Finally, Papanikolaou and Mavromoustakos (2006) introduce critical success factors for learning app engineering processes, while Kumar and Mohite (2018) suggest approaches for testing their usability. From an organizational perspective, the factors for successfully adopting digital technologies in higher education institutions are discussed (e.g., Chuchu and Ndoro 2019). It is accentuated that mobile learning initiatives are not limited to purchasing and deploying digital technologies but require a holistic consideration of diverse factors related to people, technology, or pedagogy (Krotov 2015). Thereby, principal factors that may impact user satisfaction, the intention to use, and the actual usage of mobile applications in higher education are examined (e.g., Almaiah and Alismaiel 2019; Chuchu and Ndoro 2019). As an example, Almaiah and Alismaiel (2019) focus on Jordanian universities and analyze two apps—one that provides student services (e.g., a timetable) and another one enabling “open virtual classes”—in light of the abovementioned factors. Thereby, so-called “quality factors” and “individual factors” that have been adapted from Delone and McLean (1992) and Davis (1989) seem to have a positive effect. Besides, also the variable “intention to use” was examined for this specific student group (cf. Almaiah and Al Mulhem 2019). Further, Chuchu and Ndoro (2019) present indicators that the “perceived usefulness” and “perceived ease-of-use” of a mobile learning app are central factors in creating a positive attitude among the target group and in ensuring its acceptance. An overview of critical success factors for mobile learning in organizations is provided by Krotov (2015). This study integrates the perspectives “organization” (e.g., executive involvement), “people” (e.g., personal innovativeness), “pedagogy” (e.g., quality of content provided), and “technology” (e.g., quality of mobile system) to arrive at a list of success factors from a socio-technical perspective (cf. Krotov 2015). Considering the complex process of establishing mobile education technologies in organizations, a pedagogical and educational requirements model was proposed by Sarrab etal. (2018), which supports when searching for a suitable solution to deliver content for mobile learning. Besides, the role of mobile apps in facilitating the inclusion of students with handicaps into the university environment is a subject of investigation. For instance, Ok etal. (2016) introduce an evaluation scheme to purposefully select apps for students with learning disabilities. Moreover, people with developmental disabilities can benefit enormously from mobile apps, which hold true for educational, communication, and leisure purposes, helping them connect with their environment (Stephenson and Limbrick 2015). In addition, Bravou and Drigas (2019) reflect the suitability of mobile devices and apps for students with sensory, physical, and cognitive disabilities. In this respect, a comprehensive literature review on digital technologies for people with learning or cognitive disabilities was performed by Williams and Shekhar (2019). Researchers are also engaged in theory building (cf. Hevner and Chatterjee 2010) to guide the purposeful usage of mobile apps in higher education. However, a widely accepted mobile learning theory has not yet been established (cf. Bernacki et al. 2020; Curum and Khedo 2021). Therefore, Park (2011) refers to the transactional distance theory (cf. Moore 1991), which defines “distance” as a pedagogical concept, and combines this theory with applications of digital 635 1 3 What impacts learning effectiveness ofamobile learning app… technologies to arrive at a “pedagogical framework of mobile learning”. The framework distinguishes between four types of mobile learning depending on whether a (1) high or (2) low transactional distance is given and (3) an individualized or (4) socialized activity is to be solved. Thereby, the transactional distance is defined as the psychological gap between the learner and the instructor, whereas the activity type (i.e., individualized or socialized) assesses the importance of social aspects for a particular learning environment (Park 2011). Another mobile learning framework was introduced by Motiwalla (2007), who proposes to integrate the concepts “mobile connectivity” and “e-learning” for being able to delineate application requirements for mobile learning. Furthermore, a metaframework to guide the establishment of mobile learning frameworks can be found in Liu etal. (2008). This meta-framework is, for instance, referenced by Nordin etal. (2010) as a theoretical base to create a lifelong, continuing learning framework. Future developments of mobile learning apps will essentially emphasize the integration of Artificial Intelligence (AI) with learning environments (cf. Alzahrani etal. 2021; Chong 2019; Diaz etal. 2015; Kabudi etal. 2021). The purpose is to improve students’ learning performance via personalization of learning, facilitate the evaluation of student knowledge, or systematically assess learner requirements (Kabudi etal. 2021). Besides, the use of virtual reality (VR) and augmented reality (AR) to progress students’ learning experiences is intensively discussed (e.g., Fradika and Surjono 2018; Nicolaidou etal. 2021). For instance, Nicolaidou etal. (2021) show that a VR learning environment can positively affect vocabulary acquisition and learners’ experience when studying foreign languages. Further, the readiness of students to adapt VR technology to achieve learning goals is high (Ismail and Hashim 2020). Moreover, the use of chatbots and conversational agents is also rising (Hwang and Chang 2021; Liu etal. 2020; Smutny and Schreiberova 2020). Chatbots can serve as efficient information retrieval tools for specific domains to facilitate learning (cf. Liu et al. 2020). In this context, various platform-specific chatbots for learning (e.g., for the Facebook Messenger platform) at different maturity levels have been developed in recent years (cf. Smutny and Schreiberova 2020). Having said that, chatbots for education are primarily found for language courses as well as the disciplines of “engineering” and “computers”, while topics like “arts” or “mathematics” are less accentuated (Hwang and Chang 2021). Thus, chatbots may not be suitable for all types of courses alike, especially in case students’ hands-on competencies (i.e., arts) or computations and problem-solving skills (i.e., mathematics) are to be promoted. While the effectiveness of chatbots for learning purposes is usually measured by pre-/post-test questionnaires, profound insights on chatbots’ impact on behavioral aspects of the student learning process are still elusive (Hwang and Chang 2021). To conclude this overview, our study aims to analyze factors contributing to the success of a mobile app, which was designed to meet the needs of first-year students in the “transition-in” phase of the student lifecycle. A particular interest is in the ability to positively impact their learning effectiveness, user satisfaction, and intention to reuse the app. To the best of our knowledge, a corresponding study 636 F.Johannsen et al. 1 3 concerning this stage of the student lifecycle has not been done yet. We provide insights that can help establish a mobile learning theory in the “transition-in” phase. 2.2 The student lifecycle andthe“transition‑in” phase Throughout university life, students experience an evolution of their “student identity”, which goes along with a shift of priorities and agendas (Lizzio 2011). As mentioned above, our research focuses on “commencing” students who are just about to become acquainted with the university system and have an increased interest in opportunities for social interaction, active engagement, and early formative feedback (Matheson 2018). Generally, various propositions regarding the development stages of students exist (cf. Burnett 2007; Morgan 2013) that primarily differ in their conception of student transition (Gale and Parker 2014). A widely acknowledged proposition for an integrative framework was introduced by Lizzio (2011), which is depicted in Fig.1 and differentiates between four major stages. Whereas future students (“transition-towards”) are engaged in finding an appropriate study program and university, commencing students (“transition-in”) work on the integration into the student world (Lizzio 2011). In the “transition-through” phase, continuing students work on developing graduate attributes and seek challenges by authentic curricula and assessments (Lizzio 2011; Matheson 2018; Msomi and Bansilal 2019). Finally, the “transition-up, out & back” stage addresses students that are graduating or returning for postgraduate studies to further strengthen their skills for employability (Lizzio 2011; Matheson 2018). Against this background, most premature dropouts are observed in the “transition-in” phase (Chen 2012; Isleib etal. 2019; Neugebauer etal. 2019). An empirical study focused on German higher education institutions (60 universities and universities of applied sciences) identified a lack of social and academic integration as a Graduate & Alumni 7. Focusing on future success 8. Partnering and connuing Connuing Students 5. Working for early success 6. Building on success Commencing Students 3. Comming and preparing 4. Joining and engaging Future Students 1. Aspiring and exploring 2. Clarifying and choosing TransionTowards TransionIn TransionUp, Out & Back TransionThrough Fig. 1 The student lifecycle according to Lizzio (2011) 643 1 3 What impacts learning effectiveness ofamobile learning app… • H1a: System quality will have a positive effect on first-year students’ intention to reuse the app. • H1b: System quality will have a positive effect on first-year students’ satisfaction with the app. 3.1.2 Service quality The construct “service quality” refers to the overall support for users offered by a service provider (Delone and McLean 2003). In terms of “e-learning”, Aparicio etal. (2017) emphasize the importance of the willingness and readiness of the support staff to resolve students’ difficulties at any time because this positively influences the intention to use the system. This positive effect was also confirmed in earlier studies (e.g., Chiu etal. 2016; Huang etal. 2015, among others). Generally, “service quality” may be interpreted from different angles and refer to concepts such as assurance, empathy, or flexibility—just to mention a few (Urbach and Müller 2012). In alignment with the propositions of Aparicio etal. (2017) and Urbach etal. (2010), we see the willingness of the service personnel to provide support upon request immediately, the personal attention offered to students, the timeliness of the service response as well as the competence and knowledge of the service personnel as central factors for the app’s success. Considering this, we claim: • H2a: Service quality will have a positive effect on first-year students’ intention to reuse the app. • H2b: Service quality will have a positive effect on first-year students’ satisfaction with the app. 3.1.3 Information quality Information quality addresses the system output or the information that is produced by a system (Delone and McLean 1992). According to Almarashdeh etal. (2010), information quality is the most crucial factor when determining the success of educational technology systems (Almaiah and Alismaiel 2019). Hence, the positive impact of information quality on the “intention to (re-)use” and “user satisfaction” is confirmed by manifold studies that focus on e-learning or mobile learning systems (e.g., Aparicio etal. 2017; Cidral etal. 2018; Wang etal. 2019b). However, there are also studies in which information quality played a subordinate role for the acceptance of a system (cf. Chiu etal. 2016). Once more, the construct “information quality” can be reflected from various perspectives such as data accuracy, adequacy, or completeness (cf. Klier 2008; Urbach and Müller 2012). For our app, we determine information quality based on the reliability and understandability of the information provided and its usefulness and relevance for the target group (cf. Aparicio etal. 2017; Urbach etal. 2010). • H3a: Information quality will have a positive effect on first-year students’ intention to reuse the app. 644 F.Johannsen et al. 1 3 • H3b: Information quality will have a positive effect on first-year students’ satisfaction with the app. 3.1.4 Perceived enjoyment Davis etal. (1992) summarize enjoyment in the information systems context as “the extent to which the activity of using the computer is perceived to be enjoyable in its own right, apart from any performance consequences that may be anticipated” (p.1113). Against this background, the construct of “perceived enjoyment” is increasingly getting attention when it comes to the measurement of IS success (cf. Kim etal. 2007; Wang etal. 2019b). Therefore, it is suggested that technology adoption is more likely in cases where users experience immediate pleasure or joy through mere use (Kim etal. 2007). Since the positive influence of perceived enjoyment on users’ attitudes is well examined in the mobile services and mobile commerce context (cf. Tseng and Lo 2011; Wang and Li 2012; Wang etal. 2019b), it is increasingly discussed in terms of e-learning technologies, as well (cf. Balog and Pribeanu 2010; Hussein 2018; Khalid 2014). Hence, we also assume a positive effect on “user satisfaction” and “intention to (re-)use”. In this regard, gamification elements may purposefully impact the hedonic motivation to engage with mobile apps and, as a positive side-effect, impact users’ perceived enjoyment (cf. Beatson etal. 2020; Pechenkina etal. 2017; Wang etal. 2019b). Generally, gamification is seen as a means to overcome a lack of motivation among students to deal with study-related content (cf. Kiryakova etal. 2014). Thereby, principles such as “freedom to fail”, “rapid feedback”, “progression”, or “storytelling” play a decisive role for the successful application of gamification elements in learning environments (Stott and Neustaedter 2013). Hence, specific mechanisms that are traditionally used in game design (e.g., Laine and Lindberg 2020) to increase user engagement and hedonic motivation have found their way into modern pedagogy, although their purposeful selection should be made in regards to the target group (Stott and Neustaedter 2013). For the design of mobile education apps, corresponding principles need to be purposefully transferred to corresponding design requirements (cf. Herrington etal. 2009; Laine and Lindberg 2020). Section2.3 presents the design requirements of our mobile app (see also Fig.3), whereas these are taken up in Sect.5.2 once again and reflected against the findings of the study. In summary, we determine perceived enjoyment based on the fun and enjoyment experienced by app users (cf. Kim etal. 2007; Wang etal. 2019b) and the abilities of entertaining and playful features to enhance users’ learning experience and structure their learning efforts (cf. Suki and Suki 2007). • H4a: Perceived enjoyment will have a positive effect on first-year students’ intention to reuse the app. • H4b: Perceived enjoyment will have a positive effect on first-year students’ satisfaction with the app. 645 1 3 What impacts learning effectiveness ofamobile learning app… 3.1.5 Intention toreuse, perceived user satisfaction, andlearning effectiveness “Intention to use” is specified as users’ intent to perform a defined behavior (Davis 1989). The construct is acknowledged to be strongly associated with the acceptance of an information system (Almaiah and Alismaiel 2019) and it largely depends on the users’ attitude towards the system (Agrebi and Jallais 2015). However, there is a distinct difference between “intention to use” and actual “use”, because the former represents an attitude, whereas the latter concept describes a concrete behavior (Delone and McLean 2003). To resolve the closed-loop relationships between user satisfaction, intention to use, and use in the original IS success model (Wang etal. 2019b), “intention to reuse” is commonly proposed as a worthwhile measure (Delone and McLean 2003; Wang 2008). In line with the proposition of Wang (2008), “intention to reuse” thus represents the favorable student attitude towards our app in this study. In addition, “perceived user satisfaction” helps to measure the successful interaction of users with the IS (Delone and McLean 1992). Generally, user satisfaction can be interpreted as “the extent to which users believe the information system available to them meets their information requirements” (Ives etal. 1983, p.785). Thereby, perceived user satisfaction leads to an increasing “intention to reuse” in the post-use situation (Wang 2008). Either way, the major purpose of mobile learning technologies is to increase knowledge acquisition (cf. Wang etal. 2019b) and, hence, improve learning outcomes (cf. Noesgaard and Ørngreen 2015). Generally, the beneficial individual or organizational impact of IS, which is supposed to be measured by the IS success model may occur in many ways (e.g., awareness/recall, competitive advantage, etc.) (cf. Delone and McLean 2003; Urbach and Müller 2012). Considering this, there is a lively discussion on how to operationalize the individual benefits of using e-learning technologies that cumulate in better knowledge acquisition and learning outcomes in the end (e.g., Chiu etal. 2016; Noesgaard and Ørngreen 2015; Wang etal. 2019b; Zhang etal. 2006). In that context, “learning effectiveness” (cf. Noesgaard and Ørngreen 2015) has become a commonly accepted measure to assess the success of technology-assisted learning for individuals (Smith etal. 2006; Wang etal. 2019b; Zhang etal. 2006). The variable builds on the recognition that effective learning asks for learners’ engagement, motivation, awareness, and an individualized learning process, which can be enabled by offering access to content randomly or repeatedly on demand for instance (Zhang etal. 2006). This, in turn, promotes learning skills (e.g., enhanced problem-solving or critical thinking abilities; Zhang etal. 2006) and leads to an improved understanding of study-related content, which can be recollected any time (cf. Chiu etal. 2016; Gable etal. 2008; Wang etal. 2019b). Hence, improved knowledge acquisition and learning outcomes emerge from a general point of view. To properly address these considerations, the literature proposes to ask for students’ perceptions of learning performance, efficiency, motivation (cf. Liaw 2008), awareness, and recollection of study-related information (Gable etal. 2008) along with their understanding of the course content (cf. Chiu etal. 2016). Accordingly, 646 F.Johannsen et al. 1 3 these aspects determine the items of our questionnaire to assess the variable “learning effectiveness” (see Appendix). As evident from the above explanations, a rather broad spectrum of factors (e.g., awareness, motivation, etc.) is required to describe “learning effectiveness” comprehensively. Nevertheless, the variable allows students to carefully reflect on the achieved individual (net) benefits (cf. Delone and McLean 2003) when using a mobile learning app to cope with the challenges of the “transition-in” phase. Therefore, the variable is used hereafter to measure students’ (net) benefits since we believe that other variables that have been proposed in the context of the IS success model (e.g., recall, job simplification, etc.) (cf. Urbach and Müller 2012) would not comply with the multidimensionality of first-year students’ learning success in the “transition-in” phase and may not be adequately transferred to our context. We formulate the following hypotheses: • H5: The perceived user satisfaction will have a positive effect on first-year students’ intention to reuse the app. • H6: The intention to reuse will have a positive effect on first-year students’ learning effectiveness. • H7: The perceived user satisfaction will have a positive effect on first-year students’ learning effectiveness. Figure4 summarizes the proposed research model, variables, and hypotheses. 3.2 Design ofthequestionnaire We developed a questionnaire based on the abovementioned established and validated scales from previous studies and modified them accordingly for the mobile learning context to test our hypotheses. As previously described, the constructs of Fig. 4 Proposed research model 647 1 3 What impacts learning effectiveness ofamobile learning app… “system quality”, “service quality”, and “information quality” were adapted from Aparicio etal. (2016) and Urbach etal. (2010) and are all measured by four underlying items. “Perceived enjoyment” consists of five items, three being adapted from Kim etal. (2007) and Wang etal. (2019b), and two used by Suki and Suki (2007). Three items were adapted from Wang etal. (2019b) and Wang (2008) and are complemented by one item each from Chiu etal. (2016) and Sun etal. (2008) to measure the construct “intention to reuse”. “Perceived user satisfaction” was measured by four underlying items used by Liaw (2008). Finally, we adopted three items from a previous study of e-learning effectiveness from Liaw (2008) and added one item each from Chiu etal. (2016) and Gable etal. (2008) in order to measure “learning effectiveness”. Initially, we developed the survey in English, in accordance with prior research, and then translated it to German through a professional translation service in order to ensure a low-threshold participation opportunity and, thus, a high number of participants. Subsequently, a different professional translator translated it back into English to ensure conversion correspondence (Brislin 1970). As previously described, the constructs were unanimously measured with four or five items each. All items were Table 2 Descriptive statistics (N = 113) Characteristic Items Frequency Percentage Gender Male 50 44.25 Female 63 55.75 Age (in years) < 20 22 19.47 20–25 85 75.22 26–30 5 4.42 > 30 1 0.88 Study course Business studies 80 70.8 Economics 17 15.04 Engineering and management 12 10.61 Information systems and management 4 3.54 Course attendance (Almost) always (> 90%) 18 15.93 Often (90–60%) 14 12.39 Sometimes (60–40%) 10 8.85 Rarely (40–10%) 28 24.78 (Almost) never (< 10%) 43 38.05 Exam performance Very good (1.0–1.3) 8 7.08 Good (1.7–2.3) 12 10.62 Satisfactory (2.7–3.3) 14 12.39 Sufficient (3.7–4.0) 14 12.39 Insufficient (5.0) 64 56.64 Missing 1 0.88 648 F.Johannsen et al. 1 3 assessed on a seven-point Likert scale (from 1 = “strongly disagree” to 7 = “strongly agree”). Table4 in the Appendix presents the final survey consisting of the mentioned items used in our research model. 3.3 Data collection andsample selection The mobile learning app was initially implemented in a mandatory introductory accounting course in the Winter semester 2020/21. Because of the COVID-19 pandemic, the social distancing requirements, and the sudden closures of university campuses, all lectures were held digitally in an asynchronous format via screencasts. Complementing the lectures, students could participate in synchronous, live tutorials and submit exercise sheets. Additionally, preparatory courses were also offered synchronously via Zoom. For our study, we invited all students who used the mobile learning app at some point in the Winter semester 2020/21 to participate in the online survey conducted during the final week of teaching (i.e., before the final exam) and administered on the university’s LMS. We did not offer any additional (e.g., monetary or extra course credit) incentives for participating, and the students were informed of the research purpose and their voluntary participation in the study. Even if they took part in the survey, they had the possibility to refuse to answer any question. Subsequently, one member of the research team, who was not involved with the empirical analysis, merged and pseudonymized the data from students’ questionnaires with data from several other sources, including students’ demographics being collected through another survey in the first week of the semester, students’ course attendance during the semester, and the academic performance data. More specifically, the students’ attendance at tutorials and workshops has been manually evaluated via Zoom participation protocols. Finally, the central examination office provided the student’s exam performance. In our analysis, we only use the final pseudonymized dataset, which does not allow identification of individual students. Table 3 Summary of the research results Hypothesis Relationship Β f2Result H1a System Quality → Intention to Reuse 0.137 0.029 Not supported H1b System Quality → Perceived User Satisfaction 0.317 0.174 Supported H2a Service Quality → Intention to Reuse −0.011 0.000 Not supported H2b Service Quality → Perceived User Satisfaction 0.050 0.007 Not supported H3a Information Quality → Intention to Reuse −0.023 0.001 Not supported H3b Information Quality → Perceived User Satisfaction 0.308 0.152 Supported H4a Perceived Enjoyment → Intention to Reuse 0.407 0.286 Supported H4b Perceived Enjoyment → Perceived User Satisfaction 0.346 0.247 Supported H5 Perceived User Satisfaction → Intention to Reuse 0.445 0.207 Supported H6 Intention to Reuse → Learning Effectiveness 0.426 0.200 Supported H7 Perceived User Satisfaction → Learning Effectiveness 0.448 0.222 Supported 649 1 3 What impacts learning effectiveness ofamobile learning app… The students were asked to answer the questionnaire according to their user experience throughout the semester. Thereby and due to the requirements of the IS success model, we were ex-ante limited to the population of 367students who used the app during the semester and participated in the final exam to draw our sample. Our initial sample consists of 131 students who participated in our survey regarding their user experience. Out of the initial sample, we exclude 10 observations due to missing values in their survey responses and 1 without any variation in the responses. Hence, we received 120 usable responses, bringing our usable response rate to 91.60%. Further, we exclude 7 students because of missing values in their demographics, resulting in a final sample of 113 students who participated in the final exam1 of the mandatory introductory accounting course and used the mobile learning app for learning purposes. Accordingly, our sample represents 30.79% of the underlying population that could be used for a study of this type.2 The final sample comprises 63 female and 50 male students, with the overwhelming majority (94.69%) being 25years old and younger. Table2 presents the summarized descriptive statistics for the final sample of 113 students.3 3.4 PLS‑SEM approach Our research model was evaluated using PLS-SEM as the most favorable method to validate multistage models with complex relationships, interdependencies, constructs, and indicators (Hair etal. 2011; Sarstedt etal. 2016, 2021). We thereby followed recent recommendations as suggested by Hair etal. (2019) and Sarstedt etal. (2021). The minimum sample size was ascertained by multiplying the total number of constructs by ten (Hair etal. 2011; Marcoulides etal. 2009) and met with our 113 participants. The construct indicators in our model represent reflective measurements caused by latent variables (Churchill Jr 1979). We used SmartPLS (v. 3.3.2; Ringle etal. 2015) and applied a path weighting scheme with 300 iterations with 10 − 7 as the stop criterion. Bootstrapping was done via two-tailed bias-corrected and accelerated (BCa) confidence interval method with 4,999 subsamples followed by blindfolding with an omission distance of 7 (Henseler etal. 2016). 1 Exam performance is coded according to the German grade scale from 1.0 (best) through 5.0 (fail). 2 In order to ensure the representativeness of our drawn sample, we conducted two-tailed t-tests for differences in means between the group of students included in the sample and the underlying population of app users. The results indicate that the characteristics are essentially similarly distributed. The only documented significant differences are in the share of students in Business Studies and Engineering and Management, in the share of (almost) always and (almost) never attending students, as well as in the share of students with a very good exam performance. However, the significance level is only slightly pronounced (p < 0.1) for most differences. We present a comparison between the sample and the underlying population with the conducted t-tests in Table9 (Appendix). 3 The high proportion of never and rarely attending students is likely due to the conitions of COVID-19 induced online teaching. In order not to disadvantage any students, we provided the recordings of the zoom sessions of tutorials and workshops afterwards. However, we were not able to assess which students accessed the recordings. Moreover, the distribution of the exam performance in our sample, which documents a high level of insufficient performance and thus failure, is in line with both the exam performance of the whole population of the course and the distribution of the exam performance in previous cohorts. 650 F.Johannsen et al. 1 3 4 Results Initially, we ensured that the indicator loadings are above the threshold of 0.708. Slightly weaker indicators were only kept if they contribute to content validity and are relevant on the grounds of measurement theory (Hair etal. 2011). Internal consistency was given with Cronbach’s alpha, composite reliability, and Rho_A with values greater than 0.7 (Diamantopoulos etal. 2012; Dijkstra and Henseler 2015; Drolet and Morrison 2001; Hair et al. 2019). Convergent validity was measured via average variance extracted (AVE) with values greater than 0.5 (i.e., at least half the variance of the construct’s items is explained; Fornell and Larcker 1981; Hair etal. 2019; Henseler etal. 2016). Table5 in the Appendix presents the detailed results of the reliability and validity measurements. The Fornell-Larcker criterion (Fornell and Larcker 1981) was examined to assess the discriminant validity, which can be assumed as the square root of AVE is greater than any inter-factor correlation (see Table7 in the Appendix; Fornell and Larcker 1981). Common method bias (CMB) was examined via Harman’s one-factor test for a full collinearity assessment approach. The values for the variance inflation factors (VIF) were below the threshold of 3.30 (see Table8 in the Appendix; Kock 2015). Finally, we analyzed crossloadings to rule out misassigned indicators (Henseler etal. 2016). Statistical significance was provided with p-values lower than or equal to 0.05 and t-statistics greater than 1.96 (Greenland etal. 2016). Cohen’s f2 indicates statistical relevance, where effect sizes are considered small, 0.02 < f2 ≤ 0.15; medium, 0.15 < f2 ≤ 0.35; or large, f2 > 0.35 (Cohen 1988). The exploratory power of the model was measured using R2, which ranges between 0 and 1 where higher values indicate greater explanatory power (Hair etal. 2011; Reinartz etal. 2009). The Stone-Geisser Q2 measure was calculated to support explanatory significance, that is, explaining how well the data could be (artificially) reproduced by the research model (Geisser 1974; Stone 1974). We achieved predictive accuracy with results above 0 where values greater than 0, 0.25, and 0.5 are considered as small, medium, and large effect sizes, respectively (Hair etal. 2019). Goodness of fit (GoF) is assessed using AVE and the adjusted R2. Our value of 0.78 is above the threshold of 0.36 (Wetzels etal. 2009), which indicates a valid model. We finally controlled our model using the participants’ age, grade, courses of study, and current semester, which we unanimously found not to be significant for the research question at hand. The final results of our evaluation are presented in Fig.5, and Table3 provides an overview of the results for the hypotheses. 5 Discussion andbenefits forresearch andpractice 5.1 Factors thatcontribute tothesuccess ofamobile app tosupport first‑year students inthe“transition‑in” phase inhigher education As mentioned, our app’s key user group are students who have just entered the university system. This focus on the “transition-in” phase of the student lifecycle differentiates our study from prior literature in this field (e.g., Almaiah and Al Mulhem 651 1 3 What impacts learning effectiveness ofamobile learning app… 2019; Cidral etal. 2018; Wang etal. 2019a). Furthermore, we focus on students at a German university and, hence, in a typical Continental European higher education setting. The main differences between the Continental European and the AngloSaxon model of higher education arise primarily through universities’ funding and tuition fees. The Continental European model is characterized by state sponsorship of universities and free or very low tuition (e.g., Jongbloed 2004). As a result of this greatly reduced financial burden, students from disadvantaged backgrounds can also participate in higher education, and, therefore, the student population might be more (economically) diverse (Lenzen 2015). At the same time, limited state funding results in large lectures and a high student-lecturer ratio, which probably disadvantages students in need of greater guidance and, thus, increases the need for technological learning solutions. At first, the research indicates a positive effect of system quality on perceived user satisfaction (H1b), a finding in line with prior results (e.g., Almaiah and Alismaiel 2019; Aparicio etal. 2017). As expected, factors like ease of use, easy navigation, structuredness, and the ability to efficiently retrieve relevant information help to increase user satisfaction. However, this effect could not be observed regarding the impact of system quality on the intention to reuse the app (H1a). A possible explanation for this finding, which is in line with prior literature (Aparicio etal. 2017; Chiu etal. 2016) is that students tend to use a system independently from its perceived quality, in case a university has committed to this particular system. Transferred to our context, the developed mobile app is the only solution of its kind. Fig. 5 Research model with results (N = 113). *p ≤ 0.05; **p ≤ 0.01; ***p ≤ 0.001; n.s. = not significant 652 F.Johannsen et al. 1 3 As such, first-year students obviously use it—regardless of their perception of system quality—as a means to (potentially) increase their learning performance. Nevertheless, further analysis of further factors seems promising to better understand the relationship between system quality and the intention to reuse the app (cf. Almaiah and Al Mulhem 2019). Second, contrary to the proposed expectation in hypotheses H2a and H2b, service quality neither significantly impacts the intention to reuse nor perceived user satisfaction (see Table3 or Fig.5). Thus, service quality seems to be a minor issue in evaluating the mobile app’s benefits in terms of learning effectiveness. This is also reasonable as we noticed in the field that app users rarely contact the administrative or teaching staff for help with respect to mobile app usage. Likewise, previous research finds evidence that service quality is relatively less important in the context of knowledge-orientated information system success (e.g., Wang etal. 2019b; Wu and Wang 2006). Third, information quality positively impacts the perceived user satisfaction in our study (H3b). Accordingly, the reliability, understandability, and relevance of the information provided by the app for students in the “transition-in” phase are greatly appreciated by our target group. However, no significant impact on the intention to reuse could be observed (H3a). A similar finding is presented by Chiu etal. (2016). As a potential explanation, students may believe that using the app is decisive for a successful start of studies, and therefore, they do not draw the presented information into question. This assumption aligns with the observation that students’ critical thinking abilities are only starting to take shape in their first year at university and significantly increase in the subsequent semesters (cf. Ralston and Bays 2015; Wallace and Jefferson 2015). Hence, analyzing students from more advanced semesters might lead to different results. Fourth, the effects of perceived enjoyment on both the intention to reuse (0.407) and user satisfaction (0.346) are significantly positive and greater than the impact of system quality, service quality, and information quality. These findings support our hypotheses H4a and H4b. Therefore, since perceived enjoyment is the only construct that influences both intention to reuse as well as perceived user satisfaction, it can be stated that the enjoyment and joy-related aspects are of utmost importance to promote learning effectiveness in our research setting. This result is quite striking and provides further evidence on the value of gamification in higher education, a field of research which is still quite in its infancy. Fifth, we find support that perceived user satisfaction mediates the effects of system quality, information quality, and perceived enjoyment on the intention to reuse (H5). This is also reasonable since it might be more likely that students intend to reuse the app when their level of satisfaction is high. Additionally, this mediating effect might be why the service quality (H2a) and the information quality (H3a) are not directly associated with the intention to reuse. We also find evidence that intention to reuse significantly positively affects learning effectiveness (H6). In other words, the greater the likelihood to reuse the app, the greater the app’s positive effect on students’ learning effectiveness. 659 1 3 What impacts learning effectiveness ofamobile learning app… Presumably, the impact of our learning app would have been even stronger in the prepandemic period than in the online teaching period. This is because the app functionalities purposefully complement the attendance of face-to-face lectures. In a pure online semester, the “value” provided by the app, which is still positive for users, may be less than in the era of traditional teaching. Though, this proposition needs to be explored in more detail in future studies. Besides providing several benefits for both research and practice, this study has some noteworthy limitations. First, the discussed findings and the implications drawn are limited to a specific context of an app adapted to first-year students’ particular needs in a mandatory introductory accounting course at the Universityof Bremen (Germany). However, in terms of the topics, structure, and practicalities, the course setting is similar to most foundational undergraduate courses in Continental European study programs in business and economics. Second, since we rely on self-reported data to examine the mobile learning app’s success, this may introduce the risk of common method and response bias. Having said that, as we assured participants of the confidentiality of their responses and offered no monetary rewards or other incentives for participation, we assume that the risk of systematically biased responses is minimal. Third, we employ a cross-sectional approach, which causes possible feedback links from learning effectiveness to perceived user satisfaction and the intention to reuse could not be considered in this study. Finally, our research model largely builds on the initial elements of the IS success model. This was done to receive design principles that are based on widely accepted elements for success, which may positively affect the general acceptance of such principles in the DSR community. Moreover, to the best of our knowledge, corresponding studies based on the IS success model have not been done for “type2 apps” according to the “pedagogical framework of mobile learning” of Park (2011). In future research, a longitudinal design to take these possible feedback links into account and, thus, enhance the understanding of the causality and interrelationships of the research elements in the context of mobile learning app success will be performed. Going forward, the app will be continuously developed further and is planned to be fully integrated into the entire undergraduate curriculum at our faculty. In this course, the integration of AI-based conversational agents to further improve students’ learning experience will be investigated more closely. Particularly their impact on students’ learning behavior is to be considered since the literature on accounting education as well as information systems lacks theoretical foundations in this respect. Moreover, the research model will be extended by additional elements in the next step to identify additional influencing factors that may positively affect student performance (e.g., base competencies or grit; cf. Aparicio etal. 2017; Zehetmeier etal. 2014). Appendix See Tables 4, 5, 6, 7, 8, and 9. 660 F.Johannsen et al. 1 3 Table 4 Measurement items Constructs Items Description References System quality SQ1 The app is easy to navigate Aparicio etal. (2017) / Urbach etal. (2010) SQ2 The app allows me to find easily the information I am looking for SQ3 The app is well structured SQ4 The app is easy to use Service quality SV1 The responsible service personnel is always highly willing to help whenever I need support with the app Aparicio etal. (2017) / Urbach etal. (2010) SV2 The responsible service personnel provides personal attention when I experience problems with the app SV3 The responsible service personnel provides services related to the app at the promised time SV4 The responsible service personnel has sufficient knowledge to answer my questions in respect of the app Information quality IQ1 The information provided by the app is useful Aparicio etal. (2017) / Urbach etal. (2010) IQ2 The information provided by the app is understandable IQ3 The information provided by the app is interesting IQ4 The information provided by the app is reliable Perceived enjoyment PE1 I have fun interacting with the app Wang etal. (2019a, b) / Kim etal. (2007) PE2 Using the app provides me with a lot of enjoyment PE3 I enjoy using the app PE4 Entertaining features of the app enhance my learning experience Suki/Suki (2007) PE5 Playful features of the app help me to structure my learning efforts Intention to reuse RU1 Assuming that I have access to the app, I intend to reuse it Wang etal. (2019a, b) / Wang (2008) RU2 I will reuse the app in the future RU3 I will frequently use the app in the future RU4 I will recommend that fellow students use the app Chiu etal. (2016) RU5 If I had an opportunity to take another course via this app, I would gladly do so Sun etal. (2008) 661 1 3 What impacts learning effectiveness ofamobile learning app… Table 4 (continued) Constructs Items Description References Perceived user satisfaction US1 I am satisfied with using the app as a learning assisted tool Liaw (2008) US2 I am satisfied with using the app functions US3 I am satisfied with learning contents US4 I am satisfied with multimedia instruction Learning effectiveness LE1 I believe the app can assist learning efficiency Liaw (2008) LE2 I believe the app can assist learning performance LE3 I believe the app can assist learning motivation LE4 The app enhances my awareness and recollection of study-related information Gable etal. (2008) LE5 The app helped me understand the contents of the course “Accounting & Accounts” Chiu etal. (2016) 662 F.Johannsen et al. 1 3 Table5 Reliability and validity measurements CA = Cronbach’s Alpha, RA = Rho_A, CR = Composite Reliability, AVE = Average Variance Extracted CA RA CR AVE Age 1.000 1.000 1.000 1.000 Course 1.000 1.000 1.000 1.000 Perceived enjoyment 0.917 0.925 0.942 0.804 Grade 1.000 1.000 1.000 1.000 Information quality 0.840 0.840 0.904 0.758 Learning effectiveness 0.912 0.916 0.935 0.741 Intention to reuse 0.930 0.932 0.950 0.827 Perceived user Satisfaction 0.767 0.793 0.865 0.682 Semester 1.000 1.000 1.000 1.000 Service quality 0.941 0.947 0.957 0.849 System quality 0.840 0.853 0.893 0.676 Table 6 Major effects measurements O = Original Sample, M = Sample Mean, SD = Standard Deviation, t = t-Statistic, p = p Value O M SD t p Age → Learning effectiveness 0.010 0.012 0.052 0.202 0.840 Age → Intention to reuse −0.015 −0.011 0.055 0.276 0.783 Age → Perceived user satisfaction −0.025 −0.021 0.052 0.478 0.633 Course → Learning effectiveness 0.112 0.113 0.052 2.151 0.132 Course → Intention to reuse −0.033 −0.030 0.045 0.728 0.466 Course → Perceived user satisfaction −0.087 −0.094 0.053 1.658 0.097 Perceived Enjoyment → Intention to reuse 0.407 0.412 0.086 4.757 0.000 Perceived Enjoyment → Perceiveduser satisfaction 0.346 0.345 0.086 4.025 0.000 Grade → Learning effectiveness −0.011 −0.011 0.053 0.212 0.832 Grade → Intention to reuse −0.044 −0.048 0.051 0.873 0.383 Grade → Perceived user satisfaction −0.045 −0.044 0.051 0.870 0.384 Information Quality → Intention to reuse −0.023 −0.024 0.096 0.235 0.814 Information Quality → Perceived user satisfaction 0.308 0.304 0.065 4.743 0.000 Intention to Reuse → Learning effectiveness 0.426 0.418 0.096 4.454 0.000 Perceived User Satisfaction → Learning effectiveness 0.448 0.453 0.100 4.462 0.000 Semester → Learning effectiveness 0.445 0.423 0.106 4.180 0.000 Semester → Intention to reuse −0.002 0.002 0.055 0.038 0.969 Semester → Perceived user satisfaction −0.005 −0.011 0.064 0.072 0.943 Service Quality → Intention to reuse 0.034 0.035 0.067 0.500 0.617 Service Quality → Perceived User Satisfaction −0.011 −0.008 0.072 0.155 0.877 System Quality → Intention to reuse 0.050 0.047 0.067 0.740 0.459 System Quality → Perceived user satisfaction 0.137 0.153 0.093 1.472 0.141 663 1 3 What impacts learning effectiveness ofamobile learning app… Table 7 Fornell–Larcker criterion (1) Perceived Enjoyment, (2) Information Quality, (3) Learning Effectiveness, (4) Intention to Reuse, (5) Perceived User Satisfaction, (6) Service Quality, (7) System Quality (1) (2) (3) (4) (5) (6) (7) (1) 0.897 (2) 0.667 0.871 (3) 0.778 0.685 0.861 (4) 0.801 0.680 0.806 0.909 (5) 0.756 0.773 0.809 0.834 0.826 (6) 0.458 0.439 0.532 0.442 0.489 0.922 (7) 0.621 0.715 0.697 0.708 0.767 0.481 0.822 Table 8 Inner variance inflations Learning effectiveness Intention to reuse Perceived user satisfaction Age 1.192 1.275 1.273 Course 1.254 1.319 1.287 Perceived enjoyment 2.549 2.044 Grade 1.040 1.091 1.082 Information quality 3.021 2.622 Learning effectiveness Intention to reuse 3.295 Perceived user Satisfaction 3.281 3.207 Semester 1.369 1.394 1.389 Service quality 1.450 1.440 System quality 2.862 2.439 664 F.Johannsen et al. 1 3 Table 9 Differences between sample and underlying population of app users Table9 shows the composition of the sample on which this study is based and the composition of the entire app user population as well as thedifferences between these both with the corresponding results of the two-sided t-tests for mean differences * and ** represent significancelevels of 0.10 [or 10%] and 0.05 [or 5%], respectively Characteristic Items Sample App User Population Difference Frequency Percentage Frequency Percentage Gender Male 50 44.25 193 52.59 −8.34 Female 63 55.75 174 47.41 8.34 Age (in years) < 20 22 19.47 80 21.80 −2.33 20–25 85 75.22 271 73.84 1.38 26–30 5 4.42 12 3.27 1.15 > 30 1 0.88 2 0.54 0.34 Study Course Business Studies 80 70.80 217 59.13 11.67** Economics 17 15.04 68 18.53 −3.49 Engineering and Management 12 10.61 62 16.89 −6.28* Information Systems and Management 4 3.54 20 5.45 −1.91 Course Attendance (Almost) always (> 90%) 18 15.93 31 8.45 7.48* Often (90–60%) 14 12.39 33 8.99 3.4 Sometimes (60–40%) 10 8.85 40 10.90 −2.05 Rarely (40–10%) 28 24.78 89 24.25 0.53 (Almost) never (< 10%) 43 38.05 174 47.41 −9.36* Exam Performance Very good (1.0–1.3) 8 7.08 10 2.72 4.36* Good (1.7–2.3) 12 10.62 29 7.90 2.72 Satisfactory (2.7–3.3) 14 12.39 45 12.26 0.13 Sufficient (3.7–4.0) 14 12.39 48 13.08 −0.69 Insufficient (5.0) 64 56.64 235 64.03 −7.39 Missing 1 0.88 – – 0.88 665 1 3 What impacts learning effectiveness ofamobile learning app… Funding Open Access funding enabled and organized by Projekt DEAL. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. References Agrebi S, Jallais J (2015) Explain the Intention to use smartphones for mobile shopping. J Retail Consum Serv 22:16–23 Ahlers E, Quispe Villalobos V (2022) Fachkräftemangel in Deutschland: Befunde der WSI-Betriebsund Personalrätebefragung. WSI Report Ajibade P (2018) Technology acceptance model limitations and criticisms: exploring the practical applications and use in technology-related studies, mixed-method, and qualitative researches. Library Philosophy and Practice 9 Al-Bashayreh M, Almajali D, Altamimi A, Masa’deh RE, Al-Okaily M (2022) An empirical investigation of reasons influencing student acceptance and rejection of mobile learning apps usage. Sustainability 14(7):4325 Al-Sharhan S, Al-Hunaiyyan A, Alhajri R, Al-Huwail N (2020) Utilization of Learning Management System (LMS) among instructors and students. In: Zakaria Z, Ahmad R (eds) Advances in electronics engineering, Singapore, 2020//2020. Springer Singapore, pp 15–23 Alberti MG, Bazán AM, González-Rodrigo B, Feijoo JM (2019) Gamification and question-driven learning aided with immediate response systems. Some experiences from Civil Engineering Students. In: ICERI2019 Proceedings, 2019. IATED Ally M (2005) Using learning theories to design instruction for mobile learning devices. Mobile Learning Anytime Everywhere, pp 5–8 Almaiah MA, Al Mulhem A (2019) Analysis of the essential factors affecting of intention to use of mobile learning applications: a comparison between universities adopters and non-adopters. Educ Inf Technol 24(2):1433–1468 Almaiah MA, Alismaiel OA (2019) Examination of factors influencing the use of mobile learning system: an empirical study. Educ Inf Technol 24(1):885–909 Almaiah MA, Hajjej F, Lutfi A, Al-Khasawneh A, Alkhdour T, Almomani O, Shehab R (2022) A conceptual framework for determining quality requirements for mobile learning applications using Delphi Method. Electronics 11(5):788 Almarashdeh IA, Sahari N, Zin NAM, Alsmadi M (2010) The success of learning management system among distance learners in Malaysian Universities. J Theor Appl Inf Technol 21(2) Alzahrani AS, Gay V, Alturki R, AlGhamdi MJ (2021) Towards understanding the usability attributes of AI-enabled ehealth mobile applications. J Healthc Eng Aparicio M, Bacao F, Oliveira T (2016) Cultural Impacts on E-Learning systems’ success. Internet Higher Educ 31:58–70. https:// doi. org/ 10. 1016/j. iheduc. 2016. 06. 003 Aparicio M, Bacao F, Oliveira T (2017) Grit in the path to E-learning success. Comput Hum Behav 66:388–399 Baars G, Arnold I (2014) Early identification and characterization of students who drop out in the first year at university. J Coll Stud Retent: Res Theory Pract 16(1):95–109 Baird J, Gordon G (2009) Beyond the Rhetoric: a framework for evaluating improvements to the student experience. Tert Educ Manag 15(3):193–207 Balog A, Pribeanu C (2010) The role of perceived enjoyment in the students’ acceptance of an augmented reality teaching platform: a structural equation modelling approach. Stud Inform Control 19(3):319–330 Baskerville R, Baiyere A, Gregor S, Hevner A, Rossi M (2018) Design science research contributions: finding a balance between artifact and theory. J Assoc Inf Syst 19(5):358–376 666 F.Johannsen et al. 1 3 Beatson N, Gabriel C-A, Howell A, Scott S, van der Meer J, Wood LC (2020) Just Opt In: how choosing to engage with technology impacts business students’ academic performance. J Account Educ 50. https:// doi. org/ 10. 1016/j. jacce du. 2019. 100641 Behr A, Giese M, Teguim KHD, Theune K (2020) Dropping out from higher education in Germany an empirical evaluation of determinants for bachelor students. Open Educ Stud 2(1):126–148 Behr A, Giese M, Teguim Kamdjou HD, Theune K (2021) Motives for dropping out from higher education: an analysis of bachelor’s degree students in Germany. Eur J Educ 56(2):325–343 Bernacki ML, Greene JA, Crompton H (2020) Mobile technology, learning, and achievement: advances in understanding and measuring the role of mobile technology in education. Contemporary Educational Psychology 60 Blüthmann I, Thiel F, Wolfgramm C (2011) Abbruchtendenzen in den Bachelorstudiengängen. Individuelle Schwierigkeiten oder mangelhafte Studienbedingungen? Die Hochschule: J für Wissenschaft und Bildung 20(1):110–126 Bravou V, Drigas A (2019) A contemporary view on online and web tools for students with sensory & learning disabilities. Int J Online Biomed Eng 15(12):97–105. https:// doi. org/ 10. 3991/ ijoe. v15i12. 10833 Brislin RW (1970) Back-translation for cross-cultural research. J Cross Cult Psychol 1(3):185–216. https:// doi. org/ 10. 1177/ 13591 04570 00100 301 Burnett L (2007) Juggling first year student experiences and institutional changes: an australian experience. In: The 20th international conference on first year experience Castek J, Beach R (2013) Using apps to support disciplinary literacy and science learning. J Adolesc Health 56(7):554–564. https:// doi. org/ 10. 1002/ JAAL. 180 Chen R (2012) Institutional characteristics and college student dropout risks: a multilevel event history analysis. Res High Educ 53(5):487–505 Chiu P-S, Chao I-C, Kao C-C, Pu Y-H, Huang Y-M (2016) Implementation and evaluation of mobile E-Books in a cloud bookcase using the information system success model. Library Hi Tech 34(2):207–223 Cho W, Fan M, Yoo B, Zhang H (2021) Special issue on digital transformation: challenges and opportunities. Inf Syst e-Bus Manag 19(2):387–388. https:// doi. org/ 10. 1007/ s1025702100535-2 Chong EK (2019) Teaching and learning music theory in the age of AI and mobile technologies. Int J Digit Soc 10(3):1505–1509 Chou P-N, Feng S-T (2019) Using a tablet computer application to advance high school students’ laboratory learning experiences: a focus on electrical engineering education. Sustainability 11(2):381 Chuchu T, Ndoro T (2019) An examination of the determinants of the adoption of mobile applications as learning tools for higher education students. Int J Interact Mobile Technol 13(3):53–67 Churchill GA Jr (1979) A paradigm for developing better measures of marketing constructs. J Mark Res 16(1):64–73 Cidral WA, Oliveira T, Di Felice M, Aparicio M (2018) E-Learning success determinants: brazilian empirical study. Comput Educ 122:273–290 Clayton K, Murphy A (2016) Smartphone apps in education: students create videos to teach smartphone use as tool for learning. J Media Literacy Educ 8(2):99–109 Cohen J (1988) Statistical power analysis for the behavioral sciences, 2 edn. Routledge Member of the Taylor and Francis Group Criollo-C S, Guerrero-Arias A, Jaramillo-Alcázar Á, Luján-Mora S (2021) Mobile learning technologies for education: benefits and pending issues. Appl Sci 11 (9) Curum B, Khedo KK (2021) Cognitive load management in mobile learning systems: principles and theories. J Comput Educ 8(1):109–136 Damyanov I, Tsankov N (2018) Mobile apps in daily learning activities. iJIM 12(6) Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 13(3):319–340 Davis FD, Bagozzi RP, Warshaw PR (1989) User acceptance of computer technology: a comparison of two theoretical models. Manage Sci 35(8):982–1003. https:// doi. org/ 10. 1287/ mnsc. 35.8. 982 Davis FD, Bagozzi RP, Warshaw PR (1992) Extrinsic and intrinsic motivation to use computers in the workplace. J Appl Soc Psychol 22(14):1111–1132 Delone WH, McLean ER (1992) Information systems success: the quest for the dependent variable. Inf Syst Res 3(1):60–95 Delone WH, McLean ER (2003) The DeLone and McLean model of information systems success: a tenyear update. J Manag Inf Syst 19(4):9–30 667 1 3 What impacts learning effectiveness ofamobile learning app… Diacopoulos MM, Crompton H (2020) A systematic review of mobile learning in social studies. Comput Educ 154. https:// doi. org/ 10. 1016/j. compe du. 2020. 103911 Diamantopoulos A, Sarstedt M, Fuchs C, Wilczynski P, Kaiser S (2012) Guidelines for choosing between multi-item and single-item scales for construct measurement: a predictive validity perspective. J Acad Mark Sci 40(3):434–449. https:// doi. org/ 10. 1007/ s117470110300-3 Diaz JCT, Moro AI, Carrión PVT (2015) Mobile learning: perspectives. Int J Educ Technol High Educ 12(1):38–49 Dijkstra TK, Henseler J (2015) Consistent partial least squares path modeling. MIS Q 39(2):297–316 Diliberto-Macaluso K, Hughes A (2016) The use of mobile apps to enhance student learning in introduction to psychology. Teach Psychol 43(1):48–52 Dorobat I (2014) Models for measuring E-learning success in universities: a literature review. Informatica Economica 18(3):77 Drolet AL, Morrison DG (2001) Do we really need multiple-item measures in service research? J Serv Res 3(3):196–204. https:// doi. org/ 10. 1177/ 10946 70501 33001 Efiloğlu Kurt Ö (2019) Examining an e-learning system through the lens of the information systems success model: empirical evidence from Italy. Educ Inf Technol 24(2):1173–1184 Fornell C, Larcker DF (1981) Evaluating structural equation models with unobservable variables and measurement error. J Mark Res 18(1):39–50. https:// doi. org/ 10. 2307/ 31513 12 Fradika H, Surjono H (2018) ME science as mobile learning based on virtual reality. In: Journal of Physics: Conference Series, 2018. vol 1. IOP Publishing, p 012027 Freeze RD, Alshare KA, Lane PL, Wen HJ (2010) IS Success model in e-learning context based on students’ perceptions. J Inf Syst Educ 21(2):173–184 Gable GG, Sedera D, Chan T (2008) Re-conceptualizing information system success: the IS-impact measurement model. J Assoc Inf Syst 9 (7). https:// doi. org/ 10. 17705/ 1jais. 00164 Gale T, Parker S (2014) Navigating student transition in higher education: induction, development, becoming. Universities in Transition: Foregrounding Social Contexts of Knowledge in the First Year Experience:13–39 Gangaiamaran R, Pasupathi M (2017) Review on use of mobile apps for language learning. Int J Appl Eng Res 12(21):11242–11251 Geisser S (1974) A predictive approach to the random effect model. Biometrika 61(1):101–107. https:// doi. org/ 10. 2307/ 23342 90 Goksu I (2021) Bibliometric mapping of mobile learning. telematics and informatics 56:101491. https:// doi. org/ 10. 1016/j. tele. 2020. 101491 Goldkuhl G, Karlsson F (2020) Method engineering as design science. J Assoc Inf Syst 21(5):4 Gómez-Galán J, Martínez-López JÁ, Lázaro-Pérez C, Sarasola Sánchez-Serrano JL (2020) Social networks consumption and addiction in college students during the COVID-19 pandemic: educational approach to responsible use. Sustainability 12(18):7737 Goodyear VA, Armour KM, Wood H (2019) Young people learning about health: the role of apps and wearable devices. Learn Media Technol 44(2):193–210. https:// doi. org/ 10. 1080/ 17439 884. 2019. 15390 11 Gordon N, Brayshaw M, Dixon J, Grey S, Parker D (2021) The role of gamification in a software development lifecycle. In: INSPIRE XXVI. delivering global education and impact in emergencies using E-Learning, 2021. pp 81–94 Goundar MS, Kumar BA (2022) The use of mobile learning applications in higher education institutes. Educ Inf Technol 27:1213–1236. https:// doi. org/ 10. 1007/ s1063902110611-2 Green B, Seshadri S (2013) AngularJS. O’Reilly Media, Inc Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG (2016) Statistical tests, p values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol 31(4):337–350. https:// doi. org/ 10. 1007/ s106540160149-3 Gregor S, Hevner AR (2013) Positioning and presenting design science research for maximum impact. MIS Q 37(2):337–355 Gregor S, Jones D (2007) The anatomy of a design theory. J Assoc Inf Syst 8(5) Gupta Y, Khan FM, Agarwal S (2021) Exploring factors influencing mobile learning in higher education: a systematic review. Int J Interact Mob Technol 15(12) Hair JF, Ringle CM, Sarstedt M (2011) PLS-SEM: indeed a silver bullet. J Market Theory Pract 19(2):139–152. https:// doi. org/ 10. 2753/ MTP10 6966791 90202 Hair JF, Risher JJ, Sarstedt M, Ringle CM (2019) When to use and how to report the results of PLS-SEM. Eur Bus Rev 31(1):2–24. https:// doi. org/ 10. 1108/ EBR1120180203 668 F.Johannsen et al. 1 3 Hassan MA, Habiba U, Majeed F, Shoaib M (2021) Adaptive gamification in E-learning based on students’ learning styles. Interact Learn Environ 29(4):545–565 Heinze D (2018) Die Bedeutung der Volition für den Studienerfolg. Springer Henseler J, Hubona G, Ray PA (2016) Using PLS path modeling in new technology research: updated guidelines. Ind Manag Data Syst 116(1):2–20. https:// doi. org/ 10. 1108/ IMDS0920150382 Herrington A, Herrington J, Mantei J (2009) Design principles for mobile learning Heublein U (2014) Student drop-out from German higher education institutions. Eur J Educ 49(4):497–513 Hevner A, Chatterjee S (2010) Design science research in information systems. Springer, New York Hevner AR (2007) A three cycle view of design science research. Scand J Inf Syst 19(2):87–92 Holotescu V, Vasiu R, Andone D (2018) A critical analysis of mobile applications for learning. Study case: virtual campus app. BRAIN Broad Res Artif Intell Neurosci 9:110–117 Holsapple CW, Lee-Post A (2006) Defining, assessing, and promoting E-learning success: an information systems perspective. Decis Sci J Innov Educ 4(1):67–85 Huang Y-M, Pu Y-H, Chen T-S, Chiu P-S (2015) Development and evaluation of the mobile library service system success model. The Electronic Library Huber L, Hellmer J, Schneider F (2009) Forschendes Lernen im Studium: Aktuelle Konzepte und Erfahrungen, vol 10. Univ.-Verlag Hussein Z (2018) Subjective norm and perceived enjoyment among students in influencing the intention to use E-learning. Int J Civ Eng Technol 9(13):852–858 Hwang G-J, Chang C-Y (2021) A review of opportunities and challenges of chatbots in education. Interactive Learning Environments, 1–14 Ibrahim AK, Kelly SJ, Adams CE, Glazebrook C (2013) A systematic review of studies of depression prevalence in university students. J Psychiatr Res 47(3):391–400 ifo Institut (2022) Fachkräftemangel steigt auf Allzeithoch Isleib S, Woisch A, Heublein U (2019) Ursachen des Studienabbruchs: Theoretische Basis und empirische Faktoren. Z Erzieh 22(5):1047–1076. https:// doi. org/ 10. 1007/ s1161801900908-x Ismail SM, Hashim H (2020) Virtual reality-based education (VRBE): understanding students’ readiness and expectancies. Int J Innov Technol Explor Eng 9(3):172–176 Ives B, Olson MH, Baroudi JJ (1983) The measurement of user information satisfaction. Commun ACM 26(10):785–793 Jongbloed B (2004) Tuition fees in europe and australasia: theory, trends and policies. In: Smart JC (ed) Higher education: handbook of theory and research. Springer Netherlands, Dordrecht, pp 241–310. https:// doi. org/ 10. 1007/140202456-8_7 Johannsen F, Knipp M, Loy T, Voshaar J, Zimmermann J (2021) A mobile app to support students in the "transition-in" phase. ECIS 2021 Kabudi T, Pappas I, Olsen DH (2021) AI-enabled adaptive learning systems: a systematic mapping of the literature. Comput Educ: Artif Intell 2:100017 Kehm BM, Larsen MR, Sommersel HB (2019) Student dropout from universities in Europe: a review of empirical literature. Hungar Educ Res J 9(2):147–164 Khaddage F, Lattemann C, Bray E (2011) Mobile apps integration for teaching and learning. (Are Teachers Ready to Re-Blend?). In: Society for information technology & teacher education international conference, 2011. Association for the Advancement of Computing in Education (AACE), pp 2545–2552 Khalid N (2014) The role of perceived usefulness and perceived enjoyment in assessing students’ intention to use LMS using 3-tum. Global Summit on Education GSE Kim H-W, Chan HC, Gupta S (2007) Value-based adoption of mobile internet: an empirical investigation. Decis Support Syst 43(1):111–126. https:// doi. org/ 10. 1016/j. dss. 2005. 05. 009 Kiryakova G, Angelova N, Yordanova L Gamification in Education. In: 2014. Proceedings of 9th international Balkan education and science conference Klier M (2008) Metriken zur Bewertung der Datenqualität – Konzeption und praktischer Nutzen. Informatik Spektrum 31(3):223–236 Kock N (2015) Common method bias in PLS-SEM: a full collinearity assessment approach. Int J E-Collaboration 11(4):1–10