Gaining Student Engagement Through Project-Based Learning: A competitive 2D Game Construction Case Study

Received November 29, 2021, accepted December 29, 2021, date of publication December 30, 2021, date of current version January 6, 2022. Digital Object Identifier 10.1109/ACCESS.2021.3139764 Gaining Student Engagement Through Project-Based Learning: A Competitive 2D Game Construction Case Study INIGO LOPEZ-GAZPIO Department of Informatic, Electronic and Communication Technologies, University of Deusto, 20012 Donostia-San Sebastian, Spain e-mail: [email protected] ABSTRACT In this work we consider an open artificial intelligence game as a matter of study within the lectures of artificial intelligence to combat lack of motivation and increase engagement within the classroom. During formation, students in computer science can deal with moderately complex projects, nevertheless, dealing with such problems is relegated to the Degree Final Project. In this investigation we show the procedural steps of how project-based learning combined with game construction can effectively be used to promote engagement in informatic lectures at university. For the task, we build a 2D game engine and propose students to enroll in factitious research teams with the aim of programming intelligent agents that play the game employing artificial intelligence techniques. The intended principal outcome is to show evidence of the application of project-based learning in artificial intelligence within the lectures, and how it can be combined with game construction to increase motivation in the classroom. Project-based learning has the students learn, organize, and solve challenges while students themselves remain their own responsible for the investigation and process of work. We propose to follow a series of sequential phases that conform a set of milestones that incorporate a project-based learning approach to the lectures. Through this work we show that the use of project-based learning combined with game construction provides reliable evidence that a much deeper understanding about artificial intelligence is attained by students participating in the challenge. Student evaluation questionnaires and final grade results attained by students indicate that students remained more engaged during the semester in comparison to previous semesters in which lack of motivation was reported. INDEX TERMS Action research, active learning computing education, cross-disciplinary skills, engagement, game construction, project-based learning (PBL), reinforcement learning. I. INTRODUCTION Research in the field of educational methodologies and their applications suggest that predefined laboratory practices are not as much a significant contributor towards student performance as lecturers can at first imagine [1]–[5]. Actually, it is an open research field to understand both, the components and constructs that enable significant student engagement in order to achieve high quality education [6]. Lamborn et al. [7] define student engagement as the ‘‘psychological investment in and effort directed towards learning, understanding or mastering the knowledge.’’ The cited statement has been further redefined such that student engagement The associate editor coordinating the review of this manuscript and approving it for publication was Aasia Khanum . is described as a process in which the student ‘‘develops an affective and behavioral participation in the learning experience’’ [5], [8]. Thus, these investigations claim that focused personal effort in learning activities is responsible for better understanding and performing in academic work. If we want students to acquire new knowledge, skills and competencies that allow them to face current reality, lecturers must assure them meaningful learning [9]. Meaningful learning only occurs when students are motivated and give personal meaning to what they are trying to learn. For this, it is essential to incorporate new methodologies in the classroom and offer alternative scenarios that enable experimentation and locates the students out of their comfort zone to offer new engaging experiences. In 1973, the American psychologist Bruner developed the theory of learning by discovery [10] in VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ 1881 I. Lopez-Gazpio: Gaining Student Engagement Through Project-Based Learning: Competitive 2D Game Construction Case Study which he referred to a pedagogical dynamic through which the learner discovers the concepts and their relationships and rearranges them to its own cognitive schema. More recent research suggests that incorporating team working into this methodology provides even more benefit to learners. Rush et al. noted that ‘‘when students are part of a learning community, they become positive about their identity as a member of the group and remain focused on learning’’ [11]. In this sense, we, the authors, believe that the usual activity of engineers is experimentation and research, therefore, experimentation and research must be the main tool for teaching. At the same time, the project selected for experimentation must be sufficiently engaging so to motivate students and promote commitment. If a sufficiently engaging project is selected, research performed by students will eventually lead to a meaningful learning. In the particular case of engineering educators, these also have the challenge of cultivating a diverse set of characteristics to students, such as rapidly evolving technical competencies [12], [13]. This rapid development of information and communication technologies has obliged education systems to adapt to new requirements in little time [14]. Teaching is an art that needs to be engaging so that both students and lecturers can collaborate in the learning process. For several decades, the process of collaborative learning has been studied as a matter of great interest [15], [16]. Even more, considerable effort has been put into correlating the level and quality of student engagement and personal development with the academic program and activities carried out during lectures throughout the semester [1], [5]. Distinct alternatives for increasing student engagement have been proposed, including project-based learning, problem-based learning and learning by teaching. As stated by Larraza-Mendiluze et al. [17] these proposals consist of three constructionist approaches to human learning. It is well discussed by Ploetzner et al. that an effective learning process must involve a self-construction step where opportunities for learning must be self-created [18]. In order to learn, students must work in an engaging environment to construct knowledge rather than obtain it from the external world. It is well documented that the educative effect is greater when knowledge is constructed in this way. More classical teaching methodologies aim to deliver concepts to students while constructionist methodologies are focused on knowledge building experiences. In this context, project-based learning (PBL) encourages constructionist learning [19]–[21] in which students build knowledge instead of being simple recipients of it, and encourages them to develop critical and reflexive thinking. According to the cited works PBL is defined as a student-centered pedagogic methodology in which dynamic classroom approaches are used. The student-centered pedagogy assures that students acquire a deeper knowledge of the matters addressed through active exploration of real challenges or problems. In the application of the methodology students work for relatively long periods of time in an active style trying to investigate or respond complex challenges [22]. As a result, PBL is a challenge not only for students but also for lecturers as it moves away from the more traditional teaching practices, such as: paper based laboratory practices or teacher-led instruction practices. PBL implies a change of roles, work methodology and evaluation in which students actively form teams and face a complex challenge that must be solved working altogether. PBL invites students to identify what they should learn, to get involved in the search for information to solve the problem, to define the strategies to be used and to agree on common solutions. In this context, the lecturer becomes a guide or tutor accompanying students and helping them when required, but never as the sole source of wisdom. From our point of view, the aim of PBL is to have future competent professionals who are innovative and full of confidence in their abilities and reasoning. The topic of artificial intelligence (AI) is an important field within computer science engineering. It is of great importance that computer engineers understand how an intelligent agent behaves and how such agents can be developed. Students in computer science can deal with moderately complex artificial intelligence laboratory projects; nevertheless, dealing with a real AI problem is relegated to the final degree project or to a master program. In this article we consider an open artificial intelligence game as a matter of study within the AI lectures. The previous teaching methodology goes no further than the classical approach in which students receive explanations and are assigned several predefined laboratory practices which leads many students to relate artificial intelligence with an unfashionable topic. In this work we address the problem of increasing student engagement through the employment of modern approaches such as project-based learning combined with game construction. Such an approach can be successfully used to promote informatics at University. For the task, we propose to follow a series of sequential phases that conform a set of milestones that help incorporate a project-based learning methodology to the artificial intelligence lectures. Through the completion of the proposed phases, teams formed by students develop intelligent agent players that solve the proposed game employing artificial intelligence techniques. As regards the implementation of the game, we previously build a 2D game engine from scratch based on a popular classic game that will ultimately interface with the players constructed by students. Throughout the proposed phases students are asked to enroll in fictitious research teams to construct the required players, based on artificial intelligence search and exploration techniques discussed in the lectures. The experience also help students acquire leadership and knowledge of cross-disciplinary competencies, such as code development or software process competencies. Based on student evaluation questionnaires and final grades, we show that engagement within the classroom has been higher when PBL combined with game construction is employed. Real engagement allows students to develop skills at a greater extent and when properly structured, engagement can also lead to positive interdependence between peers and 1882 VOLUME 10, 2022 I. Lopez-Gazpio: Gaining Student Engagement Through Project-Based Learning: Competitive 2D Game Construction Case Study academic staff. This in turn can create a sense of connectedness and personal investment in the process of study. This article is structured as follows. Section II presents the state-of-the-art analysis comprising distinct approaches and methodologies employed in teaching computer science, as well as game construction challenges addressed previously in the academia, Section III describes the game engine construction and summarizes the course in which the challenge is introduced and the proposed methodology is applied. Once the challenge and the course subject to modification are described, Section IV describes the application of the new methodology, and Section V presents the evolution of the teams during the proposed phases. Finally, Section VI presents the conclusions of the investigation. II. STATE-OF-THE-ART ON METHODOLOGIES USED TO TEACH COMPUTER SCIENCE The state-of-the-art (SOA) reports a varying set of styles of teaching [23], these methodologies range from purely theoretical approaches to completely practical ones. For the case of computer science education our SOA analysis revealed that a wide bunch of learning methodologies is being employed to increase engagement and attract the attention in various science domains. For instance, an interesting analysis performed by Larraza-Mendiluze et al. to extract how computer architecture is being taught throughout well-known worldwide universities determined that the most employed teaching strategies fold into one of the following set of categories [24]: i) purely descriptive approaches, ii) performance-based approaches, iii) practical approaches based on programming assignments and iv) open approaches. To obtain this categorization the cited work analyzed the educational process carried out in 36 Universities in which examination questions, course syllabi, textbooks and course materials were exhaustively analyzed. i) Purely descriptive approaches only show students a description of the matter under analysis. Students are expected to understand the lecture, and the concepts being described by them. Purely descriptive practices also consider that students can be asked to perform some summative assignment during the semester. This is known to be an easy way to introduce topics in the classroom, although the level of attraction for the students is not high. Purely descriptive approaches lack from obtaining high attraction since they barely consist of text essay realizations. ii) Performance-based approaches require students to realize some kind of computation, so that the results obtained are then compared to some reference result known by the lecturer. This approach requires the application of certain grade of knowledge to determine the appropriateness of the techniques used by the student [25]. This approach is also known in the literature as illustrative experiments [26] which involve experiences accompanied with explanations given by the teacher. iii) Programming practical approaches are based on a long-term project that must be elaborated by student individuals or groups. This approach helps to introduce abstract concepts regulated by the lecturer when defining the project scope. It is widely believed that hands-on programming helps make concepts clearer. However, it turns to be a tough experience if the student lacks the necessary ability to overcome the project and, consequently, engagement can be rapidly lost, especially in small groups or working alone. iv) Open approaches are related to non-guided research [26] in which the approach combines the acquisition and construction of concepts, the resolution of practical problems and practical work usually performed by students in cooperating teams. Open approaches aim to make students become the directors of their own learning by promoting a new active learning process through cooperation. In this model, lecturers act as facilitators of the learning that must come from within the research performed by the team. From the didactic point of view, it is the guided research, and not the static practices that allow a complete and effective learning of the procedural, attitudinal and conceptual concepts. Students are given the opportunity to experience the usual challenges that computer engineers encounter in a daily basis. Open guided research approaches have shown that they significantly improve quality of students to face real problems [27]. According to the cited investigation open guided research approaches are capable to stimulate critical thinking, offering numerous advantages over traditional static approaches due to their inherent open nature. Without a strict follow-up of a detailed script students are given the chance to work on small groups creating the optimal scenarios to grow concepts. On the contrary, as it is also stated in [27], even PBL involves several advantages it encompasses various difficulties along the implementation of the procedure, such as: the complexity to handle numerous projects in the classroom and its difficulty to standardize the evaluation. If difficulties are overcome most of the lecturers agree that teaching with PBL has numerous advantages, particularly in the field of computer science [27]. Yet, there is not a one-fits-all-solution on how to apply open guided research in a particular scenarios [28], and the specific procedure on how to apply the open guided research methodology varies among the 500 projects investigated in [27]. As an example, the authors in [29], [30] evaluated distinct student groups taught under different methodologies and indicate that student groups involved in open guided research activities obtained higher grades and a greater ability in the extent to which they could use core ideas to engage science practices. Open guided research approaches have also already been successfully tested in distinct domains of engineering sciences, for instance, [1], [31] report that for engineering lectures open simulation exercises have been widely adopted with great success. In this direction, [32], [33] describe works that employ simulation models to support education and offer more interesting and realistic open environments to students. In the recent past PBL practices have been successfully employed in distinct areas of knowledge to solve VOLUME 10, 2022 1883 I. Lopez-Gazpio: Gaining Student Engagement Through Project-Based Learning: Competitive 2D Game Construction Case Study open challenges. Some investigations on PBL combined with open problems are those of [34], [35] in which the authors tried to provide a solution to the poor motivation of students in laboratories with few resources by employing student-centered methodologies and questionnaires. The student-centered methodology is proven to be of use allowing for more enjoyable, interesting, creative, and motivating practices. There are many references in the literature related to PBL that stress its importance and effectiveness, as an example the investigations of Pucher et al. in [27] enumerates more than 500 projects involving PBL with various outcomes. Nevertheless, the work remarks that project ideas suggested by students turn out to yield exceptional good results in most of the cases. This remark suggests previous investigations that suggested an effective learning process must involve a self-construction step where opportunities for learning must be self-created. The work also states the benefits and difficulties of the methodology and poses the concern that many times the beneficial effect of the methodology is based on the context of the problem and the subjective vision of the teacher. More specific investigations on the applications of PBL include those of Larraza-Mendiluze et al. in which the authors provide an extensive enumerate of plausible projects that could be carried out in the area of computer architecture [36]. In [37], [38] the authors elaborate on distinct projects from the computer architecture domain in which PBL methodology has been incorporated to lectures with success confronting students with real world open challenges that must be resolved. The proposal of open challenges based in games also fits particularly well with PBL in the sense they define and contextualize a real or simulated challenge that must be addressed. In this regard, the work described in [39] collects experiences from the past two decades in which computer games have been used for research in the particular case of computer science and computational intelligence, among others. According to the authors games have always been a popular testbed not only for scientific research, but also educational research due to the fact games offer a big range of flexibility and variable problem complexity. The work states that games settled in an open world environment and consisting of a relatively small set of actions can pose an interesting challenge for controlled research. III. COURSE OVERVIEW AND CONSTRUCTION OF THE GAME ENGINE Artificial intelligence is a fifth-year second semester course in the Department of Informatic, Electronic and Communication Technologies at the University of Deusto (San Sebastian faculty, Spain). The course introduces classic elements of artificial intelligence to students, such as: tree and graph search heuristics, local search optimization, population-based optimization and introductory machine learning concepts. The course provides an initial understanding of what actually is inside an intelligent agent implementation, and it is based on a classical approach based on the well-known reference FIGURE 1. Initial disposition of the fifteen sticks layered in rows. The game always starts with seven, five and three sticks organized in three rows. Players, in turns, keep on removing sticks until the last stick is taken from the board, which dictates the game loser. by Russel and Norvig [40]. The course comprises four contact hours per week, and an additional tutoring hour. It has traditionally been delivered in face-to-face descriptive format using a chalk-and-talk style, aided with some laboratory programming assignments. The learning outcomes of the course are that upon successful completion of the course students should be able to: (i) identify, analyze and define the significant elements making up a problem in order to solve it with criteria and effectively; (ii) use one’s own experience and criteria in the analysis of the causes of a problem and build up a more effective and efficient solution; (iii) understand the fundamental principles and basic techniques of intelligent systems and their practical applications; (iv) formulate search problems and identify and apply an appropriate solving technique; (v) define and apply good heuristics to solve different problems considered difficult; (vi) apply machine learning techniques as a way for an intelligent system to gain a certain degree of autonomy; and (vii) analyze problems whose resolution requires empirical knowledge and to design knowledge-based systems. Thus, individuals who graduate in computer engineering are able to apply their knowledge to solve AI problems in different areas related to information technologies, providing the most appropriate solutions in each case. The main contribution of the subject Intelligent Systems to the degree in computer engineering is the resolution of problems and design of applications, based on existing requirements and applying the precise criteria of effectiveness, efficiency, costs and benefits involved. In particular, the subject of Intelligent Systems emphasizes the resolution of difficult problems, many of them NP-complete, through the design and use of heuristics in artificial intelligence algorithms; the design and use of machine learning techniques, and knowledge-based systems. For many years, artificial intelligence has been taught using a traditional approach based on laboratory practices, in which students complete well-known artificial intelligence 1884 VOLUME 10, 2022 I. Lopez-Gazpio: Gaining Student Engagement Through Project-Based Learning: Competitive 2D Game Construction Case Study problems from the state of the art, such as the N-Queens problem or the Travelling Salesman Problem (TSP). Despite the fact the classical algorithms for solving these problems need to be discussed in class, they offer an old-fashioned approach that fails in engaging students. Regarding the subject recent feedback reported that a significant number of students do not engage with the content of the lectures. Reports mainly focus on the unattractiveness of classical AI methods seen during the first weeks of the semester. Students just do not feel the classical approaches are of any more use in the recent or nearby future. This results in a drop of motivation to study AI algorithms which repercusses in poor class performance. Some students even manifest the non-usefulness of the classical AI approaches when machine learning or deep learning is currently attaining such a great impact on societal applications. Based on both, trying to make a more friendly and attractive learning style, and a more engaging learning process, the authors considered the opportunity to incorporate a project-based learning approach based on game construction to engage students. In this setting lecturers build a 2D game engine that represents a classic well-known challenge and propose students to enroll in factitious research teams with the aim of programming intelligent agent players that play and solve the game employing artificial intelligence techniques. Following [39] in order the game to expose an interesting research environment, the proposed game should be open, with a relatively small amount of actions and pose a real world challenge to students. Yet, in order to maintain student engagement high the proposal of how to code the intelligent AI player must come within the student groups [27]. To this aim, the game engine is designed so that it can interface with any kind of player that follows a defined interface to perform game actions (e.g. remove a certain amount of sticks from the board). We selected the topic for the game so as to familiarize and engage the age of the students (23-24 years). For the task, we selected an artificial intelligence game that is being popularized by the well-known ‘‘El Hormiguero’’ maximum audience television broadcast, which among others intends to deliver science to young adults: the fifteen sticks game. The fifteen sticks game consists of an open AI problem that can only be solved by complex math-based algorithms as it is a rather complex game with a high number of statespace outcomes, but a relatively low action space set; in contrast to games like Chess or Go!, the number of actions and state-space outcomes is yet tractable computationally and makes the resolution feasible for students. The fifteen sticks game1serves a perfect scenario to challenge students as it consists of a classical 2-player competitive game that consists of removing sticks from a board in turns, as many as requested by players, but only from one of the three available rows at a time. The player who catches the last stick is 1The fifteen sticks game is a redesign of the popular Nim game https://en.wikipedia.org/wiki/Nim FIGURE 2. Full sequence of a game played by two human agents that communicate through the keyboard with the game engine. Top left subfigure shows the initial game state and each of the following subfigures represent an action taken by each of the human players (player 1 selects sticks to be removed in shaded gray and player 2 selects sticks to be removed in red). At each time step any number of sticks can be removed from a single row, until a single stick is left. The player that removes that final stick is the game loser. the one who loses the match. The game always starts with fifteen sticks, distributed in three rows as shown in Figure 1. We implemented the game engine using python programming language and the pyGame2game development library. The game engine is designed in such a way that different actor players can be employed to launch the game, at first 2https://www.pygame.org/news VOLUME 10, 2022 1885 I. Lopez-Gazpio: Gaining Student Engagement Through Project-Based Learning: Competitive 2D Game Construction Case Study FIGURE 3. Block diagram describing the defined milestones for the application of the PBL methodology. we implemented a random player and a human player that communicates with the game engine through the keyboard. A full sequence of a game played using two human players is shown in Figure 2. The challenge proposed to students is then to enroll in fictitious research teams to analyze the problem and develop an intelligent agent that is able to play the game with a high win ratio, preferably with superhuman performance. For the task, they must employ AI search techniques discussed within the lectures (e.g. tree-search based heuristics, graphsearch based heuristics, local searches, machine learning, . . .) From the methodological point of view, this practice aims for students to make a first contact with the world of research, as well as fostering a critical and innovative spirit. It intends that groups of motivated students apply the concepts of the Intelligent Systems subject in the specific game context that has been described, and develop an intelligent agent able to efficiently play the game. IV. APPLICATION OF THE PBL METHODOLOGY Incorporating game interaction is a very flexible and efficient solution to construct a sufficiently complex problem that poses some degree of controlled challenge for the students FIGURE 4. Kick-off meeting of team participating in the challenge. while it allows to elaborate particular self defined solutions for the AI players. In the recent past, games and gaming platforms are becoming more and more relevant in educational environments. For instance, in the computer science and computer engineering domain the majority of the literature cites games designed to teach programming languages [41]–[44]. Additionally, similar attempts have also been performed in other computer science domains in order to teach computer architecture [1], [36] or to teach computer memory [45]. As reported by [27], most of the student group failures come from the nonexistence of organization within student groups or the lack of team working experience. To overcome these difficulties, guided by the lecturer the students employ the phases proposed in the methodology presented in Figure 3to guide their project life-cycle. The PBL methodology for the construction of the AI player is composed of seven phases, as denoted in the mentioned block diagram. These seven phases involved in the methodology cover the most relevant aspects of agile software development and are organized in sequential milestones that student teams must adopt in their timeline in order to develop the project. The phases are enumerated as a series of sequenced milestones: 1) Problem exploration, 2) State-of-the-art and relevant technology analysis, 3) Design of the solution, 4) Implementation of the solution, 5) Testing and evaluation of the solution, 6) Documentation of the solution, and 7) Dissemination of documentation. During the life-cycle of the project lecture notes on AI to build the software agent along other computer science competencies and knowledge are to be used, as the phases define tasks related to conforming groups, determining roles and objectives, coding, testing, deploying and documenting the solution. Among others, it is expected to bring at least software process and quality, software design and programming competencies forefront. The project also benefits from collaborative learning as students act as teachers of their teammates. Taking the dual roles of learners and teachers help students further their knowledge of informatics [46]. A. DESCRIPTION OF MILESTONES AND EVOLUTION OF THE STUDENT TEAMS 1) PROBLEM EXPLORATION In this phase the team concerns about the problem to be addressed. As the problem is a challenge that motivates the 1886 VOLUME 10, 2022 I. Lopez-Gazpio: Gaining Student Engagement Through Project-Based Learning: Competitive 2D Game Construction Case Study FIGURE 5. The figure shows three game states from the fifteen sticks challenge that will eventually lead the agent to lose. team chances to lose interest during the life cycle of the challenge are low, and the team soon starts delimiting the scope of the work to be done. At this stage it is important to start understanding and defining clear objectives. Figure 4 shows one practical session in which a team is brainstorming on the problem exploration of the game. 2) STATE-OF-THE-ART AND RELEVANT TECHNOLOGY ANALYSIS In this phase, the team works on exploring a possible solution and gathering useful techniques and knowledge, meetings involving brainstorming are especially relevant at this point. Exploring AI techniques or collecting further requirements regarding the functionality of the software to be built are example outcomes of this phase. In the proposed challenge, the outcome consisted of a broad exploration of search techniques used in the state of the art in the domain of AI. 3) DESIGN OF THE SOLUTION In this phase teams focus on designing a software architecture that responds to the requirements identified in the second phase. Unified Modeling Language (UML) diagrams and other types of software maps is the key to a successful solution design phase. The intelligent agent to be developed in the team must be able to model the search state-space tree of the possible occurrences of the game, and it must be able to classify game-state representations according to some utility function (used to determine goodness or badness of states and actions). In that sense the agent must be able to determine useful game moves that evolve towards the agent winning the play eventually. Identifying the game states that lead the agent to lose is also an interesting aspect discussed within teams. At this phase, teams formed by students were able to model game states and identify the losing states as shown in Figure 5. During this phase validation with the lecturer is necessary in order to maintain the scope of the work and not overload the objectives of the team and keep them realistic and doable. 4) IMPLEMENTATION OF THE SOLUTION During this phase, the team focuses on implementing the design made using various programming languages learnt throughout the informatics degree. At this phase, the team focuses on scheduling and burning milestones regarding identified functionalities. For the specific case of the described practice, many teams decided to implement the code in Python programming language and employ some kind of heuristic search or reinforcement learning approach so that the agent was able to find the best action set in order to win the game. Equation 1shows an example of a team making use of reinforcement learning (Temporal difference online learning) approach, which employs the VS function as the value function to obtain the utility (goodness or badness) of states from the game. The VS function is shown below, and it can be shortly summarized as a relaxation of the Bellman equation [47]. The VS function grades goodness or badness of states between 0 and 1 according to how near there is a winning or losing state from the actual game state. Winning and losing states as defined by a student team are shown in Figure 5, and Figure 6shows a play sequence for a full episode of the game. VS(s)=VS(s)+alpha ∗(VS(snext )−VS(s)) (1) where s denotes the current state, snext denotes the next state and alpha is the velocity to learn, a model hyper parameter. The mathematical model lets the agent learn backpropagating the value function of states once the episode is finished unrolling the performed steps. 5) TESTING AND EVALUATION OF THE SOLUTION At this phase teams experimented with several factors, such as the iteration number to train the agent and its success rate to win, the learning rate hyper parameter and other exploration hyper parameters. For solution evaluation, teams reported different kinds of experiments and results such as the effect the agent has been training with regard to success rate. The team discussions originated at this step are very enriching for the whole classroom as they analyze performance and gain practice organizing and showing experiment results. 6) DOCUMENTATION OF THE SOLUTION At this phase students are asked to overcome the tunnel vision acquired during previous phases and document the solution explaining the AI techniques employed. One of the best performing teams managed to produce a scientific paper in a journal from the University of the Basque Country. For this particular team the extent of the effort was greater than for other teams, but the reward of working through the peer VOLUME 10, 2022 1887 I. Lopez-Gazpio: Gaining Student Engagement Through Project-Based Learning: Competitive 2D Game Construction Case Study FIGURE 6. VS example values for a sequence play performed by the intelligent agent developed by a team. Note that after each action the opponent must also perform some action, that action is unknown and not reflected in the agent’s history. review process of the journal and have it published [48] consisted of a great reward for their effort. 7) DISSEMINATION OF DOCUMENTATION At this phase teams communicate their work to teammates and a general discussion was carried out in class. Distinct solutions are explained, analyzed, and discussed by team members and students learn how a scientific discussion is carried out. Some teams also participated in a video recording to show other students and future incoming students what kind of activities they performed in class. For this, students established contact with the marketing and communication department from the Faculty of Deusto in San Sebastian and Bilbao. The video recorded is expected to be useful for opening days or dissemination activities related to the degree. V. STUDENT ENGAGEMENT EVALUATION This section describes the four evaluation scenarios employed to measure the student engagement during the PBL practice. For the evaluation we have taken into account: A) the internal conclusions of the student teams, which corresponds to the intra-evaluation reports of the students extracted from assignments, B) impressions of the lecturers throughout the semester, C) final lecture grades, and D) student engagement evaluation reports collected using questionnaires by the University of Deusto. A. INTERNAL CONCLUSIONS OF STUDENT TEAMS The aim of the challenge proposed has been non-trivial as it involved analyzing, studying and applying the use of artificial intelligence in the context of a game. To do this, we first brainstorm and argue about different ways artificial intelligence could be applied in this context, such as what rational behavior is and by what means we could possibly reflect this behavior in an intelligent agent. After explaining classical methods from the state of the art, we were given additional instructions on techniques, which could be applied in the fifteen sticks game. The current article describes all the work performed during the semester, including both: the description of the game and the artificial intelligence techniques used to solve it. We also shared the code in Github for both: the game engine programmed in python and the intelligent agent class file, which we believe to be easily adaptable to solve any other similar game or use in other contexts for teaching and learning. We also developed an evaluation metric to measure the performance of our intelligent agent, based on the probability to navigate to good and bad states. The results obtained showed that our agent performed at superhuman level, with very low probabilities to lose. B. LECTURERS’ IMPRESSIONS There is no doubt that methodologies aimed to engage students need further evaluation, and that for this, educational research is necessary. Many tools have been developed, but most of them have not been evaluated nor have clear outcomes. In this experiment, we have challenged students to form small units of fictitious research groups and solve a challenge that was never computationally tackled before employing project-based learning and game construction. Overall, the development of the project has been positively evaluated by the authors because of the following outcomes: 1) It generates a high interest as students face with a real challenge they have been watching recently on the media. 2) Several different specific, generic and transversal competences from the computer science degree are applied to solve the challenge. 3) Students practice coding theory lectures seen in class, and act as facilitators of the knowledge as needed by teammates. They construct knowledge as the team evolves. In this regard, communication and conflict 1888 VOLUME 10, 2022 I. Lopez-Gazpio: Gaining Student Engagement Through Project-Based Learning: Competitive 2D Game Construction Case Study solving skills are developed within all team discussions. 4) Students simulate a research group and can try to publish their results, which brings them an extra motivation to finish the work and present results. Also, they get to know how the research community work, the peer reviewing process and they obtain feedback from experts distinct to their habitual lecturers. In-class discussions about all these aspects also favor communication between students. To be more specific, during the challenge we noticed that having an external evaluation board from the journal resulted in an additional engagement component in teams that decided to present their work as it extends the barriers of the lecture. It is also of interest to mention that due to the nature and characteristics of the challenge, an agile software methodology can be further defined to carry out the project. Agile methodologies focus on fostering communication and the iterative and incremental development of prototype-based software. This is achieved by assigning roles and responsibilities among team members work, which favors the consolidation and integration of the team. Agile software development is currently a matter of great interest in computer science degrees. Overall, we think it has been a very rewarding experience to see students manage to build an intelligent agent that solves an open game challenge that is currently being broadcasted on TV. C. STUDENT’S FINAL GRADE PERFORMANCE ANALYSIS In order to assert that the instructional strategy involving project-based learning with game construction is an effective approach to help students learn subjects, we compare among distinct student performances across two consecutive academic courses in which the artificial intelligence lecture has been taught following the same schedule and evaluation criteria but have been taught under different learning processes. One of the courses has been taught involving traditional practices (2018/2019) while the second course has been taught with the PBL methodology (2019/2020). We think that quality and quantity of evidence used to support the stated assertion of instructional effectiveness is highly related to the final performance of students measured as lecture grades once the learning process is finished. Final evaluation grades have been extensively used to determine course overall performances and measuring student engagement through the analysis of final grades is a common practice in the state-of-the art within educational research [1]. The distribution of students alongside obtained statistics for grades is shown in Table 1for the academic courses 2018/2019 and 2019/2020. Assuming an underlying normal distribution on the grades obtained by students as indicated by a Shapiro-Wilk test of normality (p-value >0.05), we conducted a one-sample Wilcoxon signed-rank non-parametric test to determine whether the performance mean values were significantly different among distinct academic years. The results of the Wilcoxon test (V =17, p-value <0.001) indicate that student average performance has been higher when project-based learning activities have been carried out with regard to the classical approach. Significantly better student performance suggests that a higher level of engagement has been attained in academic course 2019/2020, in which project-based learning methodology was employed. D. STUDENT ENGAGEMENT QUESTIONNAIRES In addition to the final grades obtained by students we also include the Student Engagement Evaluation Reports (SEER) that the University of Deusto collects every academic year with regard to all subjects taught at the University collected by means of questionnaires designed and offered to students by third-party professionals. SEER questionnaires include a total of 30 items of evaluation regarding the impressions of students with regard to engagement intervention. SEER questionnaires are completely anonymous to lecturers for which they can only access the final mean result of each item under analysis. The fundamental purpose of this evaluation is to asses lecturers in their continuous improvement. In this regard, the assessment made by students on different aspects or dimensions of the teaching methodology can be a valuable aid as a basis for reflection and identification of strengths and weaknesses, as well as to quantify student engagement along the semester. The items under evaluation fold into one of the following categories: 1) Design and Planning: definition and organization skills, content, resources, materials, methodologies, tutoring systems and evaluation that, according to the University of Deusto Training Model (MFUD), ensures an autonomous and significant learning process. 2) Learning Management: develop the set of activities that according to the MFUD actively involve students in the learning process. 3) Tutoring and Evaluation: to carry out effective monitoring and support during the student’s learning process that allows to verify, communicate and facilitate the improvement of the degree of development and achievement of competences throughout the process and as a final result. 4) Review and Improvement: verification of the degree of compliance with the planning and the adequacy of the resources used, making the consequent decisions for their improvement. 5) Collegiality: activities that lecturers carry out together, which are common and have a collective and community orientation (all that refers to work between teachers as members of the department, degree, faculty and university). Active participation with other teachers who influence the unit to develop the different parts of the process, ensuring autonomous and meaningful learning on the part of the students. VOLUME 10, 2022 1889