An Academic Reinforcement Methodology Based on Multimedia Resources

AN ACADEMIC REINFORCEMENT METHODOLOGY BASED ON MULTIMEDIA RESOURCES Vicente Arévalo1, José García-Nieto2, Juan J. Sánchez-Sánchez3, Rafael LuqueBaena2, Gerardo Gómez3, Francisco Velasco-Álvarez4, Rafael Godoy-Rubio3 1Dpto. de Ingeniería de Sistemas y Automática 2Dpto. de Lenguajes y Ciencias de la Computación 3Dpto. de Ingeniería de Comunicaciones 4Dpto. de Tecnología Electrónica University of Malaga (SPAIN) [email protected], [email protected], jjsa[email protected], rm[email protected], [email protected], [email protected], [email protected]a.es Abstract This paper describes an academic reinforcement methodology resulted from the research activity carried out by a group of lecturers from the University of Malaga (UMA) with teaching in different engineering degrees (e.g., computer science, industrial and telecommunication engineering). The research, developed in the context of the UMA Lecturer Training Plan 2010/20111, stems from the concern to solve a common problem detected in their daily work: students in the first courses have problems to understand certain key concepts, which usually are fundamental for their basic knowledge as engineers. Our methodology is based on the early identification of these “difficult” concepts and the creation of explanatory multimedia content available online, which is iteratively improved according to the feedback provided by the students. This reinforcement methodology will be implemented in diverse subjects of the new degrees given by the authors during the current and future academic years. Keywords: Academic reinforcement methodology, Multimedia resources, Engineering degrees, Lecturer training plans. 1 INTRODUCTION Most subjects in an engineering degree contain fundamental concepts that students must acquire for a proper development and progressive learning, and whose understanding may be difficult. On the one hand, students do not have time enough to mature the key concepts; on the other hand, despite the efforts of the lecturer, students have difficulties to identify those concepts, and how they are related to each other [1]. If we also consider that, in the European Higher Education Area (EHEA), the autonomous learning is promoted at the same time the number of hours of lectures is reduced, the result is a deficit in the acquisition of basic concepts. This causes students to fail to properly follow the classes and consequently to need a further effort in order to grasp the main concepts in the subject [2]. This situation leads students to decline in attendance in classrooms and hence to reduce their presence in the final evaluation. We believe that conducting a series of multimedia materials dedicated to the explanation of fundamental concepts, as a complement of the study hours out of the class (e-learning, b-learning), can reinforce the conceptual foundations related to a specific subject in the new educational system, producing a qualitative and quantitative improvement of the learning process of students [3,4]. Nevertheless, and despite the intrinsic attractive and potential benefits of such resources, it is clear that the content of the material must be carefully designed for the suitable development of the student learning process [5, 6, and 7]. This need, one, together with the difficulty of checking that the designed resources meet their purpose (i.e., to improve the academic outcomes of the students) result in the need of designing a methodology that allows us assessing the student results due to the usage of the reinforcement material and, if it is necessary, improving them in an appropriate way. 1 http://www.uma.es/formacionpdi/ We can find a broad quantity of recent works that employ different types of multimedia resources as support for the face-to-face classes. For example, in [2] a series of videos are used to explain the mathematical concepts related to the vectorial algebra, in [8] education videos are employed in teaching occupational medicine, in [9] a blending methodology is presented to combine traditional practices and e-learning in the field of microbiology, or in [10], where several audiovisual materials for analysing food are proposed. Although the majority of these works provide an evaluation of the improvement in the student learning, none of them proposes a mechanism for assessing the performance and usefulness of the employed resources. That is, they make an evaluation of the benefits of the materials from the point of view of the student academic results (whether the students have better academic outcomes with the prepared material, or not), but they do not introduce any methodology for redesigning them if they do not meet its goal. With the above-mentioned difficulties and deficiencies in mind, we have designed a methodology that iteratively improves the content according to the educational needs of students. The structure of the paper is as follows. Section 2 presents our methodological approach to improve the learning of the students by creating and redesigning the multimedia resources. The current state implementation is showed in Section 3, together with some practical examples of the new material created. Finally, conclusions are drawn in Section 4. 2 DESCRIPTION OF THE METHODOLOGY Our proposal mainly consists of the following stages: first, critical concepts of interest for a proper learning are identified; second, the identified concepts are then explained throughout multimedia resources such as flash contents, animations, etc.; third, the teaching results are evaluated at the end of each lecture by means of the ad-hoc evaluation tools applied on reduced groups of students statistically representative; and fourth, the evaluation results are analyzed in order to obtain statistic information and remarks that allow us to redesign the contents and so, addressing the learningteaching process properly. Fig 1. Cycle lifetime of the proposed reinforcement learning methodology. Multimedia resources are iteratively updated and improved according to previous evaluation and analysis phases. Fig. 1 shows a summarized scheme of our proposed methodology. As we can observe, the cyclic procedure of identifying complex concepts, evaluating, analyzing, and redesigning material, help us to continuously improve and increment the multimedia repository. All the generated material will be available for future courses, in which the reinforcement methodology will be also applied. Obsolete material will be move away from the current repository and located in a historical archive of resources, keeping the accessible (to students) material up-to-day, for the sake of a better understanding. A comprehensive description of our proposal is given in the following subsections. 2.1 Identification of concepts Our proposal is based on the identification of complex concepts and the creation of on-line multimedia material for students. This task is primarily performed by the teachers, who must elaborate a list of key concepts based on: 1) their own experience, 2) frequently asked questions from the students, or 3) typical mistakes in exams. In addition, there are many concepts in engineering degrees whose complex and dynamic nature (e.g., a wave oscillation) are hardly explained in a conventional blackboard; such concepts are good candidates for being reinforced with additional multimedia resources like videos, interactive materials, etc. Anyhow, this first step shall be considered as an iterative process, in which the list of concepts can be dynamically updated according to the feedback obtained from the evaluation tools. With this adaptive approach, we achieve two different objectives: in the first place, the material will perfectly fit the students' necessities. In the second place, and equally important, the participation of these students in the decision of the contents increases their involvement in the learning process. 2.2 Creation of the multimedia resources Once the main concepts have been identified, the next step is based on the selection and creation of multimedia resources. We recommend the use of different types of resources depending on several factors like: type of concept to be reinforced, target audience, the need of quick feedback or not, etc. In particular, the following resource categories have been selected: • Videos: the use of online videos makes it possible for students to acquire the concepts at their own pace outside of the class. In addition, due to the complexity and dynamism of some engineering processes (e.g. wave propagation along the time), the development of videos may definitely support students for a quicker learning process. • Interactive material: interactive teaching material follows the “learn-by-doing” rather than just “read-and-memorize”, thus providing an educational benefit to the students compared to passive learning tools. Through the use of multimedia development tools such as Flash and Java (e.g., Applets), teachers can create material that engages students to learn through active participation instead of passive absorption. This type of resources is highly recommended in engineering courses, e.g., for assisting the students in the use of complex software or hardware tools. As complementary resources, we propose the use of online self-evaluation tests, which may serve as regulators of the learning process as they inform the students what is already mastered or still needs further work. It is very important that the tests provide the students with some type of feedback, which can be both qualitative and quantitative. The feedback information can direct the student back to the insufficiently mastered parts of the course, towards new exercises or further explanations, according to the way the tests have been developed by the teacher. Finally, we think that it could be interesting to encourage the students to create their own multimedia resources. This way, they can reach a higher level of concept acquisition by means of a “learn-byteach” process, since an almost perfect understanding is necessary to be able to explain the concepts as clearly as possible. 2.3 Learning evaluation Although the advantages of using multimedia resources and interactive approach in lecturing could appear straightforward, it is always necessary to carry out an evaluation process to provide insights into how and what the students are learning, and to give information about what might be improved in practice [11]. Innovation not always leads to improve the student learning and, therefore, it is important to perform a carefully designed evaluation, which determines in a rigorous way the influence of new curricula in student learning [12]. Thus, what we want to know is the impact of our proposal on student’s learning experience including both, the enhancement in their learning process and their feelings towards the innovations applied in the course. Our aim is to evaluate throughout the process as it gives us the opportunity to make changes in order to improve the material, while the course is ongoing thanks to the continuous feedback. Two types of evaluation tools will be used: • Tests of acquired knowledge (Summative evaluation): the use of summative evaluation provides evidence of the effectiveness of the proposed lecturing practice. We plan to carry out several tests through the course to evaluate the acquired knowledge by individuals in focus and control groups. Thus, we will check if the students have learned what they were supposed after using the provided material (focus group) or following a traditional lecturing (control group). • Test of satisfaction (Formative evaluation): formative evaluation is typically conducted during the development or improvement of a program or product (or person, and so on), and it is performed to validate and ensure that the goals of the instruction are being achieved, and to improve the instruction, if necessary, by means of identification and subsequent remediation of problematic aspects. Formative evaluation will allow us to make adjustments to our practices during the course in order to improve the learning environment. This kind of tests shows the students that their opinion is valued and it enhances their involvement in the learning process. We have to be careful in the sense that student satisfaction should not always be identified with better learning outcomes, although improved satisfaction is usually a positive outcome [12]. One of the most common means of comparison for a formal experiment is to split up the population into a focus and a control group. In our opinion, the fairest approach in this case is asking students to voluntary take part on the experience pilot program. Those who do not volunteer can be a built-in ‘control group’, a group against which to assess the performance, that is, the responses of those who take part in the new program or approach. Making their participation on the experience voluntary, we avoid a situation in which there exists the possibility of inequity of opportunity for the students. However, there are some inconveniences regarding this approach. On the one hand, it is plausible that students who volunteer are the ones with a great motivation or drive, that is, the ones who are likely to obtain good results in any case. On the other hand, it is impossible to ‘quarantine’ the two student groups, and some interchange of teaching materials can be expected [12]. In an ideal test environment, the proposal would be tested in a controlled setting where control group students are not likely to have access to the same resources than the focus group. In a real setting, as mentioned before, expecting that kind of isolation between two different groups of students would be naive. In any case, although the assessment of the impact on student’s learning experience would be problematic when the control group also can access to the provided material. This would not mean that it was not effective, but only that its effects are measured using a weak contrast. Fig 2. Scheme of the proposed cross comparison between the attained results by focus and control groups during the application of our proposal. The cross comparison between the attained results by the other students coursing the same subject during the previous year is also considered. In order to compensate the possible dispersion caused on students’ results, a comparison with previous year results will be carried out. Thus, we first study the performance of two different groups of students, coursing the subject in which we want to apply our proposal, during the previous year in order to discover correlation between them. The groups should be created following an arbitrary criterion not related to the academic performance in any way (e.g., using surnames to order the students). Then, we study the performance during the current year of both, the focus group (volunteers) and the control group, to extract correlations (see Fig. 2). Finally, we will carry out a crossed comparison between previous and current year groups. The study will allow us to obtain a better insight on the impact of our proposal, once we have somehow compensated the aforementioned dispersions. 2.4 Data analysis and resources redesign As stated above, in our proposal we consider the evaluation of teaching innovation as a systematic attempt to determine the effects of an educational initiative, innovation or experiment on student learning [12]. We also try something new on a syllabus or in the classroom risks providing a poor educational experience for the students [13], and we feel it is professional responsible for academic staff to make sure that the innovations really enhance student’s learning experience. The analysis of the data gathered through evaluation will serve us to discover flaws in our proposal, which we can amend timely with corrective actions, e.g., though adjustments and improvements on the provided material. 1. Results stats: knowing the level of achievement reached by the focus and control groups will provide a first insight into the performance of our proposal. Thanks to these tests, we will identify which materials are fulfilling their purpose improving student’s learning experience and which ones fail to do so, hence needing improvement or redesign. Besides, the comparison with results obtained by former students during the previous year will help us to discover any dispersion in the results (see Fig. 2). 2. Usage stats: data on student patterns of usage will be collected through the course. The analysis of these data will provide to us pointers to further questions about what works and what does not, and most importantly, why it works or not. These questions can be followed up through formal interviews or informal discussions with students as proposed in [12]. 3 CURRENT STATE IMPLEMENTATION In this section, we will provide a brief summary of the developed contents, by way of example. To date, they have not been yet applied; it is during the present 2011/2012 course when the materials will be available for the students in the way it was mentioned above. The reader can get access to these contents in the following link: https://sites.google.com/site/resourcesamples/ 1) Doppler effect: this example consists of a video illustrating one basic concept used in most of engineering degrees. After a common theoretical explanation of the physics underlying the effect, the video shows an audiovisual demonstration in which, a given student can hear the change in the sound pitch depending on the relative speed of the emitter and the receiver (see Fig. 3). The next tools were used to make the video: Mathematica (for the audiovisual demo), Camtasia Studio (to record both the audio and the video of the demo), PowerPoint (to add some explanatory slides) and Windows Movie Maker (for the final layout and the voice synchronization). Fig 3. Snapshot of the audiovisual demonstration of the Doppler effect. 2) Flash tutorial of a Pspice based simulation: with this interactive application, the student can go through a step-by-step example where every action is explained in detail. The interaction is achieved by means of buttons that need to be pushed in order to go forward or backward. Wink has been used as a software tool that allows the user to record the simulation, split it into single actions (e.g., every single click of the mouse), and edit the final application. Fig. 4. Step of the interactive Pspice simulation. 3) Solution of a differential equation using Simulink: the last example is a YouTube video that shows how to solve a differential equation by means of the block diagram oriented software, Simulink. Besides this tool, PowerPoint was used for the explanatory slides, TextAloud as the voice synthesizer, Camtasia Studio to record the video and Sony Vegas for the video editing. Fig 5. Snapshot of the YouTube video demonstration of the solution of a differential equation using Simulink. 4 CONCLUSIONS In this paper, we present an academic reinforcement methodology based on the usage of multimedia resources. This methodology is the result of a research work accomplished by a group of lecturers from the University of Malaga, with teaching in different engineering degrees (e.g., computer science, industrial and telecommunication engineering), worried by the difficulties of students to assimilate and put in practice the conceptual basis of their subjects. The main goal of the proposed methodology is twofold: to improve the academic outcomes of students, and, if such a goal is not achieved, to redesign the resources according to the information collected along the evaluation stage. As final product, we provide to our students, as well as to the academic community in general, with a huge repository of multimedia didactic material that will be online available. This approach is novel in comparison with the proposals found in the literature, which only evaluate the academic impact of the multimedia resources, but not their redesign. This methodology will be put in practice in diverse technical subjects given by the authors through the next academic year. The paper also includes, as illustration, some snapshots of the materials which are being currently designed. REFERENCES [1] K. Wilson, and J.H. Korn (2007). Attention During Lectures: Beyond Ten Minutes. Teaching of Psychology, 34(2), 85-89. [2] A. Rovai, M. Ponton, M. Wighting, and J. Baker (2007). A Comparative Analysis of Student Motivation in Traditional Classroom and E-Learning Courses. International Journal on ELearning, 6 (3), pp. 413-432. [3] P. Rodríguez-Cielos, G. Aguilera-Venegas, J.L. Galán-García, M.A. Galán-García, and Y. Padilla-Domínguez (2010). Utilización de Vídeos Educativos en la Enseñanza del Análisis Vectorial. XIII Congreso de Enseñanza y Aprendizaje de las Matemáticas. [4] A. Almansa-Martínez, F. Almeida-García, C. Lara Rallo, R. Malpartida-Tirado, M. León-Vegas, I. Pineda-Hernández y C. Toledo-Báez (2011). Integración de Recursos Multimedia para la Docencia: Nuevas Perspectivas en el EEES. IV Jornadas de Innovación Educativa y Enseñanza Virtual de la UMA. [5] G. Krippel, A.J. McKee, and J. Moody (2010). Multimedia use in Higher Education: Promises and Pitfalls. Journal of Instructional Pedagogies, pp. 1-8. [6] A.L. Yen-Low, K.L. Teng-Low, and V. Chet-Koo (2003). Multimedia Learning Systems: a Future Interactive Educational Tool, The Internet and Higher Education, Volume 6, Issue 1, pp. 25-40. [7] C.S.M. Turney, D. Robinson, M. Lee, and A. Soutar. (2009). Using Technology to Direct Learning in Higher Education: The Way Forward? Active Learning in Higher Education Volume 10, Issue 1, pp. 71-83. [8] L.C. Escobar-Pinzon, B. Rossbach, M. Claus, S. Letzel, (2010). "Die Praxis" – Experiences with Educational Videos in Teaching Occupational Medicine, 3rd International Conference of Education, Research and Innovation (ICERI 2010), Madrid (Spain). [9] P. Sancho, R. Corral, T. Rivas, M.J. González, A. Chordi, and C. Tejedor (2006). A Blended Learning Experience for Teaching Microbiology. American Journal of Pharmaceutical Education 70 (5) pp. 1-9. [10] M.A. Zapata-Revilla, C. Díez-Marqués, M. García-Mata, M.L. Pérez-Rodríguez, A. RedondoCuenca, D. Tenorio-Sanz, and M.J. Villanueva-Suárez (2010). Developing Audiovisual Learning Materials in Food Analysis. 3rd International Conference of Education, Research and Innovation (ICERI 2010), Madrid (Spain). [11] I. Moore (last access 25/09/2011). A Guide to Practice: Evaluating your Teaching Innovation. The National STEM programme. Engineering Innovations Projects. http://www.hestem.ac.uk/sites/default/files/evaluating_your_teaching_innovation.pdf. [12] R. James (last access 25/09/2011). Evaluating Teaching Innovation. Centre for the Study of Higher Education. http://www.cshe.unimelb.edu.au/resources_teach/feedback/docs/teaching_innovation.pdf. [13] A.M. Boggs (2010). Evaluation of & Innovation in Teaching – Creating a Critical Feedback Loop in Higher Education. Department of Education, University of Oxford. Discussion paper for League of European Research Universities (LERU) BRIGHT Conference, April 15-19, 2010 (Munich, Germany).