Combining multimedia and self-assessment CAD tools in an interactive web environment to learn engineering drawing

Combining multimedia and self-assessment CAD tools in an interactive web environment to learn Engineering Drawing Pablo Pando Cerraa,*, Javier Gracia Rodrígueza, Humberto Fernández Álvarezb, Bernardo Busto Parraa a Departamento de Construcción e Ingeniería de Fabricación, Universidad de Oviedo, Campus De Gijón, Gijón, 33203 Asturias, Spain b Departamento de Ingeniería Eléctrica, Electrónica, de Computadores y Sistemas, Universidad de Oviedo, Campus De Gijón, Gijón, 33203 Asturias, Spain *Corresponding author E-mail address: [email protected] Telephone number: +00-34-985181948 FAX: 00-34-985 182433 Departamento de Construcción e Ingeniería de Fabricación Universidad de Oviedo, Campus de Gijón Gijón, 33203 Asturias (Spain) Biographical notes Pablo Pando Cerra ([email protected]) Pablo Pando Cerra received his PhD in Industrial Engineering from the University of Oviedo in 2006, where he is currently professor of several subjects related to computer-assisted design. His current research interests include computational geometry, medical image analysis, and interactive learning environments, specifically innovative ways to teach engineering concepts with multimedia software and to improve spatial perception. Javier Gracia Rodríguez ([email protected]) Javier Gracia received his PhD in Architecture from the University of Navarra in 2008. He is currently professor at University of Oviedo. His current research interests include computational geometry, web applications and interactive environments to improve teaching of continuum mechanics by means of virtual laboratories or multimedia software. Humberto Fernández Álvarez ([email protected]), Humberto Fernández Álvarez was born in Oviedo, Asturias (Spain), in January 1989. He received the Telecommunication Engineering degree in 2014 from the University of Oviedo (Asturias, Spain). He received the best final degree research project award in 2015 from Cátedra Telefónica and a research grant from the Education and Culture Office of the Principado de Asturias. Currently, he is working on his PhD degree in the area of signal theory and communications. His research interests include metamaterial absorbers, metasurfaces (frequency selective surfaces, artificial magnetic conductors…), antenna design and propagation, computational electromagnetics, measurement techniques and interactive learning environments. Bernardo Busto Parra ([email protected]) Bernardo Busto Parra, Mechanical Engineer and PhD (University of Oviedo, 2015) specializes in information systems and collaborative manufacturing environments. He is an industry professional in technical departments and a vocational training instructor for technical drafting. His current research interests include the industrial application of drone technology, Discrete Elements Methodology (DEM) and combining SI with interactive learning environments. Acknowledgments The authors would like to acknowledge the support from the Statistical Consulting Unit and from the Scientific and Technical Services of University of Oviedo. Combining multimedia and self-assessment CAD tools in an interactive web environment to learn Engineering Drawing Interactive Web Environments and multimedia tools are very common in engineering studies. However, when the subject to be studied requires graphic resolution for its learning, the use of this type of resources is very limited. A learning environment that combines video and self-assessment CAD (ComputerAided Design) tools was developed to analyse this problem and to determine if the combined use of these two tools may be important in learning these subjects. A comparative study of 124 engineering students was carried out. They were randomly assigned into three groups: Group-A using only traditional tools, Group-B using traditional tools and CAD tools and Group-C using traditional tools, CAD tools and video training. The statistical analysis of the results showed that a greater number of students of Group-C passed the test and improved their scores. Therefore, the combined use of these two types of resources can be an important aid in the study of subjects requiring graphic tools for their learning. Keywords: interactive learning environments; evaluation of CAL systems; Engineering Drawing; computer aided assessment; media in education Introduction E-learning environments have become quite popular because they increase student motivation and allow teachers to customise content for each student. In this vein, students choose what, when, and where to learn (Ding, Xiong, & Liu, 2015). Therefore, e-learning is a less expensive environment than conventional ones because it does not need physical classrooms prepared for a specific subject. Moreover, as stated by Blackburn (2017), it is common to “incorporate multimedia elements and interactive properties that provide improved educational experiences that prepare students for the digital age.” E-learning environments have proven themselves as effective teaching tools in areas such as informatics and engineering (Hamada, & Hassan, 2017). However, when the subject requires the use of graphical methods to solve exercises, these environments are quite limited. This is the case for engineering drawing, a traditional subject which can not be quite popular among students. It requires new, innovative tools and methodologies to improve the learning experience. In this regard, Min and Xu (2015) claims that “we have to solve the problem of how to explore a set of practical effective teaching methods, which can not only inherit the traditional essence of an engineering drawing course, but also cultivate students' innovation ability.” In many cases, e-learning environments for engineering drawing are conventional courses with the addition of a standard e-learning tool available on the Web. However, often these tools have not been developed specifically for that subject; as a result, student interaction with the e-learning platform is low. Self-study is based only on pdf class notes, video tutorials, reinforcement simulations, and learning support. However, this type of subjects requires more; it requires “active learning.” This term was defined by Bonwell and Eison (1991) as “anything that involves students in doing things and thinking about the things they are doing”. E-learning environments for subjects which require graphic methods for solving exercises are also quite limited when evaluating students work. In general, selfassessment is reduced to multiple choice tests, essay exams, or uploaded homework assignments (Hettiarachchi, Huertas, & Mor, 2015). However, none of them has proven itself as an effective solution when the problem-solving process requires graphical methods. In this sense, Rodriguez, Diaz and Carriegos (2014) claim that to “create Moodle questionnaires in drawing subjects is challenging, as problems are solved in a graphic manner.” Therefore, in current e-learning environments, teacher participation is high because they can continually monitor student learning process by means of student homework evaluation. Moreover, if the number of students is high, the time the teacher will spend on each student is reduced, and the learning process is compromised. Therefore, how can a student know if the exercises assigned have been correctly solved? As mentioned before, current e-learning environments do not provide any specific tool for self-assessment of graphical exercises. Online videos are also a powerful tool when used as a complement to the learning process. Their main purpose is to provide clear explanations of fundamental concepts of the subject. They have been supported by many authors (Giannakos, Jaccheri, & Krogstie, 2016; Wang, 2016; Verdú, Pelayo G-Bustelo, Martínez-Sánchez, & Gonzalez-Crespo, 2017). In the same vein, Onime, Uhomoibhi and Ieee (2013) study the impact of online videos in engineering education and conclude that they are useful learning resources both inside and outside the classroom. Regarding engineering drawing or any other subject which requires graphical solving methods, online videos have proven themselves as an effective tool for explaining the steps required to solve an exercise and to learn how to apply a specific graphic methodology. However, there is no agreement regarding whether a student who watches a video he will learn how to solve the exercise. This research emphasises the benefit of multimedia resources combined with self-assessment CAD tools in interactive learning environments. Purpose of the study Although video resources on e-learning environments are not a novel concept, combining them with self-assessment graphical tools is a new one. The benefits of this combination have not been studied in detail before. This research studies this combination and how to integrate these tools in the teaching-learning process of subjects which requires interactive graphic tools for their study. Therefore, this paper wants to answer the following question: Does the combination of video and self-assessment Computer-Aided Design (CAD) tools improve the teaching-learning process of engineering drawing? Related works The use of traditional methods of study (master classes, theoretical documents, etc.) combined with the use of CAD applications is common in subjects which require graphical solving methods (Martin-Gutierrez, Trujillo, & Acosta-Gonzalez, 2013). However, studies such as the one conducted by Tsuei and Lai (2015) have demonstrated the positive effects of an online engineering drawing platform for learning this subject. Teachers who require environments for rendering graphical models often resort to platforms such as Moodle, but the great majority of these platforms are not prepared for teaching this type of subject. Other teachers have resorted to interactive graphical environments as support tools for graphic visualisation (Andraphanova, 2015). Some of them have gone one step further by incorporating automatic reasoning tools as a complement for the learning process (Abanades, Botana, Kovacs, Recio, & Solyom-Gecse, 2016). However, in most cases, interaction is reduced to changing the view and the analysis of the dimensions and properties of the model. Furthermore, these interactive graphical environments typically do not allow students to represent the exercise and much less conduct a selfassessment. Additional tools used in e-learning environments are augmented reality (Gutierrez de Rave, Jimenez-Hornero, Ariza-Villaverde, & Taguas-Ruiz, 2016), virtual reality (Girbacia, Beraru, Talaba, & Mogan, 2012) or even both (Martin-Gutierrez, Garcia-Dominguez, Sanjuan-Hernanperez, Roca-Gonzalez, & Romero-Mayoral, 2015). However, these technologies allow a student to visualise contents, interact with them or even emulate their behaviour, but often these technologies still do not allow one to draw, design, or generate new contents and self-assessments of exercises. There are many e-learning environments where graphic rendering plays a prominent role in the teaching-learning process. In some cases, graphic rendering features are used to improve spatial visualisation skills (Pedrosa, Barbero, & Miguel, 2014). Van den Einde, Delson and Asee (2014) introduce touch screens for a selfguided learning process. In other cases, graphic rendering was used to represent and simulate mechanisms (Hren, 2010) or electrical schemes (de Jesus Ramirez-Rojas, & Aviles-Cruz, 2016). Interactive visualisation tools have also been developed as a complement for Learning Management Systems (LMS) (e.g., Kuosa et al, 2016). However, this type of environment often fails to provide tools for self-assessment and for interactive graphical solving of exercises. If we focus on the two types of tools selected in this article, we find fewer studies which address improving the instruction of engineering drawing. In the case of tools to revise graphical exercises, we can point to De la Torre, Saorin, Contero, Dorribo-Camba and Ieee (2013). This research presents an interactive tool that combines theoretical contents with sketching tools. However, assessment of emailed homework is done manually by a teacher. Furthermore, in most cases they resort to well-known e-learning platforms whose self-assessment tools are not prepared to revise exercises that need a graphical resolution. Previous researches by the authors of this paper proposed some tools to assess students’ works, obtaining quite interesting results (Cerra, Penin, Morales, & Garrido, 2011; Cerra et al, 2014). Researches done by Baronio, Motyl and Paderno (2016) and Speranza, et al (2017) are also remarkable; they developed self-learning tools to help students to understand fundamental concepts of engineering drawing. As stated by Khan (2012), “educators have shown that short, 5-15 minute videos are optimally suited for the attention retention for students.” Multimedia tools and videos in engineering drawing are frequently used only to support theoretical contents of the subject. Nevertheless, they have recently been used in some interesting ways. Akhtar, Warburton, Xu, and Ieee (2013) have combined interactive multimedia environment with videos and student mentoring tools to improve computer-aided drawing learning. Camba, Contero, and Salvador-Herranz (2014) have used videoconferences to support development of CAD models in a collaborative workspace. Finally, Serrano et al (2014) combine video and sketching tools in industrial design teaching-learning process. However, no research has been found which combines both tools, video and graphical self-learning tools, within the scope of engineering training and, more specifically, in engineering drawing, except in the research conducted by Baronio, Motyl and Paderno (2016). Nevertheless, in this study students do not have tools to interactively sketch and self-assess exercises. The proposal of this article is to go a step beyond the current trends of innovative teaching-learning methodologies in engineering drawing and analyse the benefits of combining both tools, tools that are not usually combined for this subject. Material and methods Participants One hundred and twenty-four students from the class “Engineering Drawing,” which is an introductory course for the industrial engineering degree, participated in this study during the 2016-2017 academic year. They were randomly assigned to three groups (Table 1):  Group-A (GrA): students that used traditional tools (pdfs, paper, pencil, etc.) to learn the subject.  Group-B (GrB): students that used traditional tools and a self-learning CAD tool to train themselves.  Group-C (GrC): students that used traditional tools, a self-learning CAD tool, and training videos. Group Male(M) Female(F) M+F Description GrA 42 8 50 Trad. tools GrB 20 12 32 Trad. tools + self-assessment CAD GrC 36 6 42 Trad. tools + self-assessment CAD + Video Training Total 124 Table 1. Distribution of groups. Self-assessment learning environment The authors of this paper have developed an interactive web-learning tool called DIBUTEC. This tool improves the learning process of subjects with high dependency on graphical solving methods. It is a multi-device and cross-platform web application developed with HTML5. DIBUTEC integrates an online 2D CAD module to sketch drawings like in similar desktop CAD applications. The CAD module has the usual tools in this type of applications (visualisation and pan, drawing and editing of entities, references to objects, etc.). It also features responsive design, interfaces automatically, and resizes itself to fit on any device. However, it has a huge advantage over other CAD applications: a self-assessment module. This module evaluates a proposed response and Figure 6. The scores of the previous test. First, the PRE-SCORE variable was examined to determine if its behaviour differs according to the different levels of the GROUP variable (Figure 6). It was necessary to consider in advance if this variable satisfies the hypothesis of normality and the hypothesis of homogeneity of variance. The hypothesis of normality was rejected for all the behaviours within the GROUP variable (Shapiro-Wilk test: GrA pvalue=0.022 < 0.05; GrB p-value=0.023 < 0.05; GrC p-value=0.022 < 0.05). However, due to the sample size, a homogeneity of variance was accepted (Bartlett test, p-value=0.679 > 0.05). After completing the preliminary analysis, the difference between the averages of the PRE-SCORE variable with respect to the GROUP variable was analysed (Table 4). The conclusion was that there were no significant differences between the variables being tested (ANOVA test, p-value = 0.434 > 0.05). Therefore, the hypothesis “there is no difference between the averages of the previous scores” was not rejected, validating the homogeneity of variance established for this research. Group Number of samples Average Median Std. deviation GrA 50 6.42 6.25 1.94 GrB 32 5.82 5.62 2.24 GrC 42 6.16 6.25 2.07 Table 4. Results of the PRE-SCORE variable by group. The results obtained from the post-test are shown in Table 5 and Figure 7. The Shapiro-Wilk and Bartlett test were again applied to the POST-SCORE variable. According to this, it was concluded that the hypothesis of normality was not rejected (GrA p-value=0.15 > 0.05; GrB p-value=0.44 > 0.05; GrC, p-value=0.1 > 0.05). The hypothesis of homogeneity of variance was not rejected either (p-value = 0.67 > 0.05). Group Average Median Std. deviation GrA 5.55 5.65 1.92 GrB 5.70 5.95 1.99 GrC 6.54 7.00 1.73 Table 5. Results of the POST-SCORE variable by group. Figure 7. Distribution of students by post-test scores. After that, the POST-SCORE variable was studied in detail to check if the behaviour of this variable differed with respect to other levels of the GROUP variable (Figure 8). By applying an ANOVA test, a p-value=0.03<0.05 was obtained. According to the data results, the equal averages in the three groups were not accepted, and so a statistically significant difference between results exists. Figure 8. The relation between the POST-SCORE and GROUP variables. In addition, the Tukey test indicates that in some pairs of levels there are significant differences between them. If ordered by order of significance, the following priority results: C and A (p-value = 0.03 < 0.05). In contrast, there were no differences between the following levels: C and B (p-value = 0.14 > 0.05) and B and A (p-value = 0.93 > 0.05), respectively. Finally, the Pearson correlation test was carried out to study the relationship between the PRE-SCORE and POST-SCORE variable for each of the groups. The correlation coefficient and the significance obtained are shown in Table 6. It obtains a significant positive relation in all cases - a higher previous higher mark and a lower previous mark lower grade obtained. Group Correlation coefficient (r) p-value GrA 0.712 <0.001 GrB 0.674 <0.001 GrC 0.416 0.006 Table 6. The correlation coefficients between PRE-SCORE and POST-SCORE variables. Discussion From the primary results, students from GrC (traditional way plus DIBUTEC and videos) achieved the best scores in the final test (average score of 6.54 points). This result is even more interesting if it is compared with the other two groups. Neither GrA (traditional way) not GrB (traditional way and DIBUTEC) reach six points of the average score. Also differences between averages are significant, 0.84 points with respect to GrB and 0.99 with respect to GrA. There is a subtle improvement of GrB with respect to GrA; nevertheless, this improvement is even better when combined with training videos, as shown by GrC. If we focus on final test pass rates, the results are quite similar. The pass rate of GrC is 88%. Only 5 out of 42 students did not pass the final test, while rates for GrA and GrB are 74 % and 71% respectively. To complete the analysis, student’s evolution from the beginning of the course is studied. Students from GrA have the best background on the subject (6.43 points average score on initial test), followed by GrC (6.16 points) and then GrB (5.82 points). Although the contents of the initial and final tests are not comparable, it is remarkable that the only group that has improved results, ultimately, the one who has evolved best, was GrC. Nevertheless, it should be mentioned that students from GrB had the worst background on the subject, yet even they were able to improve more than students from GrA. Therefore, in view of the previous results, it can be stated that the group which combined traditional lectures with self-learning tools and training videos achieved the best final scores, the best rate of pass tests, and the best improvement with respect to initial background. As the unique difference between groups was the teaching methodology applied, it can be concluded that combination of a self-assessment CAD tool with online training videos could be a reason for the improvement. Looking further into the results and if we focus on the students who passed the final test, the average score of students from GrC was nearly seven points (6.99), quite above group GrB (6.68) and GrA (6.47). GrA was again the group with worst results. If we focus on students with higher marks (Figure 7), 23 of 37 students from GrC got a final mark equal or superior to seven points (62% of passing students) against 43% from GrA (16 of 37 students) and 39% from GrB (16 of 23 students). However, it is remarkable that the results from GrA, who received a “traditional” education, are slightly better than results from GrB. Nevertheless, results concur with the ones stated before; students from GrC did better than the other groups. As stated by Hung and Chou (2015), it is very important that the “shift from being a traditional passive classroom learner to being an active online inquirer,” is the target of the self-assessment tool presented in this research. Moreover, it is essential that the learning system was as fluid and interactive as possible. Definitely, this will benefit students. In fact, research by Wei, Peng and Chou (2015) concluded that “interactivity has been viewed as playing an essential role in the learning process among learners, instructors, and learning content.” Similar conclusions were presented by Bolivar Baron, Gonzalez Crespo, Pascual Espada, and Sanjuan Martinez (2015). Results shown in these papers suggest that the combination of self-graphic-learning tools and training videos provides solid improvement in the engineering drawing learning experience. This combination increases interaction with students and provides real time feedback. This experience is not easy to achieve with conventional learning systems unless there is a dedicated teacher constantly evaluating students’ work. In many situations, this is impractical because of the number of students and the schedules of students and teachers. If we focus on students’ attitude towards the integration of new learning tools in the learning process, the high-level of satisfaction detected have been noticed through tutorships during the academic year. Time dedicated by students to study the subject increased in the group who used DIBUTEC and even more in GrC who also used training videos. Although it is not possible to accurately measure the time dedicated by those students who received the course in a traditional lecture form (we only have the result of the final test of GrA), it is true that the number of on-site tutorships and the number of questions sent by email was bigger for GrC (40% more in both cases). Students of GrA spent most of their on-site tutorship on checking and correcting proposed exercises. In current e-learning environments, it would be quite difficult to achieve a level of student work evaluation as high as that achieved in this study. Additionally, as stated by Al-Musharraf, and Alkhattabi (2016) “there is increasing interest in analysing valuable information in educational databases to learn more about student behavior and the factors that have an effect on learners’ performance.” The combination of tools presented in this article motivates student, and at the same time, is a powerful resource to track the learning evolution of each student. This tracking is useful for both the student and the teacher. However, it is difficult to get this information without proper tools, especially in subjects with high use of graphical methods. The tools used in this paper could solve this problem. On the other hand, the use of videos as complementary support for the learning process has been recognised by students as a very useful tool for recognising or understanding interactively how a certain technique is applied to the resolution of an exercise. In addition, the fact that the video playback tool is integrated into the platform was an additional advantage because it prevented them from switching between applications and, in short, avoided a certain source of distraction. Students could pause the video playback at any time and continue the exercise process without causing discomfort or altering his routine work. This was one of the aspects that was most valued by students. They were accustomed to switching their attention between two different applications, which slowed down the learning process and reduced students’ motivation. Another great advantage of interactive e-learning environments is the ease of customising content for each student. Research presented by Jose Fabregas-Ruesgas, Hernandez-Abad, Hernandez-Abad and Rojas-Sola (2015) referred to the need “to adapt the learning-teaching model to the profile of the students”. Developing content adapted to a student’s educational needs and not forcing the student to adapt himself to contents provided could be another interesting way of improving the current learning process. However, customised content is hard to implement in subjects which require graphical resolution methods. In the end, most of the teachers abandon this idea. The interactive environment presented in this article, DIBUTEC, has tools to easily customise and adapt contents for each student. It is important to remark that the time required to customise content is mainly invested in generating videos and not so much in preparing self-evaluation exercises. For this reason, DIBUTEC has been prepared, if necessary, to use external visual resources hosted on other platforms such as YouTube, Vimeo, etc. In this sense, teachers can take advantage of the resources of other authors as long as they have the appropriate permissions. The wide spread of the Massive Online Open Course (MOOC) for self-learning that we have witnessed in recent years is in undeniable. This type of course has an increasing educative value. Access to resources (documents or multimedia) and schedules are more flexible than conventional lectures. However, as MOOC are currently developed, learning of subjects which require graphical methods for solving exercises will be slow and impractical. As mentioned before, it will require an expert to evaluate student work. Self-assessment tools provided by MOOC platforms are not ready to fulfil requirements of this type of subject. Therefore, the self-assessment tool proposed in this paper would be of great value for these platforms. Finally, it is important to indicate that teacher support was crucial for the learning process and for all the groups. However, its support was most critical for GrA. While for GrB and GrC the main task of the teacher was to clarify any doubts regarding proposed exercises, with GrA he also had to correct exercises. Therefore, the methodology proposed does not try to substitute teachers with self-learning interactive tools. They only change their role in the learning process. Conventional lectures are still adequate to teach fundamental concepts, but then its role becomes minor. This is in line with the role of the teacher as proposed by the European Higher Education Area (EHEA). Additionally, as mentioned before, a teacher has access to all of a student’s session records. With this information, the teacher can conduct personalised follow-ups with each student, identify the most challenging concepts, prepare customised reinforcement tasks, and even motivate students to continue with their learning. Conclusion Current e-learning environments, such as Blackboard and Moodle, are not prepared to support subjects which need graphical assessment tools. Self-assessment tools are necessary to automatise evaluation in a precise and narrow manner. However, current self-assessment tools are not effective, and most of the time, the teacher must manually evaluate each exercise. Nowadays, online video is a common teaching and self-learning tool. However, it alone cannot guarantee the learning process in subjects where students must solve problems graphically. A student, after watching a video, can try to resolve a similar exercise, but until it is revised by an expert, he will not know the degree of accuracy achieved or the mistakes made. The research proposed in this article combined a web-based self-assessment and training CAD enviroment with videos to improve the learning process of subjects with a high dependency on graphical solving methods. An e-learning environment combining both tools has been developed to identify advantages and disadvantages of this innovative methodology. Considering the results achieved, this methodology can improve the learning-teaching process for subjects such as engineering drawing. Moreover, it can be used in any subject with high graphic demands for human-computer interaction. Additionally, this combination can spawn positive synergies between teachers and students. Students can get real-time feedback on their mistakes without human intervention. This is an outstanding improvement as students can track their learning process and identify which videos are more useful and which concepts are harder to learn. In the end, this methodology increases student motivation.