Instructional Design for a Virtual Teaching-Learning Environment (VTLE): Process, Structure And Validation By Experts

Copyright © 2022 by Author/s and Licensed by IADITI. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Journal of Information Systems Engineering and Management 2022, 7(4), 18027 e-ISSN: 2468-4376 https://www.jisem-journal.com/ Research Article Instructional Design for a Virtual Teaching-Learning Environment (VTLE): Process, Structure And Validation By Experts Norka Bedregal-Alpaca1 *, Doris Tupacyupanqui-Jaén1, Lucy Delgado-Barra1, Karim Guevara1, Leticia Laura-Ochoa1 1 Universidad Nacional de San Agustín de Arequipa, Arequipa, Perú * Corresponding Author: [email protected]du.pe Citation: Bedregal-Alpaca, N., Tupacyupanqui-Jaén, D., Delgado-Barra, L., Guevara, K., & Laura-Ochoa, L. (2022). Instructional Design for a Virtual Teaching-Learning Environment (VTLE): Process, Structure And Validation By Experts. Journal of Information Systems Engineering and Management, 7(4), 18027. https://doi.org/10.55267/iadt.07.12535 ARTICLE INFO ABSTRACT Received: 02 Sep 2022 Accepted: 15 Oct 2022 To design and implement quality training actions, teachers follow a conscious and nonroutine process, so it is useful to have a model that standardizes and guides this process. Courses designed for virtual environments must respond to models focused on student learning, so a design based on the constructivist approach is proposed that identifies the learning objectives, groups the contents in to units, involves the student in learning and evaluation activities; proposes timely feedback and promotes the transfer of knowledge and the development of skills and attitudes. To validate the proposed design, 10 experts were selected. To qualify them, t he coefficient of expert competence was calculated and to collect their appreciation of the proposed instructional design, a questionnaire with Likerttype scale questions was applied. The results show that the general assessment of the experts is positive and consistent with the proposal. Keywords: Instructional design; virtual teaching-learning environments; constructivist approach; expert judgment. INTRODUCTION The development of ICT has generated a new global order in different areas, the educational in its training process has been strongly influenced by this phenomenon. ICT-supported training processes are a training alternative at all educational levels. “With the advent of new technologies, the emphasis of the teaching profession has changed from a teacher-centered approach to primarily student-centered training within an interactive learning environment” (UNESCO, 2005). To facilitate this transition, technology is being integrated into training processes, either to complement the face-to-face model or to implement a distance model. For (Gros Salvat et al., 2011) repeating a traditional teaching model, based on the master class, is no longer viable. The integration of technologies poses challenges to students and teachers, the virtual teachinglearning environments that are formed are more complex, so teachers must be in constant training and educational institutions in constant renewal. Only then will they be able to meet the demands of today's society (Meneses, 2009). The tools linked to web 2.0 such as Virtual Educational Platforms (LMS) allow the implementation of cooperative learning activities through strategies such as forums, wikis, among others. “ICTs support changes —or innovations— in traditional pedagogical practices. A good practice of ICT use is one that not only replaces previous practices, but also fundamentally contributes to a change that provides added value to the pedagogical process” (Sunkel et al., 2012). For (Llorens Largo, 2012), ICTs are among the external elements with the greatest impact on education; however, they must be assumed and used as support tools in the training processes. Bedregal-Alpaca N. et al. / J INFORM SYSTEMS ENG, 7(4), 18027 2 / 9 There are several studies related to VTLE, (Herrera, 2005) presents an instructional model focused on the basic functions of learning technologies. (Carrillo, 2018) designs a virtual learning environment mediated by the Moodle platform, to support the teaching of mathematics in secondary education. (Oswaldo Troncoso et al., 2010) use Web technologies as enriching elements of the training process. (Vásquez, 2016), describes the Salamanca Model (MoSal-b) to implement blearning in higher education. (Juca Maldonado et al., 2020) analyze the phases that must be implemented in a b-learning model and affirm that Moodle is the ideal platform to achieve what the model proposes. (Bedregal-Alpaca, 2019) designs a virtual environment based on b-learning to favor virtual tutoring. On the other hand, according to (Salinas, 2005), to ensure that work in an VTLE favors learning, the concurrence of three functions is required: pedagogical, technological and organizational. In addition, in a VTLE it is possible to comply with the recommendations given by (Twigg, 2007) in relation to the principles that must be taken into account for the implementation of an effective virtual course. Thus, contact between students and teachers can be promoted and reciprocity and cooperation between peers can be developed using some features of the LMS such as email, electronic video conference, chat, forums, blogs and wikis. Furthermore, functionalities that make it possible to provide timely feedback, interactivity and provide personalized features according to the learning levels achieved by students are of the utmost importance. In this order of ideas, the objective of this work is to present the process followed in the design, structure and validation of a Virtual Teaching-Learning Environment (VTLE) so that interaction spaces are generated that facilitate the teachinglearning processes, spaces that transcend beyond the physical classroom and that provide characteristics of interactivity that place the student at the center of the training process. METHODOLOGY Type of research: This is applied research as it aims to improve the quality of life and contribute to the construction of new knowledge. The case study method was chosen based on the proposal of (Stake, 2007), which allowed us to take as a particular case an online course under the b-learning modality. Phases: The first phase comprises a documentary review of primary and secondary sources, the theories and models that are the theoretical basis of the research were consulted. Information on virtual teaching-learning environments, instructional design and didactic strategies was structured. In the second phase, the most important aspects that could contribute to the design of the Virtual Classroom were identified: analysis of the educational problem, design and development. In the third phase, the technique of expert judgement was used to validate the proposed instructional design. Selection of experts: According to (Maxwell, 1996), a deliberate selection was made in order to identify potential experts, i.e. the people selected met the criteria of accessibility, greater proximity to the subject of instructional design and to virtual teaching-learning environments. Ten experts were selected who met the above criteria. Collection techniques: A survey was applied to the experts to find out their opinion on the characteristics of the proposal. Data analysis: Process that consists of systematically organizing the data collected during the research, in order for the researcher to achieve the findings (Flores and Valenzuela, 2012). Thus, in order to qualify the experts, the coefficient of expert competence was calculated and to collect their appreciation of the proposed instructional design, a questionnaire with 5-level Likert-type scale questions was applied. The questions were organized into 6 dimensions: Teaching objectives and competences, pedagogical approach, actors and communication, strategies and contents, academic activities and evaluation and ICT integration. The answers obtained in the survey were analyzed using statistical techniques. THE PROPOSAL The implementation of VTLE is an opportunity to innovate in the formative process. For VTLE to be efficient, according to (Eradze and Laanpere, 2014), it must be designed, taking into account pedagogical aspects such as: teaching methods, organization and presentation of contents, techniques and processes for evaluation, type of interaction between students and teachers. The design of an VTLE must have a pedagogical approach that guides the didactic and methodological strategies to promote meaningful learning in students (Aguirre and Griffin, 2012). The ADDIE model was followed for the development of the proposal; this model is associated with quality design (McGriff, 2000). Figure 1 summarizes the proposed instructional design, the elements analyzed and the different work moments. 3 / 9 Bedregal-Alpaca N. et al. / J INFORM SYSTEMS ENG, 7(4), 18027 Figure 1 . Instructional Design Proposal for Virtual Teaching-Learning Environments Table 1. Learning practices 1 Learning practice Resource Purpose Readings Digital texts, links, downloadable files Present a study document (in PDF format) to guide the student. It should be proposed in each of the study units. It will serve as the basis for evaluating the appropriation of concepts and/or the performance of tasks. Research Links to web sites, files, downloadable, shared documents, videos, search engines The research can be done individually or in teams. If the research is individual, a monograph, an essay, an information organizer can be commissioned. If the research is carried out in teams, the creation of a wiki, a website, a blog can be proposed; students work collaboratively on a specific topic. Guided dialogue Discussion forums, e-mail, messaging. In a forum, the professor must propose a topic and ask for voluntary or compulsory participation of the students. The professor also acts as moderator of the students' participation. Case study Links to websites, downloadable files, videos, search engines, etc. It can be done individually, but it is preferable to work in teams. The professor proposes a relevant case, the students must apply the knowledge acquired to the solution of the case. The structure of the report must be defined in advance. The report must be made available to all students participating in the experience. Table 2. Educational material for the virtual classroom Material Purpose Examples Digital files Files whose content is related to the subject of study. It can include text, images, diagrams, etc. Material prepared by the professor. Material elaborated by other authors. Academic articles. Infographics Videos, audios Audiovisual material whose content is related to the topic of study and allows for an in-depth study of it. Audiovisual material prepared by the professor. Audiovisual material elaborated by other authors. Links URL that serves as a link to some element on the Web: website, downloadable file. Links to material that expands the basic information for the subject: books, articles, magazines, etc. Links to specialized websites on the subject. Forums, chat, instant messaging, email Synchronous and asynchronous communication between classroom users. Forums for discussions, questions, queries, file uploading, etc. Chat and mail for communication and messaging, sending of tasks, etc. Graphics, images, navigation buttons. Concise graphic material that conveys information visually, graphically, quickly, and easily assimilated. Presentations. Infographics. Visual organizers. Statistical graphs. Comics Search engines and databases Tools for searching (on the web) for specific information useful for the student's training. Search engines for scientific and academic information. Libraries, institutional repositories, etc. 1 Own design, adapted from Peter Van Pol, 2012. A typology of e-learning practices. Virtual EDUCA Bedregal-Alpaca N. et al. / J INFORM SYSTEMS ENG, 7(4), 18027 4 / 9 Analysis of the educational problem The investigation begins by identifying the problem and its possible causes, the context is described, and a solution is proposed. In general, the training processes are carried out under the face-to-face model, the meetings between teachers and students are held in classrooms, generally with the master class strategy, supported by projection equipment and audiovisual material. In the current context of the COVID-19 pandemic, the confinement forced to migrate towards an emergency remote education model in which physical classrooms were replaced by synchronous sessions conducted through a videoconferencing system. In this context, the need to integrate the b-learning modality and the design and implementation of virtual teaching-learning environments (VTLE) was detected. The next step is to define the instructional goals and the topics to be taught, the minimum contents to be developed should be defined considering the time and the content units. Also, the material and human resources necessary for the implementation of the VTLE must be identified. VTLE design Based on the results of the analysis phase, the training strategy is planned (Meza, 2013), for which the following objectives should be considered: • Design activities, focused on student learning. • Plan the design, structure, resources and activities so that they respond to the learning outcomes defined in the curriculum • Identify the resources and activities offered by the LMS in which the VTLE will be worked and that will support the development of the subject. Suggested learning practices are listed in Table 1. It is important to identify the educational material that will facilitate learning and will be placed in the virtual classroom. Table 2 shows examples that can be used. Considering the four environments that occur in an VTLE: knowledge, collaboration, counseling, and experimentation, following (Herrera, 2005), it is necessary to define the conceptual elements of virtual learning-teaching environments: instructional design and interface design. Table 3 presents the guidelines on which the instructional design is based. Table 4 shows the guidelines followed in the design of the virtual classroom interface. Table 3. Instructional model guidelines Instructional Design Guidelines Suggested activities Activate the processes of assimilation and accommodation of knowledge. Encourage the student to search and process information. Propose challenges that can be overcome by students. Confrontation of previous knowledge or common sense. Use of the debate technique. Proposal of information sources with opposing approaches, etc. Search, analysis, synthesis, and comparison of information. Elaboration of sustained personal opinions. Activities that are challenging and related to knowledge, time conditions, resources and possibilities. Establish dynamic interaction processes. Promote the development of critical thinking skills Stimulate self-learning and the development of lifelong learning skills. Activities that engage substantiated personal or group opinions. Timely feedback. Activities involving processes of observation, relationship, comparison, deductiveinductive reasoning, etc. Activities that involve previous experiences and learning. Activities that develop ICT skills for self-learning. Activate the processes of assimilation and accommodation of knowledge. Encourage the student to search and process information. Propose challenges that can be overcome by students. Confrontation of previous knowledge or common sense. Use of the debate technique. Proposal of information sources with opposing approaches, etc. Search, analysis, synthesis, and comparison of information. Elaboration of sustained personal opinions. Activities that are challenging and related to knowledge, time conditions, resources and possibilities. Establish dynamic interaction processes. Activities that engage substantiated personal or group opinions. Timely feedback. Promote the development of critical thinking skills Stimulate self-learning and the development of lifelong learning skills. Activate the processes of assimilation and accommodation of knowledge. Activities involving processes of observation, relationship, comparison, deductiveinductive reasoning, etc. Activities that involve previous experiences and learning. Activities that develop ICT skills for self-learning. Confrontation of previous knowledge or common sense. Use of the debate technique. Proposal of information sources with opposing approaches, etc. Encourage the student to search and process information. Search, analysis, synthesis, and comparison of information. Elaboration of sustained personal opinions. 5 / 9 Bedregal-Alpaca N. et al. / J INFORM SYSTEMS ENG, 7(4), 18027 Table 4. Interface design guidelines Interface design guidelines Required elements Promote access to the social environment, considering the elements of the LMS that will allow the student to interact. Encourage access to the natural environment, considering the elements of the LMS with which the student will identify the activities to be performed. Provide access to the documentary context, considering documents that allow the student to have references of the topic to be covered. Interaction tools such as email, discussion forums, videoconference, chat, etc. Images that have educational purposes or that activate the student's memory. Animations to explain some content or concept. Simulators and virtual reality when practical activities are needed within the learning environment. Information bases with the necessary contents for the learning. Hypertext when there are readings or when it is necessary to go deeper into a topic. Recommended readings. Management of attentional resources, considering dosed information and diverse stimuli or perceptual channels to capture the student's attention. Elimination of unnecessary information. Use of simple and coherent navigation schemes. Avoidance of difficult to read texts. Management of motivational resources, considering the elements that should be incorporated into the LMS to motivate student work. Emphasis on relevant aspects. Dosage of information. Inhibition of interference from the environment. Varied stimuli. Different perceptual channels. Intentional use of animations. Elimination of unnecessary elements. Planned management of visual elements. Motivational elements that do not become distractors. Interface design guidelines Promote access to the social environment, considering the elements of the LMS that will allow the student to interact. Required elements Interaction tools such as email, discussion forums, videoconference, chat, etc. Encourage access to the natural environment, considering the elements of the LMS with which the student will identify the activities to be performed. Images that have educational purposes or that activate the student's memory. Animations to explain some content or concept. Simulators and virtual reality when practical activities are needed within the learning environment. Provide access to the documentary context, considering documents that allow the student to have references of the topic to be covered. Information bases with the necessary contents for the learning. Hypertext when there are readings or when it is necessary to go deeper into a topic. Recommended readings. Figure 2. Proposed structure for units and topic Virtual Classroom Structure Aspects such as interaction, language use and visual communication are considered. Both the structure and the interface are supported by didactic proposals that promote student learning outcomes. At least 5 types of modules have been considered, being possible to add other types depending on the nature of the training program. The modules considered are presentation, learning, final project, assessment, and surveys (Figure 1).The presentation module serves to publicize the characteristics of the subject, the way in which it will be developed and other elements that constitute the pedagogical and didactic proposal. The following elements have been considered: introductory paragraph and/or welcome to the course, identification of the teacher who will teach the course, presentation of the subject syllabus, explanation of the virtual classroom model and explanation of the communication tools. The learning modules will be used to develop a unit and/or topic (Figure 2). The student will use the contents of the module to acquire knowledge of the topic in question. For a learning module, the following elements have been considered: title, objectives and/or summary, educational material related to the content to be developed, activities in accordance with the didactic strategy defined for the course and to provide spaces for interaction. Follow-up and learning evaluation activities should be contemplated. The purpose of the final or integrative project module is for the student to develop or apply the knowledge acquired. The project can be done individually or in teams. The teacher will determine the structure of the module according to the nature of the subject and the project. Basically, it will gather the informational elements of the presentation module and the structure of the learning modules. The design of this integrating activity must contemplate the integration of the contents developed, promote the transfer of knowledge, and the development of the competencies established in the syllabus. It is convenient to include an evaluation module whose structure will depend on the design of the evaluation process. The purposes of the evaluation in teaching-learning processes have common characteristics both in the face-to-face modality (synchronous) and in the virtual modality (asynchronous). The main characteristic is that it must be designed considering the principles of the institutional educational model and the learning objectives defined in the curriculum. The following can be considered: questionnaires, electronic portfolios, integrative activities, discussion forums, elaboration of posters or academic articles, etc. An optional module is proposed, that of opinion surveys, which is useful for gathering the students' perception regarding an activity, the educational material provided, the teaching strategies used, etc. The structure of this module is free and is conditioned by the nature of the information that the teacher wishes to collect. Bedregal-Alpaca N. et al. / J INFORM SYSTEMS ENG, 7(4), 18027 6 / 9 Usability aspects For Molenda (1997), it is convenient to consider some aspects that guarantee a friendly interaction with the work environment, so the following aspects have been considered: a) Minimum graphic elements: Main header of the course, header of topics, units or modules, labels for subtitles or subtopics that group the resources on the platform. b) Uniformity: Define and apply a common graphic style for the entire course and define a format for texts, tables, and description of activities, establish the font(s) to be used, the size of titles and subtitles (Fonts such as Arial, Verdana, Tahoma are recommended). c) Ease of location: Use an internal title in the resources for information and to propose activities. In these titles, use capital letters only at the beginning of the titles. Place the course management blocks (people, calendar, credits) on one side of the screen, on the right or on the left. d) Global navigability: Use labels to separate and group common resources presented in the modules. The labels allow a clear presentation of the content of the module and facilitate the navigation of the users, as well as a better visualization of the space. Documents and information resources should open in a pop-up window. Planning of Teaching-Learning Processes The terms and conditions to carry out the activities must be clearly established, taking into account that collaborative activities require more time. Following (Twigg, 2007), a set of guidelines is proposed: (a) Encourage contact between students and teachers, (b) Create spaces for cooperation, (c) Implement active learning methodologies, (d) Give timely feedback, (e) Consider the time required to complete an activity, (f) Establish high performance standards, (g) High expectations of individual and collective achievement and (h) Consider the existence of diverse talents and learning styles in students. RESULTS AND DISCUSSION In order to assess the proposed model, we used expert judgement, a strategy used to contrast content validity. This estimates the degree of agreement between experts, who give their assessments of the indicators or items of the tool (Bruna et al., 2019; Souza et al., 2017). For (Almenara and Cejudo, 2013), the number of experts depends on the ease of access to them or the possibility of knowing enough experts on the object of the research. (Escobar-Pérez and Cuervo-Martínez, 2008) propose that the number of evaluators depends on the level of expertise and the diversity of their knowledge. The selection of experts was made by means of non-probabilistic purposive sampling, including criteria such as accessibility, availability, impartiality, independence, teaching and research experience in the field of didactics and pedagogy in Higher Education. Based on the information related to the experts' backgrounds, they were invited to participate by e-mail. A first instrument was then sent to them to collect the information necessary to determine the Expert Competence Coefficient. In order to calculate this coefficient, the people selected as experts were asked to indicate their level of knowledge about the object of research (knowledge coefficient (kc)), as well as the sources that allow them to argue and justify this level (argumentation or substantiation coefficient (Ka)). The results obtained are shown in Table 5. The level of competence of each expert was regulated by the following levels: if 0. 8 < k ≤ 1.0, then the level of competence is high, if 0.7 ≤ k ≤ 0. 8, then the level of competence is medium and if 0.5 ≤ k < 0.7, then the level of competence is low. To define and conceptualize the criteria that serve as evaluation parameters for the experts, the work of (Villa and Figueredo, 2015) and (Harris et al., 2010) were considered. Twenty-five questions grouped into six dimensions were developed. It was decided to use a Likert-type scale to collect expert opinion and following (Villa and Figueredo, 2015) a 5level Likert scale was defined. The levels identified and the weights assigned to each level were: Never (1), Almost never (2), Sometimes (3), Almost always (4), Always (5) points. Table 6 shows the number of times the experts selected a level for each item of the questionnaire and the average score obtained for the item. The maximum score an expert could give the classroom was 125 points. Figure 3. Score given by the experts to the virtual classroom The maximum score an expert could give the classroom was 125 points. Figure 3 shows the scores awarded by each of the experts. When calculating the average of the scores obtained for the 25 items, the value 4.392 is obtained; a value that is well above the scale average of 3 points, 17 of the 25 items exceeded the average (4.392), one of them obtained a score very close to the average and the remaining 7 were below the overall average. However, the lowest rated item (item 15) exceeds the scale average. It can therefore be concluded that the overall assessment of the experts is positive and in agreement with the proposal. We agree with the works of (Bedregal-Alpaca et al., 2020b) and (Cornejo-Aparicio et al., 2019) in that it is imperative that teachers design activities aimed at the appropriation of knowledge and its creative application in daily practice. An advantage of implementing a virtual course on an LMS is that, due to the large amount of data it generates, data mining processes can be implemented to help understand the characteristics of the learning process (Bedregal-Alpaca et al., 2020a). 7 / 9 Bedregal-Alpaca N. et al. / J INFORM SYSTEMS ENG, 7(4), 18027 Table 5. Determination of the coefficient of expert competence (k) and the level of competence of each expert. Expert kc ka k Competence level Expert kc ka k Competence level E-01 0.8 0.85 0.825 High E-06 0.8 0.85 0.825 High E-02 0.9 0.99 0.945 High E-07 0.9 0.98 0.94 High E-03 0.9 0.98 0.94 High E-08 0.8 0.96 0.88 High E-04 0.7 0.83 0.765 Medium E-09 0.8 0.9 0.85 High E-05 0.9 0.95 0.925 High E-10 0.9 1 0.95 High Table 6. Scores obtained for each questionnaire item Dimensions (1) (2) (3) (4) (5) Average Dimension 1: Teaching objectives 01 The VTLE has a clear pedagogical objective and intention 0 0 0 5 5 4.5 02 The objective of the VTLE is expressed in the content 0 0 0 1 9 4.9 03 The VTLE proposes the development of some competence for the 21st century 0 0 0 0 10 5 04 The competence(s) are articulated with the academic activities 0 0 0 5 5 4.5 Dimension 2: Pedagogical approach 05 The VTLE is based on a learning approach 0 0 0 6 4 4.4 06 The learning approach is evidenced in the academic activities of the VTLE 0 0 0 4 6 4.6 07 The VTLE facilitates meaningful learning and contains an activity that enhances it 0 0 0 0 10 5 08 The VTLE promotes discovery learning 0 0 0 10 0 4 Dimension 3: Actors and communication 09 The VTLE has and demonstrates the role of the student 0 0 0 5 5 4.5 10 The VTLE has and demonstrates the role of the teacher 0 0 0 4 6 4.6 11 The VTLE favors communication between the participants 0 0 3 6 1 3.8 12 In the VTLE an appropriate academic language is perceived for the students and the level of studies 0 0 0 0 10 5 Dimension 4: Strategies and contents 13 Didactic techniques and strategies are evident in the VTLE 0 0 0 6 4 4.4 14 The VTLE follows a sequence and is coherent with the pedagogical intention 0 0 0 6 4 4.4 15 The central themes are more relevant within the VTLE and are emphasized 0 0 6 4 0 3.4 16 The contents of the VTLE correspond to the programmatic content 0 0 0 2 8 4.8 17 The contents, which are not the property of the teacher, copyright is recognized 0 0 1 9 0 3.9 Dimension 5: Academic activities and evaluation 18 The academic activities of the VTLE mention the learning objective 0 0 4 6 0 3.6 19 The academic activities of the VTLE are motivating for the student 0 0 2 4 4 4.2 20 The VTLE describes the dynamics of the activity and provides clear instructions for its development 0 0 0 0 10 5 21 The VTLE contains information on how academic activities will be evaluated 0 0 4 6 0 3.6 22 The VTLE uses elements that promote self-assessment, co-assessment and hetero-assessment 0 0 3 7 0 3.7 Dimension 6: ICT integration 23 The educational resources used are justified for the teaching of the subject 0 0 0 5 5 4.5 24 The use of technological tools supports the teaching strategies 0 0 0 3 7 4.7 25 Content, teaching strategies and technological tools are articulated with each other within the didactic planning 0 0 0 2 8 4.8 Bedregal-Alpaca N. et al. / J INFORM SYSTEMS ENG, 7(4), 18027 8 / 9 CONCLUSIONS To approach the design of virtual teaching-learning environments requires knowledge of the rudiments of content development, as well as a mastery of contemporary pedagogical foundations and trends and of taxonomies that describe the design process. Instructional design is a key factor in the teaching-learning process; the success and retention of the student in virtual courses depends to a large extent on the correct selection of the instructional model, as well as on creativity, adequate writing, presentation of clear instructions in the learning guide or learning path. For the teacher, designing and implementing a course supported by ICT should generate a space of motivation and incentive for the development of educational materials aimed at strengthening student learning. Acknowledgements This work was carried out in the framework of the project "Adaptive SPOC for the training of university teachers in active methodologies" with contract Nº IBA-26-2021-UNSA. The authors would like to thank the Universidad Nacional de San Agustín de Arequipa for the support received in the execution of this project. REFERENCES Aguirre, I., Griffin, Y., 2012. Diseño de un modelo pedagógico– didáctico para el aprendizaje en línea. Presented at the XIII Encuentro Internacional Virtual Educa, Panamá. Almenara, J.C., Cejudo, M. del C.L., 2013. La aplicación del juicio de experto como técnica de evaluación de las tecnologías de la información y comunicación (TIC). Revista Eduweb 7, 11–22. Bedregal-Alpaca, N., 2019. Tutoría virtual y blendedlearning en el posgrado: Orientaciones y resultados de una experiencia. Presented at the Industry, Innovation, and Infrastructure for Sustainable Cities and Communities: Proceedings of the 17th LACCEI International MultiConference for Engineering, Education and Technology. https://doi.org/DOI: 10.18687/LACCEI2019.1.1.220 Bedregal-Alpaca, N., Aruquipa-Velazco, D., Cornejo-Aparicio, V., 2020a. Data mining techniques to extract academic behavior profiles and predict university desertion. Revista Ibérica de Sistemas e Tecnologias de Informaçao 592–604. Bedregal-Alpaca, N., Padron-Alvarez, A., Castañeda-Huaman, E., Cornejo-Aparicio, V., 2020b. Design of cooperative activities in teaching-learning university subjects: Elaboration of a proposal. International Journal of Advanced Computer Science and Applications 11. Bruna, C.E., Villarroel, V.A., Bruna, D.V., Martínez, J.A., 2019. Experiencia de diseño y uso de una rúbrica para evaluar informes de laboratorio en formato publicación científica. Formación universitaria 12, 17–28. https://doi.org/10.4067/S0718-50062019000200017 Carrillo, J.S.A., 2018. Entorno virtual de aprendizaje: una herramienta de apoyo para la enseñanza de las matemáticas. Revista de Investigación en Tecnologías de la Información: RITI 6, 34–39. Cornejo-Aparicio, V., Flores-Silva, S., Bedregal-Alpaca, N., Tupacyupanqui-Jaén, D., 2019. Capstone courses under the PBL methodology approach, for engineering. Presented at the 2019 IEEE World Conference on Engineering Education (EDUNINE), IEEE, pp. 1–6. Eradze, M., Laanpere, M., 2014. Interrelation between pedagogical design and learning interaction patterns in different virtual learning environments. Presented at the International Conference on Learning and Collaboration Technologies, Springer, pp. 23–32. Escobar-Pérez, J., Cuervo-Martínez, Á., 2008. Validez de contenido y juicio de expertos: una aproximación a su utilización. Avances en medición 6, 27–36. Flores, M., Valenzuela, J., 2012. Fundamentos de investigación educativa. México: Editorial digital Tecnológico de Monterrey. Gros Salvat, B., Sancho Vinuesa, T., Borges Sáiz, F., Bautista Pérez, G., García González, I., López Pérez, C., Mas García, X., Lara Navarra, P., 2011. Evolución y retos de la educación virtual: construyendo el e-learning del siglo XXI. Harris, J., Grandgenett, N., Hofer, M., 2010. Rúbrica de evaluación de integración de tecnología. Herrera, B., 2005. Consideraciones para el diseño didáctico de ambientes virtuales de aprendizaje: una propuesta basada en las funciones cognitivas del aprendizaje. Revista Iberoamericana de educación 38, 2. Juca Maldonado, F., Carrión González, J., Juca Abril, A., 2020. BLearning y Moodle como estrategia en la educación universitaria. Conrado 16, 215–220. Llorens Largo, F., 2012. Tendencias TIC para el apoyo a la Docencia Universitaria. Conferencia de Rectores de las Universidades Españolas (CRUE). Maxwell, J.A., 1996. Qualitative research design: An interactive approach. Sage publications. McGriff, S.J., 2000. Instructional system design (ISD): Using the ADDIE model. Retrieved June 10, 513–553. Meneses, E.Q., 2009. Recursos didácticos digitales: medios innovadores para el trabajo colaborativo en línea. Revista Electrónica Educare 13, 47–62. Meza, J., 2013. Modelo pedagógico para proyectos de formación virtual. Giz. Oswaldo Troncoso, R., Avila, M.C., Chourio, E.D., 2010. El modelo b-learning aplicado a la enseñanza del curso de matemática I en la carrera de ingeniería civil. Revista Electrónica" Actualidades Investigativas en Educación" 9 / 9 Bedregal-Alpaca N. et al. / J INFORM SYSTEMS ENG, 7(4), 18027 10, 1–28. Salinas, J., 2005. La gestión de los entornos virtuales de formación. Seminario Internacional: La calidad de la formación en red en el Espacio Europeo de Educación Superior 1–21. Souza, A.C. de, Alexandre, N.M.C., Guirardello, E. de B., 2017. Psychometric properties in instruments evaluation of reliability and validity. Epidemiologia e servicos de saude 26, 649–659. Stake, R., 2007. Investigación con estudios de caso (Cuarta Edición ed.). Madrid: Ediciones Morata, SL. Sunkel, G., Trucco, D., CEPAL, N., 2012. Las tecnologías digitales frente a los desafíos de una educación inclusiva en América Latina: Algunos casos de buenas prácticas. Twigg, C.A., 2007. Improving learning and reducing costs: Program outcomes and lessons learned from the roadmap to redesign. Saratoga Springs, NY: National Center for Academic Transformation. UNESCO, 2005. Las tecnologías de la información y la comunicación en la enseñanza: Manual para docentes o Cómo crear nuevos entornos de aprendizaje abierto por medio de las TIC. Vásquez, M., 2016. Propuesta de modelo pedagógico b-learning para educación superior. Revista de Educación Andrés Bello 29–53. Villa, E.J.E., Figueredo, O.R.B., 2015. Hacia una propuesta para evaluar ambientes virtuales de aprendizaje en educación superior. Academia y virtualidad 8, 14–23.