Integrated value model to assess the sustainability of active learning activities and strategies in architecture lectures for large groups

Sustainability 2019, 11, 2917; doi:10.3390/su11102917 www.mdpi.com/journal/sustainability Article Integrated Value Model to Assess the Sustainability of Active Learning Activities and Strategies in Architecture Lectures for Large Groups Oriol Pons 1 ,*, Jordi Franquesa 2 and S. M. Amin Hosseini 3 1 Department of Architectural Technology, The Polytechnic University of Catalonia (UPC), Av. Diagonal 649, 08028 Barcelona, Spain 2 Department of Urbanism and Regional Planning, UPC, Av. Diagonal 649, 08028 Barcelona, Spain; [email protected] 3 Department of Civil and Environmental Engineering, UPC, Jordi Girona 1-3, 08034 Barcelona, Spain; [email protected] * Correspondence: [email protected]; Tel.: +34-93-4016391 Received: 9 April 2019; Accepted: 14 May 2019; Published: 22 May 2019 Abstract: At present, university professors lack the tools to know which is the most sustainable activity and/or strategy that should be incorporated into large-group theoretical classes in order to improve our students’ learning process whilst taking each scenario into account. These scenarios have different order thinking levels, numbers of students, available time, classroom size and professor skills, among other factors to consider. In architecture schools we have this problem in theoretical lectures. This project has developed and applied a new multi-criteria decision making tool incorporating a mathematical algorithm in order to choose the best set of active learning activities for each case for these lectures in architectural technology courses. This process has relied on seminars involving experts and the use of The Integrated Value Model for Sustainable Assessment. This tool has been very useful to solve the aforementioned problems because architecture professors have been able to choose the most sustainable activity for each scenario considering the alternative sustainability indexes. This first application has been highly useful to assist professors to incorporate active learning methodologies in their classes and to promote lecturers’ management of their course contents and time. Future improved versions of this tool will increase its interactivity and broaden its scope. Keywords: MCDM; MIVES; AHP; Knapsack; architecture; active learning; lectures; Bloom Taxonomy 1. Introduction Large group lectures are an ancient and traditional way of teaching [1,2] but continue to be an important part of university teaching activities at present. This occurs partly because universities have pragmatic reasons such as student ratios per course and professor, hours per course and space per course, especially in first-year classes [3–5]. An important part of these theoretical classes are given mainly as one-way teaching, in large groups of 100 students or more [6] and have a duration from 50 to 180 min [7]. These historic classes have been analyzed in numerous studies for decades [8,9] from diverse perspectives such as the duration of lectures depending on student attention capacities [10], lectures assessment and its improvement [11], student engagement [12] and classroom space configuration and size [13,14]. The educative community’s satisfaction is diverse, with studies in favor [15] and studies against these theoretical classes [16]. One of their main weaknesses are students’ passive role and their consequential low learning performance [17] to which some studies suggest new approaches for university courses based on laboratory lessons [18] and group work [19], or Sustainability 2019, 11, 2917 2 of 19 online editions [20] could prove beneficial, while numerous studies and resources suggest the need to introduce active strategies and methodologies [21]. In this sense, the main objectives in this project were: (a) to find the most sustainable active methodologies, tools, activities and strategies to promote deep learning and active roles by students during large group lectures and (b) develop a new tool to help professors choose the most sustainable active learning strategies for large groups attending lectures, taking into account different possible scenarios. This new model has been defined in detail and applied to the “Construction II” course at The Polytechnic University of Catalonia (UPC)-Barcelona Tech. In this sense, this paper presents previous related projects, the specific study case, the research methods for developing the new tool, the results of its application and analyzes these results to achieve first conclusions, recommendations and define future projects. 2. Literature Review There are numerous previous related studies focusing on improving large group lecture learning processes. There are books, articles and numerous conference papers found in technical literature about how to improve large group lectures. These publications collect resources, strategies, methodologies, activities and tricks to improve this type of pedagogical style, dating from the early 1990s. Table 1 classifies a representative sample of the most relevant publications for this research project. Table 1. Classification of scientific articles and books on improving learning processes for large group lectures. Main Research Area References Lecturer experience and training influence on lectures [22] Guide to organize, manage and teach a large group lecture [23,24] List of tips and strategies to improve lectures [4,25–32] Active learning to improve students’ participation [25,33–41] Active learning with limited resources [42] Advantages of large classes [9,43] Large classes assessment [11,44,45] There are also numerous universities which have online resources to improve the student learning process in classes which have large student attendance; for instance the University of Bath, University of Waterloo and Vanderbilt University [46–48]. These publications and resources are available to professors so they can study them and follow their recommendations. But this present study has not found in this literature review any index or assessment for these activities depending on numerous aspects that would be interesting to assess simultaneously. Nor have these researchers found any active resources either which can help professors to choose the best strategy or activity for each case considering the specific conditions of each scenario. 3. Identification of the Problem 3.1. Initial Diagnosis As said in the introduction, this research paper focuses on architecture schools and departments, where professors commonly lecture nowadays, mainly for theoretical courses. During the history of teaching and learning architecture there have been numerous and different approaches, such as BeauxArts and Bauhaus [49]. These approaches are still under debate at present [50] as well as the theory of architecture learning process and its relation to other disciplines [51]. Architecture studies have several areas of knowledge that commonly divide their study plan into courses such as design studio, history, aesthetics and technology, among other topics. This project aims to be applied to a broader sample in the future but has started with the architectural technologies practiced at the Barcelona Architecture School (ETSAB), Polytechnic University of Catalonia (UPC), because the authors of this project currently are professors and researchers at this institution. In this school, the aforementioned technologies have an important role, not predominant, but balanced in Sustainability 2019, 11, 2917 3 of 19 relation to the other areas so that students learn architecture from a holistic point of view [52]. Specifically, this project started with construction courses at this architectural school. In the last century several professors have prepared rigorous materials for these courses [53–56] and have carried out research about the teaching methods used, their problems and possible solutions [57]. This project analyzed the particular case study of “Construction II”. This is a compulsory undergraduate third year course, which aims that students understand the importance to incorporate constructive issues which buildings have during their design process. In this sense, this subject includes a high amount of crucial concepts related to architecture construction, from foundations to slab floors, including retaining walls and load bearing systems [58]. This course has four sessions per year, two each semester, one morning shift and one afternoon shift. Each shift has 80 students, with a total of 320 students per year. This research paper reconsidered the previous learning methodologies this course has employed, which have been three consecutive hours of lectures and two-hour practical sessions. These lectures have had three endemic problems during recent decades: (1) student attendance was really low, (2) students’ learning results during these lessons were low and superficial; and (3) students did not participate, did not take an active role. These problems had been detected and confirmed by professors’ observations during classes, students’ exams and practical exercises and specific research projects, as explained in detail by Pons et al. [59]. The first response in order to solve this problem was the introduction of active learning activities and strategies for theoretical classes designed for large groups, as presented in depth by Pons and Franquesa [60]. The professors who teach this course studied the publications and resources presented in the previous section in order to incorporate active strategies to this subject to solve the previously mentioned three endemic problems. This new strategic approach implied the use of videos, flipped classes, online contests, online questionnaires, theatrical explanations and cooperative activities, among others. The aforementioned incorporation followed an innovative method based on Bloom’s Taxonomy revised by Anderson [61]. This implementation collected several indicators about the three endemic problems which obtained satisfying results that still left room for further improvement, as shown in Table 2. Table 2. Some indicators assessed in 2017-2018 academic year and in previous courses. Criteria Indicators Before 2017–2018 Course 2017–2018 Student attendance Attendance average (%) 35% 70% Student satisfaction Participation (%) 21% 34% Lecturer is a good professor 1 2.2 3.9 Lecturer is receptive to students’ queries 1 2.5 4.1 Final results, grades 2 Students with grade ≥9 0% 3% Students with grade ≥7 and <8 1.8% 26% Students with grade ≥5 and <7 59.6% 58% Students with grade <5 24.6% 3% Number of students dropping out of the course 11.2% 10% 1 0–5 scale, 5 maximum satisfaction; 2 0–10 scale, 10 maximum satisfaction. The aforementioned incorporation also collected open satisfaction questionnaires from students regarding the following seven indicators: both students’ and professors’ dedication both in class and outside, students’ satisfaction and attendance and professors’ feedback time. The analysis of these questionnaires and seven issues results concluded that the applied active strategies had different strengths and weaknesses, which should be taken into account in future learning methodology applications. For example, this outcome could be considered by lecturers in order to decide the best active strategies for their courses. Another conclusion based on this research was that in order to integrate each of these seven variables, and even more, into a professor’s teaching strategy, a new methodology was necessary, as stated in the introduction to this article. Sustainability 2019, 11, 2917 4 of 19 3.2. Case Study The definition and application of this new tool has focused on the first semester “Construction II” 2018–2019 course, in both its morning and afternoon sessions. This course had 14 instruction sessions, 11 which were 3 h long and three sessions 2 h long. The morning section had 77 students while the afternoon section had 52 students. There were two different professors, one gave nine lectures and the other five. 4. Research Methods This new research project tool to assist lecturers has been defined during two seminars when experts participated, on 22 May 2018 and on 29 October 2018, at which different phases were discussed and the participants agreed to use tools such as analytic hierarchy process (AHP; Appendix A presents a complete list of abbreviations) (Saaty, 1990). These discussions relied on specific technical literature and previous research [60]. These seminars were comprised of members from several institutions: ETSAB university professors with different specialties, the vice director and two students; one La Salle architecture university professor; members from the Gilda research group on innovation in architecture learning and management; members of the UPC Science Education Institute ICE; members of the Education Department Cesire and a pedagogue from the research group Pedagogy, Society and Innovation with the support of the Information and Communication Technologies (PSITIC), part of the Blanquerna Education Faculty. These members were chosen because of their related and complementary expertise and experience in teaching architecture, specifically coordinating and studying at ETSAB, teaching architecture and other disciplines in other universities, carrying out research education and their knowledge about pedagogy respectively. In consequence, these experts were able to assist the authors in defining and improving this new tool, from the general perspective of higher education, research and pedagogy and, at the same time, take into account its first application in architectural studies. These seminars defined the six-phase tool presented in Figure 1, citing the specific actions, actors and methods in each phase. Figure 1. Six phase tool to assist lecturers and the actions, actors and methods in each phase. These six phases are explained in depth in the following paragraphs, while a general synthetic introduction is as follows: during the first two phases, the professor or teaching team analysed the course contents and classified the alternatives following the aforementioned new method; in phase three, the alternatives were assessed in seminars whose participants were experts who use a multi- Sustainability 2019, 11, 2917 5 of 19 criteria decision making (MCDM) method in order to be able to take into account multiple indicators; in phase four, the teaching team defined the available time in each session to incorporate active methodologies; in the fifth phase, a mathematical algorithm proposed sets of alternatives so that, in the last phase, the teaching team analysed the given proposals and made its final decision. 4.1. Phases 1 and 2: The New Method The first two phases followed a new method [60], which was introduced in the initial diagnosis section because it was previously defined and applied by the authors in a previous project. These phases assessed the course contents analysis and classified the active alternatives based on Bloom’s Taxonomy revised by Anderson. In consequence, this analysis and classification considered three order thinking levels (OTL) included in this taxonomy: lower order thinking level (LOTL), middle order thinking level (MOTL), high order thinking Level (HOTL). These OTLs differ from the thinking complexity required for students which is: low in LOTL—for example remembering and recounting concepts; middle level in MOTL—for instance applying and understanding ideas and information; and high in HOTL— such as analysing, evaluating and creating their own proposals. These first phases are presented in Figure 2, Table A2 summarizes the course contents and Table A3 lists the 25 alternatives. Figure 2. Classification of the 25 active alternatives based on Bloom’s Taxonomy revised by Anderson. L01–L08 were the most used LOTL alternatives because they are easy resources for professors to activate students’ participation and attention; they include alternatives such as: variations in professors’ explanation speed, tone and volume; questions to students and repetitions. M01, Moodle tests about the course contents, was the most used activity to improve MOTL, consisting of tests about the course contents prepared previously by professors using the Moodle platform so students could do them during the class individually or in groups, with the assistance of the professors and thus receiving immediate results. H02, individual practical exercise, combined with H05, project basic learning (PBL), were the most used alternatives in HOTL, as a type of PBL specifically designed for large groups. They Sustainability 2019, 11, 2917 6 of 19 consisted of a brief PBL the students had to solve individually based on the class contents. For instance, they had to solve a construction solution for a specific architectural work in due course by detailing in hand on a small piece of paper like they were an architect on site solving their building team queries in real time. 4.2. Phase 3: Multi-Criteria Decision Making (MCDM) Model As previously stated, in order to be able to integrate different indicators, a MCDM model was chosen because in the current case study, the most sustainable alternative could not be defined by professors directly. This occurred because the considered indicators did not have the same value or tendency for each alternative so that the best alternative according to one issue was not necessarily the best one according to another aspect. In this situation by using a MCDM it was possible to reach an integrated solution that took into account the different issues while, at the same time, it enabled the ranking of these issues according to the case study particularities [62]. To define this new MCDM tool method, the authors reviewed the use of MCDM methodologies for similar research projects. MCDM methodologies applied for university issues have been found in technical literature but they are applications and solve problems which are different from the scope of this project, which proves the novelty of this research project. For example, there are MCDMs to assess learning program quality, teaching quality, learning spaces, curriculum, students’ preferences to choose their universities, etc. In this line there is a review on publications about multi-criteria methodologies for university engineering education [63]. This article concludes that until now, most university educational problems studied using MCDM are about resource efficiency (27%), resource location (18%) and assessment (18%). There are numerous MCDM applications to assess sustainability in the field of construction and building technologies [62]. This project used the Integrated Value Model for a Sustainable Evaluation (Modelo Integrado de Valor para una Evaluación Sostenible (MIVES)). The use of MIVES occurred because this methodology allowed researchers to define specific and agile MCDM tools as they have already been successfully carried out in numerous successful research projects [64–68] and this was the type of model needed for this research project. As has been explained in the previous sections, the new tool university professors require should take into account all the specific issues involved in this case study while being easy to use and giving a quick and useful response adaptable to their specific needs and context. This methodology has the following sub phases: (1.1) establish system limits; (1.2) build the decision making tree composed of requirements, criteria and indicators; (1.3) establish the relative weight each indicator, requirement and criteria has; (1.4) define value functions for each indicator; and (1.5) assess the alternatives. As previously mentioned, these phases relied on technical literature and experts’ seminars. This analysis with MIVES was done in three separate groups of alternative active methodologies and, therefore, generated three sustainability rankings: one for higher, another middle and another low order thinking levels. From the different MCDM methods and sustainability assessment tools available, MIVES was chosen because it allowed a complete evaluation, it is agile, can be configured for this case study and has already been combined with other methods and the aforementioned specific mathematical algorithm [69,70]. The boundaries of this MIVES system were based on the initial diagnosis and case study defined in Section three. Therefore, this new tool evaluated active methodologies and strategies for large group lectures within the “Construction II” course. The implementation of this project used resources and devices currently available in most university classes. These instruments are a computer connected to a projector, a blackboard, Wi-Fi network, a Moodle and at least one computer or smart phone for every two students. Table 3 presents the decision-making tree for this new MIVES tool, defined by experts in seminars using AHP and relying on extensive literature review as explained in the previous sections. This tree exclusively included the most important and discriminatory indicators [71,72], since for a wellmeditated decision, an appropriate requirements tree is of great importance, in which the number of indicators is not excessive. This requirements tree was organized according to the sustainability approach under the three main sustainability requirements: economic, environmental and social [73– Sustainability 2019, 11, 2917 7 of 19 75]. As a result of the recommendations provided by expert panels at seminars, the applicability requirement was added. Table 3. Decision making requirements tree with weights in percentage, defined in the experts’ seminars. Requirements Criteria Indicators R1. Applicability (25%) C1. Application (60%) I01. Easiness to apply (50%) I02. Flexibility to adapt (50%) C2. Transferability (40%) I03. To other professors (60%) I04. To other disciplines (40%) R2. Economic (15%) C3. Time (100%) I05. Dedication in class (40%) I06. Professor’s dedication outside (30%) I07. Students’ dedication outside (30%) R3. Environmental (10%) C4. Impact (100%) I08. Extra environmental impact (100%) R4. Social (50%) C5. Learning process (Chickering and Gamson principles among others) (45%) I09. Feedback to students’ time (20%) I10. Encouraging cooperative work (20%) I11. Students and faculty contact (20%) I12. Talents and ways of learning (25%) I13. Number of concepts (15%) C6. Innovation (20%) I14. University learning (55%) I15. Large group theoretical classes (45%) C7. Satisfaction (35%) I16. Students’ (55%) I17. Professor’s (45%) This decision-making tree included the following four requirements, seven criteria and 17 indicators. First, the applicability requirement (R1) had two criteria and four indicators. I01 and I02 assessed two important application aspects: (1) the ease of activities to be prepared, organized, explained and carried out and (2) flexibility of activities to adapt to each class, dedication availability, classroom size and internet connection. I01 did not include application agility because it was included in economic indicators while I02 did not include flexibility to adapt to different professors or disciplines. Indicators I03 and I04 evaluated the transferability strategies have to other professors and other knowledge disciplines respectively. I04 included transference to other schools and faculties, universities, etc. Second, economic requirement (R2) had three indicators that assessed each professor’s and student’s dedication during class and outside class. I05 assessed the amount of time spent to carry out each activity and strategy during class time. I06 evaluated each professor’s required time in order to prepare and give feedback to students outside of classroom. First year preparation time can be longer than the following years as we can see in Table A2 in Appendix B. This extra time was divided between the different years during which each alternative was expected to be applied. Finally, I07 assessed each student’s required time for doing each activity outside the classroom. The required cost to implement these active methodologies and activities was not assessed because, as stated in this research project boundaries, this project considered the available resources. In this sense, this project considered that these new alternatives will use similar materials and resources, such as paper, internet connection, classroom… rather than those materials and resources used before these new alternatives developed. To sum up, the authors of this project considered that during its implementation, there would not be any extra cost or it would be very low and, therefore, no extra cost was taken into account. Third, I08 unique environmental requirement (R3) indicator assessed the extra impact of each alternative including: extra energy consumption from educational devices such as projectors or clickers etc. and extra waste generation during and outside of class, both by students and professors. This impact was very low compared to the energy consumption from other machines within buildings, such as heating and cooling systems, and the waste generation during other students’ and professors’ daily activities. However, this requirement and indicator was considered because it contributed to our society environmental impact, increasing crucial parameters such as CO2 emissions. Some activities required additional use of computers that have an extra energy consumption, while other activities Sustainability 2019, 11, 2917 8 of 19 used an important amount of extra materials, although these materials were usually paper that can be recycled and recyclable. Fourth and finally, social requirement (R4) was the most important requirement in this research project that included nine indicators distributed in three criteria: C5 to C7. C5 studied students’ learning process assessing: (I09) professor’s feedback time to students’ queries and exercises, (I10) contribution to teamwork, (I11) promotion of students and professors’ relationship and students’ sense of belonging to their institution, school, faculty, university [76], (I12) allowing different talents and ways of learning and (I13) number of concepts learned per time unit. It did not include if it promoted deep learning because this new tool was used for the aforementioned three order thinking levels depending on necessities. Nor did this tool incorporate active learning contributions either because the assessed alternatives were active methodologies and strategies. I11 could include students’ attitude and interest issues, which in the case study was not crucial because most students were interested but in other courses this aspect could be crucial. This would be assessed in I11. C6 evaluated the contribution each alternative innovation makes to university learning processes in general [77] (I14) and large group theoretical classes specifically (I15). C7 assessed students’ satisfaction and their high expectations (I16) as well as individual professor’s satisfaction (I17). This satisfaction did not include each alternative feasibility for this case study, which was large groups, because this was also part of the research project boundaries. These 17 indicators took into account the assessment parameters and the data sources presented in Table 4. These 17 values for each active alternative and strategy are shown in Table A4. Table 4. Assessment parameters and data sources for each indicator. Main Assessed Parameters Data Sources I01 Requires work before class, during and/or after class CXP I02 Adaptable to students’, time, space and resources particularities CXP I03 Available literature relation to case study and easiness to use LT I04 Related 6-digit UNESCO nomenclature areas of expertise LT I05–7 Average dedication per class CXP I08 Hardware energy consumption and activities waste generation CXP and LT I09 Average feedback time CXP I10 Encourages cooperative work CXP and LT I11 Promotes students and faculty contact CXP and LT I12 Allows different styles, approaches, learning and pacing and presentation methods, cultures, recognizes reward and respects creativity CXP and LT I13 Average number of concepts CXP and LT I14 University previous projects and literature about this alternative LT I15 Large groups previous projects and literature about this alternative LT I16 Satisfaction questionnaires CXP I17 Focus groups about satisfaction CXP Legend: CXP—2017 to 2018 course experience; LT—literature review. Then, value functions [71] for each 17 indicators were defined based on numerous rigorous bibliographical references which were discussed in the second seminar. All these functions varied between 0 and 1, being 0 the minimum satisfaction and 1 the maximum satisfaction for each indicator, as a response to the 17 indicator values that have different units as described in the previous paragraphs. These adimensional values VI,k could be added and thus the seven criteria satisfaction values VCRi,K were obtained, then the four requirements satisfaction values SIRi,K were obtained and finally the global sustainability index GSK was obtained. These additions follow Equation (1), Equation (2) and Equation (3) respectively. 𝑉, = 𝜆,   .𝑉,(𝑥) (1) Sustainability 2019, 11, 2917 9 of 19 𝑆𝐼, = 𝜆,   .𝑉, (2) 𝐺𝑆= 𝜆,   .𝑆𝐼, (3) These 17 value functions depended on five parameters, as presented in Equation (4). By giving values to these parameters it was possible to define their shape and, consequently, how each variation of the indicator value was translated into the adimensional scale. For example, if the form was in S, the initial and final variations would have a variation in smaller adimensional values than the central variations.           −⋅+=        − ⋅− Pi Ci XXalt ki ind eBAV max 1 (4) In Equation (2), A is the value generated by Xmax, the abscissa for the indicator, and Xalt is the abscissa for the evaluated indicator that generates a Vind value. Pi is a form factor that defines whether the curve is concave, convex, lineal or “S” shaped. Ci establishes, in curves with Pi > 1, the value of the abscissa at which the inflection point occurs. Ki defines the value for the ordinate of point Ci. B is the factor for the function to be maintained in the range of 0 to 1 and is defined by Equation (5) 1 minmax 1 −        − ⋅−           −= Pi Ci XX ki eB (5) Table 5 presents each indicator function shapes, the definition of which relies on previous steps in this research project and experts’ seminars. In these seminars it was decided to define linear functions for all indicators for the first applications of this tool. Then, after these initial applications, researchers would decide if it was convenient to define concave functions for the most crucial indicators and convex functions for less important indicators that needed to be promoted. Table 5. New tool indicators functions shapes. Indicators Function Shape X min. X max. C K P Code Unit I01, I02, I03, I04, I10, I11, I12, I14, I15, I16, I17 Points IL 0 100 50 0.5 1.25 I05 Minutes DL 180 90 I06 Minutes DL 60 30 I07 Minutes DL 90 45 I08 Points DL 100 50 I09 Hours DL 168 84 I13 Concepts/hour IL 25 12 Legend: IL means increasing lineal, and DL means decreasing lineal. 4.3. Mathematic Algorithm The Knapsack algorithm was chosen for phase five because it was able to generate sets of active methodologies and strategies designed for specific cases of classes and took into account the contents of each class, its order thinking level and the 17 chosen indicators. Moreover, Knapsack has already been successfully used to do so in combination with MIVES as previously stated. This algorithm defines sets of alternatives, maximizing some values according to the required measures [78]. Knapsack results are one or more sets that comply with the total measurement options equal to or less than the required Sustainability 2019, 11, 2917 16 of 19 H08 80 50 4 88 90 30 0 0 0 100 50 67 13 99 0 77 80 Legend: Alter.—Alternatives. References 1. Chaudhury, S.R. The lecture. New Dir. Teach. Learn. Available online: https://onlinelibrary.wiley.com/doi/abs/10.1002/tl.464 (accessed on 6 May 2019) 2. Tynjälä, P. Towards expert knowledge? A comparison between a constructivist and a traditional learning environment in the university. Int. J. Educ. Res. 1999, 31, 357–442. 3. The world University Rankings and ElSevier. World University Rankings 2019. 2019. Available online: https://www.timeshighereducation.com/world-university-rankings/2019/worldranking#!/page/0/length/25/name/madrid/sort_by/rank/sort_order/asc/cols/stats. (accessed on 2 February 2019). 4. Ludvigsen, K.; Krumsvik, R.; Furnes, B. Creating formative feedback spaces in large lectures. Comput. Educ. 2015, 88, 48–63. 5. Tai, H.H. The features of world-class universities. In Proceedings of the First International Conference on World-Class Universities (WCU-1), Shanghai, Chine, 16–19 June,2015, pp. 65–73. 6. Waugh, G.; Waugh, R. The value of lectures in teacher education: the group perspective. Aust. J. Teach. Educ. 1999, 24, 35–51. 7. Matz, R.L.; Rothman, E.D.; Krajcik, J.S.; Holl, M.M.B. Concurrent enrollment in lecture and laboratory enhances student performance and retention. J. Res. Sci. Teach. 2012, 49, 659–682. 8. Bligh, D.A. What’s the Use of Lectures? Intellect: Bristol, UK, 1998. 9. French, S.; Kennedy, G. Reassessing the value of university lectures. Teach. High. Educ. 2017, 22, 639–654. 10. Bradbury, N.A. Attention span during lectures: 8 seconds, 10 minutes, or more? Adv. Physiol. Educ. 2016, 40, 509–513. 11. Broadbent, J. Large class teaching: How does one go about the task of moderating large volumes of assessment? Saga J. 2018, 19, 173–185. 12. Tolley, L.M.; Johnson, L.; Koszalka, T.A.; An intervention study of instructional methods and student engagement in large classes in Thailand. Int. J. Educ. Res. 2012, 53, 381–393. 13. Brooks, D.C. Space matters: The impact of formal learning environments on student learning. Br. J. Educ. Technol. 2011, 42, 719–726. 14. Stoltzfus, J.R.; Libarkin, J. Does the room matter? Active learning in traditional and enhanced lecture spaces. CBE Life Sci. Educ. 2016, 15, ar68. 15. Gysbers, V.; Johnston, J.; Hancock, D.; Denyer, G. Why do students still bother coming to lectures, when everything is available online? Int. J. Innov. Sci. Math. Educ. 2011, 19, 20–36. 16. Charlton, B.; Marsh, S.; Gurski, N. Are Lectures the Best Way to Teach Students? The Guardian: London, UK, 2015. 17. Kaur, G. Study and analysis of lecture model of teaching. Int. J. Educ. Plan. Adm. 2011, 1, 9–13. 18. Wildermuth, S.; Tammy, F.; Edward, F. Finding the right fit: Assessing the impact of traditional v. large lecture/small labe course formats on a general education course. Int. J. Scholarsh. Teach. Learn. 2013, 7, 1–21. 19. Wilhelm, S.; Förster, R.; Zimmermann, B.A. Implementing competence orientation: towards constructively aligned education for sustainable development in university-level teaching-and-learning. Sustainability 2019, 11, 1891. 20. Atchley, W.; Wingenbach, G.; Akers, C. Comparison of course completion and student performance through online and traditional courses. Int. Rev. Res. Open Distance Learn. 2013, 14, 104–116. 21. Sumera, A. Large group teaching , an effective and efficient teaching methodology. J. Asian Sci. Res. 2014, 4, 1–5. 22. Deslauriers, L.; Schelew, E.; Wieman, C. Physics class. Science 2011, 332, 862–864. 23. Heppner, F. Teaching the Large College Class: A Guidebook for Instructors with Multitudes;Willey: Hoboken, NJ, USA, 2007. 24. Carbone, E. Teaching Large Classes: Tools and Strategies; SAGE:Thousand Oaks, CA, USA, 1998. 25. Lloyd-Strovas, J. Tips for Teaching Large Classes; Available online: http://www.depts.ttu.edu/tlpdc/Resources/Teaching_resources/TLPDC_teaching_resources/TipsForTeach ingLargeClasses.pdf (asscessed on 6 May 2019) 26. Jackson, M.W.; Prosser, M.T. Less lecturing, more learning. Stud. High Educ. 1989, 14, 55–68. Sustainability 2019, 11, 2917 17 of 19 27. Cooper, J.L.; Robinson, P. The argument for making large classes seem small. New Dir. Teach. Learn. 2002, 2000, 5–16. 28. Gedalof, A.J. Teaching Large Classes; Office of Dalhousie University: Halifax, Nova Scotia, 1998. 29. Felder, R.M. Beating the numbers game: Effective teaching in large classes. In Proceedings of the ASEE Annual Conference, Milwaukee, VI, USA, 15–18 June, 1997. 30. Morton, A. Lecturing to large groups. In A Handbook for Teaching and Learning in Higher Education, Enhancing Academic Practice; Fry, H., Ketteridge, S., Marshall, S., Eds.; Rotledge: London, UK, 2007; pp. 58–71. 31. Rodriguez-Falcon, E.; Evans, M.; Allam, C.; Barrett, J.; Forrest, D. The Inclusive Learning and Teaching Handbook. Available online: https://www.sheffield.ac.uk/polopoly_fs/1.18989!/file/The-inclusive-learningand-teaching-handbook.pdf (accessed on 6 May 2019) 32. Wammes, J.D.; Ralph, B.C.W.; Mills, C.; Bosch, N.; Duncan, T.L.; Smilek, D. Disengagement during lectures: Media multitasking and mind wandering in university classrooms. Comput. Educ. 2019, 132, 76–89. 33. Mulryan-Kyne, C. Teaching large classes at college and university level: Challenges and opportunities. Teach. High. Educ. 2010, 15, 175–185. 34. Walker, J.D.; Cotner, S.H.; Baepler, P.M.; Decker, M.D. A delicate balance: integrating active learning into a large lecture course. CBE Life Sci. Educ. 2008, 7, 361–367. 35. Huerta, J.C. Getting active in the large lecture. J. Polit. Sci. Educ. 2007, 3, 237–249. 36. JCarpenter, M. Effective teaching methods for large classes. J. Fam. Consum. Sci. Educ. 2006, 24, 13–23. 37. Knight, J.K.; Wood, W.B. Teaching more by lecturing less. Cell Biol. Educ. 2005, 4, 298–310. 38. Allen, D.; Tanner, K. Infusing active learning into the large-enrollment biology class: seven strategies, from the simple to complex. Cell Biol. Educ. 2005, 4, 262–268. 39. Caldwell, J.E. Clickers in the large classroom: current research and best-practice tips. CBE Life Sci. Educ. 2007, 6, 9–20. 40. Morell, T. Multimodal competence and effective interactive lecturing. System 2018, 77, 70–79. 41. Petrović, J.; Pale, P. Students’ perception of live lectures’ inherent disadvantages. Teach. High. Educ. 2015, 20, 143–157. 42. Renaud, S.; Tannenbaum, E.; Stantial, P. Student-centered teaching in large classes with limited resource. Eng. Teach. Forum 2007, 3, 12–18. 43. Wolfman, S.A. Making lemonade: Exploring the bright side of large lecture classes. SIGCSE Bull. 2002, 34, 257–261. 44. Angelo, T.A.; Kross, K.P. Classroom assessment techniques: A handbook for college teachers. J. Higher Educ. 1995, 66, 108–111. 45. López, D. Artículo invitado Si elimino el examen ¿mis alumnos dejarán de aprender? Una Experiencia de Diseño de Actividades Educativas Alternativas al Examen 2013, 6, 10–17. 46. University of Bath. Centre for Learning & Teaching. Available online: https://www.bath.ac.uk/professional-services/centre-for-learning-teaching/ (accessed on 6 May 2019) 47. University of Waterloo. Activities for Large Classes. Available online: https://uwaterloo.ca/centre-forteaching-excellence/teaching-resources/teaching-tips/educational-technologies/all/activities-large-classes (accessed on 6 May 2019) 48. Vanderbilt University. Large group teaching. Availablr online: https://cft.vanderbilt.edu/guides-subpages/teaching-large-classes/ (accessed on 6 May 2019) 49. Ramzy, N.S. Between the école des beaux-arts and the bauhaus: Modern Architecture as an outcome of the enlightenment philosophy. Ain Shams J. Mech. Eng. 2010, 2, 53–65. 50. Colomina, B. Radical pedagogies. 2018. Available Online: http://radicalpedagogies.com (accessed on 6 May 2019). 51. Ayiran, N. Some criticisms on theory-driven architectural education. In International Conference Unspoken Issues in Architectural Education; Eastern Mediterranean University (EMU): Famagusta, North Cyprus. 2014, pp. 65–73. 52. ETSAB. Degree in Architecture Studies. 2018. Available online: https://etsab.upc.edu/en/studies/garqetsab/curriculum (accessed on 6 May 2019). 53. Casals, A.; Gonzalez, J.L. Les Claus per a Construir L’arquitectura; Gustavo Gili: Barcelona, Spain, 2002. 54. Paricio, I. La Construcción de la Arquitectura. 2. Los Elementos; Institut de Tecnologia de la Construcció de Catalunya—ITeC: Barcelona, Spain, 2009. 55. Mañá, F. El Gros de L’obra. Uns Apunts de Construcció (Aula D’ARQUITECTURA); Universitat Politecnica de Sustainability 2019, 11, 2917 18 of 19 Catalunya, Iniciativa Digital Politecnica: Barcelona, Spain, 2000. 56. De Llorens, J.I.; Pons, O. El Terreny i L’estudi Geotècnic; Universitat Politècnica de Catalunya: Barcelona, Spain, 2015. 57. Moreno-Navarro, J.L.G.; Balagué, A.C. Las estrategias docentes de la construcción arquitectónica. Inf. Construcción 2001, 53, 5–19. 58. ETSAB. Syllabus 2018 Schedule 210126—Construction II. 2018. Available online: https://etsab.upc.edu/en/studies/garqetsab/syllabus (accessed on 6 May 2019). 59. Pons-Valladares, O.; González-Barroso, J.M.; López-Olivares, R.; Arias, I. Educational project to improve problem-based learning in architectural construction courses using active and co-operative techniques. Rev. Constr. 2015, 14, 2. 60. Pons, O.; Franquesa, J. Actividades y estrategias de aprendizaje activo para clases teóricas en grupos numerosos (Active learning activities and strategies for theoretical classes in large groups). Available online: https://upcommons.upc.edu/handle/2117/124466 (accessed on 6 May 2019) 61. Anderson, L.W.; Krathwohl, D.R.; Airasian, P.W.; Cruikshank, K.A.; Mayer, R.E.; Pintrich, P.;R.; Wittrock, M.C. A Taxonomy for Learning, Teaching, and Assessing: Pearson New International Edition: A Revision of Bloom’s Taxonomy of Educational Objectives, Abridged Edition; Pearson Education Limited, Edición, Pearson New International: London, UK, 2013. 62. Zavadskas, E.; Antucheviciene, J.; Vilutiene, T.; Adeli, H. Sustainable decision-making in civil engineering, construction and building technology. Sustainability 2017, 10, 14. 63. Ahmad, S.; Syed, H.; Tan, S.C. MCDM for engineering education: Literature review and research issues. In Engineering Education for a Smart Society; Springer: Cham, Switzerland, 2018; Volume 2. 64. Pons, O.; de la Fuente, A.; Aguado, A. The use of mives as a sustainability assessment mcdm method for architecture and civil engineering applications. Sustainability 2016, 8, 460. 65. Cuadrado, J.; Rojí, E.; José, J.T.S.; Reyes, J.P. Sustainability index for industrial buildings. Proc. Inst. Civ. Eng. Struct. Build. 2012, 165, 245–253. 66. Zubizarreta, M.; Cuadrado, J.; Iradi, J.; García, H.; Orbe, A. Innovation evaluation model for macroconstruction sector companies: A study in Spain. Eval. Program Plann. 2017, 61, 22–37. 67. Antonio, A.; del, C.A.; Pilar, d.M.; Diego, G.; Alejandro, J. Sustainability Assessment of concrete structures within the spanish structural concrete code. J. Constr. Eng. Manag. 2012, 138, 268–276. 68. José, J.S.; Garrucho, I.; J, C. The first sustainable industrial building projects. Proc. Inst. Civ. Eng. Munic. Eng. 2006, 159, 147–153. 69. Hosseini, S.M.A.; Pons, O.; de la Fuente, A. A combination of the Knapsack algorithm and MIVES for choosing optimal temporary housing site locations: A case study in Tehran. Int. J. Disaster Risk Reduct. 2018, 27, 265–277. 70. De la Cruz, M.; Castro, A.; del Caño, A.; Gómez, D.; Lara, M.; Cartelle, J. Comprehensive methods for dealing with uncertainty in assessing sustainability, Part 1: The MIVES—Monte Carlo Method. In Soft Computing Applications for Renewable Energy and Energy Efficiency; IGI Global: Hershey, PA, USA, 2015. 71. Alarcon, B.; Aguado, A.; Manga, R.; Josa, A. A value function for assessing sustainability: Application to industrial buildings. Sustainability 2011, 3, 35–50. 72. Viñolas, B. Aplicaciones y Avances de la Metodología MIVES en Valoraciones Multicriterio. (Applications and Advances of MIVES Methodology in Multi-Criteria Assessments); UPC: Wallisellen, Switzerland, 2011. 73. Comesaña-Campos, A.; Cerqueiro-Pequeño, J.; Bouza-Rodríguez, J.B.; The Value Index as a decision support tool applied to a new system for evaluating and selecting design alternatives. Expert Syst. Appl. 2018, 113, 278–300. 74. Liu, L.; Lee, L. Sustainability: Living within one’s own ecological means. Sustainability 2009, 1, 1412–1430. 75. Shen, L.; Hao, J.L.; Tam, V.W.; Yao, H. A checklist for assessing sustainability performance of construction projects J. Civ. Eng. Manag. 2007, 13, 273–281. 76. Beattie, I.R.; Thiele, M. Connecting in class? College class size and inequality in academic social capital. J. Higher Educ. 2016, 87, 332–362. 77. Mayhew, M.J.; Selznick, B.S.; Zhang, L.; Barnes, A.C.; Staples, B.A. Examining curricular approaches to developing undergraduates’ innovation capacities. J. Higher Educ. 2018, 1–22. doi.org/10.1080/00221546.2018.1513307. 78. Martello, S.; Toth, P. Knapsack Problems: Algorithms and Computer Implementations; John Wiley & Sons, Inc.: New York, NY, USA, 1990. Sustainability 2019, 11, 2917 19 of 19 79. Ibarra, O.H.; Kim, C.E. Fast approximation algorithms for the knapsack and sum of subset problems. J. ACM 1975, 22, 463–468. 80. Berkovich-Ohana, A.; Jennings, P.A.; Lavy, S. Chapter 15—Contemplative neuroscience, self-awareness, and education. In Meditation; Srinivasan, B.R. N. B. T.-P., Ed.; Elsevier: Amsterdam, The Netherlands, 2019; Volume 244, pp. 355–385. 81. Chassignol, M.; Khoroshavin, A.; Klimova, A.; Bilyatdinova, A. Artificial intelligence trends in education: a narrative overview. Procedia Comput. Sci. 2018, 136, 16–24. © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).