FROM VR CONTROLLERS TO MULTI-USER IMMERSIVE EDUCATIONAL EXPERIENCES IN DIGITAL CULTURAL HERITAGE

565 | ASCAAD 2024 FROM VR CONTROLLERS TO MULTI-USER IMMERSIVE EDUCATIONAL EXPERIENCES IN DIGITAL CULTURAL HERITAGE The Case of Sheikh Isa House in Bahrain EIMAN ELGEWELY, MOHAMED ALI School of Design, College of Architecture, Art and Design (AAD), Virginia Tech University Burruss Hall, 121, 800 Drillfield Dr, Blacksburg, VA 24061, USA [email protected] [email protected] AND DEEPAK GUPTA Department of Computer Science, College of Engineering, Virginia Tech University 185 Kent St, Blacksburg, VA 24061 [email protected] Abstract This study employs advanced virtual reality (VR) technology to reinterpret the sustainable architectural practices of Sheikh Isa House in Bahrain, offering valuable insights into immersive learning environments. Focusing on the passive solar systems of historic vernacular architecture across the Arab world, Sheikh Isa House serves as a pivotal case study. Through detailed building performance simulations, architectural features such as windcatchers, interconnected courtyards, and seasonal rooms are analyzed for their contributions to sustainability and thermal comfort. A significant advancement of this research is the transition from individual VR experiences to a multi-user immersive environment, enabling classes and groups to engage collectively within the virtual space. The VR application integrates immersive digital storytelling to facilitate an interactive exploration of regional architectural heritage and sustainable practices. Future work involves forming student focus groups to evaluate the E. ELGEWELY, M. ALI, AND D. GUPTA 566 | ASCAAD 2024 effectiveness of these multi-user immersive experiences compared to individual VR, aiming to enhance the educational impact of the application. Keywords: Digital heritage, Virtual Reality, 360 Immersive environments, Education, Sustainable heritage. 1. Introduction The Sustainable Arab Heritage in VR project (Sustainable Heritage in VR, 2024), initiated in 2022, seeks to harness the architectural wisdom of the past by focusing on the sustainable passive solar systems used in historic vernacular architecture across the Arab world. By analyzing these systems, the project addresses contemporary challenges related to heat management and energy consumption in today's buildings. The House of Sheikh Isa in Bahrain was chosen as the first case study for developing an educational virtual reality (VR) application designed to bring this valuable knowledge into classrooms in an engaging and interactive way. This selection underlines the strong potential of VR and augmented reality (AR) applications in design and architecture education, particularly their ability to enhance understanding of scale, proportionality, color, and materiality within a virtual environment(Guazzaroni and Pillai, 2020; Bazavan et al., 2021; Vilar, Filgueiras and Rebelo, 2022; Hou et al., 2024). Building upon the foundational research from the Educational Virtual Model of Sheikh Isa House project, this paper further explores VR's transformative potential in digital cultural heritage. The initial project offered a comprehensive understanding of the building's interaction with its environmental context. It showcased the efficacy of advanced visualization techniques in communicating complex scientific data within a virtual environment. The workflow and framework established during this phase laid a solid foundation for a versatile methodology with significant implications for heritage and sustainability education. It also provided valuable insights into sustainable architectural practices applicable to contemporary contexts. This research advances the discussion by shifting the focus from individual VR experiences to a multi-user immersive environment. This transition is pivotal in enhancing the inclusivity and accessibility of digital cultural heritage applications, particularly within educational settings such as classrooms and museums. By enabling simultaneous participation, this approach maximizes the educational FROM VR CONTROLLERS TO MULTI-USER IMMERSIVE EDUCATIONAL EXPERIENCES IN DIGITAL CULTURAL HERITAGE 567 | ASCAAD 2024 impact of the VR application, broadening its reach and fostering deeper engagement with cultural heritage content. By using the House of Sheikh Isa as a case study, this research aims to bridge historic architectural knowledge with contemporary technological applications. The insights gained are intended to be both practically and academically relevant, contributing to the ongoing discourse on sustainable design and digital heritage. 2. Historical Background The house of Sheikh Isa in Bahrain was selected as the first case study for this multi-phased project. It is a historic 19th-century house located in Muharraq, the old capital of Bahrain. It is preserved in excellent condition as part of the UNESCO Pearling Path, which includes approximately 17 other buildings and extends for 2.7 miles(Pearling, Testimony of an Island Economy, no date; Karimi, no date). The timeline of the house saw many alterations, but the most significant period was when it served as the residence of Bahrain's ruler from 1869 to 1932, playing a significant role in Bahrain's political history (al-Khalifa and Rice, 1993; Yarwood, 2000; Yarwood and El-Masri, 2006; Al-Sulaiti, 2011). The excellent condition of the existing structure, in addition to documentation efforts by the Bahrain Authority for Culture and Antiquities (BACA) and an international research team led by Barazzetti et al., made the house an abundant subject of study for sustainable practices in the Arab world, particularly in the Gulf region, and the methods of adaptation to the arid climate in this region(Barazzetti, Mezzino and Quintero, 2017). The structure of the house includes four connected courtyards to help circulate air inside the building, essential for privacy and internal social interaction. The house also includes a windcatcher (Figure 1), a structure commonly used in the Middle East to cool the air inside rooms. The windcatcher in the House of Sheikh Isa wasn't an integral part of the structure until it was added on top of "Al Majlis," the main living room, to help cool it down (Figure 2). This house is also a significant example of the use of native building materials, such as mudbrick, for building thick walls to help improve insulation against heat(Bayt al-Shaykh, n.d.; Lalande Hardy-Guilbert, 1981; Yarwood and El-Masri, 2006) E. ELGEWELY, M. ALI, AND D. GUPTA 568 | ASCAAD 2024 Figure 1: 3D model of the main courtyard of Sheikh Isa's house, known as the Women's Courtyard, before(left) and after(right) adding textures. The scene shows the windcatcher connected to the main living room, Al Majlis. Figure 2: 3D visualization of the main family room, Al Majlis, directly connected to the windcatcher. 3. Project Workflow and Methodology The workflow and framework used in the Sustainable Heritage in VR project, particularly in developing the educational model of Sheikh Isa House, commenced with the creation of a highly accurate 3D model of the House of Sheikh Isa in Bahrain. The model development involved using various 3D modeling software, FROM VR CONTROLLERS TO MULTI-USER IMMERSIVE EDUCATIONAL EXPERIENCES IN DIGITAL CULTURAL HERITAGE 569 | ASCAAD 2024 including Autodesk 3DS Max and Rhino 3D, which were pivotal in ensuring the precision of the architectural details. This model was then exported to Twinmotion, where textures, materials, and lighting were applied to enhance the visual realism of the structure. Subsequently, the model was integrated into Unreal Engine 5.3('Unreal Engine 5', 2022), which served as the primary tool for developing VR educational applications, enabling further refinement and interactive capabilities. In parallel, a second model was created using DesignBuilder software('DesignBuilder', 2023) to conduct comprehensive building performance simulations. These simulations offered critical insights into the sustainable features of the house, including thermal comfort analysis, airflow patterns through Computational Fluid Dynamics (CFD) (Figure 3), and daylighting analysis. The detailed analysis of airflow patterns, both inside and outside the house, revealed how the courtyards and windcatcher work in tandem to facilitate natural ventilation, efficiently cooling the interior rooms. Additionally, temperature simulations demonstrated the thermal performance of the building's materials, showcasing how traditional construction techniques effectively maintained comfortable indoor environments. Such analyses were vital in understanding the building's environmental interactions and assessing its sustainable design aspects. The results of this sustainability study are being disseminated through separate publications, providing an in-depth exploration of the findings. (Fardeheb, 2007; Hensen and Lamberts, 2011; Attia, 2014; Rajapaksha et al., 2018; Al-Sallal and Rahmani, 2019; Olawale-Johnson, Ajwang and Ondimu, 2021; Ozarisoy and Altan, 2021). Figure 3: 3D visualization of the external CFD velocity of the house of Sheikh Isa created using DesignBuilder software. Upon acquiring the results of the sustainability study, these findings were seamlessly integrated into the virtual environment developed in Unreal Engine E. ELGEWELY, M. ALI, AND D. GUPTA 570 | ASCAAD 2024 5.3. This process facilitated the creation of a versatile, multi-purpose model adaptable to various outputs depending on the intended use and audience. Potential outputs include a VR application, a 360-degree video installation, and a mobile application. This workflow serves as a replicable model for similar sustainable heritage projects (Figure 4). Figure 4. A flow diagram illustrating the integration of 3D visualization and building performance simulation results into a virtual environment, producing outputs such as a VR application, 360 video installation, and mobile application. 4. Integration with VR Environment The building performance simulation study results were translated into a visual format and audio narration within Unreal Engine 5.3, guiding users as they explored the virtual environment. After visualizing the data, interactive elements were developed to enhance user engagement with these simulations in the VR environment. At the beginning of the experience, users are greeted with an instruction screen outlining the controls. Given the building's large size, we developed an interactive navigation map that can be opened using the buttons on the left controller. This map allows users to explore the building by selecting points of interest. By pointing at specific rooms or courtyards on the map with the suitable controller and pressing the trigger, users are instantly teleported to that area (Figure 5). FROM VR CONTROLLERS TO MULTI-USER IMMERSIVE EDUCATIONAL EXPERIENCES IN DIGITAL CULTURAL HERITAGE 571 | ASCAAD 2024 Users can also interact with doors, opening or closing them by pointing and pressing the trigger button on the suitable controller. Sound narration is a key feature that accompanies users through different areas, providing contextual information and enhancing the user's understanding of the environment. Background audio and these audio cues play automatically to explain the environment, further enhancing immersion. The storytelling narrative embedded within the VR application provides users with information about the house's historical background and daily usage. It also highlights the sustainable design features that contributed to its adaptation to harsh weather conditions and improved thermal comfort. The navigation map features a sustainability mode, enabling users to preview BPS results and interact with a "See in 3D" button that overlays these results onto the 3D model for enhanced immersion and understanding (Figure 5). This approach simplifies the complex scientific data generated by the building performance study, seamlessly integrating it into the VR environment to create an effective educational tool. The goal is to make intricate architectural design and sustainability concepts accessible to a broad spectrum of users, allowing students and other learners to explore these ideas in an immersive and interactive way. Figure 5. A screenshot of the VR walkthrough created in Unreal Engine 5.3showcasing the navigation interface map inside the educational VR model of Sheikh Isa House. E. ELGEWELY, M. ALI, AND D. GUPTA 572 | ASCAAD 2024 Figure 6. A screenshot showing the navigation interface map within the sustainability mode of the educational VR application, displaying a solar gain map of the main courtyard (left). The interactive feature "See in 3D" maps the solar gain data onto the 3D model, facilitatin g a b etter inter p retation of the results ( ri g ht ) . 5. Inclusive VR Immersive Experiences: Enhancing Accessibility and Engagement The need for more inclusive VR immersive experiences became evident following rigorous testing for the demo VR application. The application was showcased at several events held at Virginia Tech University. These events attracted thousands of visitors from diverse backgrounds, including undergraduate and graduate students and individuals from the public with varying ages and experiences. This wide-ranging audience highlighted some limitations of our current VR application. Many visitors, particularly those with visual impairments, discomfort using VR headsets with eyeglasses, or physical disabilities that made it challenging to operate controllers, faced difficulties navigating the virtual environment. Additionally, in a classroom setting, the time limitations of a 75minute lecture pose challenges in providing each student with an individual VR experience. These factors underscore the importance of developing more accessible and user-friendly VR experiences to accommodate a broader range of users and the practical constraints of educational environments (Mott et al., 2019; Williams, 2022; Creed et al., 2023; Dudley et al., 2023). The challenges above inspired us to explore alternative methods to involve multiple users within the immersive environment and provide a true sense of scale and proportionality. This was made possible by the availability of the Cyclorama display system, a key facility of the Institute for Creativity, Arts, and Technology (ICAT) at Virginia Tech University. Situated at the Moss Arts Center on Virginia Tech's main campus. The design of the exhibition ensured it provided a collective and inclusive experience accessible to those unable to use traditional VR devices. This immersive, interactive 360 video installation allowed a broader audience to FROM VR CONTROLLERS TO MULTI-USER IMMERSIVE EDUCATIONAL EXPERIENCES IN DIGITAL CULTURAL HERITAGE 573 | ASCAAD 2024 engage with the VR content without the need for headsets, promoting inclusivity and accessibility for all visitors. Additionally, due to the availability of two projectors, the display system can also be previewed using stereoscopic headsets, allowing for 3D stereoscopic vision and providing users with 3D glasses. This system opens many possibilities for architecture and interior design educators who might use this system to provide their students with an unconventional method of learning about built environments, where the entire classroom could be held inside the Cyclorama(Argyriou, Economou and Bouki, 2017, 2020; Woletz et al., 2018; Škola et al., 2020; Weber, Weibel and Mast, 2021; Cinnamon and Jahiu, 2023)(Figure 7). Figure 7. A 3D model depicting the structure of the Cyclorama featuring projectors, a 180-degree curved screen, and the suggested Interactive touch table 5.1. Enhancing the Immersive Experience Through Audio Narration and Storytelling The immersive 360-degree environment is enriched by voice narration and contextual music, offering detailed explanations of key areas such as the courtyards, summer and winter rooms, the women's section, and the guest section. These narrations help participants better understand the spatial relationships and cultural significance of these spaces and their everyday use. Additionally, the voice narration succinctly presents the key findings from the building performance simulation study, providing insight into the sustainability aspects of E. ELGEWELY, M. ALI, AND D. GUPTA 580 | ASCAAD 2024 source platforms, facilitating broader dissemination and collaboration. These efforts strive to create more inclusive, engaging, and educational immersive experiences, benefiting diverse educational, cultural, and museum applications. Funding: This project was funded by Virginia Polytechnic Institute and State University through a Faculty Initiated Research Grant (FIRG) from the College of Architecture, Arts, and Design (AAD) and an ICAT Major SEAD Grant from the Institute for Creativity, Arts, and Technology (ICAT). Acknowledgments We gratefully acknowledge the research conducted by Barazzetti et al., supported by the Bahrain Authority for Culture and Antiquities (BACA) in 2017. Their comprehensive work on the House of Sheikh Isa provided essential documentation and insights that have guided our understanding and analysis for this study. Our sincere gratitude extends to our esteemed collaborators: Thomas Tucker, Associate Professor of Creative Technology at the School of Visual Arts; David Hicks, Professor of Education at the School of Education; and Todd Ogle, Executive Director of The Applied Research in Immersive Experiences and Simulations (ARIES) program at the University Libraries, Virginia Tech. We also extend our heartfelt thanks to the Institute for Creativity, Arts, and Technology (ICAT) team for their unwavering support and assistance. Additionally, we deeply appreciate the contribution of our graduate student, Emily Wilson, a Master of Architecture candidate, whose efforts have been pivotal to the project's success. References al-Khalifa, S. A. bin K. and Rice, M. (1993) Bahrain Through the Ages: The History. Al-Sallal, K. A. and Rahmani, M. (2019) 'Vernacular Architecture in the MENA Region: Review of Bioclimatic Strategies and Analysis of Case Studies', in. Springer Nature Switzerland AG. doi: 10.1007/978-3-030-06185-2_3. Al-Sulaiti, A. (2011) 'A view of the defence strategy of Muharraq, a Tribal town in the Gulf (poster)', in Proceedings of the Seminar for Arabian Studies. Argyriou, L., Economou, D. and Bouki, V. (2017) '360-Degree Interactive Video Application for Cultural Heritage Education', Third Immersive Learning Research Network Conference. Argyriou, L., Economou, D. and Bouki, V. (2020) 'Design methodology for 360° immersive video applications: the case study of a cultural heritage virtual tour', Personal and Ubiquitous Computing, 24(6). doi: 10.1007/s00779-020-01373-8. Attia, M. K. (2014) 'Sustainability in Saudi vernacular built environment: The case of Al-Ahsa', in Vernacular Architecture: Towards a Sustainable Future. doi: 10.1201/b17393. Barazzetti, L., Mezzino, D. and Quintero, M. S. (2017) 'Digital workflow for the conservation of Bahrain built heritage: The sheik ISA bin Ali house', in International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives. International Society for Photogrammetry and Remote Sensing, pp. 65–70. doi: 10.5194/isprsarchives-XLII-2-W5-65-2017. FROM VR CONTROLLERS TO MULTI-USER IMMERSIVE EDUCATIONAL EXPERIENCES IN DIGITAL CULTURAL HERITAGE 581 | ASCAAD 2024 Bayt al-Shaykh' Issa bin' Ali al-KhilafaMuharraq, Bahrain (no date) https://www.archnet.org/sites/3798. Available at: https://www.archnet.org/sites/3798 (Accessed: 22 August 2024). Bazavan, L. C. et al. (2021) 'Virtual Reality and Augmented Reality in Education', in Proceedings of the 2021 30th Annual Conference of the European Association for Education in Electrical and Information Engineering, EAEEIE 2021. Institute of Electrical and Electronics Engineers Inc. doi: 10.1109/EAEEIE50507.2021.9531005. Cinnamon, J. and Jahiu, L. (2023) '360-degree video for virtual place-based research: A review and research agenda', Computers, Environment and Urban Systems. doi: 10.1016/j.compenvurbsys.2023.102044. Creed, C. et al. (2023) 'Inclusive Augmented and Virtual Reality: A Research Agenda', International Journal of Human-Computer Interaction. doi: 10.1080/10447318.2023.2247614. 'DesignBuilder' (2023). DesignBuilder Software Ltd. Dudley, J. et al. (2023) 'Inclusive Immersion: a review of efforts to improve accessibility in virtual reality, augmented reality and the metaverse', Virtual Reality, 27(4), pp. 2989–3020. doi: 10.1007/s10055-023-00850-8. Fardeheb, F. (2007) 'Examination and Review of Passive Solar Cooling Strategies in Middle Eastern and North African Vernacular Architecture', in Proceedings of ISES World Congress 2007. Proceedings of ISES World Congress 2007. doi: https://doi.org/10.1007/978-3-540-759973_508. Google (no date) Google Arts & Culture, https://artsandculture.google.com. Available at: https://artsandculture.google.com/explore (Accessed: 10 October 2024). Guazzaroni, G. and Pillai, A. S. . (2020) Virtual and augmented reality in education, art, and museums. PA: Engineering Science Reference, an imprint of IGI Global. Hensen, J. and Lamberts, R. (2011) 'Building performance simulation for sustainable building design and operation', in 60th Anniversary of Environmental Engineering Department. Hou, N. et al. (2024) 'Virtual reality space in architectural design education: Learning effect of scale feeling', Building and Environment, 248. doi: 10.1016/j.buildenv.2023.111060. Karimi, A. I. (no date) A String of PearlsBahrain's Pearling Path offers a two-decade retrospective on history, identity, and architectural narrative. Available at: https://www.archpaper.com/2024/08/bahrain-pearling-path-retrospective-history-identityarchitectural-narrative/ (Accessed: 22 August 2024). Lalande Hardy-Guilbert, C. (1981) LA MAISON DE SHAYKH ISA A BAHRAYN (MÉMOIRE. ERC. Mott, M. et al. (2019) 'Accessible by design: An opportunity for virtual reality', in Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019. Institute of Electrical and Electronics Engineers Inc., pp. 451–454. doi: 10.1109/ISMAR-Adjunct.2019.00122. Olawale-Johnson, O. P., Ajwang, P. and Ondimu, S. N. (2021) 'Reducing cooling demands in subsaharan africa: A study on the thermal performance of passive cooling methods in enclosed spaces', Journal of Sustainable Development of Energy, Water and Environment Systems, 9(4). doi: 10.13044/j.sdewes.d7.0313. Open Heritage 3D Open Heritage 3D, https://openheritage3d.org/. Available at: https://openheritage3d.org/ (Accessed: 10 October 2024). Open Stage Control (2023) 'Open Stage Control'. Libre and modular OSC / MIDI controller. Available at: https://openstagecontrol.ammd.net/ (Accessed: 15 July 2024). Ozarisoy, B. and Altan, H. (2021) 'Systematic literature review of bioclimatic design elements: Theories, methodologies and cases in the South-eastern Mediterranean climate', Energy and E. ELGEWELY, M. ALI, AND D. GUPTA 582 | ASCAAD 2024 Buildings. doi: 10.1016/j.enbuild.2021.111281. Ozarisoy, B. and Altan, H. (no date) Systematic Literature Review of Bioclimatic Design Elements: Theories, Methodologies and Cases in the South-eastern Mediterranean Climate. Pearling, Testimony of an Island Economy (no date) https://whc.unesco.org/en/list/1364/. Privitera, A. G., Fontana, F. and Geronazzo, M. (2024) 'The Role of Audio in Immersive Storytelling: a Systematic Review in Cultural Heritage', Multimedia Tools and Applications. doi: 10.1007/s11042-024-19288-4. Rajapaksha, I. et al. (2018) 'Exploring thermal comfort in the context of historical conservation. A study of the vernacular architecture of Pompeii', Architectural Science Review, 61(1–2), pp. 4–14. doi: 10.1080/00038628.2017.1405790. Škola, F. et al. (2020) 'Virtual reality with 360-video storytelling in cultural heritage: Study of presence, engagement, and immersion', Sensors (Switzerland), 20(20). doi: 10.3390/s20205851. Sustainable Heritage in VR (2024) https://sheikhvr.house/. Available at: https://sheikhvr.house/ (Accessed: 10 October 2024). 'Unreal Engine 5' (2022). Epic Games, Inc. Vilar, E., Filgueiras, E. and Rebelo, F. (2022) Virtual and Augmented Reality for Architecture and Design, Virtual and Augmented Reality for Architecture and Design. CRC Press. doi: 10.1201/9781003051381. Weber, S., Weibel, D. and Mast, F. W. (2021) 'How to Get There When You Are There Already? Defining Presence in Virtual Reality and the Importance of Perceived Realism', Frontiers in Psychology, 0, p. 1538. doi: 10.3389/FPSYG.2021.628298. Williams, S. (2022) Examining the Effects of Disabilities on VR Usage and Examining the Effects of Disabilities on VR Usage and Accessibility Issues for Persons with Disabilities Accessibility Issues for Persons with Disabilities. Available at: https://arrow.tudublin.ie/scschcomdis. Woletz, J. et al. (2018) 'Interfaces of Immersive Media', In Interface Critique Journal, 1. doi: 10.11588/ic.2018.0.44743. WorldViewer (2023) 'WorldViewer'. The Elumenati. Available at: https://www.elumenati.com/ (Accessed: 22 August 2024). Yarwood, J. and El-Masri, S. (2006) Al Muharraq. Miracle Publishing. Yarwood, J. R. (2000) Al Muharraq: Architecture of a Traditional Arabian Town in Bahrain. Arts & The Islamic World .