Deliverable D2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments.

Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 1 Deliverable D2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 2 Document Information Document Identification Status Final Due Date 31 October 2024 Version 6.0 Submission Date 30 October 2024 Related WP WP1 Document Reference D2.1 Related Task(s) 2.1 Document Type Report Related Deliverable(s) D2.3 Dissemination Level PUBLIC Lead Participant NKUA Lead Author Dimitris Charitos Contributors NKUA Reviewers Adnan Hadziselimovic (UM) FBKW Reviewers Bruno Vandermeulen (KU Leuven) K8 Exploded View Author(s) First Name Last Name Partner Dimitris Charitos NKUA Penny Papageorgopoulou NKUA Charalampos Rizopoulos NKUA Caterina Antonopoulou NKUA Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 3 Document History Version Date Modified by Modification reason 1.0 27/8/2024 Dimitris Charitos First draft 2.0 27/9/2024 Charalampos Rizopoulos Internal review 3.0 4/10/2024 NKUA Team Internal reviewer’s comments implementation 4.0 18/10/2024 Adnan Hadziselimovic, Bruno Vandermeulen Second internal review George Anastassakis NKUA Iouliani Theona NKUA Evangelia Kovatsou NKUA Björn Stockleben FBKW Evgeny Kalachikhin FBKW Cihan Biyikli K8 Michael Schmitz K8 Jan Tretschok K8 Kristin Ullmann K8 Soenke Zehle K8 Marcin Klimek ExV Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 4 5.0 21/10/2024 NKUA Team Second Internal reviewer’s comments implementation 6.0 30/10/2024 Łukasz Pieczonka (JU); Żaneta Żegleń (JU). Quality Control and final version Quality Control Role Who (Partner short name) Approval Date Deliverable leader Dimitrios Charitos (NKUA) 24/10/2024 Quality manager Łukasz Pieczonka (JU) 30/10/2024 Project Coordinator Żaneta Kubic (JU) 30/10/2024 Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 5 Abstract In line with the aims of WP2 of the IMPULSE project, this report highlights the current technological landscape for the purpose of facilitating the development of decentralised, open-access solutions for reusing, recycling, and / or upcycling existing digitised cultural heritage content in a way that will bolster user engagement via the adoption of novel eXtended Reality (XR) technologies by cultural institutions. The second chapter of the document revolves around the concept of interconnected multi-user virtual worlds that constitute the Metaverse. It examines the transition from existing standards and practices towards what may be named ‘Web 4.0’, a new landscape that incorporates virtual worlds in ways that reinforce interoperability and reflect the values and principles of the European Union. This delineation of current and projected future trends is important for the requirements analysis of the system to be developed in the context of WP2. The third chapter presents a number of indicative case studies of multi-user VR for the presentation of cultural heritage content. Limitations that are present in traditional interaction methods are identified and ways of reducing their impact via the adoption of novel interaction paradigms are highlighted. Selected case studies, including the ones designed and developed with the participation of project partners, are analysed in greater detail in order to illustrate the opportunities that arise through the use of environmental storytelling in multi-user virtual worlds that feature cultural heritage content. The chapter ends with an analysis of the above-mentioned case studies according to an extensive list of criteria, by which multi-user virtual environments for cultural heritage may be presented, described, and analysed. These criteria are identified to be in line with the aims and objectives of IMPULSE. The following chapter examines the design trends and the interaction modalities that enhance the User Experience of social VR interfaces; communication methods and technological aspects that underpin effective social VR experiences are also examined in this context. The chapter in question highlights the tendency of providing means of incorporating nonverbal communication into the intended use of social VR platforms through the use of Head Mounted Displays, motion controllers, immersive auditory setups, face, hand and body tracking, motion capture, and other advanced interaction methods (e.g. Brain-Computer Interaction). In the next chapter, existing content aggregators are examined in light of the objectives of WP2 and the IMPULSE project as a whole. An exploratory research into open databases for cultural heritage identifies the advantages and limitations of current databases. Key findings reveal significant challenges in the digital cultural heritage ecosystem, including repository fragmentation, heterogeneous metadata standards, and varying degrees Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 6 of data accessibility (and therefore discoverability). Furthermore, the APIs of Sketchfab and Europeana are examined in greater detail, concluding that their integration into IMPULSE’s pilots would be possible but would require custom development in order to overcome the differences of these two platforms in terms of data structures and content types. Following up from the preceding chapter, a detailed overview of the state of the art of current software solutions for the development of social VR applications and multiuser virtual environments is provided. This review leads to the identification of two main categories, reflecting respective creative approaches: (i) online, proprietary platforms available as services and (ii) development platforms, tools and components for in-house application implementation. The benefits and drawbacks of each option are then identified, along certain key axes pertaining to the IMPULSE project, the concept of the Metaverse for Cultural Heritage, education and artistic creation, while taking into account the current availability of numerous development aids. This investigation leads to a preliminary recommendation according to which the in-house development of a low TRL platform via the IMPULSE project, is an appealing option. The concluding chapter of this report highlights a number of insights gained through the investigation of the current technological landscape, as detailed in the preceding chapters: (i) a preference for visual fidelity over designing and implementing effective and efficient collaboration among users, (ii) comparatively low user interactivity with the featured content (as complex editing remains elusive), which prevents large-scale citizen involvement in the preservation and (re)interpretation of cultural heritage content, (iii) a predominance of custom-made solutions for specific projects, (iv) an increased tendency to incorporate more natural and intuitive interaction modalities for user communication in the context of social VR, and (v) the need for standardisation across cultural heritage content repositories, databases, and platforms. Key words: Cultural Heritage, Virtual Reality, eXtended Reality, Social Virtual Reality, Multi-user Virtual Environments, Virtual Worlds, Metaverse, API, Embodied and Multimodal Interaction, Content Aggregators. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 7 Abbreviations and Acronyms Abbreviation / acronym Description DX.X Deliverable number X belonging to WP X EC European Commission WP Work Package CH Cultural Heritage 3D Three-dimensional VE Virtual Environment VR Virtual Reality XR eXtended Reality MUVE Multi-User Virtual Environments CVEs Collaborative Virtual Environments VWs Virtual Worlds IVWPs integrated virtual world platforms API Application Programming Interface DDL Direct Download Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 8 Table of Contents Document Information .................................................................................................................. 2 Abstract ........................................................................................................................................... 5 Abbreviations and Acronyms ........................................................................................................ 7 Table of Contents ........................................................................................................................... 8 1 Introduction ............................................................................................................................... 10 2 Definition of the Metaverse, Virtual Worlds & Multi-User Virtual Environments (MUVEs 12 2.1The evolution of Social Virtual Reality (VR) ............................................................................................................. 12 2.2 Definitions of Social VR environments .................................................................................................................... 14 3 Investigating case studies of using MUVEs for presenting Cultural Heritage content ...... 18 3.1 A general overview of case studies of developed VEs as contexts for the presentation of CH content ............................................................................................................................................................................................... 20 3.2 An analysis of selected case studies of developed VEs as contexts for the presentation of CH content ....................................................................................................................................................................................... 29 3.2.1 Case study 1: BRIDGES ............................................................................................................................................ 30 3.2.2 Case Study 2: "The Lost Time" - A VR Journey through History .......................................... 344 3.2.3 Case Study 3: Digital Renaissance in the Wake of Tragedy - The Restoration of Notre-Dame de Paris ..................................................................................................................................................... 366 3.2.4 Case Study 4: Underwater Malta Virtual Museum ...................................................................... 388 3.2.5. Case Study 5: WWII Game: Defending the Island Fort (Work in progress) ................. 411 3.3 Cultural Heritage in Multi-User VR Applications ............................................................................................ 444 3.3.1 Case Study 6: CREATE project ....................................................................................................................... 444 3.3.2 Case Study 7: Alt-Segeberger Bürgerhaus Virtual Museum ...............................................466 3.3.3 Case Study 8: The Scottish Lewis Queen Chess Piece .............................................................. 477 3.3.4. Case Study 9: The FIRB Project and The Virtual Museum of Ancient Via Flaminia ........................................................................................................................................................................................................ 499 3.3.5. Case Study 10: ArkaeVision .......................................................................................................................... 500 3.4 Identifying criteria/questions according to which case studies will be presented and analysed............................................................................................................................................................................................. 511 4 State of the art of XR interfaces used to interact with MUVEs .......................................... 577 4.1 Introduction ......................................................................................................................................................................... 577 4.2 Design Choices in Social VR ........................................................................................................................................ 599 4.3 Communication in Social VR ......................................................................................................................................... 60 4.4 Multimodal Human-Computer Interaction ......................................................................................................644 Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 9 4.5 Technological infrastructure for Social VR ....................................................................................................... 666 4.6 Techniques of integrating facial and gesture recognition in XR aspects of MUVEs (EV) ........... 71 4.6.1 Introduction..................................................................................................................................................................... 71 4.6.2 Benefits of using facial and gesture recognition and tracking .........................................722 4.6.3 Review of existing techniques and hardware technologies ............................................... 733 4.6.4 Review of existing techniques and software technologies ................................................. 755 4.6.5 Opportunities and Limitations of Current VR Technology ..................................................... 777 5 Integration of IMPULSE with existing content aggregators ............................................... 799 5.1 How to access files via Internet ............................................................................................................................... 799 5.2 State of the Art-Overview of CH Platforms with 3D Assets ........................................................................ 81 5.2.1. Sketchfab .................................................................................................................................................................... 822 5.2.2. Europeana ................................................................................................................................................................. 844 5.2.3. Smithsonian 3D Digitization ......................................................................................................................... 855 5.2.4. CyArk ............................................................................................................................................................................. 877 5.2.5 Open Heritage 3D .................................................................................................................................................888 5.2.6. Metropolitan Museum of Art .......................................................................................................................... 90 5.2.7. The British Museum ............................................................................................................................................... 90 5.2.8. Wikimedia ................................................................................................................................................................. 923 5.2.9. Morphosource ........................................................................................................................................................ 933 5.2.10. Virtual Curation Lab ......................................................................................................................................... 944 5.2.11. Zamani Project and Sudan Memory ..................................................................................................... 955 5.2.12. Other relevant initiatives………………………………………………………………………………………………………………………..95 5.3. Chapter summary ........................................................................................................................................................... 977 5.4. Investigating APIs afforded by content aggregators .................................................................................988 5.4.1 Definition of an API ................................................................................................................................................988 5.4.2 API Access ..................................................................................................................................................................... 101 5.4.3. API Comparison Europeana vs. Sketchfab ......................................................................................... 113 5.4.4. Metadata Comparison ...................................................................................................................................1154 5.4.5. Potential for AI-Generated Metadata ................................................................................................. 1176 5.4.6. API Compatibility Conclusion .................................................................................................................... 1176 6 State-of-the-art of existing software solutions supporting the development of MUVEs .................................................................................................................................................... 1187 7 Discussion and preliminary conclusions .............................................................................. 124 References .................................................................................................................................. 127 Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 16 The European Commission adopts the term virtual worlds (VWs) in relation to these online digital communication environments. According to the “Communication from the Commission to the European Parliament, The Council, the European Economic and Social Committee and Committee of the Regions” (European Commission, 2023)2: “Virtual worlds are persistent, immersive environments, based on technologies including 3D and extended reality (XR), which make it possible to blend physical and digital worlds in real-time, for a variety of purposes such as designing, making simulations, collaborating, learning, socialising, carrying out transactions or providing entertainment.” In the above-mentioned report of the EC, virtual worlds are also related to the following terms: ● Web 3.0 is the third generation of the World Wide Web. Its main features are openness, decentralisation, and users’ full empowerment enabling them to control and realise the economic value of their data, manage their online identities and participate in governing the web. Semantic web capabilities allow linking data across web pages, applications and files. Decentralised technologies and digital twins enable peer-to-peer transactions, transparency, data democracy and innovation along entire value chains. ● Web 4.0 is the expected fourth generation of the World Wide Web. Using advanced artificial and ambient intelligence, the internet of things, trusted blockchain transactions, virtual worlds and XR capabilities, digital and real objects and environments are fully integrated and communicate with each other, enabling truly intuitive, immersive experiences, seamlessly blending the physical and digital worlds. (European Commission, 2023, p. 1-2) The multi-user online platforms known as virtual worlds represent, and in many ways simulate, three-dimensional spatial experiences, and provide their users with resources to personalise their communicative environments. These online platforms facilitate multimodal communication and real-time interaction in computer-generated representations of three-dimensional space by the use of avatars. The socio-technical characteristics of interaction in these virtual places provide specific affordances for verbal and non-verbal communication (Schroeder, 2011) and the use of avatars as personal mediators (Jensen in Gürsimsek, 2014). Furthermore, the visitors and creators of these spatial representations interact, socialise and cooperate for various purposes thus, socially transforming VWs into meaningful places through their interactive experiences. 2 Communication from the Commission to the European Parliament, The Council, the European Economic and Social Committee and Committee of the Regions, URL: https://digital-strategy.ec.europa.eu/en/library/eu-initiative-virtual-worlds-head-start-nexttechnological-transition Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 17 Virtual worlds are seen as an important part of this transition to Web 4.0. They are already opening up a wide range of opportunities in many societal, industrial and public sectors (European Commission, 2023, p.1). Virtual worlds can boost the cultural and creative industry, from fashion to video games, cultural heritage, music, visual arts and design, by offering new ways to create, promote and distribute European content and engage with audiences. (European Commission, 2023, p.4) Ultimately, the Commission’s vision and strategy aims for a Web 4.0 and virtual worlds that reflect EU values and principles and fundamental rights, where people can be safe, confident and empowered, where people’s rights as users, consumers, workers or creators are respected, and where European businesses can develop world-leading applications, scale up and grow. Furthermore, the Commission aims for a Web 4.0 that is powered by open and highly distributed technologies and standards that enable interoperability between platforms and networks and freedom of choice for users, and where sustainability, inclusion and accessibility are at the core of technological developments. The EU’s Single Market, rich and diverse culture, creative content, strong industrial base, excellence in research, innovation and education, and robust legislative framework should be drivers to Europe’s leadership, competitiveness and technological sovereignty in this field. (European Commission, 2023, p.4) The identification of Metaverse and characteristics of VWs, according to the abovementioned definitions, is intended to delimit the boundaries for the analysis of application requirements in the following steps of D.2.1. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 18 3 Investigating case studies of using MUVEs for presenting Cultural Heritage content The integration of immersive technologies, such as Virtual Reality (VR) and Augmented Reality (AR), represents a significant advancement in the field of cultural heritage presentation and online publication. These technologies address the limitations inherent in traditional methods by offering novel, interactive experiences that transcend temporal and spatial constraints. Recent academic research has elucidated the ways in which VR and AR can enhance the accessibility, inclusivity, and engagement of cultural content. Cecotti (2022) explores the integration of cultural heritage within fully immersive VR environments, highlighting the transformative potential of VR technology in digitally preserving and presenting tangible, intangible and natural cultural inheritance. By delving into the methodologies and technologies employed, Cecotti (2022) highlights the capacity of VR to foster immersive engagements with cultural heritage, transcending temporal and spatial constraints, allowing for broader access with improved inclusion, diversity, and equity of a wider audience. More specifically, the categorization of several applications is attempted, based on their content, ranging from broad art galleries to focused exhibits on specific artworks or artists. Moreover, various methods for assessing the performance of such applications are evaluated, including workload, usability, flow, and potential VR symptoms surveys. Challenges related to interdisciplinary collaboration, educational deployment, gamification aspects, and social interaction are discussed, emphasising the necessity for cultural heritage practices within virtual environments to encompass the inclusion, diversity, equity, access, and success (IDEAS) principles. Despite the absence of synchronous social experiences in most of the current VR applications, users can still share experiences and interact asynchronously through digital tools like discussions and reviews. Furthermore, a growing trend involves the inclusion of virtual humans in cultural-related VR applications to enhance immersion and scale perception. Li and Cesar (2023) provide an extensive overview of Social VR applications and user experiences, delineating various Social VR platforms and their attendant features, including the field of cultural heritage. More specifically, the authors provide an overview concerning the design, implementation, and real-world deployment of social VR applications across various domains, which facilitate distant interpersonal communication within contexts such as personalised healthcare, celebratory events, interactive exploration of cultural heritage, and immersive entertainment experiences. They further focus on two experimental methodologies pertaining to the creation and validation of a social VR questionnaire through a user-centric approach, as well as Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 19 the assessment of visual fidelity of photorealistic avatar representations through different degrees of freedom (3DoF and 6DoF). The authors reach the conclusion that the advancement of Social VR necessitates the establishment of standardised protocols encompassing both qualitative and quantitative measures to evaluate user interactions. Furthermore, there is a need for standardised procedures dictating the deployment of Social VR applications within real-world settings such as hospitals and museums. Giovannini and Bono (2023) present a case study on the creation of virtual reality experiences within a social virtual environment, and more specifically Mozilla Hubs, probing the fusion of physical and digital spaces to engender phygital exhibitions. The case study endeavours to create a digital replica of the temporary Phygital Exhibition held at the Sordevolo Passion Museum in the Church of Santa Marta, commencing in July 2022. Unlike the museum's permanent collection, which narrates the tradition of popular theatre in Sordevolo concerning the Passion of Christ performance, the Phygital Exhibition centres on documenting the design and construction of the scenography. Within the virtual environment, the physical space is replicated with minimal detail to emphasise the VR exhibition, featuring interactive panels showcasing historical documentation, drawings, and images. Additionally, the virtual space incorporates video content and 3D models unavailable in the physical space, providing insights into the evolution of the scenography over time. By integrating physical artefacts with digital content, Giovannini and Bono (2023) exemplify the potential of social VR to transform the way we engage with cultural heritage, blurring the boundaries between the physical and virtual worlds. The authors explore the digital curation of immersive and virtual environments within the cultural heritage sector, with their primary research objective focusing on the establishing a methodological and operational workflow for developing virtual environments using social virtual environments. Ch’ng et al. (2023) investigate the case of social augmented reality, giving insights into its capacity to facilitate communication and interaction around cultural heritage. By overlaying digital content onto physical environments, the developed augmented reality application blurs the boundaries between the physical and virtual realms, offering users unprecedented opportunities to engage with cultural artefacts and sites in situ. More specifically, within the context of this study, a mobile augmented reality application has been developed aiming to multi-user interaction, exploring the replication and expansion of discussions around cultural heritage objects, in order to shed light into social communication dynamics within augmented reality applications when co-viewing heritage objects. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 20 3.1 A general overview of case studies of developed VEs as contexts for the presentation of CH content As presented in the previous section, VR has significantly transformed the field of cultural heritage by providing innovative ways to not only document and present CH artefacts, sites and practices but also to re-interpret them, make them more accessible to a wider audience, and engage audiences in novel ways. In this section we present an overview of selected case studies of multi-user online VR environments for Cultural Heritage. The majority of these experiences allow multiple visitors from remote locations to interact and navigate inside the virtual environments, simultaneously. Some of the selected case studies support access and interaction between more than one user via local networks, rather than the internet. Finally, there are a few case studies of online single-user cultural heritage VR applications. These VR applications follow various approaches regarding the integration and use of CH content. Some of them recreate CH artefacts, monuments, and sites, ensuring that they are preserved digitally, and they can be accessed easily by a broader audience. In cases of CH items that have been destroyed or significantly altered over time, their virtual reconstructions provide a way to visualise and understand how these items looked and functioned in their original context. Additionally, in some cases, visitors can interact with the virtual CH items in ways that are often impossible with the material CH items. An additional advantage of online VR environments is that they remove geographical barriers, allowing people from all over the world to access cultural heritage sites and artefacts remotely. Visitors can also virtually experience intangible CH practices by immersing themselves in interactive environments that revive these practices. VR storytelling enhances visitors' experiences by allowing them to engage with CH content in emotionally engaging and memorable ways. Lastly, some of the presented case studies allow users to engage in collaborative activities with the objective of exploring or reappropriating CH items. The selected case studies span from informal educational environments and edutainment projects to virtual museums or exhibitions. One of the primary instances is VIVE ARTS3, which offers users a platform to explore virtual art galleries and exhibitions, transcending the constraints of physical space and fostering enriched engagement with art and culture, through the implementation of VR, XR and blockchain technologies. By adopting the aforementioned technologies and artistic content, VIVE ARTS redefines the boundaries of cultural engagement in the digital age, enabling institutions and organisations to experiment in preserving cultural heritage to democratise creation through digital innovation in the arts, reaching a worldwide audience. VIVE Arts is more broadly associated with curating and showcasing 3 Vive Arts | https://www.vivearts.com/ Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 21 cultural content in virtual reality, often through individual experiences rather than multiuser social environments. The Museum of Other Realities4 stands as a pioneering example of an immersive virtual museum, showcasing digital artworks and interactive exhibits that redefine traditional notions of museum curation and exhibition. By harnessing the power of VR, The Museum of Other Realities creates a dynamic and interactive space for cultural exploration and exchange. The Museum of Other Realities (MOR) represents a paradigm of a multiuser social virtual reality (VR) platform. Functioning as a shared virtual space, MOR enables global users to meet and partake in various activities, including exploration of digital art exhibitions, interactive engagement with fellow visitors, and immersion in curated artworks collectively. By fostering a collaborative environment conducive to interactivity and cultural exchange, MOR epitomises the integration of social dynamics within the realm of virtual reality, particularly in the context of cultural experiences and artistic appreciation. The ‘Silk Road’ VR and AR Experience aims to recreate the historical Silk Road through VR and AR technologies, offering users a virtual journey through this historical trade route and its cultural significance. Through a blend of historical narrative and immersive technology, the ‘Silk Road’ VR and AR Experience provide users with an experiential understanding of the cultural heritage embedded within this ancient trade network. The researchers created a virtual environment developed with Unity 3D game engine. The environment consisted of digitised Chinese relics, containing six photogrammetry reconstructed cultural heritage objects. The CH objects could be experienced through both an HTC Vive Head-Mounted Display and a mobile augmented reality application, serving as an interface connecting the VR and AR worlds. The experience enabled interaction between VR and AR users (Li et al, 2018). For example, when an AR user rotated an augmented object, the same object rotated in the VR environment triggering a sound effect, thus providing visual and auditory cues to draw VR users’ attention towards the object (Li et al, 2018). Using the ‘Silk Road’ Experience as a case study Li et. al. (2018) investigate hybrid VR and AR in a multi-user application and study users’ acceptance of the technologies in terms of social influence, performance expectancy, effort expectancy, and behavioural intention (Li et al, 2018). 4 Museum of Other Realities - https://www.museumor.com/ Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 22 Figure 1. The ‘Silk Road’ VR and AR Experience (Li, Y., Ch’ng, E., Cai, S., & See, S., 2018) Çatalhöyük VR offers users an immersive experience of the ancient Çatalhöyük settlement, adopting VR technology to transport users to this archaeological site and foster engagement with ancient history and culture. By reconstructing the physical environment of Çatalhöyük in virtual space, Çatalhöyük VR enables users to explore and interact with the past in novel ways (Katifori et al, 2021). The immersive experience at Çatalhöyük involves participants engaging in collaborative enactments of simplified cultural activities based on archaeological hypotheses, such as wall plastering and grave offerings. Participants are prompted to reflect on ancient and modern practices and consider their contemporary significance through open-ended questions. To enhance participant collaboration, a unique interaction mechanism called the 'high five' paradigm was developed, encouraging collaboration while granting users and VR designers greater control over the experience. In order to perform the high five, the user avatars must approach one another and simulate the high five gesture by touching their virtual palms. The high-fiving seemed to support the objective of providing a sense of shared experience of re-enactment, dialogue and reflection. It fostered participants' feeling of “acknowledgement and success for a task, thus contributing towards a positive emotional feedback loop of establishing joint attention to the space, committing together to a task, bringing it to a successful closure and then confirming this success through the gesture” (Katifori et al, 2021). The Çatalhöyük VR experience is implemented through a server based multi-user application and has been developed with the Unity 3D game engine. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 23 Figure 2. Çatalhöyük VR5 Santiago de Compostela VR provides users with a virtual tour of the historic city of Santiago de Compostela, enabling exploration of its rich cultural heritage and architectural history (Flores et al, 2000). With the adoption of VR technology, Santiago de Compostela VR creates an immersive and educational experience that transports users to the heart of this historic pilgrimage site. The experience allows for single-user and multi-user interaction, while the project was implemented with the adoption of VRML & Java technologies. 5 Çatalhöyük VR | source: narralive.itch.io/catalhoyuk-vr Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 24 Figure 3. Santiago de Compostela VR (Flores, J., Arias, J. E., Saavedra, S., Varela, E., Ferro, J. M., & Taboada, J. A., 2000) BEYOND MATTER | Cultural Heritage on the Verge of Virtual Reality is a project aimed at preserving cultural heritage through immersive VR experiences, highlighting the potential of VR to democratise access to cultural heritage and foster a deeper appreciation for the shared human history. More specifically, this practice-based research project delved into multifaceted research activities focusing on the intricacies of virtual reality. By engaging with contemporary shifts in visual art production and mediation within modern and contemporary art museums, BEYOND MATTER addresses the profound impact of rapid advancements in computer science, information technology, and the increasing utilisation of augmented and virtual reality, alongside artificial intelligence. In contrast to physical exhibition spaces, which rely on spatial properties to contextualise artworks, virtual exhibition models offer immersion through the interaction between materiality and representation, irrespective of geographical constraints. BEYOND MATTER aims to explore the interdependence between physical and virtual spaces and understand its implications on art production, curation, and mediation. By investigating the "virtual condition", the project proposes innovative approaches to preserving cultural heritage and harnessing the potentials of digital world- Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 25 making. Within the course of the project, different research activities have been conducted, including a workshop which took place in Mozilla Hub's online platform, enabling multi-user immersive VR interaction. The workshop took place in a shared immersive virtual reality space built with Mozilla Hubs. Figure 4. ‘Make and Share Stories’ PORe Workshop, Beyond Matter project.6 6 Make and Share Stories’ PORe Workshop, Beyond Matter project | source: https://beyond-heritage.aalto.fi/pore-workshop-model1/ Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 32 Figure 7. “A day in Ancient Athens” XR Experience, at the Foundation of Hellenic World. © BRIDGES project In the context of the BRIDGES project, an iterative, user-centred methodology was followed to elicit user requirements and to design various scenarios for the XR experiences. This methodology involves a four-phase process, including the following phases: 1) understanding and describing the context, 2) defining user groups through personas, 3) codifying and categorising user requirements, and 4) eliciting and prioritising user requirements. This process aimed to balance the diverse requirements of different user groups and application contexts. Furthermore, it attempted to create an inclusive XR experience considering the diversity of target users: staff and visitors of museums, cultural heritage and other kinds of informal education or recreational institutions. This target group can show a great diversity in terms of age, motivations, interests, physical condition, profession, language, level of interaction and participation, familiarity with technology, abilities, or cultural background. At the core of the BRIDGES solution lies the utilisation of the pre-existing Immersive Deck platform, initially developed by the Technical University of Vienna and subsequently enhanced by the SME Illusion Walk, a key industrial partner within the BRIDGES consortium. The Immersive Deck consists of a set of technologies and tools, assembled into a complete, low-cost platform. It includes positional tracking for a group of up to 10 people concurrently. Users wear high-end Head Mounted Displays (HMDs) connected to laptops carried in backpacks, which render locally the virtual world in the HMDs. A stereo camera attached to each HMD performs inside-out optical hand and finger tracking allowing the detection of hand movements for haptic interaction. This setup Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 33 enables natural interaction and free-roam navigation, by walking in large, even multiple rooms. The key technical and operational features of the XR platform include: 1) a mixed reality setting of any size and configuration (e.g. multiple rooms, corridors, etc.), combining physical and virtual interaction; multi-user concurrent interaction in the physical and virtual space, supporting a sizeable number (e.g. 10) of coand remotely-located participants; 2) a low cost solution for tracking the physical area and mapping the virtual world onto the built environment, including the architecture, objects, machinery, equipment and any object related to the scenario of each application; 3) intuitive interaction, where hands and fingers are tracked optically and represented in the virtual world, thus eliminating the need for hand controllers; users can shake hands as they do in real life and navigate the virtual naturally by physically walking around in the real world; 4) multi-sensorial stimulus, improving immersion and maximising the feeling of presence by incorporating effects such as wind, heat, smell, vibration, in addition to the visual, auditory and tactile (El Raheb, et. al.; Schönauer, et. al.) The platform also offers a highly immersive environment, enriching training and learning experiences through immersive XR technologies to foster engagement and retention. Additionally, its modular and customizable system ensures adaptability and flexibility, catering to diverse user requirements and scenarios, thus ensuring seamless integration within different contexts. Figure 8. “A day in Ancient Athens” XR Experience, at the Foundation of Hellenic World. © BRIDGES project Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 34 3.2.2 Case Study 2: "The Lost Time" - A VR Journey through History "The Lost Time" implemented by Film University Babelsberg KONRAD WOLF offers an immersive 6 Degrees of Freedom (6 DOF) cinematic virtual reality experience that meticulously recreates the environment of the Theresienstadt ghetto and concentration camp using advanced scanning and photogrammetry. Alongside this, the project brings to life a 1930s Berlin apartment and Auschwitz concentration camp to set the stage for a deeply moving narrative. This VR journey is anchored in the life of Margot Friedländer, a Holocaust survivor, who shares her harrowing experiences from one of history's darkest periods. The story begins in Berlin, 1943, as Margot's mother and younger brother Ralph are arrested by the Gestapo and deported. Unaware of her family's fate, Margot finds refuge in the Berlin underground. However, after 15 months in hiding, she is captured and sent to Theresienstadt, holding onto the hope of reuniting with her family. Upon her arrival, Margot is faced with the devastating reality that both her mother and brother had been murdered in Auschwitz. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 35 Figure 9. "The Lost Time" by Film University Babelsberg Konrad Wolf Figure 10. "The Lost Time" by Film University Babelsberg Konrad Wolf Figure 11. "The Lost Time" by Film University Babelsberg Konrad Wolf "The Lost Time" leverages the power of digital technology to preserve and convey historical narratives, providing an educational tool that immerses viewers in the personal Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 36 and collective tragedies of the Holocaust. Through the recreation of significant historical sites and personal stories in VR, the project offers a unique and impactful way to foster empathy and understanding, inviting a deep, interactive engagement with history. 3.2.3 Case Study 3: Digital Renaissance in the Wake of Tragedy - The Restoration of Notre-Dame de Paris The devastating fire at Notre-Dame de Paris vividly illustrated the urgent necessity for proactive preservation and underscored the vital role digital technologies play in the conservation and restoration of cultural heritage. This tragic event not only emphasised the fragility of our historic monuments but also showcased the innovative ways digital resources can aid in their recovery and preservation. A notable aspect of Notre-Dame’s restoration effort involved the utilisation of a detailed 3D model from the video game "Assassin's Creed Unity," developed by Ubisoft. Caroline Miousse, a senior level artist at Ubisoft, had dedicated years to the cathedral's digital recreation, achieving an extraordinarily detailed representation that meticulously mirrored the actual architecture, down to the texture and form of individual bricks. In parallel, the comprehensive 3D laser mapping conducted by the late art historian Andrew Tallon provided an invaluable resource. Tallon's work, capturing the cathedral with precision accuracy to within five millimetres through over one billion data points from more than 50 locations, offered a highly detailed and precise digital blueprint of Notre-Dame prior to the fire. The synergy between Miousse’s digital reconstruction and Tallon’s laser scans exemplifies the transformative potential of digital tools in cultural heritage preservation. These digital assets have become crucial to the ongoing and meticulous restoration efforts, providing accurate measurements and a deep understanding of the cathedral's structural nuances. This case study highlights the indispensable role of digital innovation in the restoration processes, demonstrating how such technologies are critical in ensuring the resilience and continuity of our cultural heritage in the face of unforeseen calamities. Through the lens of the Notre-Dame restoration, we see a broader mission crystallise: to employ digital innovation not merely as a preservation tool but as an essential instrument in the restoration, revitalization, and enduring safeguarding of the world’s cultural heritage. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 37 Figure 12. Notre-Dame’s digital restoration Figure 13. Notre-Dame’s digital restoration Figure 14. Notre-Dame’s digital restoration Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 38 The integration of VR and AR technologies and applications in the Cultural Heritage field has transformed the way users engage with the historical content. Implementations of Mixed Reality advanced technologies have created immersive environments for users to interact with, offering a more realistic experience. The following projects highlight the potential these technologies have in enhancing the users’ experience in cultural heritage virtual representations. 3.2.4 Case Study 4: Underwater Malta Virtual Museum The Underwater Malta - Virtual Museum15 is a project by the Underwater Cultural Heritage Unit of Heritage Malta. The initiative aims to make Underwater Cultural Heritage (UCH) sites accessible to the public and promote the preservation of maritime history through immersive technologies, including Virtual Reality. Underwater Malta also engages in historical research to provide context to each site, making the museum both educational and visually engaging (Gambin et al, 2001a). The sea is often described as the world’s largest museum, containing traces of human existence that have a cultural, historical or archaeological character, including an estimated three million shipwrecks, their cargoes, aircraft wrecks, submerged prehistoric cultural landscapes, submerged ports, and harbour structures (Gambin et.al, 2001). UCH offers invaluable insights into human history yet remains largely inaccessible to the broad public. While diving provides physical access to some sites, the vast majority of UCH remains out of reach. The UNESCO Convention has emphasised in-situ preservation and public sharing of these underwater sites, by balancing scientific research, protection, and the promotion of responsible access to underwater cultural heritage sites (Gambin et al, 2001b). Underwater Malta aims at addressing this challenge by using 3D photogrammetry, Virtual Reality and other digital technologies to document and display UCH sites. The Underwater Malta website is dedicated to showcasing Malta's underwater cultural heritage. It features a virtual museum where visitors can explore 3D models of underwater archaeological sites, such as shipwrecks and aircraft from various historical periods, including World War II. The Virtual Museum allows the visitors while they are exploring the sites online to decide to click on the different 3D models and zoom in and out and look at them from different angles. The digital visitors have the option to view the models via immersive VR as well. There are annotations with details and information about each underwater model. Only the models have been scanned and 15 Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 39 not the whole archaeological sites, therefore allowing a fragmented but detailed experience. Figure 15. 3D model of the SS Polynesien shipwreck © Underwater Malta - Virtual Museum The creation of virtual 3D models involves multiple steps, including remote sensing surveys, diver documentation, and data processing. Remote-operated vehicles (ROVs) and underwater robotics are used to capture data, especially for deep-water sites. Highresolution cameras, drones, and 3D modelling, and photogrammetry software are employed to reconstruct and display these cultural sites online (Gambin et al, 2001a). The data gathered throughout the data capture process was used to produce various visualisations: • A still from the 3D reconstruction of the wreck onto a matching seabed, that is presented on the landing page of Underwater Malta. • A 3D model that is simple to navigate and is fast to load. This is used for the landing page of the wreck-specific sub-site. These models are supplemented by annotations and links to other assets for that same wreck. Textual information on the background and history of the site, as well as archival photos and videos, are also included. • A video “flythrough” of the wreck, taking the viewer through a fixed path that simulates a diving experience, and allows for a diver’s perspective. • A full 3D model that can be zoomed, spun around, and examined at a detailed level. This allows the visitor to use a VR set to experience the model in a 1:1 scale. (Gambin et al, 2001a) Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 40 Various visualisations ensure that users with varying background in computer literacy could experience and interact with the online platform and access information about the displayed UCH sites. The 3D models have been created through photogrammetry. The data for the 3D models was captured using both video and stills. The sites presented on Underwater Malta lie in waters of various depths, ranging from 2 m to 120 m. The University of Malta used a low-cost 3D modelling process, using Structure from Motion (SfM) photogrammetry. In order to reconstruct models of UCH sites lying in deep waters the deep-water photogrammetry process was developed (Gambin et al, 2001a). Figure 16. 3D model of the Maryland wreck © Underwater Malta - Virtual Museum Underwater Malta bridges the gap between in-situ preservation and public engagement by offering virtual access to Malta's rich underwater cultural heritage. The way that VR has been approached so far focuses on accessibility, engagement and educational impact. More concretely, VR makes it possible for a global audience to experience Malta's rich underwater and historical sites, which would otherwise be inaccessible due to physical, logistical, or preservation constraints. VR serves as a medium for digital preservation, capturing detailed 3D models of fragile sites, which can be studied and enjoyed without risking damage to the originals. By offering immersive and interactive experiences, VR captures the interest of diverse audiences, including younger generations, who might not engage with traditional museum exhibits. Finally, the use of VR enhances learning by allowing users to explore and interact with historical Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 41 environments in a way that is both informative and memorable. This approach encourages deeper understanding and retention of historical knowledge. 3.2.5. Case Study 5: WWII Game: Defending the Island Fort (Work in progress) This case study is a work in progress by Heritage Malta, consisting of a first-person Virtual Reality game, set in St. Elmo’s fort, in Valletta-Malta during World War II. The working title of the project is ‘Defending the Island Fort’ or ‘Defending Fortress Island’. The agency’ s goal is to create hybrid experiences where digital information is overlaid on the physical world. This approach potentially enhances the educational or artistic experience by adding interactive layers to real-world objects. During the game, players operate a physical replica of a World War II autocannon, in order to defend St. Elmo’s fort from aerial attacks. The focus is placed on creating an experience that balances historical accuracy, with engaging gameplay, using cut scenes and scripted events to guide the player, while creating an educational VR game that aims to engage players in a speculative yet informative portrayal of a significant wartime event. The use of Virtual Reality is crucial in allowing visitors to experience the sights, sounds, and intensity of a WWII battlefield. By getting immersed into a fully realised 3D environment, players gain a first-person perspective of the historical setting, enhancing the authenticity and educational value of the game. VR technology is leveraged to create a sense of presence, making the player feel as though they are physically situated within the fort. The game’s mechanics are carefully designed to balance realism with accessibility. Players can control a World War II autocannon, the Bofors 40mm gun, targeting enemy aircraft. The gameplay is immersive, with the player's viewpoint restricted to what can be targeted by the cannon, emphasising the intensity and focus required in real-life combat situations. The game’s design is based on the actual layout of St. Elmo’s Fort, with attention to detail in recreating the fort’s architecture, environment, and wartime conditions. The VR experience is enhanced by historical accuracy, from the types of aircraft attacking the fort to the strategic importance of the location. More concretely, the Bofors 40mm autocannon models provide an accurate and realistic representation of this historically significant weapon. Similarly, the aircraft models within the game reflect a high level of historical accuracy, ensuring a high degree of authenticity in the game. Through VR, the game delivers a rich sensory experience. Visual elements such as the changing skies, explosions, and the fort’s surroundings are rendered in high detail, while the audio design includes authentic sounds of gunfire, aircraft, and ambient battlefield noises. The combination of visual and auditory cues in VR enhances Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 48 The utilisation of VR and AR technologies in the cultural heritage field has proven to be transformative, as it introduces users to interactive and immersive experiences that surpass the traditional methods. The projects mentioned above present the diversity of applications that can be developed using these technologies. VR and AR can revolutionise the way users approach, perceive and explore cultural heritage by enhancing their experience. The continuous advancement of these technologies could inevitably shape the future of cultural heritage visualisation. Figure 20. The Haptic Device Prototype (Source: Dima et al., 2014, p. 6) Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 49 3.3.4. Case Study 9: The FIRB Project and The Virtual Museum of Ancient Via Flaminia Forte, Pietroni and Dell’Unto developed research projects that focused on virtual archaeology and on the communication and study of cultural heritage through multi-user virtual reality applications. For this research two case studies were presented: The Virtual Museum of Ancient Via Flaminia and the FIRB (Funds for the Investments of Basic Research) project, “Integrated Technologies of Robotics and Virtual Environment in Archaeology” (funded by the Italian Ministry of Research). The first one has been available in the National Roman Museum of Rome since 2008, while the second one is still in progress but can be accessed through the web since 2008. The FIRB project allowed the creation of a multi-user web domain aimed at a multidisciplinary scientific community. Collaborating with the Department of Archaeology of the University of Pisa and Scuola S. Anna, the project focused on three archaeological sites: Teban tomb 14 in Gurna-Luxor, Temple A in Fayum Medinet Madi, and the ancient settlement of Khor Rori in Oman. These sites varied in characteristics and thus required different technologies for data acquisition, processing, and representation. The virtual environment that came as a result, enabled real-time interaction, hypothesis testing, and collaborative data sharing, supporting continuous evolution and re-elaboration of 3D models and simulations. Users could interact with the 3D models, make modifications, and create new contexts, enhancing learning and scientific communication within the virtual space. Figure 21. TT14, GurnaLuxor: 3D model of TT14, obtained from scanner laser acquisition (Source: Forte et al., 2008, p.6) Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 50 According to Forte, Pietroni and Dell’Unto, the Virtual Heritage Lab of CNR ITABC developed a research and communication project· a Multi-user Virtual Reality installation focusing on the archaeological landscape of ancient Via Flaminia, supported by Arcus S.P.A. The initial part of the road of ancient Via Flaminia was reconstructed by the Virtual Museum of Ancient Via Flaminia, digitising 4.45 acres of terrain. The Milvius Bridge, the area of Grottarossa, Livia's Villa, and the ancient Roman arch of Malborghetto, built by the emperor Constantine were the archaeological sites that were included in the communication system as monographic levels of exploration. According to the different levels of resolution, accuracy, perceptive impact and datasets available for each site, every site was documented through integrated technologies such as topographical survey with GPS, scanner laser, photogrammetry, GISs, total laser station, computer vision, digital photos from aerostatic balloon etc. The avatar-mediated VR system is planned for 4 interactive platforms and an HD stereo display for all the public present in the room. The users will interact in the same virtual space, each one using their own avatar from one platform. Inside the virtual environment, aiming to the discovery of cultural, interpretative and narrative contents associated with the 3D space users share purposes and perform joined actions. A large screen is installed and, wearing 3D glasses, the visitors can watch the real-time movements and actions of the avatars inside the virtual scenario from different points of view. Every time a user discovers a narrative content, interaction on every platform stops and users together with the public can attend a projection of a movie on the large screen. This way a collective experience of common learning is achieved. The immersive experience combines VR exploration, multi-user interaction, storytelling, and stereoscopic vision to engage both tech-savvy and general audiences. Historical characters within the VR environment narrate stories and describe daily activities, bringing the space to life and deepening visitors' understanding of cultural and historical contexts. 3.3.5. Case Study 10: ArkaeVision In a research study conducted by Bozzelli, Raia, Ricciardi, De Nino, Barile, Perrella and Palombini is shown that the ArkaeVision project aims to enhance the experience of Cultural Heritage by creating a more engaging and culturally-rich user experience through the development of a technological infrastructure. This system allows users to interact with Cultural Heritage, including virtual reconstructions of monuments and artefacts. ArkaeVision introduces a new communication model, blending game-like exploration in 3D environments with elements of digital fiction and engaging storytelling, applied to two case studies: the exploration of the Hera II Temple of Paestum with Virtual Reality (VR) technology (ArkaeVision Archeo), and the exploration of the slab of the Swimmer Tomb with Augmented Reality (AR)(ArkaeVision Art). By emotional Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 51 engagement and gamification, ArkaeVision deepens the user interaction and understanding. The system also supports intuitive interactions by offering customised and inclusive experiences with advanced interfaces. Finally, the multi-user mode allows users to share their experiences within an immersive virtual environment. Figure 22. The current Hera temple exterior (in Paestum) (Source: Bozzeli et al., 2019, p.15) 3.4 Identifying criteria according to which case studies will be presented and analysed Following the presentation of the above mentioned case studies of using personal or multi-user VR technologies for presenting CH content, an attempt is made to analyse their implementation in terms of several of their characteristics, as criteria for this analysis: Numbers of users, objective of use, type of content, technological infrastructure, user interface, the extent to which they supported the editing of CH content, the quality of graphical representation presenter and the context within which this production was created. The table below summarises the analysis of these case studies according to their main characteristics: Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 52 Numb er of Users Functionality/Obj ective Content Technologies Interface Extent of Content Editing Graphics’ Quality Context / Production Case Studies Single User MultiUser Virtual Museum Repurposi ng of Cultural Heritage Content Edutainme nt Artifacts Buildings Sites Landscape s Immersive VR Desktop VR AR/MR Other None Editing Manipulati on Compositio n with Other Forms of Content Basic Advanced Photorealistic Academic /research Commerci al Phygital Exhibition X X X X Mozilla Hubs X X X X VIVE Arts X X X X X X X X Museum of Other Realities X X X X X X Silk Road X X X X Unity Photogrammetry X X X X X Çatalhöyük VR X X X X X Unity X X X X Santiago de Compostel a VR X X X X X VRML Java X X X Beyond Matter X X X Mozilla Hubs X X X X ATON Framework X X X Node.js Three.js X X X X Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 53 ArchiSearc h X X X X web-based VR AI X X Da Vinci Effect X X X Museum of the Future X X X X X X X X Burns Beyond Reality X X X X X X Cultural Universe X X X X X x Chronosco peVR X X X X X X X X x X HeriVerse X X X X X x x BRIDGES x x x x x Immersive Deck platform, realtime Holodec, Triple-TrackTM Software x x x x X The Lost Time x x x x X x x X The Restoratio n of NotreDame de Paris x 3D laser mapping/scanning X Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 54 Underwate r Malta x x x photogrammetry x x x X WWII Game: Defending the Island Fort x x x x X x x x X CREATE Project x x X Modelling-fromimages technologies x x x x AAltSegeberger Bürgerhau s Virtual Museum x x x Terrestrial laser scanning, Digital photogrammetry, Unreal game engine x x x x x The Scottish Lewis Queen Chess Piece x x 3D Printing, Haptic Device x x The FIRB Project x x X Laser Scanning, Total Laser Station, Computer Vision, GPS, GIS, Remote Sensing, Photogrammetry, 3D Panorama, 3D Computer Graphics x x x x Ancient Via Flaminia Virtual Museum x x X Total Laser Station, Laser Scanning, GPS, GIS, x x x Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 55 Computer Vision, Photogrammetry, Digital Photos from Aerostatic Balloon ArkaeVisio n x x x X 3D Modelling, Unity Game Engine x x x x Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 56 From the analysis of the above-mentioned case studies that integrate cultural heritage content into virtual environments, we can extract a set of interesting initial remarks and “lessons learned”. Firstly, we observe that all multiuser online virtual environments use basic or advanced quality graphics. Photorealistic graphics are used in single-user and offline virtual environments, as they cannot be fully supported by multiuser and/or online VR applications. Moreover, in most case studies users can virtually navigate inside cultural heritage sites and examine CH content from different angles. In some examples, users are additionally allowed to translate or rotate small-scale CH artefacts. However, more extensive interaction with CH items, including advanced manipulation, editing or composition with other types of content is uncommon. Additionally, cultural heritage content is usually displayed in its original context. It is rarely artistically reappropriated, critically reinterpreted or inserted into new contexts. In the majority of the presented cases, users can experience the virtual environments in immersive mode, using headmounted displays. Some projects, also explore extended or mixed reality interfaces. Finally, in most of the projects, especially research-oriented ones, custom-made solutions have been developed in order to support the required functionality of the virtual environments. It is also important to stress that some of these VEs, based on third-party off-the-shelf platforms, such as Mozilla Hubs, are no longer accessible, after the closure of these platforms. Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 57 4 State of the art of XR interfaces used to interact with MUVEs 4.1 Introduction Social Virtual Reality (VR) represents a significant advancement in digital communication, offering immersive environments for social interaction, education, and entertainment. Social VR enables users to interact in virtual environments using avatars and advanced interaction modalities. This technology has rapidly grown, driven by advancements in VR hardware and software. This sub-chapter examines the design strategies of social VR interfaces, the interaction modalities that enhance user experiences and communication methods and technological aspects and equipment that underpin effective social VR experiences, drawing from key research papers in the field. The technological landscape of VR for cultural heritage applications, as far as interaction modes and modalities are concerned, has been rather aptly summarised in the following diagram by Bekele & Champion (2019, p. 4). Figure 23. Technologies that enable immersive reality (Bekele & Champion, 2019, p. 4). Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 64 presence, i.e. the sense of being in the virtual environment with someone else. Hoppe et al. (2020) showed that the sense of touch (in this case via an artificial hand) may increase the perception of human likeness of artificial virtual agents. Specifically, the perception of agents was positively influenced with respect to perceived agency, perceived copresence, and experienced sense of embarrassment. In the words of Della Longa, Valori, & Farroni (2022, p. 7), “affective touch is fundamental in giving life to the virtual experience, as it is closely linked to emotions, in a mutual influence that nurtures social encounters”. However, the authors note that, in some cases, the high degree of interaction fidelity provided by touch input and output and the resulting high degree of immersion, may in turn lead to users struggling to differentiate between the virtual and the real. Furthermore, the provision of touch when touch is not desired may lead to negative effects - see Slater et al. (2020) for a relevant discussion, as well as Maloney, Freeman, & Robb (2021) for a discussion on ethical aspects of future research on social VR. Spatial Audio Spatial audio systems allow users to experience sound in a more realistic way since they simulate direction and distance of sounds to mimic the real-world experience. The sense of presence in the virtual world is enhanced and the immersion deepens. These sound systems appear especially useful in group conversations and activities designed for multiple users (Ruiz et al., 2022). 4.4 Multimodal Human-Computer Interaction The technological advancement in VR interaction systems, such as improved HeadMounted Displays (HDMs), motion controllers and haptic devices has offered users the chance to communicate both with the environment and each other in more natural, effective and intuitive ways (Mulders & Zender, 2021). Nonetheless, limitations do exist - for instance, there is room for improvement regarding the display quality (resolution, refresh rate) of HMDs; furthermore, interaction between users and virtual objects remains a challenge (Kyrlitsias & Michael-Grigoriou, 2022). Vaz, Fernandes, & Veiga (2018) refer to various art projects, museums, and exhibitions that utilise multi-touch display and projection technology; they also list a number of VR projects in which users are not limited to simply looking around, but they can move freely as well (this ability to move around is considered a necessary characteristic of VR, as opposed to 360-degree immersive visual experiences). Furthermore, they list several implementations in which beacons (for location tracking) and wearable equipment (for various types of measurements) are used. Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 65 In the research conducted by Ruiz et al. (2022), great emphasis is given on the need of multiple interaction modalities in Human-Computer Interaction (HCI). This way more immersive and engaging applications can be developed. Visual, auditory and haptic feedback systems are required to create a richer user interaction experience in Social VR applications. Olin et al. (2020) present a VR system that supports cross-collaboration through different devices, namely an HMD and a mobile touchscreen device. Their findings indicate that handheld users managed to attain a high degree of immersion and presence despite the non-immersive nature of their device (compared to an HMD). The authors evaluated the interaction patterns by using two scenarios: conversation and collaborative building (joint construction of an abstract object); they observed that, in the former scenario, handheld users assumed similar positions as they would in the real world. Furthermore, they tended to look at the other participant much more than HMD users, who rarely did so. Also, handheld users exhibited stronger movement patterns compared to HMD users in the collaborative building task. Another interesting observation was that leadership was not related to the degree of immersion. Olin et al. (2020) highlight a number of important points when it comes to designing collaborative virtual experiences involving handheld devices. A virtual experience through an HMD allows for spatial cues similar to those normally perceived in the real world, thus facilitating spatial awareness. On the contrary, when using handheld devices, such cues are either absent or diminished (there is no depth perception / stereoscopic view when experiencing space through the screen of a handheld device). Another challenge when designing virtual experiences for handheld devices is related to the method of navigation, which is bound to be less natural (compared to that of HMD users) due to the lack of positional tracking. The study revealed the users’ preference for moving by means of a ‘pinching / spreading’ two-finger gesture (similar to the gesture typically used for zooming in / out), and rotating via the smartphone’s gyroscope (including switching between absolute and relative gyroscope mode). With respect to the conversational scenario, Olin et al. (2020) note that handheld users tended to assume a face-to-face configuration when the line of sight between the participants was unobstructed; when view was obstructed, handheld users either assumed a face-to-face configuration after positioning themselves so as to overcome the visual obstruction, or opted for other configurations, such as corner-to-corner or side-byside18. 18 The authors refer to “F-formations”, using the framework of Ciolek & Kendon (1980), with adjustments by Dim & Kuflik (2013), to describe the spatial arrangement between participants. Furthermore, the authors acknowledge a limitation in their study: participants were recruited in pairs who knew one another beforehand. Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 66 Figure 25. Three common configurations between two participants: (A) corner-to-corner, (B) face-toface, (C) side-by-side (Olin et al., 2020, p. 114). Olin et al. (2020) arrive at eight design considerations related to cross-device collaboration in social virtual environments. While all of them are important, some of them are particularly relevant to the interaction techniques employed. Specifically, the authors encourage designers of such systems to support non-verbal communication by providing appropriate interaction methods (such as gestures, body posture, and viewing direction). Furthermore, adequate field-of-view (whether using an HMD or other devices) is necessary for accurate and consistent distance perception so as to avoid invasions of personal space. These recommendations highlight the fact that research on ‘traditional’ face-to-face communication between two or more actors remains relevant when designing social virtual experiences. The importance of incorporating communicational aspects of space (in particular, the spatial arrangement and configuration of the participants) in the design of XR experiences is also highlighted by Schwajda & Anthes (2022) and Pathi et al. (2019). Kyrlitsias & Michael-Grigoriou (2022) note that immersive virtual reality and associated technologies provide a fertile ground for replicating social experiments with a high degree of ecological validity, further underlining the (ideally) close relationship between established theory on social behaviour and interaction and technological applications. Furthermore, an accurate representation of human actions - i.e. body and eye movement, facial expressions - on avatars / virtual bodies is important for inducing the sense of body ownership, as well as for communication with other users in shared immersive environments (Kyrlitsias & Michael-Grigoriou, 2022, p. 5). 4.5 Technological infrastructure for Social VR Immersive Social VR often involves tracking of body movements in order to provide a more holistic experience (Interaction Design Foundation, 2023). In general, there are two main modes of delivery for immersive virtual reality: Head-Mounted Display (HMD) and projection-based (Slater & Sanchez-Vives, 2014). Non-immersive forms of VR (e.g. VRenabled chat / messaging applications, multiplayer games, etc.) may be delivered using conventional equipment, such as a typical personal computer or mobile device. Typically, modern commercial social VR platforms incorporate high-fidelity 360-degree content and six degrees of freedom, while allowing for a multitude of verbal and nonverbal communication means (Maloney, Freeman, & Robb, 2021). This results in a richer Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 67 communication experience19 while adhering to accessibility standards and recommended practices. Although employing interaction devices that support a large number of degrees of freedom often results in performance improvements, Liarokapis et al. (2017) argue in favour of decreasing the number of degrees of freedom if the task at hand allows it, in an attempt to make the interaction easier to understand by a wider audience. Thus, the authors suggest the utilisation of hybrid interfaces, which they define as combinations of 3D input devices with a 2D device. Head-Mounted Displays (HMDs) HMDs are the primary interface for accessing social VR environments. They provide immersive visual experiences by encompassing the user's field of view with highresolution displays. Modern HMDs are equipped with advanced sensors that track head movements, enabling users to look around and interact with the virtual environment naturally (Handley et al., 2022). HMDs can be tethered or standalone (Angelov et al., 2020); the latter tend to be better in terms of ergonomics, since they do not require cables that may hinder movement - thus, they may be more suitable for applications that require a high degree of mobility20. On the other hand, such HMDs rely on their onboard hardware for content rendering, which may not always be adequate, and the same applies to battery power. Tethered HMDs rely on the computer’s processing power and are thus more suited for demanding applications. The screen-door artefact (the pixel contours forming a visible grid) is present to a lesser or greater extent in all HMDs due to the small distance of the screen to the users’ eyes, but can be mitigated through high pixel density. Additionally, the perception of the image presented to the users as natural tends to increase as the field of view increases. It should be noted that, in order to increase field of view, it is necessary to increase screen size, which may adversely affect the screen-door artefact (Angelov et al., 2020). Another important characteristic of HMDs is their refresh rate; generally, the higher, the better for performance, presence, and the quality of the user experience (including absence or reduced severity of motion sickness symptoms). While it is not uncommon to see monitors (especially high-end ones intended for gaming) achieve a very high refresh rate (over 100Hz), HMD refresh rates tend to be more modest, at least presently. Still, they are 19 Comparing social VR to traditional virtual worlds, the authors note that communication in social VR is currently not archivable (Maloney, Freeman, & Robb, 2021). This is partly due to the interaction modalities supported in modern social VR, which attempts to simulate traditional nonverbal communication. Archiving such interactions would rely on a commonly agreed-upon logging protocol that would encompass a wide variety of intentional and unintentional, verbal or nonverbal, interaction cues. 20 VR glasses are a subtype of standalone HMDs: they are a set of lenses that need a smartphone to serve as display and rotation sensor, but their lack of tools for interaction with the content may result in a reduced sense of presence. Also, they often lack positional tracking, which places limits on what the users can do. Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 68 consistently over 75Hz, with the Valve Index achieving a maximum refresh rate of 144Hz, a performance that is on par with the aforementioned monitors. While positional tracking is at present considered a basic requirement for modern HMDs, some (like the Valve Index) also provide finger tracking. This can allow for richer user interaction when fine-grained manipulation of objects is involved. In addition to tracking, weight and form factor are important in an HMD, as they directly affect the user experience, especially during prolonged use6. Motion Controllers The possibility of utilising kinesics when interacting with other users in a social VR setting brings immersive VR closer to face-to-face interaction and is dependent on appropriate body tracking hardware. In many cases, while speech tends to be the predominant mode of communication among social VR users, the users in question appreciate the system’s ability to support body movement as an input signal, enabling them to interact (e.g. via body language) with other users who cannot speak for whatever reason (Freeman & Acena, 2021). Motion controllers allow users to interact with the virtual environment through hand movements. These controllers are equipped with sensors that track the position and orientation of the user's hands, enabling precise manipulation of virtual objects. Advanced controllers also include haptic feedback to enhance the tactile experience (Ruiz et al., 2022). Another category of motion trackers uses the entire body as input (full-body motion tracking), e.g. motion capture suits. While accurate, such systems are often expensive, which prevents the majority of end users from being able to use them. Using more than one tracker for various body parts may result in an approximation of body tracking which is less accurate, but more affordable and easier to set up and use by comparison (Kyrlitsias & Michael-Gregoriou, 2022). Regarding locomotion, a standard solution often employed is teleportation, i.e. pointing at a location with a handheld controller and, upon pressing a button on the controller, moving the camera at that location. This solution was used by Soto-Martin, Fuentes-Porto, & Martin-Gutierrez (2020) in their virtual reconstruction (developed in the Unity game engine) of the church of St. Augustine in the city of San Cristóbal de La Laguna Tenerife, Spain. Regarding the efficacy of this locomotion technique, it may be said that, while potentially limiting the users’ freedom of movement, it may be advantageous from the point of view of motion sickness, as it is often not accompanied by rapid turns and movements that tend to contribute to the appearance of adverse symptoms and effects. Reimat et al. (2022) describe a cultural heritage experience using social VR. Three Microsoft Kinect sensors were used alongside an HTC Vive Pro headset for interaction Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 69 and navigation in virtual space. Users can freely interact with a specific exhibit (a costume) and take part in a curated tour. The system supports co-presence for multiple users. Haptic Devices Haptic devices, such as gloves and vests, provide tactile feedback that simulates the sensation of touch. These devices are crucial for enhancing the realism and immersion of social VR experiences. By providing physical sensations that correspond to virtual interactions, haptic devices enable users to feel the virtual environment, making interactions more engaging and intuitive (Mulders & Zender, 2021). Perret & Vander Poorten (2018) classify haptic devices in three broad categories: (i) gloves, i.e. hand-shaped garments made of flexible fabric, (ii) thimbles, i.e. devices involving actuators attached to a fingertip, and (iii) exoskeletons, i.e. articulated wearable devices capable of transmitting force. The authors provide a review of existing haptic devices from all three categories. Haptic feedback is regarded as a useful tool in bridging the gap between the real and the virtual; being able to touch virtual artefacts renders the simulation of the virtual museum more lively and intuitive. To that end, dedicated devices specifically designed to utilise touch ad input and provide tactile output (e.g. force feedback) have been used in such settings - for instance, the Novint Falcon to simulate the shape, texture, and material of virtual artefacts (Arnab et al., 2011). Auditory interaction and Spatial Audio Systems Sound is an interesting choice of interaction modality, since users (ideally) do not need to learn potentially complex interaction methods, since they could simply talk to the computer; the same applies to users who are visually impaired. Of course, the current situation in speech recognition and comprehension is still not adequately accurate for general usage of speech as input (the larger the domain of discourse, the less the degree of recognition accuracy). Regarding using sound as output, text-tospeech synthesis can be an affordable and easy to implement solution, but it tends to suffer from a low degree of naturalness (e.g. robotic voice); though more naturally sounding voices are now available, the intonation and overall speech patterns of text-tospeech system output still lags behind what might be considered natural by the wider audience. The alternative of employing voice acting talent is still a viable option (though the increase in the usage of generative AI may soon make this practice redundant). Nonetheless, Liarokapis et al. (2017, p. 381) provide a number of examples of interactive applications that employ sound as their primary modality. Spatial audio systems use advanced algorithms to create a 3D sound environment that mimics how we hear sounds in the real world. These systems are essential in social VR for enhancing the realism and immersion of the virtual environment. They help users Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 70 locate sounds in space, which is crucial for effective communication and interaction (Ruiz et al., 2022). Computer vision Computer vision is an interesting choice when it comes to positional tracking, since it can be non-intrusive (in the sense of allowing untethered interaction) and accommodate a large number of potential users and/or spectators, who may also interact with one another. Zabulis et al. (2013) describe such a system that utilises several cameras in front of a large screen. QR codes or other visual information is also a plausible choice for indoor position tracking. Brain-Computer Interfaces An HMD can be regarded as a good choice for mounting unobtrusive electroencephalography [EEG] sensors (Tremmel et al., 2019). A non-invasive BCI can be a viable option for simple interactions in some cases, but they tend to suffer from various problems, mainly having to do with detection accuracy (at least in the lower, non-medical end of the spectrum), that render their effective use somewhat problematic. Still, they can accompany and supplement other interaction techniques, for instance by increasing the total number of degrees of freedom in a way that does not make things more complicated for other methods (Liarokapis et al., 2017) Additionally, BCI can be suitable in accessibility-related use case scenarios. A BCI-based VR setup is based on software that records and classifies brain activity. From the point of view of hardware, it is important to couple the BCI device with the VR headset (if one is used) to avoid unnecessary delays in the communication between the two subsystems; if users are to move around, the use of active electrodes is recommended to avoid movement artefacts (Lotte et al., 2013). Various early applications combined VR with EEG for movement control, the general operating principle being the system training on the users’ brain activity pattern, resulting in the users being able to navigate virtual spaces after this training period (Lotte et al., 2013). Lotte et al. (2013) highlight the issue of user fatigue as a result of the need for continuous mental activity to navigate the virtual world. A potential solution is shared control: users select (via mental imagery) one from a small number (e.g three) classes in order to indicate specific navigation points, and the system ‘drives’ them there. In this manner, users are spared the low-level movement and locomotion details. A related application in which users are asked to move an object by ‘imagining’ the motion is described by Lotte, Renard, & Lécuyer (2008). Mental imagery seems to be suitable for locomotion but not so much for selection tasks due to the small number of available classes when using the former method. Evoked potentials are a much more suitable method if selection is required (Lotte et al., 2013). Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 71 Apart from its potential use as an interaction method, BCI can also be used for evaluation purposes. Škola et al. (2020) concluded that the use of EEG does not detract from the overall experience of using immersive VR. 4.6 Techniques of integrating facial and gesture recognition in XR aspects of MUVEs (EV) 4.6.1 Introduction Facial and gesture recognition and tracking technologies allow to accurately capture and recreate users' facial expressions and body gestures. It is important to start with an accurate definition of each technique and to distinguish recognition and tracking methods properly. Facial recognition is a technology that identifies a person by analysing and comparing patterns based on their facial features. It is based on capturing an image or video of a person's face, extracting unique facial features (eg. the distance between the eyes or the contours of the face), and comparing these features with a database of stored images to find a match. In multi-user virtual environments, facial recognition technology can be used to create personalised avatars or authenticate users joining experience. Gesture recognition is the technology that interprets human body movements, particularly hand and arm gestures, as input commands. By analysing the position, orientation, and motion of the body, gesture recognition systems can identify specific gestures, such as waving, or pointing, and translate them into actions or commands within a virtual environment. This allows users to interact with virtual environments more naturally and intuitively. Facial tracking refers to the continuous monitoring and analysis of a person's facial movements in real-time. It tracks changes in facial expressions, head orientation, and other dynamic facial features over time. This data can then be used in virtual environments to replicate the user's real-time expressions and movements using virtual avatar. It increases realism and enables more expressive interactions. Gesture / body tracking is the process of continuously monitoring and following the movement of a user's body, hands, or other parts over time. It ensures that the system accurately detects and responds to the user's gestures as they move within the virtual space. Gesture tracking is essential for smooth, real-time interaction in virtual environments, enabling users to perform actions and navigate the environment using their own physical movements. According to the definition above, recognition techniques are focusing mostly on the process of detecting and classifying particular facial expressions or gestures Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 72 making them an input into the system. Tracking techniques extend this single detection process into continuous monitoring of the person's facial and body movements enabling transferring them onto an avatar, that is virtual representation of the user in MUVEs. 4.6.2 Benefits of using facial and gesture recognition and tracking Facial and gesture recognition and tracking technologies are essential elements of MultiUser Virtual Environments (MUVEs) enabling more natural, immersive, and interactive experiences for the users. They also enable real-time, non-verbal communication, critical for effective social interaction and provide the ability to enhance the sense of presence and realism which is especially important in the context of experiencing cultural heritage. Gesture recognition simplifies navigation in the MUVEs and control of the avatar, allowing users to interact with the virtual environment in a much more intuitive way. Overall, facial and gesture recognition and tracking bridge the gap between the physical and virtual environment, making interactions more engaging, accessible, and personalised. Facial and gesture recognition used in Multi-User Virtual Environments (MUVEs) provide unique benefits in the context of cultural heritage, enhancing the education, preservation, and experiencing of cultural artefacts and practices. Here are some key benefits: • Enhanced Immersion, Embodiment and Presence. By using facial expressions and body gestures users interact with both virtual environments and each other in a more natural and intuitive way. This makes users feel more physically present and helps them focus on the content instead of technological aspects of the experience. Real-time facial expressions and gestures mapping onto avatars, allows users to express their feelings and emotions which can be used as an input for the virtual experience, strengthening the story and the message. This benefit is clearly confirmed by research conducted at Department of Digital Media, Ajou University in Republic of Korea by Haejung Suk & Teemu H. Laine. (2024) • Accessibility and Inclusivity for Cultural Heritage. Gesture recognition allows for hands-free control and navigation within the virtual environment, which is beneficial for users with physical disabilities or those who find traditional input methods challenging to use. This approach is confirmed in the research “Effects of interacting with facial expressions and controllers in different virtual environments on presence, usability, affect, and neurophysiological signals” conducted by the team led by Arindam Dey from the University of Queensland and Dr. Mark Billinghurst from the University of South Australia (Arindam Dey et al., 2022). Facial recognition enables the creation of personalised avatars that resemble the user’s real appearance. It promotes inclusivity and identity representation within the virtual space, which is especially important when cultural heritage is experienced by a group of users simultaneously. It fosters communication among the users and makes it more emotional. Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 73 • Immersive Educational Experiences. A multi-user virtual environment’s ability to recognize and respond to individual users' expressions and gestures allows it to offer much more personalised experiences. Virtual characters, as a part of cultural heritage experience, can react differently based on the user’s mood or actions, creating a more dynamic and interactive environment for education and a deeper emotional connection between the user and the subject matter. These technologies also allow users to engage in traditional moves, dances or rituals by mirroring the gestures of virtual instructors. Analysis of users’ emotions, may also help customise experience to improve its effectiveness and level of engagement. Similar concept was successfully validated in the research conducted in a more traditional e-learning environment (Daouadji & Bendella, 2024). • Preservation of Cultural Practices. Capturing traditional gestures and expressions can digitally preserve traditional dances, rituals or expressions that are crucial to cultural heritage. By recording and accurately reproducing these gestures and expressions in virtual environments, these practices can be secured for the future generations. Facial and gesture recognition and tracking can help recreate historical personas, bringing them to life in virtual environments. This allows users to interact with virtual representations of important cultural icons in a way that is both engaging and educational. 4.6.3 Review of existing techniques and hardware technologies Integrating facial and gesture recognition and tracking in MUVEs involves using a combining of various hardware and software technologies. To achieve proper results for particular virtual experiences it is important to select the most appropriate ones. To effectively select them for particular Multi-User Virtual Environments supporting digital representation of cultural heritage it is important to provide structure and categorization. Here is a brief review of existing face and body tracking solutions. Facial recognition and tracking Facial recognition and tracking rely on advanced algorithms and optical hardware, ranging from consumer-grade cameras to specialised sensors. The availability of powerful hardware tools makes it possible to implement facial recognition in a wide range of applications, considering different sets of requirements and budgets: • Webcams and Smartphone Cameras are commonly used for facial recognition and tracking, especially in low budget consumer applications. These kinds of devices capture video that is then processed by dedicated software. Examples include cameras on devices like Apple iPhones21 or high-quality webcams like Logitech 21 Apple iPhone | https://www.apple.com/iphone/ Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 80 or download them onto your device. For more advanced needs, you could resort to file transfer protocols like FTP, cloud storage services, or even command-line tools. Every one of these approaches has its relative strengths for use in various situations, from browsing casually to professional file management. It can be very helpful to have an idea about these alternatives and it will help one find their way around and make the most out of what the web offers. • Web browsers: Directly accessing files through URLs. • File Transfer Protocol (FTP): For transferring files between computers on a network. • HTTP/HTTPS protocols: Used by web browsers and other applications for secure file transfers. • Cloud storage services: Like Dropbox, Google Drive, or OneDrive. • Torrent clients: For peer-to-peer file sharing. • Command-line tools: Such as “wget” or “curl” for downloading files. • APIs: Programmatic access to files hosted on various platforms. • WebDAV: Protocol for collaborative editing and file management. • Rsync: For efficient file transfer and synchronisation. • Email attachments: Sending and receiving files via email. All of them have their pros and cons mostly because of file size limitations, the number of files that need to be transferred or ease of use. While Sketchfab offers Direct Download via its interface in a web browser there is no working Plugin to query and download directly to UE5 (02. September 2024). Openheritage3D is used as Data vehicle for CyArk; its rather minimal search functionality hides the often very good data behind a “send by mail” system. Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 81 5.2 State of the Art-Overview of CH Platforms with 3D Assets Examples on this list exemplify the current state of the art regarding 3D repositories and databases. Name Link Access Sketchfab Cultural Heritage & History 3D models https://sketchfab.com/categories/c ultural-heritage-history DDL, API-(Viewer, Download, Login, Data) Europeana https://www.europeana.eu/en DDL, API-(Search, Record, Download, Metadata) Smithsonian 3D Digitization https://3d.si.edu/ DDL CyArk https://www.cyark.org/ Photogrammetry Data is provided on request (selected projects partner with Open heritage) Open Heritage 3D https://openheritage3d.org/ Photogrammetry / Lidar Data is provided on request Metropolitan Museum of Art https://www.metmuseum.org/art/c ollection/search?showOnly=openA ccess 42 3D data items via Sketchfab The British Museum https://sketchfab.com/britishmuse um 269 3D data items via Sketchfab Wikimedia https://www.wikimedia.org/ DDL, API Morphosource https://www.morphosource.org/ Download Request, API Virtual Curation Lab https://vcuarchaeology3d.wordpre ss.com/ Moved to Sketchfab Zamani Project https://www.zamaniproject.org/ On Sketchfab, not downloadable During our research we have been focusing on 3D-Data of Cultural Heritage assets. The researched CH-3D Data is mostly accessible for users via Direct Links for single Download of a file through a web browser. Databases hosted by IMPULSE partners are not part of this overview and will be assessed separately. Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 82 5.2.1. Sketchfab43 Sketchfab is a web service to easily publish and explore online 3D/VR/AR content. The concept for this was given birth initially in France during 2012, from the minds of Alban Denoyel, Cédric Pinson, and Pierre-Antoine Passet, as an answer to the display of 3D models over the web. The platform relies on WebGL technology, so users are able to publish and view such content within their web browser without special software. It accepts many 3D file formats and supports embedding on other websites. Since then, Sketchfab has grown to serve every audience—from 3D artists and designers to developers. Its applications range in many sectors, such as gaming, architecture, and cultural heritage. The website also has a marketplace where users can buy or sell their 3D models. Lately, in 2021, Epic Games, creator of Unreal Engine and Fortnite, announced the acquisition of Sketchfab. That is in line with Epic's broader acquisition strategy in the 3D graphics and gaming industry. Therefore, Sketchfab might disappear after the Launch of fab.com Quote from the website “Creators offering free or for-purchase products across UE Marketplace, Sketchfab and the ArtStation Marketplace will be able to continue selling on those platforms during Fab’s Alpha period. In 2024, UE Marketplace, Quixel, Sketchfab, and ArtStation Marketplace will all roll up into one destination: Fab.”44 This is another indication that relying on existing platforms / content aggregators may prove to be problematic for IMPULSE, since the probability of support being rescinded in the future cannot be ruled out. Figure 26. Sketchfab Model Results Page 43 Sketchfab | https://sketchfab.com/ 44 Fab | https://www.fab.com/ Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 83 Although Sketchfab offers a wide range of importers45 for DCC’s, these often lack updates. For example, the latest importer for Unreal Engine has been available on Github since 2021 and there is no update for the most commonly used Version of 5.0 and upwards. Figure 27. No Result. Sketchfab's announcement to enable cultural organisations to dedicate their 3D scans and models to the Public Domain through the Creative Commons Public Domain Dedication (CC0) marks a significant advancement in the democratisation of cultural heritage. This initiative paves the way for museums and similar entities to share their 3D data more openly, contributing numerous remarkable 3D models to the public domain, many for the first time. This move significantly enhances the accessibility of ancient and modern artefacts, objects, and scenes for 3D creators worldwide, facilitating their download, reuse, re-imagination, and remixing. In a collaborative effort with 27 cultural organisations from 13 different countries, Sketchfab proudly welcomed the Smithsonian Institution alongside its open access program, highlighting the initiative's broad appeal and significant impact. The introduction of CC0 at Sketchfab not only expands the repository of freely available 3D models but also simplifies the process for 3D creators to engage with and repurpose cultural and historical data for various creative and commercial projects without the need for attribution. Since adopting Creative Commons Attribution licences in 2014, Sketchfab has seen over 300,000 3D models shared under these terms, facilitating generous reuse across artistic and academic endeavours. The shift to CC0 dedication for cultural heritage content represents a deeper commitment to fostering artistic and academic reuse of 3D data under clear, accessible terms, allowing for even broader application and innovation. Sketchfab's implementation of CC0 dedication aligns with the global trend towards open access policies amongst the world's leading cultural institutions. This initiative indeed provides a platform that makes it easier for organisations to align their digital 3D 45 Sketchfab importers: https://sketchfab.com/importers Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 84 collections with open access policies. It has to be stressed though that SketchFab is a commercial initiative and Cultural Institutions pay to publish high resolution models there. It is however, the Cultural Institutions that own the collections, which have embraced the open access movement and are actively supporting this approach by often adopting a CC0 policy, thus enhancing the global accessibility and preservation of cultural heritage in the digital era. SketchFab is merely facilitating this process by providing the context for presenting this CC0 content. The launch collaborators, including renowned museums, libraries, art galleries, and archaeological projects, exemplify the diverse and rich collections that have embraced this initiative. From ancient artefacts to scientific innovations, the public domain 3D models on Sketchfab offer an unparalleled resource for exploration, education, and creative reinterpretation, providing accessibility to our global cultural heritage. It has to be stressed though that the 3D models that his resource consists of may be of varying quality and resolution. 5.2.2. Europeana46 Europeana was launched in 2008 as an initiative of the European Union and born from a vision of making all Europe's cultural and scientific heritage accessible to all across a single multilingual online portal. It was inaugurated with a letter from six European leaders to the President of the European Commission in March 2005, calling for the establishment of a virtual European library. This is a metadata aggregator concerning millions of cultural items kept in the collections of many various European museums, libraries, archives, and audio-visual collections. The items may include books, paintings, films, museum objects, and archival records. However, Europeana does not host the digital objects but gives links to content on the providers' own sites. Over the years, the service developed from a proof of concept into a full-scale operation. The platform has suffered with the aggregation of metadata coming from various sources, which has been more streamlined with the creation and adoption of the Europeana Data Model. Next to that, it has suffered because of compatibility issues in relation to rapidly changing digital technologies. It has also expanded its service to include APIs for developers and tools for use in education. Today, Europeana remains a centre of digital European cultural heritage for the objectives of research and education, including creative reuse. It is the result of a joint effort to preserve and share their cultural wealth as a whole in the digital age on the part of the EU and its member states. Europeana is part of the consortium and a full partner for the ECCH (European Cloud for Cultural Heritage). 46 Europeana |https://www.europeana.eu/en Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 85 Figure 28. The Europeana Homepage 5.2.3. Smithsonian 3D Digitization47 The Smithsonian Institution, a group of museums and research centres administered by the U.S. government, has been working on digitising its collections since the early 2000s. This process involves creating digital versions of physical objects, specimens, and documents held by the institution. The digitization program spans the Smithsonian's 19 museums, 9 research centres, libraries, archives, and the National Zoo. It includes capturing images, 3D scans, and other digital representations of items, along with cataloguing them with metadata. In 2020, the Smithsonian launched its Open Access program48, releasing millions of digital assets into the public domain. This move allows free use of these materials, though the impact and utility of this access vary depending on user needs and interests. “The 3D Program is a small group of technologists working within the Smithsonian Institution's Digitization Program Office. We focus on developing solutions to further the Smithsonian's mission of “the increase and diffusion of knowledge” through the use of three-dimensional scanning technology, analysis tools, and our distribution platform.” 49 The Smithsonian 3D Scan Lab acknowledged a necessity for Metadata automation and API access to its collection. 47 Smithsonian 3D Digitization | https://3d.si.edu/ 48 Smithsonian Open Access program | https://www.si.edu/openaccess 49 Smithsonian Open Access program | https://3d.si.edu/about#:~:text=The%203D%20Program,our%20distribution%20platform Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 86 Additionally, the Smithsonian Institution provides the Smithsonian Voyager, an opensource 3D explorer and authoring tool suite, which affords 3D viewing on the web, quality inspection and authoring of experiences, annotations, articles and tours. Figure 29. Smithsonian 3D Digitization Homepage Figure 30. Smithsonian About Page - Partnership Advertisement50 50 Smithsonian About Page - Partnership Advertisement | https://3d.si.edu/about#:~:text=Currently%2C%20we%20are,we%20will%20create Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 87 5.2.4. CyArk51 CyArk was founded in 2003 by Ben Kacyra, an Iraqi-born engineer and entrepreneur. The inspiration for CyArk came after the Taliban's destruction of the Bamiyan Buddhas in Afghanistan in 2001. Kacyra, who had co-invented a portable 3D laser scanning system, recognized the potential of this technology to preserve cultural heritage sites digitally. The organisation's name, CyArk, is derived from "Cyber Archive," reflecting its mission to create a digital archive of the world's cultural heritage sites. CyArk uses advanced technologies like 3D laser scanning, photogrammetry, and traditional survey techniques to capture detailed 3D models of heritage sites. Over the years, CyArk has documented hundreds of sites across all six continents. These range from ancient ruins like Pompeii to modern structures like the Sydney Opera House. The data collected is used for conservation, education, and virtual tourism. The organisation's impact varies. While it has successfully created digital records of many sites, the practical applications and accessibility of this data for conservation, research, and public engagement continue to be areas of development and discussion in the heritage preservation field. In other words, they did a great job capturing the CHsites but the data is not fully accessible. Figure 31. CYARK Homepage 51 CyArk | https://www.cyark.org/ Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 88 Figure 32. CYARK Projects Page 5.2.5 Open Heritage 3D OpenHeritage3D52 is a collaborative development to advance the application of 3D technologies in documentation and preservation of cultural heritage. OpenHeritage3D started as part of a collaboration between CyArk and the National Center for Preservation Technology and Training (NCPTT), a research arm of the U.S. National Park Service. It was a 2019 project to achieve this through the development of a central repository for 3D data related to cultural heritage sites. The platform hosts an open-access database that enables researchers, cultural heritage professionals, and the general public to access high-quality 3D datasets of heritage sites and monuments from around the globe in 3D documentation formats, among others, including point clouds and mesh models. OpenHeritage3D is designed to address some critical challenges in the field of digital heritage: • Data availability: By making 3D data freely available, it allows wider use for research and education purposes, as well as in efforts of conservation. • Standardisation: The project promotes best practices in both data capture and processing methodologies. • Long-term preservation: It provides a way to store large 3D data sets, which can be difficult for individual institutions. While the search page doesn't have any necessary filters, for example to search for a date 52 OpenHeritage3D | https://openheritage3d.org/ Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 89 published, the point cloud viewer is a bit slow (but rich in features) and it's not always specific why this is cultural heritage, the original data is mostly high quality. Which is also the problem, there are only the source files (lidar or pictures), no final model, no downloadable preview model or textures. Unfortunately, it can only be accessed by email request. Only if the Openheritage link is displayed there will be a link to access the recording data at the open heritage site. Figure 33. OpenHeritage3D Data Sets Page Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 96 Name API Documentation Content Aggregator / Single Prov Formats searchable Monetised Copyright Update frequency Sketchfab Cultural Heritage & History 3D models https://sketchfab.com/deve lopers High Aggregator OBJ, FBX, BLEND, 3DS and STL and more x yes / no (Depending on Uploader and Plan) yes / no (Depending on Uploader and Plan) very high frequency Europeana https://europeana.atlassian .net/wiki/spaces/EF/pages/ 2461270026/API+Suite High Aggregator glTF, X3D, STL, OBJ, DAE, PLY, WRetc x no yes / no (Depending on Uploader) high frequency Smithsonian 3D Digitization 3590 objects Single Output: obj, glb, gltf, usbz, MD x no CC0 / Usage Conditions Apply low CyArk 40 Sites Single x no depends on Project low Open Heritage 3D 460 Datasets Single Output: jpg, raw, e57 x no Attribution low Metropolitan Museum of Art https://metmuseum.github. io/ ?? Single x no Extensive CC0 collection nA The British Museum 269 3D data items Single x no Attribution nA Wikimedia https://api.wikimedia.org/w iki/Main_Page ?? Aggregator no 3D x no very high frequency Morphosource https://morphosource.stopl ight.io/docs/morphosource -api/rm6bqdolcidctmorpho-source-rest-api ?? Aggregator a lot x no yes / no (Depending on Uploader) nA Virtual Curation Lab https://sketchfab.com/virtu alcurationlab 3800 data items Single x Deliverable D 2.1 IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 97 5.3. Chapter summary Challenges of 3D Cultural Heritage Assets The digital revolution has transformed cultural heritage preservation, with 3D scanning and modelling technologies enabling unprecedented access to collections. However, as we explore the landscape of 3D asset repositories, we encounter significant challenges that hinder the seamless discovery and utilisation of these digital resources. Fragmentation and Standardization The digital landscape is highly fragmented, with platforms like Sketchfab, Europeana, and OpenHeritage3D operating independently, apart for the wide variety of institutional repositories. This decentralisation, while fostering innovation, creates a complex journey for users seeking comprehensive access to 3D cultural assets. Moreover, the lack of standardisation in metadata formats, file types, and quality standards across platforms makes it difficult to compare assets or integrate them into unified research projects. Many smaller-scale case studies within research frameworks have been developed. However, there is a significant gap between developing case studies and integrating standardized 3D digitization workflows (data creation, data publication, and data preservation) into the daily operations of heritage institutions. Initiatives such as the Smithsonian 3D Digitization Program or the British Museum’s efforts to scan their entire collections require policy frameworks that not only commit to these goals but also provide the necessary resources to support them. The absence of standards, guidelines, and clearly defined workflows continues to hinder the adoption and implementation of such initiatives. Accessibility and Quality Issues While many repositories promote open access, the reality often falls short. Email requests for data access, as required by OpenHeritage3D, create barriers to immediate use. Institutions not always share datasets under Public Domain, CC 0 or CC By mark. In terms of legal frameworks, there can be huge differences in IPR related legislation between countries, even in Europe. Additionally, the quality and consistency of 3D assets vary significantly across and within repositories as technology, software and file formats evolve at fast pace. The British Museum's Sketchfab collection, for instance, showcases wide disparities in file sizes and resolutions among key artefacts. Paradata, information about the processes and methods used in the creation, collection, or transformation of data, offering transparency about how the data was produced or manipulated, is close to non-existent for legacy data sets. Direct Download Links and lack of APIs makes it harder for industry professionals to maintain larger amounts of data. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 98 Technical Challenges and User Experience Many repositories struggle with the technical aspects of hosting and displaying 3D content. Slow loading times, compatibility issues, and limited viewing options can impede research and frustrate users. These technical hurdles highlight the gap between data capture and creating structured, usable resources for research and education. Preservation and Long-term Accessibility As we digitise cultural heritage, we face new challenges in preserving the digital assets themselves. The rapid evolution of file formats and viewing technologies raises questions about long-term accessibility. Projects like the Virtual Curation Lab, which relocated its assets after discontinuation, underscore the need for sustainable storage solutions. Commercial initiatives and platforms raise questions about data storage, data preservation, long term sustainability. Outlook and Recommendations Addressing these challenges requires greater collaboration between institutions to standardise approaches to metadata and file formats. Investments in user-friendly interfaces and robust search capabilities could dramatically improve discoverability. Furthermore, providing processed, ready-to-use 3D models alongside raw data could enhance the utility of these digital assets for a broader range of users. In conclusion, while the landscape of 3D cultural heritage assets holds immense potential, significant hurdles remain. Overcoming these challenges is crucial to creating a truly accessible, standardised, and user-friendly ecosystem that can serve researchers, educators, and the public alike, unlocking the full potential of these digital treasures for future generations. 5.4. Investigating APIs afforded by content aggregators 5.4.1 Definition of an API API stands for Application Programming Interface. In simple terms, an API is a set of rules and protocols that allows different software applications to communicate with each other. Here's what you need to know: • Function: An API acts as a messenger that takes requests, translates them, and returns responses between different software systems. • Analogy: Think of an API as a waiter in a restaurant. You (the user) ask the waiter (the API) for something, and the waiter goes to the kitchen (the system) to retrieve it for you. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 99 • Purpose: APIs enable developers to access specific features or data from another application or service without needing to understand all of its internal workings. • Examples: When you use a weather app on your phone, it's likely using an API to fetch weather data from a service. When you log into a website using your Google account, that's also facilitated by an API. • In context: For e.g. Europeana or Sketchfab, their APIs allow developers to access and retrieve information about 3D objects and their metadata from their respective databases, enabling integration of this data into other applications or services. While the “big three” (Sketchfab, Europeana and Wikimedia) offer API access most of the other researched Databases don't. In the following sections, we provide a comprehensive overview of the key aspects of the APIs offered by Europeana, Sketchfab, and other relevant platforms. It aims to highlight the core functionalities, strengths, and limitations of each API, particularly in the context of working with digital content and 3D assets. A deeper examination will be conducted on the search and model APIs, as these are critical components for discovering, filtering, and accessing the content across these platforms. We explore how each API handles search queries, filtering options, and metadata retrieval, with a specific focus on their applicability to 3D assets and cultural content. By taking a closer look at the search capabilities and the data structure of these APIs, we will assess how well each platform supports advanced content discovery and interaction. Through a comparative analysis of the APIs, we will identify the key differences and similarities in their design, functionality, and usability. The comparison will focus on aspects such as the richness of the metadata, flexibility of search filters, support for 3D content, and integration options provided by each platform. Sketchfab • Purpose: A platform for hosting, sharing, and selling 3D models. • Key Features: Web-based 3D viewer, VR/AR support, and a marketplace for 3D assets. • Community: Artists, designers, and developers share and explore 3D content. • Uses: 3D visualisation in education, gaming, animation, and virtual experiences. Europeana • Purpose: A digital platform for accessing European cultural heritage. • Key Features: Digitised collections from museums, libraries, and archives across Europe. • Content: Art, literature, music, and historical records available for public access. • Uses: Education, research, and cultural exploration with open access to many resources. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 100 Wikidata • Purpose: A free, open knowledge base providing structured data. • Key Features: Linked open data for use in Wikimedia projects and external applications. • Data: Interconnected, machine-readable information on a wide range of topics. • Uses: Supports Wikipedia, data queries, AI, and applications in semantic web projects. Feature/Platform Sketchfab Europeana Wikidata Primary Focus 3D model hosting, sharing, and viewing Access to European cultural heritage Structured data repository Content Type 3D models, VR/AR content Digitised cultural items (art, books, music, archives) Data on a wide range of topics (people, places, concepts) Community 3D artists, designers, developers Cultural institutions, educators, researchers Volunteers, data scientists, researchers Data Accessibility Public and paid access to 3D assets Free access, open licences for many items Open data under Creative Commons CC0 licence Interactivity Web-based 3D viewer, VR/AR support Interactive exhibitions, thematic collections SPARQL query service for advanced data analysis Integration Integrates with 3D software (Blender, Maya) Integrated collections from over 3,000 cultural institutions Integration with Wikipedia, AI applications, semantic web Monetization Marketplace for buying/selling 3D models Not focused on monetization, primarily for public access Not focused on monetization, open data for reuse Educational Use Used for teaching 3D design, VR/AR experiences Used for teaching European history, art, and culture Used for research, education, data science Licensing Mix of open and commercial licences Mostly open access with some restrictions All data is open and available under public domain (CC0) Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 101 APIs and Developer Tools API for embedding 3D models APIs for accessing cultural heritage data SPARQL query service and APIs for data retrieval Search and Discovery Searchable database of 3D models Search by keyword, thematic browsing, filtering options Searchable data with query capabilities Each platform plays a distinct role: Sketchfab focuses on 3D content, Europeana on European cultural heritage, and Wikidata on structured, accessible data for both human and machine use. Wikidata, as a collaborative knowledge base, offers a vast repository of structured data on a wide range of topics. While it excels in providing rich metadata and interlinking concepts across various domains, its primary focus is not on the hosting or distribution of 3D assets. Unlike Sketchfab, which specialises in 3D models, or Europeana, which provides digitised cultural content with visual representations, Wikidata is more concerned with cataloguing and interlinking information across the web. Given that our project is primarily focused on 3D content, including the discovery, use, and integration of 3D models, the deeper exploration of Wikidata is beyond the scope of our current research. This is why we will be demonstrating and introducing Sketchfab and Europeana in particular, as they serve as excellent examples of platforms that are well-suited for working with 3D assets and digital cultural content, which aligns closely with our objectives. 5.4.2 API Access In this section, we will dive into the technical details of accessing and utilising the APIs provided by Europeana, Sketchfab and Wikidata. The goal is to understand how these APIs enable interaction with their vast repositories of digital content, with a particular emphasis on 3D assets and cultural heritage data. We will explore the authentication methods, available endpoints, and the overall structure of the APIs, highlighting how developers can efficiently retrieve and manipulate data. This exploration will set the foundation for deeper comparisons and practical demonstrations of each platform's capabilities and inform the design of the content administration features of the IMPULSE platform. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 102 Platform API Types Key Features Authentication Use Cases Documentation Sketchfab Viewer API, Data API, Download API Embedding 3D models, managing accounts, VR/AR support OAuth2 3D content integration, virtual/augment ed reality Comprehensive docs, community support Europeana Search API, Entity API, Annotations API Access to cultural heritage data, contextual info API key Cultural heritage exploration, educational apps Detailed docs, developer portal Wikidata SPARQL Query API, REST API, MediaWiki Action API Querying structured data, editing and retrieving items OAuth, API tokens Knowledge graphs, AI, semantic web projects Extensive docs, tutorials, query editor Sketchfab APIs Viewer API This allows the user to embed and control Sketchfab's 3D viewer on your website or application. It is possible to manipulate models, control cameras, interact with the scene, and even integrate with VR/AR. In the context of our project, while the Sketchfab Viewer API is a nice feature, it is not a crucial component for our needs. Although it provides useful functionality for embedding and interacting with 3D models, it does not play a significant role in the core objectives we are focusing on. Sketchfab Data API Overview The Sketchfab Data API provides access to the platform's extensive collection of 3D models, allowing users to manage assets, retrieve metadata, and search through digital content. The API supports functionality such as uploading, updating, and organising models, as well as querying for specific content based on tags, categories, and licences. It enables integration of 3D assets into applications and automation of workflows. Key capabilities of the API include: Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 103 • Model Search: Query the Sketchfab database for models using keywords, tags, or categories. • Model Details: Retrieve detailed metadata about specific models, including creator information, tags, and licensing. • User Data Access: Obtain information about users and their uploaded models. The API offers a wide range of endpoints for interacting with various platform resources, including: • Models • Users • Me (authenticated user details) • Collections • Avatars • Categories • Skills • Environments • Tags • Relationships • Backgrounds • Matcaps • Thumbnails • Likes • Search • Comments • Licences • Purchases • Organisations (Orgs) • Projects These endpoints provide developers with extensive possibilities for integrating Sketchfab's 3D content into their applications. The following sections will focus on the Search and Models endpoints, with detailed examples of their usage.72 Sketchfab Search API Overview The Sketchfab Search API provides endpoints to efficiently search across different resource types, including models, collections, users, and organisational projects. These 72 A documentation with all endpoints and models: https://docs.sketchfab.com/dataapi/v3/index.html Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 104 endpoints enable developers to access and filter Sketchfab's extensive content effectively. Available Search Endpoints 1. Search Collections o Endpoint: GET /v3/search?type=collections o Description: Search for collections, which are curated groups of 3D models organised around specific themes or topics. 2. Search Models o Endpoint: GET /v3/search?type=models o Description: Search for 3D models, with the ability to refine results using filters such as keywords, tags, and categories. 3. General Search o Endpoint: GET /v3/search o Description: Perform a general search across multiple resource types, including models, collections, and users. 4. Search Organisational Projects o Endpoint: GET /v3/orgs/{orgUid}/search?type=projects o Description: Search for projects within an organisation, which group and manage multiple models under specific initiatives. 5. Search Organisational Models o Endpoint: GET /v3/orgs/{orgUid}/search?type=models o Description: Search for models associated with a specific organisation, useful for enterprise-related content. 6. Search Users o Endpoint: GET /v3/search?type=users o Description: Search for users on the Sketchfab platform, including creators and contributors. Performing a Search Query Using Postman To demonstrate the usage of the Sketchfab Data API, let's perform a search query using Postman73, an API development and testing software. In this example, we will search for the keyword "apple" to retrieve relevant 3D models from Sketchfab. Let’s make a GET request with the URL: https://api.sketchfab.com/v3/search?q=apple 73 Postman | https://www.postman.com Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 105 The response from the API will be displayed in JSON format, containing details about the models that match the search query. Here, we get 3420 lines of JSON. Figure 39. Response from the Sketchfab Data in JSON format. When a request gives many results, results are paginated using cursors. Each response will contain these fields you can use to make subsequent requests: ● next: full URL containing the next results. ● previous: full URL containing the previous results. ● cursors: an object containing the previous and next cursor that you can use to build the URL to the previous/next results. By default, pages contain 24 items. You can use the count parameter to change the number of items per page. This parameter is capped to 24: it will be ignored if a higher value is passed; the default value will be applied instead. Models Endpoint Overview Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 112 Figure 42. Response of the search query “stone bowl” and the media-type filter set to “3D” on the website of Europeana77 Record API: Used to retrieve detailed information about specific items in the Europeana collection. It provides all data and metadata for a single Cultural Heritage object using a unique Europeana ID, which consists of a dataset ID and a record ID. Both are extractable from the object's URL on the Europeana website, e.g. https://www.europeana.eu/de/item/181/share3d_998 has the dataset ID 181, and the record ID share3d_998 (see Image (2)). Request: Endpoint: https://api.europeana.eu/record/v2/[EUROPEANA_ID].[FORMAT] The EUROPEANA ID is typically in the format of "/DATASET_ID/RECORD_ID", and the file extensions FORMAT is one of the following output formats: .json (default), .jsonld, or .rdf. Example: Stone Bowl with the DATASET_ID= 181 and RECORD_ID= share3d_998 https://api.europeana.eu/record/v2/181/share3d_998.json?wskey=APIKEY 77 Response of the search query “stone bowl” and the media-type filter set to “3D” on the website of Europeana | (https://www.europeana.eu/en/search?page=1&qf=TYPE%3A%223D%22&query=%22stone%20b owl%22&view=grid ) Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 113 Response: The Europeana Record API returns comprehensive metadata about a specific cultural heritage object representing the EDM metadata record. It includes its unique identifier, basic details (such as title, or date), descriptive information, rights status, provider information, and links to digital representations. Figure 43. Example of a Cultural Heritage item “Stone Bowl” on the website of Europeana78 Entity API: Offers access to contextual information about entities such as people, places, concepts, and time periods related to items in the Europeana collection. 78 Example of a Cultural Heritage item “Stone Bowl” on the website of Europeana | https://www.europeana.eu/en/item/181/share3d_998 Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 114 Annotations API: Allows creation, retrieval, and management of annotations associated with Europeana resources, enabling users to add additional context or information to items. IIIF API: Provides access to high-quality images using the International Image Interoperability Framework (IIIF) standard, allowing for advanced image manipulation and presentation. User Set API: The User Set API from Europeana enables users to manage and interact with their personalised collections of digital items. It allows users to create, update, and delete sets of items from Europeana's vast cultural heritage collections. Users can organise items into sets, retrieve information about their collections, and adjust the contents as needed. 5.4.3. API Comparison Europeana vs. Sketchfab The following table provides a side-by-side comparison of the key search parameters and filters available in the Europeana, Sketchfab, and Wikidata APIs. By examining these features, we can better understand how each platform supports content discovery and management, with a particular focus on 3D assets and digital cultural content. This comparison highlights the strengths and limitations of each API, helping to determine which platform is most suitable for specific use cases. Feature/Filter Europeana API Sketchfab API Main Search query for text search in metadata fields q for text search in titles, descriptions, tags Media Type Filter type (e.g., IMAGE, TEXT, VIDEO, etc.) category and tags (e.g., architecture, nature) Licence Filter reusability (e.g., Open, Restricted) licence (e.g., Creative Commons, Public Domain) Geospatial Filter place for filtering by location No direct geospatial filter Downloadable Filter Not available downloadable filter for downloadable models Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 115 Animated Filter Not available animated filter for animated models Faceting Facets by year, provider, country, type No faceting options Sorting By timestamp_created, score, etc. By likes_count, view_count, published_at Pagination rows and start per_page and cursor for pagination Embedding Not focused on embedding embed option for 3D model embedding Complexity Filter Not available min_face_count / max_face_count for model complexity Advanced Filters None staffpicked, store_item for curated and store items 5.4.4. Metadata Comparison The following table provides a comparison of some of the technical metadata fields79 available through the Europeana and Sketchfab APIs. This side-by-side comparison highlights the strengths and focus areas of each platform. Metadata Field Europeana API Sketchfab API Title ✓ ✓ Description ✓ ✓ Creator ✓ ✓ (as part of user data) Providing institution ✓ ✗ URL to object ✓ ✓ Thumbnail URL ✓ ✓ 79 The descriptive metadata doesn’t follow any standard such as Dublin Core. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 116 Views count ✗ ✓ Animation count ✗ ✓ Face count ✗ ✓ Vertex count ✗ ✓ File size ✗ ✓ File format ✓ ✓ Language ✓ ✗ Cultural context ✓ ✗ Geolocation ✓ ✗ Time period ✓ ✗ Related items ✓ ✗ Comments ✗ ✓ Sketchfab is a platform specifically designed for publishing, sharing, and discovering 3D content. Its API provides detailed technical information about 3D models and focuses on user interaction data. Sketchfab excels in providing: • Comprehensive 3D model specifications (face count, vertex count, animation details). • User engagement metrics (views, likes, comments). • Multiple resolution options for thumbnails and model previews. • Embed codes for easy integration into websites. • Categories and tags specific to 3D modelling and design communities. Europeana provides much more detailed information about the objects themselves, which is a key differentiator from platforms like Sketchfab. This rich, object-specific metadata is crucial for understanding the cultural and historical significance of the items in Europeana's collection. Here are some key points about Europeana's object-specific metadata: • Cultural Context: Europeana provides detailed information about the cultural background of objects, which is often not available on Sketchfab. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 117 • Historical Information: There's typically more information about the time period, historical significance, and provenance of objects in Europeana. • Physical Characteristics: Europeana often includes details about materials, techniques used, and physical dimensions of objects. • Institutional Information: As items come from cultural institutions, there's often information about acquisition, conservation status, and exhibition history. • Academic Context: Europeana may provide links to related literature or research about the objects. This level of detail reflects Europeana's focus on cultural heritage and its partnerships with museums, libraries, and archives. Sketchfab, being more of a general 3D model platform, typically doesn't provide this depth of cultural and historical context. 5.4.5. Potential for AI-Generated Metadata To bridge the gap between these two platforms, AI could potentially be used to generate missing metadata: • For Sketchfab: o AI could analyse visual features of 3D models to infer cultural context, historical period, or potential materials used. o Natural Language Processing (NLP) could extract more detailed object information from titles, descriptions, and user comments. o Machine Learning models could suggest related cultural or historical items based on visual similarities. • For Europeana: o AI could generate technical 3D model specifications (face count, vertex count) by analysing the 3D files. o Computer Vision techniques could be used to detect and count potential animations in 3D models. o Predictive models could estimate user engagement metrics based on object features and historical data. While AI-generated metadata could enhance interoperability between these platforms, it's important to note that such data would be probabilistic and may not match the accuracy of curated information. Any AI-generated fields should be clearly labelled as such and potentially include confidence scores. 5.4.6. API Compatibility Conclusion In conclusion, while each platform offers valuable API capabilities for accessing and managing digital content, Sketchfab and Europeana stand out in their specific support for 3D assets and cultural heritage data, respectively. Through a detailed Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 118 comparison of their search functionalities, filtering options, and data structures, it's clear that both platforms provide powerful tools for content discovery and interaction, making them highly relevant to our project’s focus on 3D content. While Wikidata offers rich metadata for a wide range of topics, its scope doesn't align as closely with the 3D asset focus of this research. Therefore, Sketchfab and Europeana serve as prime examples of platforms that meet our specific needs. While both APIs can be utilised together in projects involving cultural content and 3D assets, their integration may require custom development to bridge their different data structures and content types. 6 State-of-the-art of existing software solutions supporting the development of MUVEs In the broader context of online social virtual environments and given the specific goals and aspirations of the IMPULSE project, a set of common features and functionalities emerges as a basis of minimum requirements that must be fulfilled by any technological solution to be adopted for the development of the project’s virtual environments: Solution architecture • Support for the deployment/development of a service providing access to a collection of online, multi-user, persistent, 3D virtual environments, with VR-enabled embodiment, social features, account management, content creation facilities and transaction support. • Development as a distributed platform involving backend services and local user clients. • Backend services include account management and authentication services as well as virtual world services. • Users authenticate once with the platform and can then enter different worlds through the client. • Users can register for the entire platform with email/password and administer their account via the client. User embodiment, navigation, interaction • Support for user embodiment through an avatar, avatar appearance is configurable (to a basic extent) by the user via the client. • Users can walk on terrain, fly or otherwise move freely within the virtual world for self-guided exploration purposes. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 119 • Users can teleport to select, predefined, world-specific locations in the world to visit key points of interest. Virtual world features, rendering • Support for basic physics, solid objects, physics. • World audio background, in-world interaction sound effects. • World rendering by client is adjustable in terms of quality vs performance. Social interaction • Multiple users can coexist within the virtual world. • Users can communicate verbally and non-verbally, in a geographically local or global scope within the virtual world or directly with other users. • Each use is uniquely identified by an alias that appears to all users in proximity within the virtual world and in all communication. Content management and creation • Copyright and intellectual property rights of all created content showcased within all virtual worlds shall be fully protected. • Users can be granted a “Creator” role which entails privileges for object creation and management as well as configuration of various parameters of a certain virtual world instance. • Creators can add, remove and transform objects available through the backend service in the world. • Creators have ownership of objects they add in the world and can only remove or modify objects they own, so that multiple co-creators can securely co-work within the same virtual world. • The client includes functionality for adding objects to the creator's palette/DB/other from available sources. • The platform includes a web-based interface for administering object sources and collections. Standards and implementation • Multi-user functionality is implemented in such a way so that no third-party services are required. • All implementations are based on well-defined architecture leveraging standard, preferred and documented patterns and practices. • Client is VR-enabled. • VR support by client based on established standards, not vendor-specific implementations. • Exhibit transactions backed by NFT technology and mechanisms. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 120 • Blockchain-secured exhibit transaction history. • Platform maintains only basic user information required for unique identification, world access and transactions. • Service databases reflecting, among other things, state of the world, backed up automatically and regularly. • Specific range of media file formats supported for compatibility with the platform. Technological solutions currently available for the development of multi-user virtual environments with features and functionalities as listed above can be grouped under two main categories, reflecting respective creative approaches: 1. Online, ready-made platforms available as services offering integrated authoring tools, interaction with objects and exhibits, user-to-user communication and other social facilities, and user authentication and management services. 2. Development platforms, tools and components for in-house application implementation. Options in the first category focus largely on the creative aspects of a virtual environment creation endeavour and include services such as the following: • Spatial80: A platform that offers tools for the creation of online multi-user virtual reality experiences. Its focus has recently been shifted towards gaming and entertainment but it still offers the Spatial Creator Tools suite for general purpose development. Integration with design and development tools such as Blender and the Unity engine is also offered as well as facilities for importing 3D models and other types of assets. Spatial supports both desktop and immersive access. • VRChat81: VRChat provides extensive world and content creation capabilities using Unity and their VRChat SDK. Its focus is on generic interactive experiences with a strong social element, including online games and social hubs. VRChat supports Meta VR equipment and offers integration with marketplace and diffusion services such as Steam and Google Play. • Sansar82: Advertised as a leading social virtual reality platform, Sansar also supports content creation via integration with the Unity engine. It has a strong focus on sophisticated world-building for social events and online gaming as well as explorable VR experiences in general. 80 Spatial | www.spatial.io 81 VRChat | hello.vrchat.com 82 Sansar | www.sansar.com Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 121 • Meta Horizon83: A generic virtual world platform for gaming, artistic events or plain social interaction, Horizon by Meta provides in-world content creation capabilities. It has a strong focus on social interaction and, naturally, supports immersion using Meta gear including Quest 2, 3, Pro and Rift S. • Second Life84: Still alive and relevant two decades since its launch, Second Life is a platform that has been extensively used for exhibiting creative and artistic work. Social interaction is a key element of the service and its user base is still large and active. However, it does not seem to keep up with technological advancements as a platform, only offering VR support, for instance, via custom client modifications and not natively. Other options such as Microsoft Mesh85 offer content creation, social interaction and immersion functionality among similar lines but are more specifically targeted towards integration with other services such as Microsoft Teams, etc. Options with an artistic orientation such as VR-All-Art86 and Artsteps87 seem to be constantly emerging, naturally leveraging both the technological progress in the field and the current hype around VR and the Metaverse; many, though, lack key features or are yet to attract the critical mass of users required to ensure their sustainability. The second category encompasses a more technical mindset as it requires in-house implementation of virtual environment software in addition to the actual content. This does by no means imply implementation from scratch as, today, numerous development platforms, tools and components are available. Game engines such as Unity and Unreal are prime candidates for a complete, integrated development environment, providing out-of-the-box an extensive range of key capabilities such as: • Scene and object management. • Interactivity and user input. • Diverse user-interface options including desktop, web and XR. • Powerful and robust object scripting in C# and C++, respectively, in conjunction with comprehensive platform-level SDKs. • Visual scripting tools that substantially ease development by non-programmers. • Support for importing multiple asset formats for 3D models, audio, video & files, images, textures, etc. • Support for audio, physics, networking, sophisticated rendering pipelines, and more. 83 Meta Horizon | horizon.meta.com 84 Second Life| secondlife.com 85 Microsoft Mesh | www.microsoft.com/en-us/microsoft-teams/ microsoft-mesh 86 VR-All-Art | vrallart.com 87 Artsteps | www.artsteps.com Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 128 Robotics and AI, 6, 91, https://doi.org/10.3389/frobt.2019.00091. 8. Biocca, F. and Levy, M.R. (1995). Communication in the age of virtual reality. Hillsdale, New Jersey: Lawrence Erlbaum. 9. Boellstorff, T. (2008). Coming of Age in Second Life. An Anthropologist Explores the Virtually Human. Princeton: Princeton University Press Princeton Gürsimsek, Remzi Ates , Online Journal of Art and Design volume 2, issue 2, 2014, p.1 10. Bolaji David Oladokun, Yusuf Ayodeji Ajani, Bernadette C. N. Ukaegbu,Emmanuel Adeniyi Oloniruha (2024). Cultural Preservation Through Immersive Technology: The Metaverse as a Pathway to the Past. Retrieved on 23rd September 2024 from https://www.degruyter.com/document/doi/10.1515/pdtc-20240015/html?lang=en 11. Bozzelli, G., Raia, A., Ricciardi, S., De Nino, M., Barile, N., Perrella, M., & Palombini, A. (2019). An integrated VR/AR framework for user-centric interactive experience of cultural heritage: The ArkaeVision project. Digital Applications in Archaeology and Cultural Heritage, 15, https://doi.org/10.1016/j.daach.2019.e00124 12. CAPHE (2024). International Conference “Enhancing Artistic Experience in Hybrid Environments” 15-17 May, La Spezia, Italy. Available: https://www.caphe.space/international-conference-enhancing-artisticexperience-in-hybrid-environments-15-17-may-la-spezia-italy/ 13. CAPHE (2024a). Architecture in VR. Opera set stage design. Available: https://www.caphe.space/architecture-in-vr-opera-set-stage-design/ 14. CAPHE (2024b). XR Festival Florence 2024. Available: https://www.caphe.space/xrfestival-florence-2024/ 15. Cecotti, H. (2022). Cultural Heritage in Fully Immersive Virtual Reality. Virtual Worlds, 1(1), 82-102. https://doi.org/10.3390/virtualworlds1010006. 16. Ciolek, T.M. & Kendon, A. (1980). Environment and the Spatial Arrangement of Conversational Encounters. Sociological Inquiry, 50(3-4), 237-271, https://doi.org/10.1111/j.1475-682X.1980.tb00022.x. 17. Charitos, D. (2005) “Virtual Reality: A new type of human – computer interface or a new communication medium?’, Issues of Communication, Athens: Kastaniotis Publications. 18. Ch’ng, E., Cai, S., Feng, P., & Cheng, D. (2023). Social Augmented Reality: Communicating via Cultural Heritage. ACM Journal on Computing and Cultural Heritage, 16(2), 1-26. 19. Chong, H.T., Lim, C.K., Rafi, A., Tan, K.L., & Mokhtar, M. (2022). Comprehensive systematic review on virtual reality for cultural heritage practices: coherent taxonomy and motivations. Multimedia Systems, 28, 711–726. https://doi.org/10.1007/s00530-021-00869-4. 20. Christou, C., Angus, C., Loscos, C., Dettori, A., & Roussou, M. (2006, November). A versatile large-scale multimodal VR system for cultural heritage visualization. In Proceedings of the ACM symposium on Virtual reality software and technology (pp. 133-140). Retrieved 10Aug. 2024, from Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 129 https://doi.org/10.1145/1180495.1180523 21. Csikszentmihalyi, M. (2014). Flow and the Foundations of Positive Psychology: The Collected Works of Mihaly Csikszentmihalyi. Dordrecht: Springer, https://doi.org/10.1007/978-94-017-9088-8. 22. Della Longa, L., Valori, I., & Farroni, T. (2022). Interpersonal Affective Touch in a Virtual World: Feeling the Social Presence of Others to Overcome Loneliness. Frontiers in Psychology, 12, https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2021.795 283. 23. Diamantaki, K., Papageorgopoulou, P., Charitos, D. & Rizopoulos, C. (2018). “Social experiences in virtual environments: a study into an emerging socio-technological phenomenon”. In Proceedings of #AoIR 2018. Montreal/Canada: Association of Internet Researchers. 24. Dim, E. & Kuflik, T. (2013). Social F-formation in Blended Reality. In Proceedings of the 3rd International Workshop on Location Awareness for Mixed and Dual Reality (LAMDa’13) in Conjunction with the International Conference on Intelligent User Interfaces (IUI’13), 19 March 2013, Santa Monica, CA, USA, 25-28, https://www.dfki.de/LAMDa/2013/accepted/LAMDa13Proceedings.pdf. 25. Dima, M., Hurcombe, L., & Wright, M. (2014). Touching the past: Haptic augmented reality for museum artefacts. In Virtual, Augmented and Mixed Reality. Applications of Virtual and Augmented Reality: 6th International Conference, VAMR 2014, Held as Part of HCI International 2014, Heraklion, Crete, Greece, June 22-27, 2014, Proceedings, Part II 6 (pp. 3-14). Springer International Publishing. Retrieved 10 Aug. 2024, from https://bura.brunel.ac.uk/bitstream/2438/19706/1/FullText.pdf 26. Dupont, L., Hubert, J., Guidat, C., & Camargo, M. (2019). Understanding user representations, a new development path for supporting Smart City policy: Evaluation of the electric car use in Lorraine Region. Understanding Smart Cities: Innovation Ecosystems, Technological Advancements, and Societal Challenges, 142, 333–346. https://doi.org/10.1016/j.techfore.2018.10.027 27. El Raheb, K., Soulis, A., Nastos, D., Lougiakis, C., Roussou, M., Christopoulos, D., Sofianopoulos, G., Papagiannis, S., Rüggeberg, J., Katsikaris, L., & Rüggeberg, J. (2021). Eliciting requirements for a multisensory eXtended Reality platform for training and informal learning. CHI Greece 2021: 1st International Conference of the ACM Greek SIGCHI Chapter, 1–8. https://doi.org/10.1145/3489410.3489428 28. European Commission (2023), “Communication from the Commission to the European Parliament, The Council, the European Economic and Social Committee and Committee of the Regions: An EU initiative on Web 4.0 and virtual worlds: a head start in the next technological transition” 29. Fang, J., Chang, V., Gao, C., & Wang, H.-C. (2021). Social Interactions in Virtual Reality: What Cues Do People Use Most and How. In Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 130 Computing (CSCW '21 Companion). Association for Computing Machinery, New York, NY, USA, 49–52. https://doi.org/10.1145/3462204.3481772. 30. Forte, M., Pietroni, E., & Dell’Unto, N. (2008). 3D Multi-user domain and virtual ecosystems for transmission and communication of cultural heritage. In DMACH 2008, Digital Media and its Applications in Cultural Heritage, 31. Freeman, G. & Acena, D. (2021). Hugging from A Distance: Building Interpersonal Relationships in Social Virtual Reality. In Proceedings of the 2021 ACM International Conference on Interactive Media Experiences (IMX '21). Association for Computing Machinery, New York, NY, USA, 84–95. https://doi.org/10.1145/3452918.3458805. 32. Fanini, B., Ferdani, D., Demetrescu, E., Berto, S., & d’Annibale, E. (2021). ATON: An open-source framework for creating immersive, collaborative and liquid webapps for cultural heritage. Applied Sciences, 11(22), 11062. 33. Flores, J., Arias, J. E., Saavedra, S., Varela, E., Ferro, J. M., & Taboada, J. A. (2000). Spreading of the cultural heritage by means of multi-configurable, low cost virtual reality techniques. In Eurographics (Short Presentations). 34. Galloway, A. (2005). Non-Probability Sampling. In K. Kempf-Leonard (Ed.), Encyclopedia of Social Measurement, Elsevier, 2005, Pages 859-864, https://doi.org/10.1016/B0-12-369398-5/00382-0 35. Gambin, T., Hyttinen, K., Sausmekat, M., Wood, J. (2021a). Making the Invisible Visible: Underwater Malta—A Virtual Museum for Submerged Cultural Heritage. Remote Sensing, 13(8), 1558. https://doi.org/10.3390/rs13081558 36. Gambin, T., Sausmekat, M., Kovacevic, D. (2021b). The Innovative and State of the Art Public Access Management of Malta’s Underwater Cultural Heritage. Heritage, 4(4), 3365–3381, https://doi.org/10.3390/heritage4040187 37. Gibson, William. Neuromancer. ACE, July 1984. p. 243-244 38. Giovannini, E. C., & Bono, J. (2023). Creating Virtual Reality Using a Social Virtual Environment: Phygital Exhibition at the Museum Passion in Sordevolo. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 48, 669-676. 39. Girvan, C. (2013). What is a Virtual World? Definition and Classification (TCDCS2013-10). Tech. rep. Dublin, Ireland: School of Computer Science and Statistics (SCSS) at Trinity College Dublin. Retrieved from: https://www.scss.tcd.ie/publications/tech-reports/reports.13/TCD-CS-201310.pdf 40. Gonzalez-Franco, M., & Peck, T. C. (2018). Avatar Embodiment. Towards a Standardized Questionnaire. Frontiers in Robotics and AI, 5. https://doi.org/10.3389/frobt.2018.00074 41. Greenhalgh, C. & Benford, S. (1995), MASSIVE: A Collaborative Virtual Environment for Teleconferencing, ACM Transactions on Computer-Human Interaction, Vol 2, No 3, September 1995, Pages 239-261. 42. Gunawardena, C. N., & Zittle, F. J. (1997). Social presence as a predictor Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 131 of satisfaction within a computer mediated conferencing environment. American Journal of Distance Education, 11(3), 8-26. 43. Gunawardena, C. N. (1995). Social presence theory and implications for interaction collaborative learning in computer conferences. International Journal of Educational Telecommunications, 1(2/3), 147-166 44. Gunkel, S., Stokking, H., Prins, M., Niamut, O., Siahaan, E., & Cesar, P. (2018, June). Experiencing Virtual Reality Together: Social VR Use Case Study. In Proceedings of the 2018 ACM International Conference on Interactive Experiences for TV and Online Video (pp. 233-238). ACM. 45. Gürsimsek, R. A. (2014), Online Journal of Art and Design volume 2, issue 2, p.1. 46. Haejung Suk and Teemu H. Laine (2024). Influence of Avatar Facial Appearance on Users’ Perceived Embodiment and Presence in Immersive Virtual Reality. Retrieved on 2nd September 2024 from https://www.mdpi.com/20799292/12/3/583 47. Han, E., & Bailenson, J. (2024, May 22). Social Interaction in VR. Oxford Research Encyclopedia of Communication. Retrieved 17 Aug. 2024, from https://oxfordre.com/communication/view/10.1093/acrefore/9780190228613.00 1.0001/acrefore-9780190228613-e-1489. 48. Handley, R., Guerra, B., Goli, R., & Zytko, D. (2022). Designing Social VR: A collection of design choices across commercial and research applications. Retrieved on 2nd August 2024 from https://arxiv.org/pdf/2201.02253 49. Hassenzahl, M., Burmester, M., & Koller, F. (2003). AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In G. Szwillus & J. Ziegler (Eds.), Mensch & Computer 2003 (Vol. 57, pp. 187–196). Vieweg+Teubner Verlag. https://doi.org/10.1007/978-3-322-80058-9_19 50. Herrera, F., Oh, S.Y., & Bailenson, J.N. (2018). Effect of Behavioral Realism on Social Interactions Inside Collaborative Virtual Environments. Presence: Teleoperators and Virtual Environments, 27 (2), 163–182, https://doi.org/10.1162/pres_a_00324. 51. Hoppe, M., Rossmy, B., Neumann, D.P., Streuber, S., Schmidt, A., & Machulla, T.-K. (2020). A Human Touch: Social Touch Increases the Perceived Human-likeness of Agents in Virtual Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3313831.3376719. 52. Hudson, M., & Cairns, P. (2014). Chapter 6 Measuring Social Presence in TeamBased Digital Games. In G. Riva, J. Waterworth, & D. Murray, Interacting with Presence: HCI and the Sense of Presence in Computer-mediated Environments (pp. 83–101). DE GRUYTER OPEN. https://doi.org/10.2478/9783110409697.6 53. IMPULSE Project proposal (2023). IMPULSE, IMmersive digitisation: uPcycling cULtural heritage towards new reviving StratEgies. PROPOSAL_101132704IMPULSE-HORIZON-CL2-2023-HERITAGE-01-PART_B_Section_1.pdf (Available: https://teams.microsoft.com/v2/) Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 132 54. Interaction Design Foundation - IxDF. (2023). What is Social VR?. https://www.interaction-design.org/literature/topics/social-virtual-reality-vr (accessed 17 August 2024). 55. Kaneva, N. "Narrative power in Online Game worlds" p.56-73 in Williams, J. P., & Smith, J. H., eds. (2007). The players' realm: studies on the culture of video games and gaming. Jefferson, N.C.: McFarland & Co. 56. Kaplan, A.M. and Haenlein, M. (2009). The fairyland of second life: about virtual social worlds and how to use them. Business Horizons, 52 (6), 563-72 57. Karunanayaka, K., Johari, N., Hariri, S., Camelia, H., Bielawski, K.S., & Cheok, A.D. (2018). New Thermal Taste Actuation Technology for Future Multisensory Virtual Reality and Internet, IEEE Transactions on Visualization and Computer Graphics, 24(4), 1496-1505, doi: 10.1109/TVCG.2018.2794073. 58. Katifori, A., Lougiakis, C., & Roussou, M. (2021). The Role of High-fiving for Sustaining Engagement in Social VR Experiences. 59. Kersten, T., Tschirschwitz, F., & Deggim, S. (2017, February). Development of a virtual museum including a 4D presentation of building history in virtual reality. In TC II & CIPA 3D Virtual Reconstruction and Visualization of Complex Architectures, 1–3 March 2017, Nafplio, Greece (pp. 361-367). Copernicus. Retrieved 10 Aug. 2024, from https://repos.hcuhamburg.de/bitstream/hcu/731/1/isprs-archives-XLII-2-W3-361-2017.pdf 60. Kosmas, P., Galanakis, G., Constantinou, V., Drossis, G., Christofi, M., Klironomos, I., Zaphiris, P., Antona, M., & Stephanidis, C. (2020). Enhancing Accessibility in Cultural Heritage Environments: Considerations for Social Computing. Universal Access in the Information Society, 19(2), 471-482, https://doi.org/10.1007/s10209019-00651-4 61. Kun Lyu, Arianna Brambilla, Anastasia Globa, Richard de Dear (2023). An immersive multisensory virtual reality approach to the study of human-built environment interactions. Retrieved on19th September 2024 fromhttps://www.sciencedirect.com/science/article/pii/S0926580523000961 62. Kyrlitsias, C. & Michael-Grigoriou, D. (2022). Social Interaction With Agents and Avatars in Immersive Virtual Environments: A Survey. Frontiers in Virtual Reality, 2, https://www.frontiersin.org/journals/virtualreality/articles/10.3389/frvir.2021.786665. 63. Li, B.J. & Bailenson, J.N. (2017). Exploring the Influence of Haptic and Olfactory Cues of a Virtual Donut on Satiation and Eating Behavior. Presence: Teleoperators and Virtual Environments (2017) 26 (3): 337–354, https://doi.org/10.1162/pres_a_00300. 64. Li, G., Lin, S. & Tian, Y. (2024). Immersive Museums in the Digital Age: Exploring the Impact of Virtual Reality on Visitor Satisfaction and Loyalty. Journal of the Knowledge Economy, https://doi.org/10.1007/s13132-024-01782-7. 65. Li, J., & Cesar, P. (2023). Social virtual reality (VR) applications and user experiences. In Immersive Video Technologies (pp. 609-648). Academic Press. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 133 66. Li, Y., Ch’ng, E., Cai, S., & See, S. (2018). Multiuser interaction with hybrid VR and AR for cultural heritage objects. In 2018 3rd Digital Heritage International Congress (DigitalHERITAGE) held Jointly with 2018 24th International Conference on Virtual Systems & Multimedia (VSMM 2018) (pp. 1-8). IEEE. 67. Liarokapis, F., Petridis, P., Andrews, D., de Freitas, S. (2017). Multimodal Serious Games Technologies for Cultural Heritage. In: Ioannides, M., Magnenat-Thalmann, N., Papagiannakis, G. (eds), Mixed Reality and Gamification for Cultural Heritage. Cham: Springer, pp. 371–392, Cham. https://doi.org/10.1007/978-3-319-496078_15. 68. Lotte, F., Faller, J., Guger, C., Renard, Y., Pfurtscheller, G., Lécuyer, A., & Leeb, R. (2013). Combining BCI with Virtual Reality: Towards New Applications and Improved BCI. In Allison, B.Z., Dunne, S., Leeb, R., Del R. Millán, J., & Nijholt, A. (eds.), Combining BCI with Virtual Reality: Towards New Applications and Improved BCI: Towards Practical Brain-Computer Interfaces: Bridging the Gap from Research to Real-World Applications, Berlin / Heidelberg: Springer, pp. 197220, https://doi.org/10.1007/978-3-642-29746-5_10. 69. Lotte, F., Renard, Y., & Lécuyer, A. (2008). Self-Paced Brain-Computer Interaction with Virtual Worlds: A Quantitative and Qualitative Study “Out of the Lab”. 4th International Brain Computer Interface Workshop and Training Course, Graz University of Technology, Graz, Austria, https://inria.hal.science/inria00304340/file/bci-wtc08.pdf. 70. Lowenthal, P. R. (2010). The evolution and influence of social presence theory on online learning. Online Education and Adult Learning: New Frontiers for Teaching Practices, 124-139. Hershey, PA: IGI Global. 71. Maloney, D., Freeman, G., & Robb, A. (2021). Social Virtual Reality: Ethical Considerations and Future Directions for An Emerging Research Space. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Lisbon, Portugal, pp. 271-277, doi: 10.1109/VRW52623.2021.00056. 72. Marini, D., Folgieri, R., Gadia, D. and Rizzi, A. (2012). Virtual reality as a communication process. Virtual reality, 16(3), 233-241 73. McVeigh-Schultz, J., Kolesnichenko, A., & Isbister, K. (2019). Shaping Pro-Social Interaction in VR: An Emerging Design Framework. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, Paper 564, 1–12. https://doi.org/10.1145/3290605.3300794. 74. Meadows, M.S. (2008). I, Avatar: The culture and consequences of having a second life. Berkeley CA: New Riders 75. Minge, M., Thüring, M., & Wagner, I. (2016). Developing and Validating an English Version of the meCUE Questionnaire for Measuring User Experience. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 60(1), 2063–2067. https://doi.org/10.1177/1541931213601468 Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 134 76. Minge, M., Thüring, M., Wagner, I., & Kuhr, C. V. (2017). The meCUE Questionnaire: A Modular Tool for Measuring User Experience. In M. Soares, C. Falcão, & T. Z. Ahram (Eds.), Advances in Ergonomics Modeling, Usability & Special Populations (Vol. 486, pp. 115–128). Springer International Publishing. https://doi.org/10.1007/978-3-319-41685-4_11 77. Minocha, S. and Reeves, A.J. (2010). Design of learning spaces in 3D virtual worlds: an empirical investigation of Second Life, Learning, Media and Technology, 35 (2), 111-137 78. Moustafa, M., & Steed, A. (2021). Communication in immersive social virtual reality: A systematic review of 10 years’ studies. Retrieved on 2nd August 2024 from https://arxiv.org/pdf/2210.01365 79. Mulders, M., & Zender, R. (2021). Enabling human interaction in virtual reality: An explorative overview of opportunities and limitations of current VR technology. Retrieved on 2nd August 2024 from https://link.springer.com/chapter/10.1007/978-3-031-21707-4_9 80. Mystakidis, S. (2022) Metaverse, Encyclopedia 2022, 2(1), MDPI, pp. 486-497. 81. Olin, P.A., Issa, A.M., Feuchtner, T., & Grønbæk, K. (2020). Designing for Heterogeneous Cross-Device Collaboration and Social Interaction in Virtual Reality. In Proceedings of the 32nd Australian Conference on Human-Computer Interaction (OzCHI '20). Association for Computing Machinery, New York, NY, USA, pp. 112–127. https://doi.org/10.1145/3441000.3441070. 82. Pathi, S.K., Kristoffersson, A., Kiselev, A., & Loutfi, A. (2019). F-Formations for Social Interaction in Simulation Using Virtual Agents and Mobile Robotic Telepresence Systems. Multimodal Technologies and Interaction, 3(4), 69, https://doi.org/10.3390/mti3040069. 83. Perret, J., & Vander Poorten, E. (2018). Touching Virtual Reality: a Review of Haptic Gloves. ACTUATOR 2018; 16th International Conference on New Actuators (Bremen: VDE), 1–5. 84. Pescarin, S., Bonano, V., Bordignon, A., & Fiorenza, G. (2023). Hybrid XR collaborative and guided experiences in Cultural Heritage: Brancacci POV prototype. In Proceedings of the EUROGRAPHICS Workshop on Graphics and Cultural Heritage, Lecce, Italy (pp. 4-6). 85. Reimat, I., Mei, Y., Alexiou, E., Jansen, J., Li, J., Subramanyam, S., Viola, I., Oomen, J., & Cesar, P. (2022). Mediascape XR: A Cultural Heritage Experience in Social VR. In Proceedings of the 30th ACM International Conference on Multimedia (MM '22). Association for Computing Machinery, New York, NY, USA, pp. 6955–6957. https://doi.org/10.1145/3503161.3547732. 86. Rizvic, S., Young, G., Changa, A., Mijatovic, B., & Ivkovic-Kihic, I. (2022). Da Vinci Effect-multiplayer Virtual Reality experience 87. Rohrer, Ch. (2022). When to Use Which User-Experience Research Methods. NNgroup.com. Available: https://www.nngroup.com/articles/which-ux-researchmethods Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 135 88. Ross, J. (2017). Speculative method in digital education research. Learning, Media and Technology, 42(2), 214-229. 89. Roth, D., Bente, G., Kullmann, P., Mal, D., Purps, C. F., Vogeley, K., & Latoschik, M. E. (2019, November). Technologies for social augmentations in user-embodied virtual reality. In Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology (pp. 1-12). Retrieved on 18nd August 2024 from https://dl.acm.org/doi/fullHtml/10.1145/3359996.3364269 90. Ruiz, S., Molina-Espinosa, J. M., Magana, A. J., & Benes, B. (2022). Systematic review of multimodal human–computer interaction. Informatics, 9(1), 13. Retrieved on 2nd August 2024 from https://www.mdpi.com/2227-9709/9/1/13 91. Schönauer, C., Kaufmann, H., Roussou, M., Rüggeberg, J., Rüggeberg, J., Katsikaris, L., Rogkas, S., & Christopoulos, D. (2023). Creating Informal Learning and First Responder Training XR Experiences with the ImmersiveDeck. 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 53–60. https://doi.org/10.1109/VRW58643.2023.00016 92. Schroeder, R. (2011), Being There Together: Social Interaction in Virtual Environments, Human Technology Interaction Series, Oxford University Press. 93. Schroeder, R. (ed.) (2002), The Social Life of Avatars: Presence and Interaction in Shared Virtual Environments. London: Springer Verlag. 94. Schroeder, R. (1996). Possible worlds: The social dynamic of virtual reality technology. Boulder, CO: Westview Press. 95. Schubert, T., Friedmann, F., & Regenbrecht, H. (2001). The Experience of Presence: Factor Analytic Insights. Presence: Teleoperators and Virtual Environments, 10(3), 266–281. https://doi.org/10.1162/105474601300343603 96. Schwajda, D. J., & Anthes, C. (2022). Utilizing F-Formations in Collaborative CrossVirtuality Analytics Scenarios. In AVI'22 Workshop Proceedings: “Enhancing Crossreality Applications and User Experiences”, ACM Press. https://crworkshop.github.io/papers/schwajda2022-utilizing.pdf. 97. Škola, F., Rizvić, S., Cozza, M., Barbieri, L., Bruno, F., Skarlatos, D., & Liarokapis, F. (2020). Virtual Reality with 360-Video Storytelling in Cultural Heritage: Study of Presence, Engagement, and Immersion. Sensors, 20(20), 5851, https://doi.org/10.3390/s20205851. 98. Slater, M. & Sanchez-Vives, M.V. (2014). Transcending the Self in Immersive Virtual Reality. Computer, 47(7), 24-30, https://ieeexplore.ieee.org/document/6861900. 99. Slater, M., Gonzalez-Liencres, C., Haggard, P., Vinkers, C., Gregory-Clarke, R., Jelley, S., Watson, Z., Breen, G., Schwarz, R., Steptoe, W., Szostak, D., Halan, S., Fox, D., & Silver, J. (2020). The Ethics of Realism in Virtual and Augmented Reality. Frontiers in Virtual Reality, 1, https://www.frontiersin.org/journals/virtualreality/articles/10.3389/frvir.2020.00001. 100. Snowdon, D., Churchill, E., Munro, A. (2001), Collaborative Virtual Environments: Digital Spaces and Places for CSCW: An Introduction, Collaborative Virtual Environments, Springer-Verlag London Limited. Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 136 101. Soto-Martin, O., Fuentes-Porto, A., & Martin-Gutierrez, J. (2020). A Digital Reconstruction of a Historical Building and Virtual Reintegration of Mural Paintings to Create an Interactive and Immersive Experience in Virtual Reality. Applied Sciences, 10(2):597. https://doi.org/10.3390/app10020597. 102. Souza, L. B. O., Cunha, A. C. S., Lima, M. M. X. de, & Ricca, D. E. P. (2023). Pragmatic and hedonic aspects of user experience in Virtual Reality: Analysis of novice users’ information mediation during their first interaction with a Metaverse platform. InfoDesign - Revista Brasileira de Design Da Informação, 20(3). https://doi.org/10.51358/id.v20i3.1068 103. Tremmel, C., Herff, C., Sato, T., Rechowicz, K., Yamani, Y., & Krusienski, D.J. (2019). Estimating Cognitive Workload in an Interactive Virtual Reality Environment Using EEG. Frontiers in Human Neuroscience, 13, 401. https://doi.org/10.3389/fnhum.2019.00401. 104. Tromp, J., Le, C., Le, B., & Le, D. N. (2018). Massively Multi-user Online Social Virtual Reality Systems: Ethical Issues and Risks for Long-Term Use. In Social Networks Science: Design, Implementation, Security, and Challenges (pp. 131149). Springer, Cham. 105. Vaz, R. I., Fernandes, P. O., & Veiga, A. C. (2018). Interactive Technologies in Museums: How Digital Installations and Media Are Enhancing the Visitors' Experience. In Rodrigues, J., Ramos, C., Cardoso, P. & Henriques, C. (eds.), Handbook of Research on Technological Developments for Cultural Heritage and eTourism Applications (pp. 30-53). IGI Global. https://doi.org/10.4018/978-15225-2927-9.ch002. 106. Zabulis, X., Grammenos, D., Sarmis, T., Tzevanidis, K., Padeleris, P., Koutlemanis, P., & Argyros, A.A. (2013). Multicamera human detection and tracking supporting natural interaction with large-scale displays. Machine Vision and Applications, 24(2), 319-336, https://doi.org/10.1007/s00138-012-0408-6 Deliverable D 2.1: Report on the review of the latest MUVE technologies, processes, formats, best practices, impediments. IMPULSE IMmersive digitisation: uPcycling cULtural heritage towards new reviving Strategies| 137