D6.3: Report on XR4HUMAN Experience Library Evaluation & Results

! 1 D6.3: Report on XR4HUMAN Experience Library Evaluation & Results Authors: F. Mushtaq (UoL), O. Schreer (HHI). M. Barngrover (XR4Europe), E. Prasolova-Forland (NTNU), M. Fominykh (NTNU) , Hannah Svennungsen (NTNU), Marianna Pizzo (NTNU) Editors: O. Schreer (HHI), F. Mushtaq (UoL) Project title: The Equitable, Inclusive, and Human-Centered XR Project Project acronym: XR4HUMAN Grant Agreement no.: 101070155 Lead contractor for this deliverable: ! 2 Deliverable factsheet: Project Number: 101070155 Project Acronym: XR4Human Project Title: The Equitable, Inclusive, and Human-Centered XR Project Title of Deliverable: D6.3 Experience Library Evaluations & Result Work Package: WP6 Due date according to contract: 30. October 2025 (M36) Editor(s): O. Schreer (HHI), Faisal Mushtaq (UoL) Contributor(s): M. Barngrover (XR4Europe), Ekaterina Prasolova-Forland (NTNU), Mikhail Fominykh (NTNU), Hannah Svennungsen (NTNU), Marianna Pizzo (NTNU), Stig Tobiassen (NTNU) Reviewer(s): Lucas Stephane (IFE) Approved by Rigmor C. Baraas, Rosemarie Bernabe ABSTRACT: This deliverable reports the outcomes of Task 6.4 within Work Package 6 of XR4HUMAN. It documents the development of a standardised, modular post-experience survey for XR evaluation, its piloting in controlled and public settings, and its integration with complementary protocols led by partners, notably NTNU. The report synthesises findings from evaluations conducted with diverse user groups, including children and young people, adults with cognitive or physical impairments, jobseekers engaged through national employment services, international students, and medical students using specialist neuroanatomy applications. Results are presented with a focus on factors that matter for evaluation—onboarding, navigation and orientation, sensory comfort, accessibility, vocational ! 3 relevance, and engagement sustainability—rather than on appraisal of specific applications. The deliverable also describes targeted software re-development to exemplify responsible and inclusive practice; and it sets out a sustainability pathway for the Experience Library, including replication on the XR4Europe website, migration to a lightweight data-collection platform, and a volunteer evaluator pool chaired by XR4Europe. Together, these actions establish an evidence-based, ethically grounded framework for inclusive XR evaluation that can be reused beyond the lifetime of the project. Keyword List: XR experience, library, submission form, self-assessment form, evaluation, benchmark ! 4 Consortium: ROLE NAME Short Name Country 1. Coordinator UNIVERSITETET I SOROST-NORGE USN NO 2. Partner XR4EUROPE XR4Europe BE 3. Partner UNIVERSITETET I OSLO UiO NO 4. Partner ETHNICON METSOVION POLYTECHNION NTUA EL 5. Partner NORGES TEKNISK-NATURVITENSKAPELIGE UNIVERSITET NTNU NTNU NO 6. Partner UNIVERSITEIT LEIDEN ULEI NL 7. Partner KARLSRUHER INSTITUT FUER TECHNOLOGIE KIT DE 8. Partner INSTITUTT FOR ENERGITEKNIKK IFE NO 9. Partner FRAUNHOFER GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG EV Fraunhofer DE 10. Partner OPEN AR CLOUD EUROPE GEMEINNUTZIGE UG OARC EU DE 11. Partner STICHTING FONTYS Fontys NL 12. AP UNIVERSITY OF LEEDS UNIVLEEDS UK Revision history: VERSION DATE Revised by Reason 0.1 12.08.2025 Original version drafted by Faisal Mushtaq 0.2 26.08.2025 Input by partners 0.3 15.09.2025 Consolidated version 0.4 01.10.2025 Internal review 0.5 16.10.2025 Final version sent to coordinator 1.0 19.10.2025 Approved by coordinator ! 5 Table of Contents 1!Introduction ............................................................................................................................................................ 6! 2!Methodological Framework for Evaluation .................................................................................................. 8! 3!Development of the Post-XR4HUMAN Survey .......................................................................................... 10! Rationale and Objectives ........................................................................................................................ 10! Literature-Derived Framework .......................................................................................................... 10! Expert-Led Iterative Refinement ...................................................................................................... 10! Pilot Testing Across Contrasting Contexts ................................................................................... 11! Data Handling and Initial Benchmarking .................................................................................... 11! Summary ............................................................................................................................................................ 13! 4!Examples of Real-World Evaluations ............................................................................................................ 14! Individuals with Cognitive and Physical Impairments ............................................................ 15! Methodology .................................................................................................................................................... 15! Results .................................................................................................................................................................. 17! Vocational and Career Exploration Evaluations .......................................................................... 18! Methodology .................................................................................................................................................... 19! Results ................................................................................................................................................................ 20! Implications for Evaluation Frameworks .......................................................................................... 21! NAV Jobseeker Evaluations ................................................................................................................... 21! Methodology ................................................................................................................................................... 22! Results ................................................................................................................................................................. 22! Implications for Evaluation Frameworks ..................................................................................... 23! 5!XR Software (Re)-Development ...................................................................................................................... 25! XR software applications VR4VET Blue Sector and Nevrolens ......................................... 25! VR4VET Blue Sector ................................................................................................................................... 25! Nevrolens ........................................................................................................................................................... 26! XR software refinement use cases and results ............................................................................ 26! Inclusion and Accessibility refinements in the VR4VET Blue Sector app ............. 27! Inclusion and Accessibility refinements in the Nevrolens app .................................... 32! Conclusions of the Software refinement ......................................................................................... 34! 6!Experience Library Future ................................................................................................................................. 36! Annex 1: Post-XR Experience Final Survey (Online Version) .................................................... 39! ! 6 1 Introduction This document presents Deliverable D6.3, the final report for Work Package 6 (WP6) of the XR4Human project, corresponding to Task 6.4: Evaluation of Example Use Cases. As outlined in the Grant Agreement, WP6 focuses on the creation, curation, and responsible dissemination of extended reality (XR) experiences, with Task 6.4 committed specifically to their systematic evaluation. The objective of this task is to assess the quality and accessibility of selected XR experiences across a range of user-centred dimensions, including cybersickness, visual (dis)comfort, accessibility, and perceptions relating to safety, security, privacy, and ethics. The evaluation work undertaken in this task was designed to produce meaningful benchmark data, grounded in empirical usage within physical XR laboratories. In keeping with the inclusive ethos of XR4Human, the project prioritised the recruitment of participants from under-represented groups, encompassing variation across age, gender, ethnicity, and sensory-motor abilities. This inclusive sampling strategy was essential to ensuring that the evaluations conducted under Task 6.4 are generalisable and reflect the real-world diversity of XR users. To support the delivery of this task, the consortium developed a dedicated evaluation survey instrument tailored to the dimensions above. The tool was deployed in conjunction with selected submissions from the XR4HUMAN Experience Library, most notably the 'da Vinci' experience, which was evaluated with a benchmark sample of 64 participants (32 adults and 32 children) at the University of Leeds. In parallel, additional pooled data and qualitative evaluations were undertaken by Ekaterina Prasolova-Førland, Mikhail Fominykh and other colleagues at the Norwegian University of Science and Technology (NTNU), providing complementary insights into user experience across use cases. Throughout this process, WP6 also supported partner institutions in refining their XR submissions to meet the evolving technical and ethical standards of the XR4HUMAN framework. Notably, the NTNU submissions were updated to align with the project's guidelines on non-commercial use, accessibility, and ethical evaluation. External partners, including stakeholders from the Smart City initiative in Kirchheim, Germany (connected via Technische Universität München), were invited to contribute both to the Delphi study on best practices and to the XR4HUMAN Experience Library. While WP6 as a whole has advanced the design, curation, and dissemination of responsible XR experiences, this document concentrates on how those experiences were evaluated in practice. Section 2 sets out the methodological framework that underpins core activities in Task 6.4. Section 3 details the development of the postexperience survey and explains how the literature-derived constructs and expert iterations shaped a modular instrument. Section 4 presents real-world evaluations across multiple settings and participant groups, with an emphasis on the factors that evaluation must capture to remain inclusive and generalisable. Section 5 summarises targeted software re-development carried out to align exemplar ! 7 experiences with XR4HUMAN guidance. Section 6 outlines the sustainability plan for the Experience Library. ! 8 2 Methodological Framework for Evaluation A key objective of Task 6.4 was to establish a robust and replicable methodology for evaluating XR experiences in ways that are both user-centred and technically informative. Central to this was the development of a standardised post-experience survey instrument that could be deployed across different XR use cases and institutional contexts. The motivation for this was twofold: to ensure consistency in how XR experiences are assessed across the XR4Human consortium, and to generate benchmark data that could support meaningful comparisons over time and between applications. The lack of standardisation in how XR experiences are currently evaluated poses a challenge to both research and practice. While many studies focus on technical performance or subjective impressions, fewer adopt a unified approach that incorporates ethical, psychological, and accessibility-oriented dimensions within a single evaluation framework. This gap was particularly evident when trying to compare or interpret user responses across different environments, technologies, or user populations. Without a common instrument, data remained siloed or anecdotal, limiting the extent to which we could draw generalisable insights or identify patterns of harm, discomfort, or exclusion. To address this, WP6 designed a modular survey (see Annex 1: Post-XR Experience Final Survey (Online Version) that could be used after any XR experience has been delivered in person. The survey was structured around key dimensions specified in the Grant Agreement: cybersickness, visual comfort, physical and cognitive accessibility, and perceived ratings of security, safety, privacy, and ethical integrity. Additional items were included to capture users’ demographic background, prior XR familiarity, and any specific barriers encountered during the experience. This structure allows the survey to serve not only as a user feedback tool but also as a vehicle for surfacing normative issues in XR design. Standardisation brings several key advantages. It enables data aggregation across sites, populations, and technologies, thus supporting the generation of pooled datasets that can be meaningfully analysed. It also enhances transparency and accountability, as developers contributing to the Experience Library can be evaluated along consistent criteria. Perhaps most importantly, it provides a foundation for benchmarking: identifying where particular applications excel or fall short in meeting inclusive and ethical design standards. In the context of XR4Human, this approach reinforces the project’s commitment to responsible innovation and provides a methodological anchor for future XR research that seeks to centre the lived experience of diverse users. The methodological framework therefore rests on two pillars: a common, construct-based survey that provides comparable post-experience data across sites, and context-sensitive protocols that capture observational and task-based indicators where these are most informative. Having established the need for a standardized evaluation approach, the next step was to develop the specific survey ! 9 instrument that would operationalise this framework. The survey development process involved literature synthesis, expert iteration and pilot testing to create a robust and modular measurement tool that could be deployed across multiple use cases, such as a public exhibition and a specialist clinical training lab, while maintaining construct comparability. ! 16 training simulation, which presented a cognitively and motorically demanding environment in which users completed tasks associated with aquaculture and fisheries work, including moving between stations using a teleportation mechanic, manipulating tools with handheld controllers, and interpreting in-world prompts (app presented later in the deliverable). The third app was AR app Nevrolens 3 for visualizaing rat brain neuroanatomy, presented later in this deliverable, deployed on both Hololens and on a tablet. While this app has been primarily developed for medical and neuroscience students, we demoed it with individuals with cognitive impairments to explore how AR interfaces, multimodal interaction, and engaging visualizations could support enjoyable and accessible learning experiences across diverse user groups. The evaluation protocol followed a consistent structure for each participant. The session began with a short verbal introduction to the activity, often with the assistance of the caretaker. This was followed by the fitting of the HMD, and — where relevant — calibration of the hand-tracking system. Each application was demonstrated briefly by the assistant or evaluator, after which the participant was encouraged to take control and interact independently for up to ten minutes. Other participants could follow the activities on laptop or big screen connected to the headsets while waiting for their turn. Observers documented the participant’s ability to navigate, initiate actions, and respond to virtual stimuli. Behavioural cues such as facial expression, vocalisation, body posture, and frequency of assistance requests were noted. The same process was repeated for the next application, with a short break in between to avoid fatigue. Following the lab visits, semi-structured interviews were conducted with a physical impairment participant and the cognitive impairment assistants. The interview of the physical impairment person focused on accessibility, embodiment, and empowerment. Users with cognitive impairments were not interviewed due to practical limitations and ethical considerations related to obtaining informed consent. They were presented with a highly simplified pictorial shortened version of the XR4HUMAN questionnaire instead. The cognitive impairment assistants were asked about their perceptions of the participant’s engagement, inclusion, accessibility and challenges, and the fit between the XR activities and the participant’s usual abilities. Qualitative approach was especially useful in this case as the participants’ impairments and diagnoses varied significantly. For ethical and privacy reasons, the researchers did not have access to diagnoses, so perceptions and observations of their assistants provided valuable insights on various aspects of cognitively challenged participants’ interaction with XR. Interviews were audio-recorded and later summarised thematically. Observational notes, questionnaire results and interview summaries were reviewed together to identify recurring patterns in both opportunities and challenges presented by the XR experiences. 3 Nevrolens https://www.ntnu.edu/imtel/nevrolens ! 17 Results The results from this evaluation were complex, illustrating both the empowering potential of XR and the barriers that can limit its accessibility, especially for neurodiverse populations and people with mobility challenges whose needs are often overlooked by developers of XR systems For the wheelchair-using participant (NTNU employee), the physical embodiment of VR proved to be a double-edged sword. While exploring the Blue Sector environment, the participant leaned forward in his chair to interact with an object on a virtual workbench, overestimating the stability of his real-world position. Immersed in the simulation and momentarily unaware of his physical surroundings, he tipped sideways and partially out of his wheelchair. The assistant intervened immediately, helping him back to the chair, but the event was a clear reminder that immersive XR can generate physical behaviours with real-world consequences, especially for users with limited core stability. According to the participant, the stability also depends on the type of wheelchair used, something that needs to take into account. Following the incident, the participant resumed the activity without hesitation, later describing the experience as “freeing” and unlike anything he could typically do in daily life. This comment reflected a key theme across the evaluations: for some individuals with disabilities, XR offered a form of agency and exploration that transcended their usual physical limitations. For the participants with cognitive impairments the hand-tracking cat game consistently elicited positive affective responses. Participants smiled, laughed, and experimented with different gestures, quickly learning that the virtual cat would react to their hand movements. The lack of controllers removed a substantial barrier for those with fine motor difficulties or who might be intimidated by unfamiliar hardware. Observers recorded longer uninterrupted play sequences, fewer calls for assistance, and a greater degree of self-initiated exploration compared to the more complex simulation. This sense of immersion, coupled with the simplicity of the interaction, made the cat game a particularly effective example of accessible XR design. The Blue Sector app highlighted several accessibility and usability challenges for the cognitive impairment group. The teleportation-based movement system was a particular point of difficulty. Participants often struggled to align the controller precisely to the intended destination, and the cognitive demand of remembering multiple buttons and triggers led to frequent pauses and confusion. Assistants found themselves offering near-continuous verbal guidance to maintain forward progress. In several cases, participants became “stuck” in a virtual space, unable to recall the sequence of actions needed to proceed, which occasionally triggered visible frustration — such as lowering the headset, sighing, or disengaging from the controls. Task complexity also compounded the navigation challenges; when an action required more than two or three sequential steps, participants were more likely to lose track of their objective and abandon the attempt. This correlates with ! 18 existing research on the use of XR controllers and associated accessibility challenges (ref.) At the same time, some participants were visibly intrigued by the simulated industrial environment, particularly when guided by a virtual assistant (NPC). They also remarked that if the tools were simpler and the places to go more obvious, it would have been easier: “ if you have to navigate over larger areas, where you have to teleport or do things like that, it's difficult for some people to realise that, when they try different things, there are slightly different mechanisms at play, and that's what makes it difficult for them.” This indicates that the setting itself was motivating but the interface presented a barrier. Similarly, participants found the rat brain visualization stimulating, but most of them found it easier to explore the app on the tablet compared to the Hololens. A cross-cutting observation from assistants was the importance of predictability and consistency. Participants with cognitive impairments tended to thrive when the environment and task structure remained stable, with cues repeated in the same format and location. Sudden changes in scene or interaction style — for example, shifting from teleportation movement to object manipulation with no transition — could be disorienting and sometimes led to withdrawal from the task. An important aspect when it comes to cognitive impairment group is the individual differences: “Some need constant guidance and repetition, while others understand things right away”. Assistants recommended that future designs incorporate optional “slow mode” features, allowing users to proceed at their own pace without time pressure or the risk of virtual penalties for mistakes. Overall, the evaluation underscored that XR for individuals with cognitive and physical impairments must be approached with careful attention to both safety and design simplicity. When interaction mechanics are intuitive, feedback is immediate, and the environment is forgiving, participants can engage with a sense of joy and agency. Conversely, when complexity, precision, or multi-step memory demands are high, the risk of disengagement — or, in physical terms, unsafe movements — increases. The lessons from this work will inform XR4HUMAN’s inclusive benchmarking framework, ensuring that evaluation criteria capture the nuanced interplay between capability, design, and experience quality for underrepresented user groups. Also, there is a need to develop alternative evaluation forms adjusted for diverse groups such as simplified pictorial surveys. To capture in depth the challenges and the needs of the diverse groups, quantitative evaluations are not always sufficient and often need to be supplied with qualitative ones. Vocational and Career Exploration Evaluations The NTNU evaluation activities in vocational and career exploration contexts were designed to examine how XR experiences can be assessed in a way that is adaptable to varied settings, participant profiles, and levels of environmental ! 19 control. The purpose was not to appraise individual applications in isolation, but to identify the methodological, practical, and experiential factors that an effective evaluation framework must address when applied to XR for vocational awareness, industry outreach, and career guidance. Two contrasting contexts were used: a high-traffic public engagement environment at the Tautdanning career fair, and a structured laboratory session for international students at NTNU. Together, these settings provided insight into how environmental conditions and participant characteristics shape the feasibility and appropriateness of different evaluation measures. Methodology At the career fair, the evaluation was embedded in NTNU’s booth over two full days. The Tautdanning education and career fair in Trondheim is one of Norway’s largest arenas for students exploring study and vocational opportunities. It gathers universities, colleges, companies, and organizations, offering young people direct insights into different career paths and educational programs. The booth contained two VR stations equipped with high-performance laptops, tethered head-mounted displays, and handheld controllers preloaded with the Blue Sector vocational training application. Recruitment was opportunistic: participants were drawn from the steady flow of visitors, most of whom were secondary school students, early-career adults, or educators attending the fair. Totally 24 participants mostly in the age group 14-17, participated in the evaluation. The evaluation process was adapted to the public nature of the event. Onboarding began with a concise verbal introduction from a staff member. Session lengths were capped at approximately 10-20 minutes per user to maximise throughput. Staff monitored each session closely, offering brief instructions when participants hesitated or deviated from the intended activity sequence. Data collection relied on questionnaire (standard XR4HUMAN one with additional questions on accessibility, self-efficacy and career guidance) and semi-structured interview involving 2 participants The second session did not have the same time and space limitations and, allowed for a more systematic evaluation, taking place in the lab and nearby classroom where the students (aged 18 +) received XR4HUMAN presentation prior to the evaluation. This session involved 28 international students, attending ISFIT festival in Trondheim. The XR laboratory and classroom provided a controlled space with adjustable lighting and ample room for standing or seated play. After an initial project briefing, participants rotated between several XR applications, with most spending at least 15 minutes in the Blue Sector environment. The evaluation process in the lab included: 1. Pre-experience briefing explaining the purpose of the activity and the basic controls. ! 20 2. Researcher-led onboarding using both verbal instruction and demonstration inside the headset. 3. Post-experience questionnaire consisting selected questions from the standard XR4HUMAN survey plus additional questions on accessibility, selfefficacy and career guidance. Results The evaluations in both contexts revealed several factors that any XR evaluation in vocational contexts must capture, each illustrated through concrete observations. Based on the interviews conducted at the career fair, students generally found the Blue sector app to be user-friendly, particularly those with prior gaming experience. Both students reported that the use of controllers was intuitive, especially with the help of clear labeling and an introductory tutorial. Virtual guides (NPCs) were seen as helpful in explaining tasks and maintaining flow, although one student believed they could manage without them, albeit more slowly. Text was easy to read in VR, and the addition of text-to-speech was considered a useful feature. One participant suggested that improved graphics could enhance the experience, particularly for extended use. In terms of learning outcomes and vocational exploration, both students felt that VR provided valuable insights into the industry and could complement physical practice. One student noted that they learned more about factory work rather than traditional aquaculture settings, while the other emphasized that VR could increase accessibility and inclusion, especially as technology becomes more affordable and widespread. Elements like directional arrows and interactive guidance contributed to a sense of mastery and engagement. Overall, both students expressed a high level of confidence in using the VR app, with potential for increased self-efficacy and motivation through further experience. Approachability and onboarding emerged as decisive in the career fair environment. Many participants at the booth had no prior VR experience and some required reassurance before agreeing to participate. Staff presence and approachable physical setup reduced hesitation. For evaluation purposes, this suggests that measuring both time-to-engagement and the number of onboarding interventions could serve as robust indicators of accessibility in publicfacing contexts. Both groups (career fair and International Students) answered the Blue Sector questionnaire, including XR4HUMAN questions. Key takeaways: The Blue Sector VR app was evaluated positively by both groups of students. Most respondents agreed that the app was easy to use, with the controls and interface generally perceived as intuitive. Confidence in navigating the environment was also rated highly, suggesting that even participants with limited VR experience were able to engage effectively with the simulation. In terms of learning outcomes, a strong majority reported that the app gave them new insights into aquaculture and fisheries, and many saw clear potential for such tools to complement more traditional forms of vocational training. Motivation and engagement scores were ! 21 also high, with students describing the experience as enjoyable and stimulating for further exploration of the sector. Although the overall pattern of responses was clearly positive, the charts also showed some variation. A minority of students remained neutral or less certain about their level of autonomy in the app, pointing to areas where navigation and task flow may be further refined. Similarly, while most participants endorsed the educational and vocational value of the app, a small proportion were less convinced, highlighting the importance of continued development and testing with diverse user groups. Taken together, the results suggest that the Blue Sector app provides an effective and engaging introduction to fisheries and aquaculture, with strong potential for use in vocational guidance and education, while also leaving room for targeted improvements. Blue Sector has also undergone extensive testing in schools and at career fairs in Shetland, UK, where it demonstrated clear impact and received positive feedback. However, the outcomes of these evaluations fall outside the scope of this deliverable. Implications for Evaluation Frameworks These findings confirm that vocational XR evaluations must be methodologically flexible while maintaining a set of core measures. Factors such as onboarding clarity, navigation success, sensory comfort, accessibility, utility, vocational relevance, and engagement sustainability can be measured across settings, but the instruments and scoring rubrics should be modular. NAV Jobseeker Evaluations The NAV jobseeker evaluations formed a key part of NTNU’s contribution to Task 6.4 by extending XR4HUMAN’s evaluation activities into the context of employment preparation and vocational re-entry. Conducted in collaboration with the Norwegian Labour and Welfare Administration (NAV), these sessions provided an opportunity to examine how XR evaluation frameworks operate when the target audience includes individuals actively seeking work, often with varied educational backgrounds, mental health and social issues, differing digital literacy levels, and diverse motivational profiles. The purpose of these evaluations was not to assess the intrinsic quality of the XR content alone, but to identify which factors in the evaluation process are most relevant when assessing XR as a tool for jobseeker engagement and skills awareness. The NAV context is particularly instructive because it brings together participants, in particular vulnerable youth, who may face structural barriers to employment and for whom accessibility, clarity, and perceived relevance are central to sustained engagement. ! 22 Methodology The evaluations were conducted at a NAV centre equipped with a designated demonstration area that could accommodate small groups. Participants were recruited from active NAV jobseeker lists, with invitations extended to those attending scheduled career support sessions. Recruitment was therefore pragmatic and contextually embedded, meaning participants represented a realistic cross-section of the service user base rather than a self-selected technology-enthusiast population. Some of the participants had mental health issues, including one with autism diagnosis. Evaluation ran in 2 blocks on 1 day, with 5 designated stations with the Blue Sector vocational training experience, to avoid waiting and provide a comfortable and dignified user experience. To facilitate a participant with autism, an individual station was set up in a separate room to avoid sensory overload, where the participant could try the app with the help from NTNU researcher and career counsellor. Headsets and controllers were sanitised between uses. Each individual session lasted approximately 10-20 minutes, balancing the need for a meaningful exploration of the different parts of the app with the need to avoid fatigue and sensory overload. The evaluation protocol comprised: • Pre-session briefing to introduce XR, explain the activity, and clarify safety considerations. • Guided onboarding with a researcher fitting the headset, explaining the controllers, and initiating the experience. • Immediate quantitative survey for participants after the sessions, including XR4HUMAN general questions and questions in accessibility, self-efficacy and career guidance aspects. • Qualitative data were captured through focus group interview with 5 participants, and focus group with 5 counsellors Results The NAV sessions brought to light several key considerations for evaluating XR technologies in jobseeker contexts, combining insights from both qualitative and quantitative data The NAV sessions provided valuable insights into how XR technologies like the Blue Sector VR app function in career guidance for jobseekers. Drawing on qualitative data from focus group interviews with users and counsellors, alongside quantitative feedback collected immediately after user sessions, the app was generally described as engaging and curiosity-provoking. Many users reported that the immersive format helped them visualize work tasks and better understand job roles. Counsellors, meanwhile, observed increased initiative and physical involvement, particularly among participants who had previously been passive or hesitant. However, some inconsistencies between ! 23 qualitative and quantitative findings suggest that user perceptions may shift depending on context, timing, and the type of feedback elicited. A key factor influencing engagement was vocational relevance. While some users felt the experience closely mirrored real work and sparked greater interest in the profession, others struggled to connect the virtual tasks to their own career goals. This variation highlights the need to assess not only perceived realism but also personal applicability—whether users see the activity as relevant to their employment trajectory. Here, the counsellor’s role was essential in helping users interpret the experience and relate it to their individual career paths. Digital confidence varied across both users and counsellors. Although most users had prior exposure to VR, the time and support needed to get started differed significantly. This underscores the importance of evaluating initial confidence and support load, especially when working with individuals who may not identify as digitally adept. Physical discomfort was minimal, though a few users required headset adjustments due to glasses, and one participant shortened their session due to headset weight. These findings suggest that ergonomics—headset fit, controller grip, and seating should be considered in future evaluations. Accessibility and inclusion features played a significant role in shaping the user experience. Tools such as text-to-speech, virtual guides, controller labelling, and high-contrast text were generally well received and often used voluntarily. However, qualitative interviews revealed mixed reactions—some users found these features helpful, while others experienced confusion or sensory overload. This points to the importance of flexible, user-centred design that accommodates diverse needs. Navigation and task comprehension were mostly smooth, supported by a clear tutorial. Still, some users needed verbal prompts, often due to unfamiliarity with the vocational (fishery) setting rather than interface design. Counsellors and assistants were instrumental in helping users overcome these challenges, reinforcing the importance of relational support in XR-based career guidance. Ultimately, while the BlueSector app shows strong potential to spark engagement and support experiential learning, its success depends not only on the technology itself but also on the relational, adaptive, and inclusive frameworks that surround it. Implications for Evaluation Frameworks The NAV jobseeker evaluations show that when XR is deployed in employment support settings, evaluation frameworks must account for a broader spectrum of user readiness, comfort, and motivation than in more homogenous participant groups. Factors such as onboarding support load, ergonomic comfort, navigation confidence, personal applicability, voluntary use of accessibility features, and engagement drivers can all influence both the experience and the data collected. The methodological takeaway is that evaluation instruments must be sensitive enough to capture these dimensions without imposing additional barriers to ! 24 participation, particularly in populations where confidence and trust in digital tools cannot be assumed. In addition, special adjustments need to be considered to participants with mental and physical health challenges that are often overrepresented in this user group. Medical student evaluation The study was conducted at Levanger Hospital on March 11, 2025, with 8 participants aged 18–29 (6 female, 2 male). Most participants used glasses or contact lenses, while none reported hearing impairments. Educational backgrounds ranged from high school to bachelor’s degree. They were presented with updated Nevrolens application and selected questions from the XR4HUMAN survey + accessibility questions. The results are summarized below: The survey results showed that participants generally found the scalable text size and contrast modes useful. Adjusting brightness and font size was reported to reduce strain during longer sessions. Although no severe visual impairments were represented, the flexibility of these features was considered important even for mild vision correction. Speech clarity and the availability of captions or subtitles were highlighted as positive additions. While none of the participants reported hearing loss, captions were noted as helpful in noisy environments or when audio quality was less than ideal, suggesting broader benefits beyond users with diagnosed impairments. Simplified navigation menus and clearer iconography were reported to improve ease of use. Icons were described as easier to distinguish, which helped reduce cognitive load. A centralized accessibility menu was also valued, as it allowed adjustments such as contrast, text scaling, and captions to be toggled quickly without interrupting the main task flow. At the same time, several limitations emerged. Participants often struggled to activate the text-to-speech function, and once running, the audio could not be stopped, which many found frustrating. Model manipulation was another challenge: users sometimes confused moving the entire brain with individual parts, and controls for whole-model adjustments were hidden behind menus. Overall, despite these challenges, the findings suggest that the new accessibility features represent meaningful progress and provide a strong foundation for future improvements. ! 25 5 XR Software (Re)-Development As part of WP6’s iterative approach, two submissions from the Experience Library were supported through targeted refinements to demonstrate alignment with XR4HUMAN’s principles on inclusivity, accessibility, and ethical self-assessment. The intent was not to redesign the applications wholesale, but to surface practical changes that improve evaluability and user experience in line with the methodology set out in Sections 2–4. XR software applications VR4VET Blue Sector and Nevrolens This subsection provides a brief description of the two XR software applications that were refined to demonstrate alignment with XR4HUMAN’s principles. VR4VET Blue Sector The VR4VET Blue Sector VR app 4 , 5 , 6 is being developed for career guidance and vocational training in the Blue Economy sector, with a focus on empowering young job seekers (Fominykh and Prasolova-Førland, 2024) 7 . Figure 1 Screenshot of the Blue Sector VR Experience 4 VR4VET Blue Sector on Meta App store, https://www.meta.com/engb/experiences/vr4vet-blue-sector/7304270832929125/ 5 The development of the VR4VET Blue Sector app was supported by the Virtual Reality for Vocational Education and Training - VR4VET project, partially funded by the European Union’s Erasmus Plus program, grant number 2021-1-NO01-KA220-VET-000028033. 6 The development of the VR4VET Blue Sector app was supported by UHI Shetland and the Department for Environment, Food, and Rural Affairs in respect of a project titled UK Seafood Fund – Skills and Training Pillar (Grant) Round 1. 7 Mikhail Fominykh and Ekaterina Prasolova-Førland (2024): Supporting Career Guidance and Vocational Training in Fishery and Maritime Professions with Virtual Reality. In the 10th International Conference of the Immersive Learning Research Network, Glasgow, UK. DOI: 10.56198/5M1RH0QNF. ! 32 Inclusion and Accessibility refinements in the Nevrolens app The Nevrolens app, developed in collaboration with Nobel Prize winning Kavli Institute for Systems Neuroscience has shown the potential to transform peer learning of neuroanatomy among neuroscience and medical students. The third use case of refinements of the Nevrolens app was originally planned along the following principles and practices of the XR4Human Code of Conduct: • Social inclusion and accessibility: access to neuroscience education everywhere and for everyone through multi-platform app • Physical health (of people and animals): a part of the motivation behind the development of the app was originally to minimize COVID-19 exposure (remote learning with AR app) and reduce use of rat cadavers for dissection Features proposed for the development in accordance with XR4Human code of conduct: • Implementation of general improvements of the UX of the app based on the completed designs. • Accessibility and user agency: Automatic sound narration all appropriate written text information, using text-to-speech, configurable in the app settings. • Accessibility and user agency: An additional UI layout for visual impaired users with higher contrast and large font size, configurable in the app settings. The results of the refinement of the Nevrolens app are illustrated in a video 12 and include the following: • User Interface re-design: First, most of the User Interface has been redesigned for screen-space, evaluated with end users, and further improved based on the feedback. The user interface is high contrast with large fonts used for text elements, following the accessibility standards and best practices. The example screenshots below show a new curtain screen “Session” for the selection of brain models (on the left) and a new system for the whole brain manipulation interaction (round buttons on the right side of the screen of the rights screenshot). 12 Accessibility and Inclusion features of the Nevrolens app https://youtu.be/RrtD_yg79ro?si=5X_yeYMMfzKWAXuM ! 33 Figure 10 Illustration of the User Interface Re-Design • Tutorial: An integrated and optional tutorial provides the user with the necessary guidance to comfortably use the application, covering the most common misconceptions and needs in use discovered in user testing. The tutorial includes (a) the Understanding of the Augmented Reality space (see screenshot below on the left), (b) the manipulation controls for the entire brain model, (c) manipulating the individual sections of the brain model, and (d) accessing the information about these sections, including the text-tospeech feature (see screenshot below on the right). Figure 11 NevroLens Tutorial ! 34 • Text-to-Speech: Information about the different parts and sections of the brain is available as text. Interacting with a specific part of the brain in the AR space will display its name in the bottom panel of the app, which then can be extended to get a more detailed description (see screenshot below, on the left). Furthermore, by tapping on the Text-to-Speech button (left side of the screen on both screenshots below), the user can listen to the same description. The speech can be stopped at any time by tapping on the same Text-to-Speech button again. When working with both rat and human brain models placed in the same AR space, the user can interact with their parts and sections at the same time and in the same way (see screenshot below, on the right). Figure 12 Implementation of Text to Speech Conclusions of the software refinement Both submissions are now presented within the Experience Library as examples of responsible and ethically aligned XR practice. While the software refinements addressed immediate evaluation needs and demonstrated practical implementation of XR4HUMAN Code of Conduct principles, ensuring the long-term impact of these improvements requires a sustainable framework for the Experience Library. The next section outlines how the evaluation framework and refined applications will be maintained and expanded beyond the project's lifetime to continue supporting responsible XR development. ! 35 ! 36 6 Sustainability plan for the Experience Library The Experience Library will be sustained and further expanded beyond the lifetime of the project through integration with the XR4Europe web platform. In its future form, the library will be fully replicated on the XR4Europe website, retaining its current branding to preserve identity and recognition, while adopting a potentially refreshed visual style to align with XR4Europe’s design framework. Functionality will remain consistent with the current implementation to ensure continuity for existing users. A direct link will be maintained from the XR4HUMAN website to the XR4Europe-hosted version, enabling seamless navigation between the two platforms. The migration and integration process is planned for completion after the conclusion of the funded project period, ensuring that the necessary development, quality assurance, and stakeholder consultations are undertaken without disrupting ongoing project activities. As part of this transition, the underlying data collection workflow will be adapted to move away from the current reliance on Qualtrics. The project team has identified a lightweight, flexible alternative in Tally. So, which offers simplified survey deployment, easier integration with the library, and lower long-term running costs. XR4Europe is already familiar with using Tally, having utilised it for more than a year for survey research and open calls, ensuring a smooth transition to this platform for future Experience Library submissions. To maintain the integrity and value of the Experience Library after the project ends, a volunteer-based pool of evaluators will be established. This group will draw on the XR4HUMAN network as well as external collaborators from XR4Europe and other allied initiatives. The evaluator pool will operate under the coordination of Professor Oliver Schreer, co-lead of WP6 and a member of the XR4Europe Board of Directors, who will provide oversight on evaluation quality, methodological consistency, and recruitment of additional contributors. The application evaluation process will build on the ethical self-assessments created by the XR4HUMAN project. Evaluations will not be a static continuation of the process trialled and established during XR4HUMAN but will continue to evolve and to integrate the project’s final outputs, namely the XR Code of Conduct, the Interoperability Guide, and the ethical selfassessment checklist. Additionally, the scope of the Experience Library will undergo annual review in the year after the XR4HUMAN project to ensure it aligns with emerging trends and needs in the marketplace. Reviews may lead to modifications such as in the scope of projects considered and featured, the requirements for application, and the variety of communication and promotional channels used to support the selected projects. Annual reviews will be conducted online and will involve persons drawn from XR4HUMAN partners, the pool of expert application evaluators, and XR4Europe's management team. ! 37 Selected applications will remain in the Experience Library so long as they confirm the accuracy of their submissions. This helps to prevent the library becoming a repository of abandoned projects in the years to come. At the conclusion of the XR4HUMAN project, annual confirmation emails will be sent to the contacts registered for each selected submission. Subject to the Experience Library's annual review, confirmation will simply require explicit confirmation from submitters that their previously submitted information remains correct and accurate. Conclusions Task 6.4 set out to evaluate example use cases in ways that are technically informative, ethically grounded, and inclusive of user diversity. The work delivered on three fronts. First, it produced a literature-derived, expert-refined, modular post-experience survey that can travel across domains and populations while retaining construct comparability. Second, it demonstrated through deployments in public, laboratory, employment, accessibility, and clinical settings how evaluation must adapt to context without losing coherence. The emphasis throughout was on factors that matter for evaluation—onboarding, orientation and navigation, sensory and cognitive load, accessibility, vocational relevance, and engagement sustainability—rather than on product-level verdicts. Third, it established a sustainability pathway for the Experience Library, including integration with XR4Europe, adoption of a lighter data-collection platform, and a governance model for ongoing evaluation. The Experience Library addresses gaps that existing catalogues and repositories rarely cover. It requires contributors to evidence accessibility, safety and ethical practice using a common instrument and self-assessment based on the XR4HUMAN Code of Conduct. Entries are described and, where feasible, evaluated against shared constructs, allowing users and practitioners to compare like with like across age groups, abilities and contexts. This gives practical value for organisations that serve vulnerable or under-represented users, providing clear indicators of onboarding demands, sensory load, navigation complexity, and available accessibility features before adoption decisions are made. Beyond collation, the Library curates examples that have been iteratively improved for inclusion and documents how those changes affected usability and engagement. This transforms it from a static repository into a learning resource, where inclusive design decisions and their user impact are transparent. Its governance model further enhances this value: the volunteer evaluator pool, annual accuracy checks, and integration with XR4Europe processes maintain quality and avoid drift towards outdated or promotional content. Migration to a lightweight, low-cost platform and multilingual submission support also reduce entry barriers for smaller teams and community organisations, extending representation beyond well-resourced developers. ! 38 Together these outputs provide XR4HUMAN and its successors with a reusable framework for measuring user experience in XR that supports responsible innovation. The framework is sufficiently standardised for pooled analysis and benchmarking, yet flexible enough to respect differences of age, ability, setting and purpose. As XR adoption widens, a maintained Experience Library and living evaluation toolkit will enable the community to identify good practice quickly, surface risks early, and ensure the lived experience of diverse users remains at the centre of design and deployment. ! 39 Annex 1: Post-XR Experience Final Survey (Online Version) ! Start!of!Block:!Opening! Thank you for participating! Your feedback is valuable to us and for helping to improve the development of XR technology. Please take a moment to fill out this short survey. If you are unsure about the terminology used in any of these questions, please ask your demonstrator for guidance. This form was developed through European Union’s Horizon Research and Innovation programme under grant agreement no. 101070155 and the UKRI through the Horizon Europe Guarantee (#10039307). End!of!Block:!Opening!! Start!of!Block:!Demographics! Age o Aged 10 or under o 11-17 o 18-29 o 30-39 o 40-49 o 50-59 o 60-69 o 70 or older ! ! 40 Sex o Male o Female o Non-Binary o Prefer not to say ! Highest level of education o Less than high school o High school graduate o Some college or vocational training o Bachelor's degree or higher o Prefer not to say ! ! 41 Visual Exceptionalities (please tick all that apply) ▢ No known visual impairments ▢ Glasses or contact lenses for distance ▢ Glasses or contact lenses for reading ▢ Color vision deficiency (color blindness) ▢ Low vision ▢ Partially sighted ▢ Blind in one eye ▢ Fully blind ▢ Other __________________________________________________ ▢ Prefer not to say ! Previous experience with XR o Never tried before o Tried once or twice o Occasionally use (a few times a year) o Regular use (a few times a month) o Frequent use (weekly or more) o Expert or professional use !