Enhancing Healthcare Remote Education with 6G and XR Technologies

Enhancing Healthcare Remote Education with 6G and XR Technologies Mohammad Rajiullah, Giuseppe Caso, Anna Brunstrom, Karl-Johan Grinnemo, Jonas Karlsson Department of Mathematics and Computer Science Karlstad University Karlstad, Sweden [email protected] Anna Nordin, J¨ orgen Jansson, Anders Sidenblad Department of Health Science Karlstad University Karlstad, Sweden [email protected] Abstract—As the adoption of Fifth Generation (5G) systems increases, efforts towards Sixth Generation (6G) systems have already started across research, standardization, and stakeholder fora. 6G is expected to support applications with immersive capabilities, with specific use case requirements from different verticals playing a critical role in solution development. Unlike current solutions in the education vertical that uses immersive technologies such as Augmented/Virtual/eXtended Reality (AR/VR/XR), which rely on pre-recorded content and lack engagement, 6G can enhance remote education by enabling real-time, AR/VR/XR-enriched interactions among students and instructors. This paper presents ongoing activities within the 6G-PATH EU project, towards the design, implementation, and testing of a 6G use case for healthcare personnel remote education/training, which aims to facilitate real-time, AR/VR/XRenhanced interactions among healthcare trainees and instructors. Index Terms—5G, B5G, 6G, AR/VR/XR, remote education, nursing, KPI, KVI I. INTRODUCTION Fifth Generation (5G) systems, e.g., the 5G New Radio (NR) standardized by the Third Generation Partnership Project (3GPP), have been commercially available worldwide for at least a couple of years, aiming to address the Quality of Service (QoS) and Quality of Experience (QoE) requirements of three main vertical categories, referred to as enhanced Mobile Broadband (eMBB), Ultra-Reliable Low Latency Communication (URLLC), and massive Machine-Type Communication (mMTC). While the usage of 5G is set to increase in the following years, work towards Sixth Generation (6G) systems has already started across research, standardization, and stakeholder fora. As a starting point, the International Telecommunication Union Radiocommunication Sector (ITUR) has issued Recommendation M.2160 in 2023, which sets the basis for developing 6G systems and technologies, i.e., International Mobile Telecommunications 2030 (IMT-2030), and also includes a description of envisioned capabilities and usage scenarios. In particular, the extension of eMBB, URLLC, and mMTC categories is expected towards immersive, hyperreliable, low-latency, and massive communication. At the same time, new use cases are also defined for integrated sensing and communication, ubiquitous communication, and the synergy between Artificial Intelligence (AI) and communication [1]. Several organizations, such as the Next Generation Mobile Networks Alliance (NGMN), the Global System for Mobile Communications Association (GSMA), and the 6G Smart Networks and Services Industry Association (6G-IA), are working on developing the 6G vision. They focus on creating systems based on specific use case requirements rather than pure technological advancements. Therefore, in addition to network Key Performance Indicators (KPIs), there is now an emphasis on vertical-specific Key Value Indicators (KVIs), to facilitate the inter-working between networking experts and verticals in the 6G design, and quantify the societal benefits derived by the technological advancements [2]. One area where 6G could significantly impact is remote education, which is a clear use case at the intersection of the immersive and low-latency communication scenarios envisioned by ITU-R. As further detailed in Section II, different levels of immersiveness and interaction are being included in current solutions for remote education via Augmented/Virtual Reality (AR/VR) and, more in general, eXtended Reality (XR) technologies; the majority of such solutions adopt, however, pre-set/recorded contents, scenarios, and feedback, hence reducing engagement and information retention. 6G has the potential to enhance such remote education environments by enabling real-time, XR-enriched interaction among students and instructors, ultimately boosting learning quality. Among a broad range of educational scenarios, healthcare personnel remote education/training is paramount, as also highlighted by ongoing research activities, considering that real-time and interactive demonstrations are essential for achieving effective educational experiences for healthcare trainees via personalized and realistic training sessions. Within the above context, this paper presents our ongoing activities in the framework of the 6G-PATH EU project1, towards the design, implementation, and testing of a 6G use case for prehospital nursing remote education/training. Our main goal is to extend current solutions for remote education by (i) designing and implementing an end-to-end solution enabling real-time, AR/VR-enriched remote interactions among nursing students and instructors and (ii) testing such solutions over Beyond-5G (B5G) and 6G systems. To do so, we adopt a use case-driven system design approach and initiate a collaborative effort between networking and health education experts at Karlstad University (KAU), ultimately deriving a multifaceted yet comprehensive use case description in terms of scenarios, key components, functional/performance requirements, mapping over B5G/6G systems, and KPIs/KVIs to consider. We plan to leverage the work done so far in the subsequent phases, which include the use case implementation and experimental testing over B5G/6G systems. We believe that our work can be further leveraged and extended by other researchers facing similar challenges in defining and evaluating 6G use cases towards establishing systematic procedures that would benefit the entire ecosystem growing around 6G systems and use cases. The paper is organized as follows. Section II summarizes the related work, while Section III introduces the considered use case, the requirements of which are described in Section IV. Section V discusses how such requirements can be supported by B5G/6G technologies, and Section VI concludes the paper. II. RELATED WORK Immersive remote education holds significant importance in transforming traditional learning approaches into dynamic, engaging, and effective educational experiences. It fosters realworld context, improves information retention, and benefits complex concept demonstration. By enabling personalized, collaborative, and skill-based training, it benefits learners with various skills while bridging geographical barriers through high-quality remote learning. While purely virtual educational environments or telemeeting-like remote education are already available, the potential of AR/VR-based solutions enabling a truly immersive remote educational experience has yet to be fully explored, particularly in combination with B5G/6G networks and edge/cloud computing infrastructures. Both are key towards enabling immersive remote education, as they aim at handling the high traffic demands and strict requirements of such services in terms of QoS/QoE KPIs and KVIs. AR/VR technologies are being explored in several educational scenarios, as they potentially enable immersive experiences that transform traditional pedagogy. Architecture enthusiasts use tools like Autodesk’s Revit2for immersive design exploration. Aspiring astronauts can traverse the cosmos with Titans of Space3, while TheBlu4and Calcflow5help learners 1https://6gpath.eu 2https://www.autodesk.com 3https://learnvr.org/portfolio/titans-of-space-2/ 4https://wevr.com/theblu 5https://tinyurl.com/4d4nbrzj in marine biology and complex math, respectively. Immersive technologies have also found significant applications in medical education. The work in [3], [4] delved into the use of AR/VR in healthcare education, focusing on offline, pre-recorded virtual learning environments. Platforms like Osso VR6allow medical students to simulate surgeries, while Cassette7and Holopatient8offer AR/VR training across diverse scenarios, from emergency situations to surgical roleplaying. Within academia, Case Western Reserve University leveraged Microsoft HoloLens to offer medical students intricate 3D anatomical insights, eliminating the need for cadavers [5]. Moreover, zappar9uses AR for diverse educational pursuits. Yet, a barrier to the ubiquitous adoption of such technologies is the considerable resource requirements. This can render them financially impractical for many educational institutions. In this context, 360◦videos have also emerged as a solution for enabling immersive education, combining rich experiences with cost-efficiency [6]. The work in [7] suggests that such videos are more suitable for disciplines requiring observation and practical applications in medical training and social care. The majority of current AR/VR solutions adopt prerecorded content. This leads to our use case, which aims at integrating and testing real-time, non-prerecorded, immersive remote education services over B5G/6G networks. It is worth mentioning that the EU project 5G Heart10 preliminary explored the potential of live remote interactions for medical training, but it did not incorporate immersive technologies. III. USE CASE In this section, we first provide a description of the scenario used in our use case. We then outline the key technological components and explain how they interact to create the necessary platform for the use case. A. Scenarios Description Our primary focus is leveraging 6G and XR technologies to deliver comprehensive training to prehospital nurses and other emergency responders. Prehospital nurses must possess a combination of cognitive and practical skills to effectively manage a wide range of medical emergencies, where experience-based knowledge is a central component of their competence [8]. This makes practical training in both simulated scenarios with manikins and under supervised participation in clinical prehospital care an essential part of their education. We aim to enhance this educational practice and explore the potential of 6G and XR in simulating situations using medical manikins and replicating real-life emergency experiences. Our use case involves outfitting prehospital nurses with video cameras and audio equipment within the ambulance. 6https://www.ossovr.com 7https://www.cassettegroup.com/products 8https://www.gigxr.com/holopatient/ 9https://www.zappar.com 10https://5gheart.org These devices will record (simulated) emergencies, transmitting the footage to a server in the ambulance. Simultaneously, the nurses’ microphones will capture their voices and ambient sounds, streaming to the ambulance server. In simulated scenarios, the manikin will generate sensor readings. In contrast, in real scenarios, sensors will be placed on the patient to collect medical data, all streamed to the server. An application server receives all the streams and performs any needed pre-processing steps, such as: •Scene creation: The different video streams are merged to form a coherent scene representing the original 3D scene. The audio streams are integrated into the scene to match the sounds in the original scene. The scene may also include sensor readings, with labels indicating the measured person or position and their respective values. •AI-based analysis: AI tools that can search the scene may be further utilized to guide learning within the classroom. The AI, for instance, could perform information extraction and augmentation, as well as offer assessment and prediction of the patient’s status by analyzing the collected data. The reconstructed scene is streamed to students in a remote classroom using audio-video peripherals, such as monitors or head-mounted displays (HMDs). In the simulated scenario, audio communication channels and manikin control are established, allowing communication in both directions. While bi-directional audio communication support is also expected for training in real medical scenarios, communication with the first responders may need to be restricted to avoid intrusion. B. Key Components Integrating B5G/6G systems, edge/cloud computing, and ML/AI is crucial for delivering high-quality, immersive, realtime healthcare education. To ensure a high QoS/QoE for the students, we envision XR healthcare education scenarios that require low-latency transmission of multimodal data flows, some of which have high data rates, for example, 360-degree video. Before being accessed by the trainees, the multimodal inputs undergo efficient processing. ML/AI can offer additional functionalities, including information extraction and augmentation and assessment/prediction of the patient’s status by analyzing the collected data. The use case consists of two locations. The first location, the medical site, is where the healthcare scenario for a specific training session takes place, for example, an ambulance where first responders operate in an emergency case. The other location is the training site, where the students receive training. A key feature of this use case is that the trainees experience the remote events at the medical site in a high-quality, immersive, and real-time manner. Therefore, the training site is where the composite real-time data flow is consumed after being augmented with additional historical information and pedagogical references retrieved from dedicated databases. The two end locations described earlier will have functionality for capture and rendering, respectively. The communication network and computing platform are anticipated to be highly distributed and flexibly implemented along the path. We anticipate the following main components: a) Multimodal capture and processing: That is hardware and software for collecting and processing various inputs to synthesize the scenario at the medical site realistically. This includes cameras for high-quality video, microphones for audio input, speakers for audio output (e.g., for bidirectional communication with the training site), and heterogeneous sensors for collecting body/environment data such as temperature, oxygen level, blood pressure, heart rate, and humidity. b) Manikin: The manikin is only used in the simulation scenario. It creates the sensor values mentioned above. c) Rendering devices: The training site has hardware and software that consume the XR flow, such as audio-video peripherals like monitors or HMDs. d) Computing (and storage) node(s): These nodes form an end-to-end distributed computing platform that provides the functionalities for generating the XR content by integrating the multimodal inputs collected at the medical site and additional non-real-time information. Different nodes may execute different functionalities towards content generation, such as data synchronization, encoding, and augmentation. Depending on each functionality’s computing and latency demands, nodes are deployed over the edge-cloud continuum at different locations. e) App servers: The App servers provide access to the XR content for end users. One or several app servers may be deployed at different locations based on the identified functional requirements (e.g., latency towards the end users). f) Communication nodes: These form the end-to-end communication network between medical and training sites. An end-to-end B5G/6G system will be deployed to meet stringent QoS/QoE requirements. The system includes end devices at both locations connecting to a hybrid radio access network (RAN) and the core network (CN), which provides the control plane and is possibly deployed across different nodes. The edge-cloud computing platform is instantiated on top of the communication network so that the functionalities are distributed and flexible. As described above, our use case comprises a medical site, a training site, communication networks, and computing platforms, as also shown on a high level in Figure 1. IV. USE CASE REQUIREMENTS AND EVALUATION METRICS The following section first outlines the important requirements for deploying and operating our use case, followed by a description of the evaluation method for our use case. A. Functional Requirements The functional requirements (FR) outline the necessary capabilities and functionalities that must be supported to realize the use case. Table I outlines the FRs for the use case, detailing the necessary functionalities for effective implementation. AR/VR glass Training site Medical site 5G Core 5G gNodeB Camera head band 360 camera 5G modem/WIFI Sensors to read vitals Edge node (MEC) Database server App server UPF Edge node (MEC) Local processing server AI/ML Fig. 1. A high-level view of the proposed use case. TABLE I USE CASE FUNCTIONAL REQUIREMENTS. FR Title Description FR1 Audio/Video Capture Capturing video and audio streams at the medical site. FR2 Data Capture Capturing medical and other sensor data at the medical site. FR3 Manikin Control Remote control of the manikin (only for simulated scenarios). FR4 Communication Communication of multimodal data between the medical site and the training site. FR5 Edged Deployment Edged deployment with data and AI processing for the creation of XR content. FR6 XR Environment XR environment for students engaged in immersive learning. B. Key Performance Indicators An important motivation for working with vertical use cases as part of the development of 6G is to understand the performance requirements that various use cases put on the network and how network performance influences user-perceived value. To improve our understanding of the requirements for immersive healthcare education, we intend to evaluate our use case by examining its correlation with network KPIs. This will provide important insights for refining 6G requirements within the context of immersive experiences. The specific KPI requirements for our use case are detailed in Table II. C. Key Value Indicators The development of 6G is guided by so-called KVIs to ensure that it not only meets technical and user requirements but also addresses societal challenges. The developed use case aims to enhance the competence and sense of authenticity of students in the specialist prehospital nursing program. By incorporating immersive technologies and 5G/6G, simulated training scenarios will gain increased flexibility and realism, addressing key issues in current practices. We will conduct interviews with participating students to better understand the key values for the use case. Two different scales have been selected to assess KVIs for our use case scenarios: the System Usability Scale (SUS) and the Paramedic Global Rating Scale (PGRS). These scales are designed for specific purposes. SUS is recommended for measuring the perceived usability of technology and has previously been used in research on digital technology in healthcare interventions [11]. PGRS is used to evaluate seven dimensions of clinical competence in prehospital nursing [12] and can be used to assess learning in education and training. The KVIs for the use case are outlined in Table III. V. MAPPING USE CASE REQUIREMENTS TO 6G The goal of our use case is to enhance authenticity, realism, and learning in prehospital nursing education through the use of advanced digital technologies. In this section, we discuss how the functionalities and requirements of the use case relate to B5G and emerging 6G technologies. Realizing our use case involves several components and traffic flows. In the manikin scenario, the manikin is placed at the simulated medical site along with cameras to collect high-quality video and microphones/speakers for audio input/output. The manikin’s control takes place at a remote site. Several multi-modal traffic flows hence need to be transferred from the medical site over the B5G network, taking into consideration the KPI requirements listed in Section IV-B. Here we can take advantage of the slicing capabilities available in B5G networks. For an initial realization of the use case in our testbed at Karlstad University, we plan to send the manikin control and audio over a high-priority slice, and the video flows over the default slice. To meet the latency requirements of the use case, upcoming B5G latency enhancements, such as radio enhancements for XR services, will also be important. For the initial realization of the use case, we will explore the use of L4S (Low Latency Low Loss Scalable Throughput) [13] to control latency, where radio access network support for L4S is part of the 3GPP Release 18 5G-Advanced standard. The definition of L4S-capable QoS flows underpins the smooth functioning and expansion of time-sensitive high-rate applications on 5G networks. The integration of Non-Terrestrial Networks (NTNs) in B5G systems will be beneficial for virtual participation in real-life emergency situations. Integrated TN-NTN infrastructures can enhance coverage and reliability, ultimately enabling virtual participation in training situations occurring in remote areas or in other situations with limited coverage. Computing nodes that generate the XR content and integrate the multimodal inputs collected at the medical site with any further non-real-time information can take advantage TABLE II USE CASE KPIS. KPI name Description Metric Uplink capacity High uplink capacity for upload of video and other types of streams. >20 Mbps Packet loss rate Low packet-loss to avoid interference in real-time streaming. <= 1% (frame loss rate) [9] Latency Low latency to provide an interactive immersive experience and allow control of the manikin. <20 ms, extended up to 50 ms if object detection is done at the edge and head movement is tracked at the HMDs [9]. 30–50 ms for the manikin feedback [10]. TABLE III USE CASE KVIS. KVI name Description Metric Increased realism Fraction of students that feel the solutions contribute to increased training realism PGRS (≥3), SUS (≥68) Increased engagement Fraction of students that feel the solutions improved their engagement in the training PGRS (≥3), SUS (≥68) Technology acceptance Fraction of students that feel 5G/6G and XR brings a positive benefit for the training SUS (≥68) Increased knowledge Fractions of students that will see a more comprehensive image of the training process PGRS (≥3) Flexibility Fraction of students that will be able to access personalized environments to train according to their necessities SUS (≥68) of emerging edge deployments to reduce latency and limit backhaul traffic demands. A main challenge in realizing the use case will be integrating all the needed hardware and software components while ensuring the required KPIs not only for individual components but also end-to-end at the system level. VI. CONCLUSION Most of the current use of immersive learning for healthcare revolves around offline virtual learning environments. 6G, however, has the potential to enable real-time, XR-enriched interaction among students and instructors. To this end, we introduce an immersive remote healthcare education use case with the aim to enhance both flexibility and learning in education. We present the scenario and key components of the use case, its requirements and associated KPIs and KVIs, and how the use case can be supported by B5G/6G technologies. We are working on first realising the use case within our B5G testbed. While we focus on prehospital nursing education, we hope that the use case design and identified requirements can inspire, help guide, and develop a broader set of 6G remote education use cases and related 6G use cases involving remote immersive interaction. ACKNOWLEDGEMENT This work is funded by the EU H2020 research and innovation programme under grant agreement No. 101139172 (6GPATH). REFERENCES [1] “ITU-R Recommendation M.2160: Framework and overall objectives of the future development of IMT for 2030 and beyond.” https://www.itu.int/rec/R-REC-M.2160/en, 2023. [Accessed 13-062024]. [2] “6G Infrastructure Association Vision and Societal Challenges Working Group Societal Needs and Value Creation Sub-Group.” https://5gppp.eu/wp-content/uploads/2022/05/What-societal-values-will-6Gaddress-White-Paper-v1.0-final.pdf, 2022. [Accessed 13-06-2024]. [3] D. King, S. Tee, L. Falconer, C. Angell, D. Holley, and A. Mills, “Virtual health education: Scaling practice to transform student learning,” Nurse Education Today, vol. 71, pp. 7–9, 2018. [4] O. L. Ch´ avez, L.-F. Rodr´ ıguez, and J. O. Gutierrez-Garcia, “A comparative case study of 2d, 3d and immersive-virtual-reality applications for healthcare education,” International journal of medical informatics, vol. 141, p. 104226, 2020. [5] “Case Western Reserve, Cleveland Clinic Collaborate with Microsoft on Earth-Shattering Mixed-Reality Technology for Education — case.edu.” https://case.edu/hololens/. [Accessed 13-06-2024]. [6] L. Jensen and F. Konradsen, “A review of the use of virtual reality headmounted displays in education and training,” Education and Information Technologies, vol. 23, pp. 1515–1529, 2018. [7] M. Ranieri, D. Luzzi, S. Cuomo, and I. Bruni, “If and how do 360◦ videos fit into education settings? results from a scoping review of empirical research,” Journal of Computer Assisted Learning, vol. 38, no. 5, pp. 1199–1219, 2022. [8] J. Wihlborg, G. Edgren, A. Johansson, and B. Sivberg, “Reflective and collaborative skills enhances ambulance nurses’ competence – a study based on qualitative analysis of professional experiences,” International Emergency Nursing, vol. 32, pp. 20–27, 2017. [9] “XR and 5G: Extended reality at scale with 5G networks — ericsson.com.” https://www.ericsson.com/en/reports-andpapers/ericsson-technology-review/articles/xr-and-5g-extended-realityat-scale-with-time-critical-communication. [Accessed 13-06-2024]. [10] A. B. Weddle, H. Yu, and M. Moreno, “The Impact of Haptic Latency on User Experience in Automotive Touchscreens.” https://tinyurl.com/2kwcmztz, 2013. Accessed: 2024-06-12. [11] J. C. S. Hvidt, L. F. Christensen, C. Sibbersen, S. Helweg-Jørgensen, J. P. Hansen, and M. B. Lichtenstein, “Translation and validation of the system usability scale in a danish mental health setting using digital technologies in treatment interventions,” International Journal of Human–Computer Interaction, vol. 36, no. 8, pp. 709–716, 2020. [12] A. Bremer, M. A. Hagiwara, W. Tavares, H. Paakkonen, P. Nystr¨ om, and H. Andersson, “Translation and further validation of a global rating scale for the assessment of clinical competence in prehospital emergency care,” Nurse Education in Practice, vol. 47, p. 102841, 2020. [13] B. Briscoe, K. D. Schepper, M. Bagnulo, and G. White, “Low Latency, Low Loss, and Scalable Throughput (L4S) Internet Service: Architecture.” RFC 9330, Jan. 2023.