Cooperative Augmented Reality: Displaying Occluded Vehicles using V2X Communications

Cooperative Augmented Reality: Displaying Occluded Vehicles using V2X Communications Andr´ e Cl´ erigo∗†, Gonc¸alo Silva∗†, Maximilian Schrapel‡, Pedro Rito∗†, Susana Sargento∗†, Alexey Vinel‡§ ∗Departamento de Electr´ onica, Telecomunicac¸ ˜ oes e Inform´ atica, Universidade de Aveiro, Portugal †Instituto de Telecomunicac¸ ˜ oes, Universidade de Aveiro, Campus Universit´ ario de Santiago, Aveiro, Portugal ‡Karlsruhe Institute of Technology (KIT), Germany §Halmstad University, Sweden Abstract—As urban mobility increasingly integrates micromobility solutions such as bicycles, innovative road safety solutions have become a priority for these Vulnerable Road Users (VRUs). This work explores the use of Augmented Reality (AR) and Vehicle-to-Everything (V2X) communications to enhance cyclist safety at intersections with obstructed visibility. We propose an AR-based system that enables cyclists to visualize occluded vehicles using an ‘X-ray’ vision effect, leveraging real-time V2X messages and edge computing for low-latency interaction. The system architecture integrates multiple communication technologies, including 5G and ITS-G5, ensuring reliable transmission of road user data. To evaluate the system’s feasibility, we conducted a real-world demonstration in the Aveiro Tech City Living Lab (ATCLL) platform using a Microsoft HoloLens 2 AR headset and an NVIDIA Jetson-based object detection pipeline. The system was tested in a real environment and results show that: (1) response time for total system latency falls within the 300 ms safety threshold defined by ETSI (which assures system safety); (2) the system operates on 3.75 FPS; and (3) increasing the video cameras’s frame rate does not significantly affect resource and power consumption. Index Terms—Vehicle-To-Everything, Augmented Reality, Vulnerable Road User, Cyclist Safety, See Through Vision. I. INTRODUCTION An increase in public awareness of climate change and green initiatives implemented by cities around the world is transforming people’s mobility. The proportion of cyclists in road traffic is increasing, reaching 41% of the population in the Netherlands, 21% in Sweden, and 15% in Germany [1]. This increase in cyclists places new requirements on traffic safety measures. According to the World Health Organization [2], in 2023, 1.19 million people died in road accidents, amongst those, more than half were pedestrians, cyclists and motorcyclists. The European Transport Safety Council (ETSC) reported that road fatalities decreased by 16% between 2013 and 2023 [3]; however, the number of cyclist fatalities remains constant, in contrast to the number of fatalities with other modes of transport [4]. The majority of these (58%) occur in urban environments, while all fatalities occur mainly in rural environments (53%). One of the most dangerous situations for cyclists is crossing scenarios, specifically, uncontrolled intersections in occluded urban environments [5]. Augmented Reality (AR) is seen as a solution to improve safety. Its high proximity with the user changes perception and interaction with the real world, particularly within urban environments [6] [7]. Another promising technology for road safety is the use of communications for Intelligent Transportation Systems (ITSs) [8]. ITSs allow connected road users to communicate with each other and smart city infrastructure, creating systems for collision avoidance and road navigation. Existing cyclist safety systems that use AR face tradeoffs in both the integration with real-world traffic and realtime performance. Prior research in collision warnings and blind-spot visualization has demonstrated that these systems enhance road users’ situational awareness. However, these are usually evaluated in controlled environments, such as simulators, which limits their ecological validity. In this work, we propose the use of AR and cooperative intelligent transportation systems, integrated into smart cities [9] to develop a system aimed at enhancing Vulnerable Road User (VRU) safety, specifically for cyclists. Figure 1 illustrates a scenario involving road users, their communications, and the User Interface (UI) presented to the VRU via an AR headset to assist with navigation through traffic. This use case addresses situations where cyclists receive assistance at intersections with limited Line of Sight (LoS), enabling the visualization of occluded vehicles through an ‘X-ray’-like view in the AR headset. This system leverages Vehicle to Everything (V2X) communications for real-time interaction, and a computational device that processes the AR headset’s video camera to intelligently control the ‘X-ray’ system based on LoS. This work expands the existing literature by combining insights from previous work regarding the use of AR to aid pedestrians navigate through traffic [7], and using a communication strategy for low-latency services [10] that encompasses multiple communication technologies, such as 5G and ITS-G5. The proposed system is tested in a real scenario, displaying occluded vehicles in real-time to the cyclist. Results also show that the representation of the ‘Xray’ vehicle is done under the 300 ms threshold set by ETSI for road safety applications, and that increasing the frame rate of the stream does not impact the energy expenditure of the real-world deployment. This paper is organized as follows. Section II presents the related work in cyclist safety. Section III discusses the proposed system architecture for road users’ interaction. Section Fig. 1. Visual representation of the use case. A cyclist wearing an AR headset interacting with traffic through vehicular messages. IV demonstrates how occluded vehicles are displayed in AR, and Section V reveals how the system determines when to trigger the ‘X-ray’. Section VI presents and discusses the system performance. Finally, Section VII concludes this work and outlines future work. II. RELATED WORK Cyclist safety has been heavily studied; however, there has not been much work in assistance systems with AR and V2X communications. This section examines existing work through the prism of conventional and novel road safety systems. Technological advances have significantly improved the safety of bicycles. Modern bicycles and helmets can be equipped with multimodal systems that enhance the cyclist’s situational awareness. The popularity of e-bikes, recognized as a sustainable mode of transportation [11], has put computational power and sensors that allow for safety and navigation systems [12], otherwise not possible in conventional bicycles. Current e-bikes integrate real-time tracking and communications, making our proposed system feasible for a market-ready implementation. Previous literature has shown the effectiveness of assisting cyclists in traffic navigation [13] and collision avoidance [14] using vibrotactile and audiovisual feedback. Cyclist warning systems are prominent in both scientific research and commercial devices. In [14] a smart sensor, a camera and an Arduino are attached to a cyclist’s helmet to warn about out-of-sight vehicles. Carried on a backpack or a bike, an NVIDIA Jetson processes the camera’s video and performs object recognition. An example of a commercial device is the Garmin Varia Rearview radar1. A device attached to the back of the bicycle detects approaching vehicles and relays audiovisual warnings to a screen on the handlebar. Focusing on motorcyclists, the SKULLY AR-12puts a Head-up Display (HUD) for GPS navigation and rear-end feed without blocking the driver’s view. These HUDs present information egocentrically, disregarding the user’s head movements. To overcome this, we explore AR technologies, which present information in the context of the environment. 1https://www.garmin.com/en-US/p/721258/ 2https://wearables.com/products/skully-ar-1 AR offers a high degree of user interaction, and studies are exploring its potential uses on road safety [15]. Schwarz et. al validated the impacts of in-vehicle AR warnings, finding that these consistently improve various measures of driving effectiveness [16]. For overtaking scenarios, Rameau et al. [17] present an AR system that ‘sees-through’ vehicles ahead by projecting an image of the road covered by a leading vehicle. There has also been an investigation into interaction with VRUs. Cl´ erigo et al. [7] revealed that AR and V2X-enabled systems aid pedestrians in crossing scenarios, with collision warnings and virtual traffic lights found to increase perceived safety and reduce cognitive workload in a real-world testbed. Although focused on pedestrian crossings, these findings underscore the potential of these technologies in more critical scenarios, such as cyclists at intersections without LoS. Focusing on cyclists at intersections with a lack of LoS, Matviienko et al. use AR to visualize vehicles through sightblocking elements, and display a countdown for when it is safe to cross [18]. Their findings show improved performance and safety of users at uncontrolled intersections. Von Sawitzky et al. demonstrated that multimodal awareness messages can alert cyclists to potential “dooring” scenarios — passing parked vehicles with occupants inside [19] [20]. They also explored three types of HUD, including seeing-through walls, smart paths and warning signs. According to their empirical study, the most preferred method was seeing through walls [21]. To date, most of the evaluations have been conducted in stationary indoor simulators using Virtual Reality (VR) technologies (e.g. wall projections and VR simulators) or Wizard of Oz experiments. Building on our previous work and the insights from these studies, our work differentiates itself by creating an AR system capable of displaying occluded vehicles in realtime in a real-world scenario. Other studies have explored the use of smart city infrastructure and connected devices to create an ad-hoc communication network that exchanges relevant information regarding road users to enhance road safety [22]. The work in [23] focuses on a scenario where pedestrians crossing the street are occluded by a parked vehicle. Their study uses V2X messages to alert vehicles through ITS-G5 and pedestrians using LTE. Similarly, Maruta et al. [24] enhances driver perception using AR to visualise blind spots, and leverages cooperative perception with millimetre-wave V2X communications. However, due to the complex communication architecture and data complexity, delays obtained go upwards of 500 ms — far beyond the safety-critical threshold defined by the European Telecommunications Standards Institute (ETSI) [25]. This excessive delay renders the system impractical in real-world scenarios. This work builds upon our previous work in microservicebased architecture for low-latency road safety applications [10]. This architecture processes data from various sources through heterogeneous communication technologies such as ITS-G5 and 5G. In contrast to related work, this solution enhances scalability, adaptability, and low End-to-End (E2E) latency (<110 ms), ensuring real-time display of occluded vehicles to warn about out-of-sight vehicles of the cyclist. By Fig. 2. An overview of the system’s architecture. using this setup, we move beyond controlled scenarios to realworld evaluations that enhance ecological validity. III. SYSTEM ARCHITECTURE To enable a system capable of real-time AR assistance of cyclists at road intersections with a lack of LoS, we design an architecture capable of seamless interaction among road users through various communication technologies. Figure 2 presents the system architecture that considers two road users: the VRU equipped with an AR headset and the vehicle. The VRU interacts via an AR headset, sharing its location and attributes through ITS-G5 or 5G, while vehicles communicate through On-board Units (OBUs) with fixed Roadside Units (RSUs) wirelessly using ITS-G5 technology. V2X messages already defined by the ETSI are used to transmit information, including position, heading, velocity, and acceleration, among others. This setup allows for medium-range wireless communications associated with vehicular networks, enabling data transmission in road traffic [9]. RSUs relay all messages from the vehicle to the services and vice-versa. This architecture builds upon previous work by Cl´ erigo et al. [10]. A microservice-based architecture processes incoming messages from both VRUs and vehicles to provide the necessary data for the AR application to work correctly. An AR application is deployed on the AR headset to consume all relevant data needed to display visual notifications. This application receives incoming messages and stores them on a middleware module, relaying the information to other modules. One of these modules is the X-ray visualizer, which uses the geographical attributes from the VRU and the vehicle. This data enables the creation of a virtual asset representing the occluded vehicle’s position in the real world, hence creating the effect of an ‘Xray’-like vision. Road safety ITS applications demand a maximum E2E latency of 300 ms [25]. To accommodate these requirements, this work is developed with a Multi-Access Edge Computing (MEC) infrastructure adapted for mobility and deployed close to users. Beyond latency, MEC is crucial for the interoperability of users communicating via heterogeneous technologies, such as ITS-G5 or cellular networks. The MEC system is represented in Figure 2 by the Processing and Communication Services block. The ITS-G5 communication stack is implemented in the system using Vanetza-NAP [26], an extension Fig. 3. Illustration of the X-ray prototype (yellow vehicle). of the Vanetza3tool, enabling interaction with ITS-G5 V2X messages through pub/sub messaging. IV. DISPLAYING OCCLUDED VEHICLES Recent advancements in AR and ITS are promising to enhance the safety of VRUs, especially in dynamic scenarios such as cycling. Cl´ erigo et al. [7] demonstrated the benefits of such systems in a real-life user study (N=20) aimed at pedestrians in crossing scenarios. The findings of this study indicate that AR significantly reduces perceived workload and increases perceived safety while achieving an almost excellent System Usability Scale (SUS). Following that work, this paper displays visual elements that represent an ‘X-ray’ of occluded vehicles to aid cyclists in road intersections with no clear LoS. Figure 3 depicts an illustration of the X-ray system viewed from the cyclist’s Point Of View (POV), a virtual asset from the Unity store that resembles a vehicle4. Although this asset will not completely match the real-life vehicle’s model, it will inform the user about a vehicle’s presence. The headset displays an AR asset in the virtual scene that matches the vehicle’s real-world position concerning the user’s location, minimizing the overwhelmingness of the UI. This is achieved through a series of calculations that translate realworld GPS coordinates into the virtual world, taking the GPS coordinates and heading from both the vehicle and the cyclist. Figure 4 showcases the communication architecture between devices and their services. Communication between devices occurs via MQTT, while intra-device communications use DDS and Zenoh. A VRU computing device runs a VRU awareness basic service [10]. This service generates Vulnerable Road User Awareness Messages (VAMs) on userend devices using information from the GPS sensor 1, and the AR headset 6 7 8 to broadcast VAMs to other road users through Vanetza 9using ITS-G5 with location, speed, and optional attributes from the ETSI standard, such as the user’s gaze heading. Additionaly, a middleware service 3also receives these VAMs through MQTT 1 8 7 . Cooperative Awareness Messages (CAMs) from vehicles are received through Vanetza 9and relayed to the headset via MQTT 8 7 . The virtual vehicle’s position 4is calculated 3https://github.com/riebl/vanetza 4https://assetstore.unity.com/packages/3d/vehicles/land/3d-low-poly-carfor-games-tocus-101652 Fig. 4. Displaying an ‘X-ray’ of an occluded vehicle. using the user’s gaze orientation 6and the messages stored in the middleware 3. Using the headset’s camera stream, objects are detected 2 on the VRU computing device. These are transmitted back to the AR headset via MQTT 8 7 , and a vehicle fusion algorithm 5verifies if the detected objects match with the vehicles’ coordinates received through ITS-G5 to trigger the X-ray system. Building on our previous work [10], we use a pre-developed microservice pipeline that processes V2X messages from road users and predicts their trajectories. While originally used for collision prediction, this functionality now smooths the movement of virtual assets. This is because CAMs can be transmitted between 1 and 10 Hz, leading to choppy and discrete movements rather than fluid motion. These microservices are placed between 9and 8. One of the technical challenges lies in placing a virtual object in a virtual scene according to the real object’s position and relation to the user. To accurately map the virtual coordinates of the vehicle based on real-world GPS data, our approach first calculates the distance between the road users in the real world using Haversine’s formula: distance =RE×2×atan2√a, √1−a a= sin2∆ϕ 2+ cos(ϕA)×cos(ϕB)×sin2∆λ 2(1) where RErepresents Earth’s radius (6371000 meters), and ϕ and λrepresent latitude and longitude coordinates in radians, respectively. Using this formula, the error from assuming Earth as a sphere might be up to 0.55% when crossing the equator, though generally, it is below 0.3%. Since this scenario is primarily applied at a 100 m scale, the resulting error is 3050 cm. However, the primary source of error is GPS accuracy (1–5 m), rather than the formula itself. The bearing between the cyclist and the vehicle received by V2X communications, is given by the following equation: θV2X = atan2 sin(∆λ)×cos(ϕ2),cos(ϕ1)×sin(ϕ2) −sin(ϕ1)×cos(ϕ2)×cos(∆λ)(2) where θV2X represents the initial bearing in radians, which ranges from −πto π. This is converted to degrees and normalized to ] 0, 360 ] to ensure compatibility with Unity. Finally, the virtual coordinates of the vehicle are calculated by offsetting the user’s Unity camera position and orientation with the value of distance (d) and bearing (θV2X). Another critical challenge for this system is to determine whether a user has LoS with the vehicle or not. To mitigate this, we use the Microsoft HoloLens25as the AR headset, which can broadcast its video camera stream, matching the user’s POV. Using a compact and powerful computing device, the NVIDIA Jetson Orin Nano6, we process this camera stream, detect vehicles in the frame, and determine their bearing. Figure 5 illustrates the process of determining whether a vehicle perceived through V2X is the same as the one detected by the camera. The Unity application 5compares the bearing calculated by the V2X messages θV2X in 4with the one obtained by the detected object θobject in 2. If the difference between these headings ∆θis larger than 15°, then the X-ray application considers them as distinct objects and enables the visualization of the virtual car received via V2X. Otherwise, the overlay is disabled, as the user is already seeing the vehicle in the real world (fused information). The threshold of 15° was empirically selected as a tradeoff between compensating for inaccuracies (i.e. GPS error and camera FOV distortion) while avoiding false matches. Fig. 5. Visual representation of the bearings considered for matching detected vehicle through the camera with a perceived vehicle by V2X communications. V. OBJECT DETECTION As previously mentioned, the AR headset used for this system is the Microsoft HoloLens 2, heavily used in commercial and research applications. Its early release to the market made it as one of the most complete devices with advanced hardware and software capabilities. Its ability to blend the real and virtual worlds makes it a valuable tool for diverse fields such as healthcare, industry, education, and ITS [7]. 5https://learn.microsoft.com/en-us/hololens/hololens2-hardware 6https://www.nvidia.com/en-eu/autonomous-machines/embeddedsystems/jetson-orin/nano-super-developer-kit/ A. Microsoft HoloLens 2 as a Sensor One key feature of this device is the “research mode”. This feature allows developers to access raw sensor data from its onboard components including the RGB and depth cameras, Inertial Measurement Unit (IMU), and microphone, among others. This unprocessed data stream allows the development of innovative computer vision applications, effectively making the AR headset a mobile video camera. Ungureanu et al. [27] demonstrated the potential of this device in high-fidelity spatial mapping and real-time object recognition. The raw data streams allowed researchers to bypass the device’s internal processing pipeline, giving full control over how data is interpreted and utilized. This capability is particularly important for applications requiring finegrained analysis, such as VRU safety applications. However, given that the HoloLens is a mobile device that operates on a battery, its processing power is limited. The work in [28] tackled this issue by offloading processing tasks to an external device. They developed and open-sourced a Python library7 that enables real-time transmission of the HoloLens’ sensors to external devices via TCP sockets. We use their library in our object detection module presented in Figure 4 marked as 2. Previous work in [29] showed that offloading the computational video feed to an NVIDIA Jetson that detects road users achieves latencies compatible ETSI’s threshold. Using a Jetson with ITS-G5 capabilities connected to the HoloLens 2 (wired or wirelessly) creates a setup where the AR device operates independently of the infrastructure, allowing it to be adapted for several use cases. B. Architecture and Operation To allow for real-time vehicle detection, the architecture in Figure 6 prioritizes low processing times and decouples the video streaming capabilities for modular use. Expanding on the previous Figure 4, this architecture highlights the architecture for object detection, more specifically, vehicle detection. The module for object detection, marked as 2, is now expanded into multiple components working together 10 . The RGB Camera Handler 11 captures the RGB camera stream from the HoloLens. This connection is done using the aforementioned Python library [28] through a TCP socket. This module is also decoupled from the remaining logic, so that other services can use this video stream if needed. The vehicle detection service 12 continuously consumes frames, sent by the RGB Camera Handler 11 , and processes them using an object detection algorithm. We use a predeveloped decoupled framework for object detection that uses an external YOLO service 13 for request-response interactions [29]. This abstracts the application-specific requirements, enabling the integration of other algorithms in the future. To detect and track vehicles, it is important to choose a diversified and state-of-the-art model. For this, we chose the YOLOv9t8 for the detection service. Even though this model has the 7https://github.com/jdibenes/hl2ss 8https://docs.ultralytics.com/models/yolov9/ Fig. 6. Architecture that offloads vehicle detection of a HoloLens’ camera. lowest number of parameters (2 ˙ M) in the YOLOv9 family, it is sufficient for our use case where camera frames have good visibility and low distortion9. The RGB frames sent by the HoloLens 2 have a size of 1504x846 with a 64.69º Field Of View (FOV) [30]. However, YOLO is best optimized at a resolution of 640x640, which means that the frames have to be resized, thus reducing the FOV. The updated FOV can be determined as: Sscale = min WYOLO WHL ,HYOLO HHL (3) FOVscaled =FOVHL ×Sscale (4) where WYOLO and HYOLO are optimized to the width and height of the YOLO’s frame (640x640), while WHL and denote HHL the width and height of the native’s HoloLens 2 camera frame (1504x846). The FOV for the resized frame is 27.53 °. Once the YOLO server detects the objects 13 , it sends this information back to 12 to filter for the vehicle class. The bearing extractor 14 continuously consumes vehicle detection results 12 and user’s gaze heading 6published through MQTT 7 8 at a frequency of 20 Hz. The vehicle detection results are represented as bounding boxes within the image. The module determines the center of each bounding box based on its horizontal width, and calculates the bearing by correlating the object’s position with the FOV, user heading, and bounding box center. The object’s bearing is determined as: xcenter,YOLO =WYOLO 2(5) 9YOLOv9 was selected as it was the most recent version of YOLO at the time of development. However, using this decoupled framework, newer versions of YOLO like YOLOv11s could be integrated, potentially lowering the detection times in approximately 5 ms. xcenter,box =xbox +Wbox 2(6) θoffset =xcenter,box −xcenter,YOLO WYOLO ×FOVscaled (7) θobject = (θuser +θoffset) mod 360 (8) where xcenter,YOLO represents the horizontal center of the YOLO frame, and xcenter,box and Wbox denotes the horizontal centre of the detected bounding box’ and its width. The angular offset θoffset accounts for the object’s relative position in the frame, allowing computation of the geographical bearing θobject based on the user’s heading θuser. When all vehicles are recognized, the respective bearings will be sent to to the HoloLens (Figure 4 marked as 3), through MQTT 8 7 . VI. EVALUATION AND DISCUSSION In this section, we assess the system’s performance through real-world testing, focusing on three key aspects: stream latency, resource and power consumption, and system performance. These are critical in determining the feasibility of using this system to enhance cyclist safety in occluded intersections. A. Stream Latency To measure the latency of the video camera’s stream, we measure the delay from the moment a video frame is captured to the moment it is displayed on a computer screen. The Jetson consumes the HoloLens’s stream and controls the colors of an external display. The camera is pointed towards this monitor and continuously processes the captured frames. When the camera detects a red frame, the monitor changes its color to green and vice-versa, taking the respective timestamps. By doing this, the measured latency accounts for frame acquisition and transmission time. The frame acquisition time includes the light capture made by the CMOS sensor and the processing made by the image signal processing. The transmission includes the encoding and wireless stream transmission time to the Jetson. This test runs continuously for 5 minutes. Figure 7 and Table I show the latency comparison between three different resolutions (1128x636, 1280x720 and 1504x846) for both 15 and 30 FPS using Wi-Fi. The Microsoft HoloLens 2 can stream its video camera in 11 different resolutions. However, we only tested resolutions above YOLO’s frame size (640x640), since downscaling a video frame achieves better results than upscaling it. Resolutions above 1504x846 were discarded, as these are higher than the HoloLens’ native resolution, resulting in an upscaling overhead. The results indicate that latency is higher for 15 FPS when compared to 30 FPS by 60-100 ms; this is to be expected since at 15 FPS, frames are produced every 66.7 ms, whereas at 30 FPS, frames are produced every 33.3 ms, naturally leading to higher perceived latency at lower frame rates. A higher frame resolution resulted in a slightly higher delay (<10 ms). However, for further tests and system usage, we have decided to stick with the native resolution (1504x846). We also tested a stream with 60 FPS (only available for the native resolution). Fig. 7. Latency comparison of HoloLens’ camera stream between different resolutions and frame rates. Fig. 8. Latency Comparison (1504x846 at 30 FPS) between Wired and Wi-Fi communication. Results showed an average latency of 170.25 ± 55.10 ms. Therefore, we discarded this option. Figure 8 shows the latency results for the stream delay at the resolution of 1504x846 with 30 FPS, comparing WiFi to wired (USB-C) connection. Results show an average latency of 152.90 ± 14.75 ms for USB-C, and 153.98 ± 17.27 ms for Wi-Fi. These two connection types show minimal differences, falling within a similar range, and indicating that a wireless connection provides comparable performance to USB-C. Therefore, we opted to primarily use Wi-Fi for convenience and flexibility in real-world scenarios without significant performance trade-offs. The small differences in response time might be due to the library’s internal buffering and encoding, which can overshadow transport-level delays. TABLE I LATENCY RESULTS FOR THE DIFFERENT RESOLUTIONS AND FRAME RATES. Resolution FPS Average (ms) Standard Deviation (ms) 1128x636 30 140.28 10.29 1128x636 15 215.06 27.75 1280x720 30 144.80 23.08 1280x720 15 216.73 27.79 1504x846 30 153.98 17.27 1504x846 15 261.10 11.89 Fig. 9. CPU usage, RAM usage and power consumption comparison for the Object Detection of streams at 15 and 30 FPS. B. Resource and Power Consumption This system is aimed to be placed in a mobile and batterypowered context, like a modified bicycle or an e-bike. This subsection evaluates the feasibility of using this system in a real-world deployment. Although results show a better latency when using 30 FPS, we analyze the resource and power consumption of the Jetson when processing video stream from the HoloLens 2 at 15 and 30 FPS. These tests explore whether a higher frame rate significantly impacts power consumption, leading to excessive energy drain and reduced operation time. To test this, we run the object detection system at the Jetson and use the jetson-stats tool10 to monitor CPU and RAM usage, as well as power consumption over a period of 5 minutes. Figure 9 presents the results using a resolution of 1504x845 and Wi-Fi. CPU load remains nearly the same between 15 FPS (49.15 ± 1.78%) and 30 FPS (49.35 ± 1.81%), indicating that increasing the frame rate does not impact the processing overhead. RAM usage and power consumption, on the other hand, indicate a slight increase for higher frame rates, 5241 MB and 7.88 ± 0.22 W, when compared to 4733 MB and 7.76 ± 0.20 W for a lower frame rate. However, this variation remains within error limits. This negligible difference demonstrates that streaming at 30 FPS does not significantly impact a battery-powered scenario, remaining the best configuration for real-world deployment. The AR headset’s power consumption was left out, as its role was limited to visualization, with most processing offloaded to the external computing device. C. System Performance To assess the system’s performance, we conduct latency measurements across the different processing stages highlighted in Figure 6. These evaluations provide insights into the efficiency of the system and its viability for real-time ARbased cyclist assistance. The primary goal is to ensure that the system remains within acceptable latency limits for safetycritical applications while maintaining a stable frame rate. The results below summarize the metrics for 100 processed frames: 1) Stream Delay (from 15 to 11 ): 152.90 ± 14.75 ms 2) RGB Handoff (from 11 to 12 ): 12.83 ± 5.80 ms 3) Vehicle Detection ( 12 to 13 ): 65.03 ± 26.73 ms 10https://github.com/rbonghi/jetson stats Fig. 10. Representation of the demonstration’s setup, location and equipment. A cyclist (left) wearing a Microsoft HoloLens 2 and carrying a computing device, and a vehicle (right) is carrying a battery-powered OBU. 4) Bearing Extractor 14 : 0.31 ± 0.08 ms 5) Object Detection 2: 100.42 ± 77.08 ms 6) E2E Delay (from 15 to 14 ): 266.46 ± 78.67 ms 7) E2E Frame Rate (from 15 to 14 ): 3.75 ± 0.11 FPS The total system latency of 266.46 ± 78.67 ms highlights a key opportunity in real-time use cases that leverage computer vision algorithms. These results demonstrate that the E2E latency for intelligently triggering the X-ray system off is done under the 300 ms threshold set by ETSI for road safety applications. As previously mentioned, this setup builds upon prior work by Cl´ erigo et al. [10] exploring an architecture for enhancing road safety with support for low-latency services. Results showed that this architecture achieves an E2E latency of 81.53 ± 8.36 ms in a setup where road users communicate in a peer-to-peer manner through ITS-G5. Consequently, both the reception of a V2X message from the vehicle — activating the virtual vehicle for the ‘X-ray’-like vision — and intelligently deactivating this vision when LoS with the vehicle is established, occur within the 300 ms threshold. The object detection processing time contributes the most to the E2E delay (100.42 ± 77.08 ms), indicating that optimizing this through hardware acceleration or model refinement would greatly enhance the latency. Unlike other detection methods, the number of objects in a frame does not significantly impact YOLO’s processing time, meaning that this methodology remains reproducible across different scenarios. Additionally, the low frame rate of 3.75 FPS suggests that real-time tracking of occluded vehicles may be hindered, especially in fastmoving traffic scenarios. Reducing computational overhead and improving the stream delay imposed by the current implementation will be crucial for enhancing the system’s throughput, making it more viable for real-world deployment. In the process of validating the system, we employed a real-life testing scenario within the Aveiro Tech City Living Lab (ATCLL) [9], in which a cyclist is positioned close to a road he must cross, with a building obstructing the LoS of an oncoming vehicle. This vehicle travels at approximately 20 km/h on the perpendicular lane, as depicted in Figure 10. The testing site is on a restricted road near the Instituto de Telecomunicac¸ ˜ oes in the University of Aveiro, therefore, no external road traffic was present, ensuring a safe environment. The VRU carries the aforementioned devices in section V, an NVIDIA Jetson Orin Nano with ITS-G5 capabilities connected to a GPS module, and a Microsoft HoloLens 2 AR headset. The vehicle carries an OBU to transmit its V2X messages. The real demonstration can be visualized in this link11. VII. CONCLUSIONS AND FUTURE WORK This work explored the integration of AR and V2X communications to enhance cyclist safety at intersections with obstructed visibility in real time. The system leverages V2X communications and edge computing to display occluded vehicles using an ‘X-ray’ vision effect in AR, allowing cyclists to perceive otherwise hidden vehicles. The system architecture integrates multiple possibilities for communication technologies, including 5G and ITS-G5, ensuring reliable data exchange between vehicles and cyclists. Through real-world testing in the ATCLL platform, we evaluated the system’s feasibility using a Microsoft HoloLens 2 AR headset and an NVIDIA Jetson-based object detection pipeline. The results show that the system’s latency for displaying virtual vehicles in the AR headset falls within the 300 ms safety threshold defined by ETSI. Moreover, objects are detected by the headset’s video at a frame rate of 3.75 FPS, to verify that the user is in LoS with the vehicle and no longer requires AR assistance, highlighting challenges in real-time object detection. Future work will expand real-world evaluations in varied urban scenarios and the incorporation of eye-tracking data to provide insights into the effects of this system on the cyclist’s cognitive load. Moreover, we will incorporate the AR headset’s Lidar capabilities to better identify if the user is within LoS with the vehicle to enhance the system’s effectiveness. ACKNOWLEDGMENT This work was supported in part by the EU’s HE research and innovation programme HORIZON-JU-SNS-2023 under the 6G-PATH project (Grant No. 101139172). REFERENCES [1] European Commission, Directorate-General for Communication, “Special eurobarometer 495: Mobility and transport (v1.00).” [Data set], 2020. Accessed: 2025-02-11. [2] W. H. O. S. D. of Health Team, “Global status report on road safety 2023,” 12 2023. [3] European Transport Safety Council (ETSC), “Ranking eu progress on road safety - 18th annual road safety performance index (pin) report,” 2024. Accessed: 2025-02-11. [4] Road Safety - European Commission, Directorate-General for Communication, “Facts and Figures – Cyclists - 2023,” 2023. Accessed: 202502-11. [5] I. Gohl, et al., “Car-to-cyclist accidents from the car driver’s point of view,” in International Cycling Safety Conference (ICSC), 2016. [6] T. Park, et al., “When mixed reality meets internet of things: toward the realization of ubiquitous mixed reality,” GetMobile: Mobile Computing and Communications, vol. 22, no. 1, pp. 10–14, 2018. [7] A. Cl´ erigo, et al., “Safearcross: Augmented reality collision warnings and virtual traffic lights for pedestrian safety,” in Proceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’24, (Stanford, CA, USA), 2024. [8] A. Silla, et al., “Can cyclist safety be improved with intelligent transport systems?,” Accident Analysis & Prevention, vol. 105, pp. 134–145, 2017. Improving cyclist safety through scientific research, ICSC2015. [9] P. Rito, et al., “Aveiro tech city living lab: A communication, sensing, and computing platform for city environments,” IEEE Internet of Things Journal, vol. 10, no. 15, pp. 13489–13510, 2023. 11https://youtu.be/837cFBvnPX4 [10] A. Cl´ erigo, et al., “Microservice-based architecture for enhancing road safety with support for low-latency services,” in 2025 IEEE/IFIP Network Operations and Management Symposium (NOMS 2025), 2025, Accepted. [11] The New York Times, “Farther, faster and no sweat: Bike-sharing and the e-bike boom,” 2021. [Online; accessed 10-February-2025]. [12] G. Kapousizis, et al., “A review of state-of-the-art bicycle technologies affecting cycling safety: level of smartness and technology readiness,” Transport Reviews, vol. 43, no. 3, pp. 430–452, 2023. [13] A. Matviienko, et al., “Navibike: Comparing unimodal navigation cues for child cyclists,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, (New York, NY, USA), p. 1–12, Association for Computing Machinery, 2019. [14] J. Solus, et al., “Iot-enabled smart bike helmet with an ai-driven collision avoidance system,” in 2023 IEEE International Conference on Electro Information Technology (eIT), pp. 175–179, 2023. [15] J. Orji, et al., “Virtual and augmented reality applications for promoting safety and security: A systematic review,” in 2022 IEEE 10th International Conference on Serious Games and Applications for Health(SeGAH), (Sydney, Australia), pp. 1–7, IEEE, 2022. [16] F. Schwarz, et al., “Augmented reality warnings in vehicles: Effects of modality and specificity on effectiveness,” Accident Analysis & Prevention, vol. 101, pp. 55–66, 2017. [17] F. Rameau, et al., “A real-time augmented reality system to see-through cars,” IEEE Transactions on Visualization and Computer Graphics, vol. 22, no. 11, pp. 2395–2404, 2016. [18] A. Matviienko, et al., “Bikear: Understanding cyclists’ crossing decisionmaking at uncontrolled intersections using augmented reality,” in Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22, (New York, NY, USA), Association for Computing Machinery, 2022. [19] T. von Sawitzky, et al., “Hazard notifications for cyclists: Comparison of awareness message modalities in a mixed reality study,” in Proceedings of the 27th International Conference on Intelligent User Interfaces, IUI ’22, (New York, NY, USA), p. 310–322, Association for Computing Machinery, 2022. [20] T. von Sawitzky, et al., “Investigating hazard notifications for cyclists in mixed reality: A comparative analysis with a test track study,” in Proceedings of the 15th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’23, (New York, NY, USA), p. 202–212, Association for Computing Machinery, 2023. [21] T. von Sawitzky, et al., “Augmentation concepts with huds for cyclists to improve road safety in shared spaces,” in Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, CHI EA ’20, (New York, NY, USA), p. 1–9, Association for Computing Machinery, 2020. [22] P. Teixeira, et al., “A sensing, communication and computing approach for vulnerable road users safety,” IEEE Access, vol. 11, pp. 4914–4930, 2023. [23] F. Gonc¸alves, et al., “Enhancing vrus safety with v2p communications: an experiment with hidden pedestrians on a crosswalk,” in 2022 14th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), (Valencia, Spain), pp. 96–103, IEEE, 2022. [24] K. Maruta, et al., “Blind-spot visualization via ar glasses using millimeter-wave v2x for safe driving,” in 2021 IEEE 94th Vehicular Technology Conference (VTC2021-Fall), pp. 1–5, 2021. [25] ETSI, “Intelligent Transport Systems (ITS); V2X Applications; Part 3: Longitudinal Collision Risk Warning (LCRW) application requirements specification .” https://www.etsi.org/deliver/etsi ts/101500 101599/ 10153903/01.01.01 60/ts 10153903v010101p.pdf (Accessed: March 22th, 2024), 11 2013. no. TS 101 539-3 V1.1.1. [26] R.Rosmaninho, et al., “Vanetza-nap: Vehicular communications and services in microservices architectures,” in 2024 IEEE Vehicular Networking Conference (VNC), pp. 297–304, 2024. [27] D. Ungureanu, et al., “HoloLens 2 Research Mode as a Tool for Computer Vision Research,” Aug. 2020. arXiv:2008.11239 [cs]. [28] J. Dibene, et al., “HoloLens 2 Sensor Streaming,” Nov. 2022. arXiv:2211.02648 [cs]. [29] M. Mendes, et al., “Real-time object and event detection service through computer vision and edge computing,” in ITS World Congress 2024. [30] Microsoft, “Locatable camera overview,” 2022. [Online; accessed 10February-2025].