Augmented Reality for Real-time Decision-Making in Flood Emergencies

© 2024 IEEE. This is the author’s pre-print version of the article that has been published in the proceedings of IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). The final version of this record is available at: https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00032 Augmented Reality for Real-time Decision-Making in Flood Emergencies Ioannis Safranoglou* School of Electrical and Computer Engineering, Technical University of Crete Alexis Stavroulakis† School of Electrical and Computer Engineering, Technical University of Crete Marcel Ebel‡ Paderborn University - Heinz Nixdorf Institute Jens Pottebaum§ Paderborn University - Heinz Nixdorf Institute Georgios Lamprinakis¶ School of Electrical and Computer Engineering, Technical University of Crete Despina Dimelli|| School of Architecture, Technical University of Crete Katerina Mania** School of Electrical and Computer Engineering, Technical University of Crete ABSTRACT Locomotion interfaces for head-worn AR, while rescuers move on site, are critical in disaster response including flood emergency management. Incorporating live, real-time data, rather than static information, in the form of forecasting of events over time, without overloading the user’s field of view, is crucial. This paper presents an innovative head-worn AR flood management system designed to support situational awareness and way-finding by enhancing decision-making for emergency responders on-the-go. Our system dynamically visualizes present and predicted water levels as well as water speed and direction at specific future time intervals in real-time, regardless of location, improving flood management while rescuers are moving. Moreover, points of interest are shown in the horizon with the ones most likely to require assistance indicated by color variation, allowing rescuers to navigate and respond more effectively to emergency scenarios. Our gaze-enabled AR system prototype was tested by professional firefighters in Innsbruck and Dortmund. Feedback from these trials highlighted the potential of providing comprehensive, real-time, AR visualization to enhance disaster response strategies. Index Terms: Augmented Reality, Flood Visualization and Forecasting, First Responders, Hololens, Flood Management. 1 INTRODUCTION Locomotion affordances while deploying head-worn Augmented Reality (AR) play a pivotal role in enhancing the effectiveness of AR navigation and way-finding. Navigation interfaces for AR while moving are critical in disaster response [30] including flood emergency management [1] and firefighting [3]. Virtual Reality (VR) is effective for rescue training simulations [8,25] and remote flood rescue planning [13], not typically deployed for on-site operations. AR, on the other hand, facilitates real-time visualization of digital information superimposed onto physical objects, without obscuring the real world [9,27]. Previous work has shown that AR can improve operational efficiency and safety in firefighting by incorporating thermal imaging and live tracking [3]. For flood emergencies, AR provides spatial awareness and efficient navigation to rescue targets for first responders, facilitating access to critical information in hazardous environments [11,15]. Past work in mobile AR-based visualization of coastal erosion and *e-mail: [email protected] †e-mail: [email protected] ‡e-mail: [email protected].de §e-mail:[email protected].de ¶e-mail:[email protected] ||e-mail:[email protected] **e-mail:[email protected] sea level rise employed static data in pre-defined locations to display water levels [10,22]. Water visualization in mobile AR has been primarily implemented using hand-held cameras, constraining rescuers’ operations [9,6,5]. AR visualization of real-time data for spatial flood forecasting, rescuer locations and points of interest (POIs), while firefighters are moving is still a research challenge. Presenting crucial information in head-worn, hands-free AR without overloading the user’s field of view is paramount for efficient situational in hazardous environments [4]. This paper presents an innovative head-worn AR flood management system designed to support way-finding by enhancing situational awareness and decision-making for emergency responders on-the-go. Our system dynamically visualizes water levels in realtime, at any location, improving flood management. Environmental monitoring is refined and rescuers are equipped with a clearer understanding of potential hazards. An integrated digital map offers a comprehensive view of the surroundings. This way, rescuers anticipate flood behavior while moving at any location. Our AR system prototype was tested by professional firefighters in Innsbruck and Dortmund. Feedback from these trials highlighted the potential of providing comprehensive, real-time, AR visualization to enhance disaster response strategies. The specific contributions of this paper include: • An AR system combining a server and head-worn AR displaying predicted water levels, water velocity, and water direction in real-time, at future time intervals, improving situational awareness and way-finding of emergency responders. • Integrating GPS data from mobile platforms with head-worn AR to visualize critical POIs (schools, hospitals) at far and hazards such as manholes in near proximity, indicating hazards by change of colour in the operational environment, enhancing navigation, safety and emergency planning. • Uncertainty visualization for predicted flood levels over time. • Gaze interaction and gestures for hands-free operation. • Proof-of-concept testing by professional firefighters in urban areas of Innsbruck and Dortmund, highlighting the system’s potential to enhance way-finding during flood emergencies. 2 RELATED WORK 2.1 AR for Emergency Response AR has been adopted in emergency response scenarios to enhance situational awareness and operational efficiency [23,24,16,26]. Campos at al. designed an AR mobile system assisting first responders during emergencies by displaying POIs and a mini-map [2]. Mobile AR systems, though, occupy users’ hands, preventing users from performing critical tasks while requiring constant shift of focus between the screen and the physical environment. Certain 1 © 2024 IEEE. This is the author’s pre-print version of the article that has been published in the proceedings of IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). The final version of this record is available at: https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00032 head-worn AR systems for emergency responders have been developed [21,29]. Nelson et al. designed head-worn AR triage tools to assist first responders during mass casualty incidents [19,14]. Their approach involved first responders in iterative prototyping and evaluation, however, it focused exclusively on triage scenarios and did not address other in-the-field tasks. In contrast, our system deploys head-worn AR for hands-free, gaze and gesture driven interaction by firefighters, visualizing real-time data including present and predicted water levels, including prediction uncertainty, enhancing situational awareness and operational efficiency for first responders. 2.2 Flood Visualization Flood visualization has provided tools for emergency responders to manage and mitigate the impact of flooding [9,6], often designed for mobile devices which can be limiting in emergency scenarios. Head-worn AR systems have also been developed for flood visualization. Sarri et al. designed a system which uses AR to visualize future sea level rise at coastal locations enhancing public awareness of climate change [22]. Their work focused on static data and pre-defined locations without locomotion. Moreover, the water visualization was stationary; if the user moved further away, the water visualization was not updated for varied locations. In contrast, our approach dynamically adjusts the scanned mesh of the environment to follow the user, allowing for continuous scanning and visualization of water levels as the user moves. Wang et al. developed a system based on head-worn AR that created an interactive 3D model of urban flood scenarios. This system, while not providing on-location visualization, allowed users to visualize and manipulate flood data for better decision-making [28]. Our AR-based flood visualization system projects flood data directly into the user’s real-world environment, visualizing flood levels at desired locations around the user indicating flood impact. A head-worn AR flood visualization system was developed by Rydvanskiy et al, evaluating the system’s usability in flood risk management [20]. Their approach integrated 3D geo-spatial data into the physical space of the user based on a map to indicate flood scenarios. Our system displays flooding scenarios in the 3D world around the user, enhancing firefighters’ situational awareness. By combining our AR system with a server infrastructure and by effectively visualizing water levels, flow velocity, and direction over time based on real-time data flow, our system supports first responders on the field while they move. 2.3 Uncertainty Visualization Uncertainty visualization is a critical component of data representation that helps users comprehend the reliability and variability of data. The uncertainty inherent in data is communicated, which is essential for making informed decisions. Visualizing uncertainty can significantly improve decision-making by providing a clearer understanding of data variability and reliability [18]. When it comes to flood management, techniques such as heatmaps, and ensemble visualizations are often used. Probabilistic flood maps show flood scenarios based on levels of rainfall [7]. A heatmap might go from blue to red to show increasing risk, while ensemble visualizations can use colors to indicate the likelihood of different outcomes [12]. This way, users can see not just the most likely result, but also the range of other possible outcomes. Deploying uncertainty visualization, in our case in AR for rescue operations on the move based on real-time data streaming, offers a clearer picture of risks and outcomes, improving planning and decision-making. Possible flood extents help emergency teams decide where to effectively focus their resources, enhancing their overall situational awareness [17]. In our system, we display the prediction certainty of the flood visualization based on a selected time, at any location, by moving a slider indicating present and future water level and indicating uncertainty utilizing a numeric probability (Figure 7). POIs’s colors are adjusted based on risk level. 3 SYSTEM OVERVIEW Our system architecture for AR flood management utilizes headworn AR, driven by real-time data integration, to enhance decision making in rescue operations. This approach provides first responders with real-time visual representation of urban floods while they are navigating around an urban area, focusing on water visuazalition (near the rescuer) and POIs visualization (at far), to support navigation. The following key components and system features were based on rescuers providing detailed user requirements. Key Components HoloLens 2 AR Headset: Enables hands-free operation and immersive visualization of flood scenarios on the move; Data Integration: Combines real-time forecast data for flood water height processed and stored in Kafka topics; Intermediate Server: Manages data processing and communication between the Kafka server and HoloLens 2 devices; Dynamic Flood Visualization: Provides real-time updates on water levels, flow velocity, flow direction, position of manholes. System Features Hands-Free Operation: Allows users to remain fully engaged with their surroundings while receiving crucial data updates, on the move; Real-Time Data: Ensures up-to-date information for effective coordination and execution of rescue operations; Interactive POIs and Map Integration: Enhances navigation and situational awareness by displaying POIs of critical infrastructure at far (hospitals, schools, etc) as a navigation aid and real-time rescuer location on an interactive map; Flood Visualization: Utilizes advanced AR capabilities to provide a realistic and detailed representation of water level and flow including uncertainty indicators of risk prediction as well as visualization of the worst case scenario. 4 IMPLEMENTATION 4.1 Data For a flooding emergency, context data is loaded in our AR system in terms of grid data for 2D overland simulations and network data for 1D simulations. Basic grid data is given in terms of Digital Elevation Models (DEM) often as governmental open data like bathymetry, Digital Terrain Models (DTM) and Digital Surface Models (DSM). DEM provide elevation values of the earth’s surface without objects such as plants or buildings, while DTM refine this data and represent terrain forms. DSM, on the other hand, capture elevation information of all visible surfaces such as vegetation and buildings to provide a complete picture of the earth’s surface. In contrast, the sewer network of a city consists of nodes like manholes or pumps as point coordinates, and links like pipes. For real-time use cases of the AR system presented in this paper, streaming data is consumed via Kafka topics like, for instance, flow velocity within links or water levels in nodes. Thus, data categories are distinguished in terms of spatial and temporal characteristics as well as their origin resp. data source. Based on these distinctions, a range of data formats is relevant to the AR system presented. •Spatial resolution from global data to local data. Global data is available almost independent from borders such as satellite images or weather forecasts, in rough resolution. Local data is available in higher resolution, but limited to a field of view such as thermal cameras or points such as gauges. •Spatial geo-references to points, grid elements, or dedicated areas. While sensor measurements such as a water level in a manhole or a river gauge can be assigned to a single coordinate, types of input are provided as rasterized data providing values per grid element. Data can be also provided for a certain area such as population density for a city district. 2 © 2024 IEEE. This is the author’s pre-print version of the article that has been published in the proceedings of IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). The final version of this record is available at: https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00032 •Absolute time as date and time of measurement, forecasting or simulation run. There is an absolute temporal reference representing the measurement or the starting time set for an algorithm producing data such as a simulation or a forecast. •Relative time refers to date and time in the future, in relation to an absolute time reference. Both simulations and forecasts provide data such as rising water levels relative to an absolute starting time in given time intervals. Such time series data can be 1D data for sensors such as gauges or 2D grid data for overland water levels, which is to be coupled with DEMs. Input such as current precipitation and wind conditions are set by weather stations or by forecasts from weather models. For rasterized grid data provided by weather services, standards like GeoTIFF, netCDF or GRIB are used. There is no consistent use of standards regarding simulation outputs. The DHI as a provider of widely established hydrological models specified res1d and the DFS (Data File System). Types of data referring to POIs and areas are represented data formats for scalars (dfs0), vectors (dfs1), matrices (dfs2) and unions of these three (dfsu). 4.2 Server Architecture and Data Handling The system leverages a server infrastructure and real-time data integration to provide dynamic visualization directly on the field as first responders are moving. The architecture of our back-end system comprises a Dockerized Kafka server and an Intermediate Node Server (Figure 1). Apache Kafka is an open-source distributed event streaming platform designed to handle real-time data feeds with high throughput, low latency, and fault tolerance. Kafka’s main functions include publishing and subscribing to streams of records, effectively storing these records in the order they were generated, and processing them in real time. Kafka uses a publishsubscribe model, where data is written to topics by producers and read by consumers, ensuring scalable and fault-tolerant data transmission. This makes it ideal for applications that require real-time processing and event-driven architectures. By initializing the Kafka server container using Zookeeper and Kafka images, we ensure message brokering logic and processing. We implemented an Intermediate Node Server to manage data processing. Node.js, selected for its asynchronous event-driven architecture, handles the data flow between Unity application and Kafka, leveraging libraries like KafkaJS for Kafka client support and ws for WebSocket functionality. The data transmission process involves two main steps: transmitting data from the Kafka topic to the Node server and transmitting processed data from the Node server to the HoloLens 2. Using KafkaJS, the Node server subscribes to the Kafka topic, retrieving and processing forecast data. This data is then sent to the HoloLens 2 device via WebSockets, utilizing the ws library on the server side and websocket-sharp on the client side. This architecture ensures that first responders receive real-time updates. Upon receiving messages from the Kafka topic, the Node server processes the data to isolate information necessary for visualization. The HoloLens 2 device, acting as a consumer in the Kafka setup, receives data and integrates it into AR. Real-time data integration allows for the dynamic display of information to the user. The HoloLens 2 device lacks a built-in GPS system, so we developed a custom Android application to provide GPS data. It sends the user’s GPS coordinates to the Node server via WebSocket. The Node server then relays this data to the HoloLens 2 device, ensuring that the virtual world is correctly aligned with the real world. This is essential for precise placement and calibration of virtual elements such as POIs and user positions. Figure 1: System architecture 4.3 AR Interaction The AR system developed for the HoloLens 2 offers hands-free operation, dynamically displaying predicted flood levels, flow velocity, direction, water level worst-case scenarios, and manhole visualization indicating danger zones. It includes POIs and a map for navigation and situational awareness. The HoloLens 2 device’s display in bright outdoor lighting becomes less visible and tracking can be unstable. The cameras are either over-saturated in bright light or unable to capture enough detail in low light, impacting performance. For optimal tracking, consistent lighting within a range of 500-1000 lux is recommended. 4.3.1 Hand Menu and User Interface The user interface (UI) of our AR system is designed to be intuitive and easily accessible through hand gestures. The main menu is accessed by facing one of the palms towards the user, utilizing the hand tracking feature of the HoloLens 2. The menu is designed in a layered structure, with the first layer providing quick access to essential toggles and a calibration button. The second layer, accessible through specific first-layer options such as the filter button opens additional menus that allow users to toggle and filter POIs. (a) Hand menu interface (b) Second layer (c) Calibration menu Figure 2: Hand menu and user interface 4.3.2 Calibration Process To accurately visualize and calibrate our AR system, GPS data and orientation are crucial. The user must be oriented to true north when the application starts, using a compass. For convenience, a virtual compass is provided by our custom Android system. The calibration process ensures that the 3D scene is correctly aligned with the 3 © 2024 IEEE. This is the author’s pre-print version of the article that has been published in the proceedings of IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). The final version of this record is available at: https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00032 real world, essential for precise placement of virtual elements such as POIs and user positions. •Initial Calibration: When the application starts, the user must follow the instructions and press the calibration button. •Recalibration: The calibration process can be initiated at any time through the main menu if the alignment is lost or needs adjustment. The calibration accuracy of the AR system depends on the user. The process requires the user to face true north using a compass and then press a button on the HoloLens to orient the application accordingly. This user-dependent step typically results in a small offset of about 4-5 degrees. While this offset may introduce minor inaccuracies, it does not significantly misalign the placement of POIs, ensuring that the AR system’s functionality remains reliable. 4.3.3 Points of Interest (POIs) The POI visualization feature in our AR system enhances situational awareness by providing real-time 3D representations of key locations such as schools, hospitals, churches, stations, and others. This feature leverages GPS coordinates to accurately place POIs within the Unity environment, ensuring that emergency responders can navigate and identify critical locations during a crisis. Static POIs: These are pre-loaded locations such as schools, hospitals, and other critical infrastructure that remain unchanged during a weather emergency; Dynamic POIs: These are real-time updates for locations such as other team members’ positions, as well as live danger occurrences or any significant events occurring in real-time, dynamically acquired and broadcast through the Kafka-Node server pipeline. Figure 3: POIs (a) (b) (c) (d) Figure 4: (a) School POI (b) POIs with filter menu open (c) POI Info panel (d) Manhole For every GPS location to be visualized in an AR environment in the correct location and distance from the user, it must undergo a conversion process. The real latitude and longitude must be converted into AR ∆Latitude and ∆Longitude as follows: •∆Latitude (∆lat): Calculated by taking the difference between the latitude of the target location and the user’s latitude, then multiplying by approximately 111,000 (the approximate length in meters of one degree of latitude). •∆Longitude (∆lon): Calculated by taking the difference between the longitude of the target location and the user’s longitude, then multiplying by the length in meters of one degree of longitude at the user’s latitude. This is approximately 111,000 ×cos(user’s latitude in radians). This conversion ensures accurate positioning of POIs in the AR environment relative to the user’s location. However, due to typical GPS accuracy limitations, the placement of these points can have an offset of around 3-5 meters under optimal conditions. This degree of accuracy is generally sufficient for most systems. In our system, especially for the placement of manholes where exact positioning is critical, this limitation can be mitigated by using the manhole representation as a general alert point. This ensures that the user is aware of the vicinity of the manhole, prompting them to stay alert and manually verify the precise location nearby. POIs are designed to be gaze-interactable. When the user focuses on a POI for a short period, an information panel opens, revealing more details about that POI, as shown in Figure 4(c). This interaction method provides first responders with critical information quickly, without manual input, maintaining hands-free operation. 4.3.4 Map The map integration leverages the MapBox Unity SDK to provide a real-time 2D map, updated to display the user’s current location based on GPS data received from our custom Android system. POIs from the 3D world are accurately represented on the map, allowing rescuers to easily navigate and identify key locations during floods. Rescuers move the map as they wish to provide better orientation. (a) (b) Figure 5: (a) Map with real-time user update and POI display (b) Map showing user interaction 4.3.5 Flood Visualization The flood visualization component of our AR system provides first responders with a real-time 3D representation of predicted flood levels. This is developed using Unity and the Mixed Reality Toolkit (MRTK3). Water and flood visualization utilizes a custom water shader applied to a plane positioned within the AR environment to depict varied water heights. Users interact with the water plane through a prediction over time slider. The forecasted values of the water heights including a worst case scenario are updated in realtime based on the desired location and GPS data, ensuring accurate and context-specific flood visualization. 4 © 2024 IEEE. This is the author’s pre-print version of the article that has been published in the proceedings of IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). The final version of this record is available at: https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00032 (a) (b) (c) (d) Figure 6: (a) Water interaction slider (b) Water colliding with wall (c) Water occluding stairs (d) Collision with complex surrounding The slider includes two buttons enabling the display of additional water visualization features. Once activated, the predicted direction of the water flow or the worst case scenario level is shown on the water. A value representing the predicted water velocity is displayed next to the button on the slider as well as a numeric value indicating certainty of prediction. These features offer critical information on both the direction and speed of flood water. Figure 7: Water direction with velocity prediction To accurately position the water within the AR environment, our system employs both environmental scanning and ground height calculation techniques. The ARMeshManager component initiates an environmental scan to generate a detailed mesh of the surroundings, based on the HoloLens 2’s spatial mapping. This mesh is crucial for ensuring that the water interacts with physical objects, providing realistic occlusion. The system calculates the ground height relative to the user’s position by casting a downward ray from the HoloLens 2’s main camera. When this ray intersects with the ground or any surface tagged as ”ground,” the intersection point’s Y-coordinate is recorded as the ground height. By combining these two processes, the water plane is precisely positioned at the correct height, taking into account the surrounding terrain and ensuring an accurate representation of the predicted flood levels. Figure 8: Scanned area with water occlusion Environmental scanning for occlusion enhances the realistic perception of water height but it is resource-intensive leading to a drop in frame rate on the HoloLens 2. Our implementation includes performance optimizations such as mesh cleanup and efficient data handling to maintain smooth performance without compromising the quality of the visualization. Initially, the AR system was unstable during water visualization because of continuous scanning. Post-optimization, the AR system is stable, but frame rate drops from 30-40 fps to 10-15 fps when water visualization is enabled, highlighting the need for further performance improvements. 5 EVALUATION The evaluation of our AR flood visualization system was conducted in Innsbruck and Dortmund by professional firefighters. These trials assessed the system’s effectiveness in real-world conditions. 5.1 Location Initially, system testing was carried out in the old town of Innsbruck. The area contains vulnerable infrastructures, such as public schools and kindergartens. The old town is located close to the river Inn and possesses the deepest point of the sewer system which could lead to extensive flooding due to a heavy rain event in the future. A similar location was selected for testing the AR system, subsequently, in Dortmund. The evaluation involved professional firefighters of Innsbruck and Dortmund and other security authorities, across different hierarchy levels. Experts had varied levels of prior knowledge in AR; the experience of head-worn AR was new for all. 11 experts participated in these trials, providing feedback. 5.2 Methodology Testing was carried out in outdoor areas in Innsbruck and Dortmund, proved to be critical points within the evolving situation over time in pluvial flood events based on heavy rain in the past. A theoretical introduction to AR and the system was offered. Participants, who had no prior experience with the AR system, were individually guided through the system’s features by our team. Features tested included real-time flood visualization, interactive maps, and navigation to critical POIs. Adjustments to the system were applied based on firefighters’ feedback. 5 © 2024 IEEE. This is the author’s pre-print version of the article that has been published in the proceedings of IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). The final version of this record is available at: https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00032 Figure 9: AR trials in Innsbruck and Dortmund 5.3 Quantitative and Qualitative User Feedback The AR trial was first carried out in Innsbruck with experts. Experts noticed that the displayed map was positioned in their immediate field of vision. The map was, then, moved at the upper edge of the field of vision. This enhanced safety, as the map did not cover the ground in the immediate vicinity. Firefighters had difficulty interacting with the main UI as the menu’s position was initially in front of their visual field. The UI was, then, re-positioned on the user’s palm, as shown on Figure 2(a). First responders reacted positively to this change. The AR trial was also carried out in Dortmund, including the AR system optimizations after the Innsbruck trial. The revised UI was well-received by the Dortmund experts who commented that the AR UI was lean and intuitive. The participants of both trials liked the water level visualization. However, the experts criticized the constant visibility of the slider for flood visualization, remedied by placing a smaller button. The firefighters found the visualization of the predicted and simulated water level to be extremely innovative. Showing the direction of the water flow was rated positively as rescue forces can assess possible dangers such as drifting objects like trees if flow direction is visualized. Doubts were expressed in terms of additional mental demand of detachment from the ”real world”, the limited contact with colleagues in the field when wearing an AR device and the AR head-worn devices not being waterproof, expensive and incompatible with protective equipment. Regarding the routing functionality the experts were positive. The experts were pleased to put forward a safe routing plan for the emergency services to the next deployment location or back to the dispatch location from a critical situation as further development. Figure 10 represents qualitative feedback regarding AR functionalities. Users rated them very positively concerning geo-references information in an operation, displayed in terms of POIs (1)-(3). Similarly, but a bit more reserved, functions regarding routing and orientation were rated (4)-(6). The boxplot shows the specific feedback on AR functionalities on a scale from 0 to 5, with 5 being the highest rating. Each box represents the ratings for a specific functionality, with the functions in (1)-(3) referring to points of interest (POI) and (4)-(6) referring to routing/orientation. The mean value of each functionality is represented by an ”X”, while the median is indicated by a horizontal line within the box. The upper and lower ends of the boxes mark the first and third quartiles of the feedback data series. Outliers are indicated by circles that lie outside this range. The function for illustrating vulnerable elements (1) receives a score of 4.0, with a score range of 3.5 to 4.5 and no outliers. The functions for illustrating manhole covers (2) and indicating critical points (3) both have a rating of over 4, with their boxes ranging from 3.75 to 5.0 and no outliers. The function for illustrating evacuation routes (4) receives a rating of over 4.0. The function for routing emergency personnel (5) has a rating of 3.65, with a rating range of 3.0 to 5.0 and one outlier at 2.0. The function for on-site orientation (6) receives a rating of around 4.35, with a range of 3.75 to 5.0. In addition, Figure 10 shows the standard deviations of the assessments indicating the spread of answers by experts. Figure 10: Specific feedback on AR functionalities (1)-(3) POI related, (4)-(6) regarding routing and orientation and their standard deviation 5.4 General Feedback Users logged features for future development, written down in the evaluation questionnaire. Flooding was mentioned, where AR could support decisions to protect objects as well as people if the location of all manholes are included as POIs in a flooding situation. The end users in Innsbruck and Dortmund agreed that the presented AR visualization system was useful and felt that with further improvements and adjustments, the technology could become a valuable tool. They saw great potential for future use within flooding scenarios as well as other scenarios e.g. forest fires. 6 CONCLUSION The development and evaluation of our AR-based flood visualization system demonstrated its potential to enhance situational awareness, and decision-making for emergency responders during flood emergencies, on the move. The head-worn AR system provides real-time 3D visualizations of flood forecast and critical POIs. The trials conducted in Innsbruck and Dortmund gathered valuable feedback from professional firefighters. Participants appreciated the easy-to-use information provided by the AR system, the water level visualization predicted over time and real-time updates on potential danger of POIs. They also noted challenges, such as the additional mental demand and the current limitations of AR hardware, being non-waterproof nature and high cost. Several enhancements are planned to address the feedback received during the trials such as pan and zoom, allowing users to interact directly with map-based POIs, voice commands for improved hands-free operation and developing pop-up windows for POIs that include links to detailed maps of the respective facilities. Additional features will allow users to manually rank POIs by risk score, providing a visual representation of the importance of POIs at risk. Further optimization will ensure a more stable and userfriendly head-worn AR system. Alternative communication methods such as Bluetooth and other long-range point-to-point or mesh networks to enhance the system’s resilience under communicationlimited scenarios, are going to be explored. By continuously refining the AR system based on user feedback, we aim to develop a comprehensive tool that significantly improves the efficiency and effectiveness of emergency response efforts. Moreover, this work contributes to the future development of an integrated, operational AR helmet for firefighters. ACKNOWLEDGMENTS This work was supported by the EU project CREXDATA under Horizon Europe agreement No. 101092749. 6 © 2024 IEEE. This is the author’s pre-print version of the article that has been published in the proceedings of IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). The final version of this record is available at: https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00032 REFERENCES [1] Barmpas Zachariadis, Nikolaos. Development of an iOS, Augmented Reality for disaster management, 2020. Student Paper. 1 [2] A. Campos, N. Correia, T. Rom˜ ao, I. Nunes, and M. Sim˜ oes-Marques. Mobile augmented reality techniques for emergency response. In Proceedings of the 16th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services, pp. 31– 39, 2019. 1 [3] T. Chalimas and K. Mania. Cross-device augmented reality systems for fire and rescue based on thermal imaging and live tracking. In 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 50–54, 2023. doi: 10.1109/ISMAR -Adjunct60411.2023.00018 1 [4] G. Daskalogrigorakis, A. McNamara, A. Marinakis, A. Antoniadis, and K. Mania. Glance-box: Multi-lod glanceable interfaces for machine shop guidance in augmented reality using blink and hand interaction. pp. 315–321, 10 2022. doi: 10.1109/ISMAR-Adjunct57072. 2022.00070 1 [5] G. V. de Andrade, V. L. Padilha, A. Vahldick, and F. H. de Oliveira. Towards an augmented reality application to support civil defense in visualizing the susceptibility of flooding risk in brazilian urban areas. In International conference on computational science and its applications, pp. 494–506. Springer, 2022. 1 [6] U. Erra, D. Mirauda, R. Agatiello, and M. Cerverizzo. Mobile augmented reality for flood events management. International Journal of Sustainable Development and Planning, 13:418 – 424, 03 2018. doi: 10.2495/SDP-V13-N3-418-424 1,2 [7] F. J. Gomez, K. Jafarzadegan, H. Moftakhari, and H. Moradkhani. Probabilistic flood inundation mapping through copula bayesian multi-modelling of precipitation products. Natural Hazards and Earth System Sciences Discussions, 2024:1–27, 2024. doi: 10.5194/nhess -2024-26 2 [8] J. Haskins, B. Zhu, S. Gainer, W. Huse, S. Eadara, B. Boyd, C. Laird, J. Farantatos, and J. Jerald. Exploring vr training for first responders. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 57–62, 2020. doi: 10. 1109/VRW50115.2020.00018 1 [9] P. Haynes, S. Hehl-Lange, and E. Lange. Mobile augmented reality for flood visualisation. Environmental Modelling & Software, 109:380– 389, 2018. doi: 10.1016/j.envsoft.2018.05.012 1,2 [10] M. Katsiokalis, L. Ragia, and K. Mania. Outdoors mobile augmented reality for coastal erosion visualization based on geographical data. In XR@ ISS, 2020. 1 [11] M. Kourogi and T. Kurata. A wearable augmented reality system with personal positioning based on walking locomotion analysis. In The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings., pp. 342–343, 2003. doi: 10.1109/ ISMAR.2003.1240751 1 [12] A. M. MacEachren, R. E. Roth, J. O’Brien, B. Li, D. Swingley, and M. Gahegan. Visual semiotics & uncertainty visualization: An empirical study. IEEE Transactions on Visualization and Computer Graphics, 18(12):2496–2505, 2012. doi: 10.1109/TVCG.2012.279 2 [13] J. M. Mol, W. J. W. Botzen, and J. E. Blasch. After the virtual flood: Risk perceptions and flood preparednessafter virtual reality risk communication. Judgment and Decision Making, 17(1):189–214, 2022. doi: 10.1017/S1930297500009074 1 [14] C. R. Nelson, J. L. Gabbard, J. B. Moats, and R. K. Mehta. Usercentered design and evaluation of artts: an augmented reality triage tool suite for mass casualty incidents. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 336–345, 2022. doi: 10.1109/ISMAR55827.2022.00049 2 [15] T. Nescher and A. Kunz. Analysis of short term path prediction of human locomotion for augmented and virtual reality applications. In 2012 International Conference on Cyberworlds, pp. 15–22, 2012. doi: 10.1109/CW.2012.10 1 [16] I. L. Nunes, R. Lucas, M. Sim˜ oes-Marques, and N. Correia. An augmented reality application to support deployed emergency teams. In Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018) Volume V: Human Simulation and Virtual Environments, Work With Computing Systems (WWCS), Process Control 20, pp. 195–204. Springer, 2019. 1 [17] N. F. Oliveira, L. C. Botega, L. C. Ferreira, and M. R. de Campos. Uncertainty visualization framework for improving situational awareness in emergency management systems. In S. Yamamoto, ed., Human Interface and the Management of Information. Information and Knowledge Design, pp. 86–96. Springer International Publishing, Cham, 2015. 2 [18] K. Potter, P. Rosen, and C. R. Johnson. From quantification to visualization: A taxonomy of uncertainty visualization approaches. In Uncertainty Quantification in Scientific Computing: 10th IFIP WG 2.5 Working Conference, WoCoUQ 2011, Boulder, CO, USA, August 1-4, 2011, Revised Selected Papers, pp. 226–249. Springer, 2012. 2 [19] C. Rae Nelson, J. Conwell, S. Lally, M. Cohn, K. Tanous, J. Moats, and J. L. Gabbard. Exploring augmented reality triage tools to support mass casualty incidents. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 66, pp. 1664–1666. SAGE Publications Sage CA: Los Angeles, CA, 2022. 2 [20] R. Rydvanskiy and N. Hedley. Mixed reality flood visualizations: Reflections on development and usability of current systems. ISPRS International Journal of Geo-Information, 10(2), 2021. doi: 10.3390/ ijgi10020082 2 [21] D. Sainidis, D. Tsiakmakis, K. Konstantoudakis, G. Albanis, A. Dimou, and P. Daras. Single-handed gesture uav control and video feed ar visualization for first responders. In Proceedings of the International Conference on Information Systems for Crisis Response and Management (ISCRAM), Blacksburg, VA, USA, pp. 23–26, 2021. 2 [22] F. Sarri, L. Ragia, A. Panagiotopoulou, and K. Mania. Location-aware augmented-reality for predicting sea level rise in situ. In 2022 International Conference on Interactive Media, Smart Systems and Emerging Technologies (IMET), pp. 1–8. IEEE, 2022. 1,2 [23] M. Sebillo, G. Tortora, G. Vitiello, L. Paolino, and A. Ginige. The use of augmented reality interfaces for on-site crisis preparedness. In Learning and Collaboration Technologies: Second International Conference, LCT 2015, Held as Part of HCI International 2015, Los Angeles, CA, USA, August 2–7, 2015, Proceedings 1, pp. 136–147. Springer, 2015. 1 [24] M. Sebillo, G. Vitiello, L. Paolino, and A. Ginige. Training emergency responders through augmented reality mobile interfaces. Multimedia Tools and Applications, 75:9609–9622, 2016. 1 [25] Y. Sermet and I. Demir. Flood action vr: A virtual reality framework for disaster awareness and emergency response training. 08 2018. doi: 10.1145/3306214.3338550 1 [26] T. Siu and V. Herskovic. Sidebars: Improving awareness of offscreen elements in mobile augmented reality. In Proceedings of the 2013 Chilean Conference on Human-Computer Interaction, pp. 36– 41, 2013. 1 [27] B.-P. Smit, R. Voˆ ute, and E. Verbree. Creating 3d indoor first responder situation awareness in real-time through a head-mounted ar device. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, V-4-2021:209–216, 2021. doi: 10.5194/ isprs-annals-V-4-2021-209-2021 1 [28] S. Wang, J. Wang, P. Gourbesville, and L. Andres. Visualization of flood simulation with microsoft hololens. In Advances in Hydroinformatics: SimHydro 2019-Models for Extreme Situations and Crisis Management, pp. 91–101. Springer, 2020. 2 [29] A. Wani. Augmented reality for fire and emergency services. 11 2013. 2 [30] F. Xu, T. Zhou, T. Nguyen, and J. Du. Augmented reality in teambased search and rescue: Exploring spatial perspectives for enhanced navigation and collaboration. Safety Science, 176:106556, 2024. doi: 10.1016/j.ssci.2024.106556 1 7