CONTEXT-GAD: A Context-Aware Gaze Adaptive Dwell model for Gaze-based Selections in XR Environments

CONTEXT-GAD: A Context-Aware Gaze Adaptive Dwell model for Gaze-based Selections in XR Environments George Ramiotis Electrical and Computer Engineering Technical University of Crete Chania, Greece [email protected] Katerina Mania Electrical and Computer Engineering Technical University of Crete Chania, Crete, Greece [email protected] Figure 1: An illustration of the system’s components for adaptive dwell selection. The elements shown are: (a) the currently highlighted target, designated by a blue arrow; (b) a graphical representation of a subset of the contextual features (red box and arrows) that are analyzed in real time to estimate cognitive load; and (c) a progress indicator (red perimeter bar) that provides feedback to the user on the dwell activation. During the experiment, only the progress indicator was visible to the user Abstract Gaze-based selection, via techniques such as gaze dwell, is one of the most common hands-free interaction performed by users in eXtended Reality (XR) environments. However, selecting a small constant dwell threshold to activate a target might lead to missinteractions, also known as the Midas Touch problem, while a large threshold leads to eye fatigue. Prior research has proposed methodologies to adapt dwell thresholds based on the probability of the user activating a certain target considering past interactions or predicting intent based on gaze features. However, utilizing past inputs or gaze features leads to a heavily biased system towards individual strategy or physiology and cannot be generalized to other XR scenarios or users. In this work, we propose a novel context-aware system that leverages visual features of the task This work is licensed under a Creative Commons Attribution 4.0 International License. VRST ’25, Montreal, QC, Canada ©2025 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2118-2/25/11 https://doi.org/10.1145/3756884.3766048 environment and user behavioral features such as the frequency of interactions, gaze speed variance, and head rotation velocity to adapt dwell thresholds across three distinct levels. We conducted a data collection experiment with twenty participants performing gaze dwell interactions in a general User Interface (UI) navigation task, and a visual search task. We trained a hierarchical machine learning model to predict and adapt dwell thresholds into three levels based on the induced cognitive load. We evaluated our system by utilizing standard machine learning metrics and by conducting a user study (n=17) based on quantitative and qualitative measures. Our system achieves a classification accuracy of 70 . 72% on the first level and 85 . 43% on the second. In addition, the system significantly reduces task completion time in less complex tasks and improves error rates in more cognitive intensive scenes. CCS Concepts •Human-centered computing →Contextual design. Keywords Gaze Interactions, Machine Learning, eXtended Reality (XR) VRST ’25, November 12–14, 2025, Montreal, QC, Canada George Ramiotis and Katerina Mania ACM Reference Format: George Ramiotis and Katerina Mania. 2025. CONTEXT-GAD: A ContextAware Gaze Adaptive Dwell model for Gaze-based Selections in XR Environments. In 31st ACM Symposium on Virtual Reality Software and Technology (VRST ’25), November 12–14, 2025, Montreal, QC, Canada. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3756884.3766048 1 Introduction Selection in immersive environments (VR/AR) is the most common action performed by users [ 2 ]. XR headsets employ various devices such as hand controllers, hand trackers and eye trackers, to facilitate seamless interactions performed by users. However, many hand controllers feature a bulky design and a high weight factor, leading to fatigue in prolonged XR sessions. Camera-based hand trackers suffer from poor gesture recognition in environments with suboptimal lighting and occlusions. For these reasons, there has been an increasing research interest for hands-free interaction methods utilizing the user’s eye movements with one of the most common techniques being dwell selection, where the user needs to fixate on a target for a fixed duration to initiate an interaction. While prior research has proven that dwell selection is a fast method for initiating an interaction with a target [ 31 ], unintentional prolonged fixations of the eye on the target are not accounted for, leading to miss-interactions, also known as the Midas Touch problem. To address this, prior research introduced a technique that adapts the dwell activation threshold based on a probabilistic system. Specifically, the system predicts the probability of the next target using gaze features, then adjusts the threshold to a shorter or longer duration based on high or low probability, respectively [ 30 ]. Another proposed solution to the Midas Touch problem is a machine learning-based intent prediction model [ 26 ], where the user’s intention to interact with an object is predicted based on gaze features, scaling the available dwell time. However, developing a model to predict user intent based on gaze features, past interactions, and past errors is too dependent on unique and variable gaze patterns of each user and user’s individual interaction strategies within the same task, leading to a biased model. This limitation is particularly pronounced in prior work which fails to incorporate the visual context of the scene, such as the size, depth, location, and density of neighboring objects. Further, the visual context can cause discrepancies between a user’s intent and their observable gaze behavior due to the induced cognitive load, which could lead to false predictions and, inadvertently, to miss-interactions. In this paper, we propose a context-aware machine learningbased model for adapting dwell thresholds for gaze-based selections in XR environments. Our model leverages visual context and behavioral features to determine the perceived cognitive load of the scene and dynamically adapt the fixation threshold needed to perform an interaction. We conducted a data collection experiment with 20 participants performing gaze dwell interactions in two different tasks, a User Interface (UI) navigation task where the user interacts with various UI buttons and a visual search task where the user needs to search for a specific target and perform an interaction. During the experiment, we collected behavioral and visual context features tied to inducing cognitive load. For each interaction, the user provided feedback for the dwell threshold in three levels: Same, Shorter and Longer, forming the ground truth. We trained a Random Forest classifier on the collected data and labels in a hierarchical manner to predict the dwell threshold needed by the user based on the context of the scene. To evaluate the trained model, we performed a standard machine learning analysis utilizing robust metrics such as accuracy, f1-score and confusion matrices to ensure that no overfitting is present. We also conducted a user study with 17 participants, to determine the effects of our system to error rate and task completion time with a quantitativelyand qualitatively-based evaluation. Our specific contributions include: • A schema for estimating cognitive load in the scene, leveraging visual context features and behavioral patterns of the user, that could also be generalized to other scenarios and users • A machine learning-based processing pipeline for classifying features and adapting gaze dwell duration into three distinct dwell thresholds based on the inferred cognitive load of the scene • A rigorous evaluation and analysis investigating the performance of our model and its impact on task efficiency and miss-interaction rate when performing gaze dwell interactions in XR environments 2 Related Work In this section, we include an overview of cognitive load in immersive environments, analyze past research in gaze-based selection as well as past techniques for adapting dwell thresholds. 2.1 Gaze-based Selection In XR environments, user input modalities are crucial for effective interaction. Gaze-based selection, defined by the use of a user’s visual focus to indicate an intended target, has become a prominent area of investigation [ 7 ] due to its hands-free nature and potential for rapid target acquisition. Past work [ 22 ] has explored various gaze interaction techniques focusing on rapid activations, being unobtrusive while performing a task to minimize cognitive effort and reliable to avoid unintentional interactions. Another focus of an effective gaze interaction technique is minimizing task completion time to boost the overall efficiency of a system [ 8 ]. The most common gaze interaction techniques that aim to minimize false interactions and improve task efficiency are dwell, dwell combined with blinks and gaze gestures [ 34 ]. With dwell activation the user needs to fixate on the intended target for a predefined set of time to trigger the interaction. Gaze interactions with dwell has seen usage in XR applications in the medical field [ 21 ], providing ALS patients with interactive capabilities, and in learning environments [ 29 ]. A 2-stage dwell-based gaze interactions has also been used in past research [ 12 ], where the user dwells on a target for a specific threshold and then fixates on a different confirmatory target with a dwell threshold to execute the interaction. Dwell combined with blinks is a two-stage approach where the user fixates on a target and performs an eye blink as a confirmatory input to perform an interaction. Providing a confirm action with blinks minimizes miss-interactions and avoids the Midas Touch Problem. Another approach is gaze gesture for CONTEXT-GAD: A Context-Aware Gaze Adaptive Dwell model for Gaze-based Selections in XR Environments VRST ’25, November 12–14, 2025, Montreal, QC, Canada spontaneous and pervasive gaze interactions due to its spatial accuracy [ 43 ]. Gaze gestures utilize gaze strokes based on the relative position of the gaze to form a distinct gesture that corresponds to specific commands. In [ 24 ], natural eye movements, that occur in moving targets intended for activation, were used in a smooth pursuit-based interaction scheme for robust object selection. The authors proposed a hidden Markov model to classify the user’s eye movement, supporting precise object selection. However, having a constant fixation threshold for dwell interactions are prone to false activations of objects that require a long fixation to adequately understand the functions of the target. Dwell combined with blinks is less prone to the Midas Touch problem due to the confirmatory action, but can lead to eye fatigue due to repetitive blinking. Moreover, blinks are a natural physiological response where involuntary blinks might be classified as confirmation for interaction with an object. Gaze gestures is a complex form of an interaction and users need training to learn and memorize the specific gaze patterns to invoke interactions. In this paper, we propose an adaptive dwell method that addresses the limitations of constant dwell thresholds and gaze gestures by utilizing a machine learning model that dynamically adjusts the dwell time based on a combination of the visual context related to the task and behavioral features such as gaze speed variance, frequency of interactions and head rotation velocity. 2.2 Adaptive Dwell While fixed dwell-based selection offers a hands-free interaction method in XR, its inherent static nature often leads to a tradeoff between selection speed and the Midas touch problem [ 6 ]. To address these limitations and enhance user experience, research has been conducted to explore adaptive dwell interactions. Eye typing in a keyboard is common scenario where users are prone to false keystrokes with dwell. Past research [ 13 ] has explored dynamic dwell adjustment based on the user’s number of past mistakes and the average selection time, leading to a smaller error rate. Mott et al. [ 23 ] proposed an adaptive dwell time strategy by cascading the dwell thresholds of keys when pressed, slowly decreasing the minimum allowable dwell time as a user enters text, leading to higher input rate. In [ 32 ], a Bayesian probabilistic model was developed using the user’s past input history to predict the probabilities of the next likely keystrokes. The proposed model achieved 41.8% faster typing for able-bodied subjects. Further, Mutasim et al. [ 25 ], proposed the Multi-Threshold Dwell (MTD) system where they adjusted dwell thresholds for consecutive selections of the same key, while keeping a constant threshold for the backspace key and adjusted the threshold for the space key based on repeated selections. Further, when eye typing a word, the dwell thresholds of the most probable subsequent letters in a word were reduced to increase input rate. The proposed MTD system increased typing accuracy and speed in novice users. In general target selection tasks, adaptive dwell systems aim to provide the user with enough dwell time to properly understand the functions of the target. In [ 28 ], an action-value solution was proposed to emulate mouse click events on buttons by adjusting dwell time values based on the user’s prior usage experience of the given button within the application and Midas touch characteristics for the given button. Further, reinforcement learning was utilized [ 7 ], to predict the number of fixations and duration required to make a gaze-based selection. The trained model can also be adapted to target size and width, as well as to noise in the motor and vision systems for more robust target selection. In [ 9 ], a logistic regression model was trained on eye tracking data from 15 participants performing item selection tasks. The model achieved a 0.76 Area Under the Curve (AUC) score in predicting the user’s intent to interact from gaze-only features like gaze velocity and saccadic dynamics. Machine learning-based intention recognition was also proposed by [ 16 ] and [ 26 ], where a Light Gradient-Boosting Machine (LightGBM) and a Long Short-Term Memory (LSTM)-based neural network were trained, respectively, on gaze-only data to predict the intent of the user when dwelling on a target and confirm selection. Both models boast high AUC ( = 0 . 90) and F1 ( = 0 . 94) scores, respectively, proving that intent recognition is feasible when having a large and high quality dataset. Other approaches for adaptive dwell thresholds in general target selection involve training a machine learning model to classify gaze features and action context outcomes to predict dwell selection as intentional or spontaneous [ 39 ]. In uncertain predictions of the model, the user had to perform a prolonged dwell to confirm target selection. Isomoto et al. [ 17 ], proposed a model human processor that derives dwell times from the number of fixations the user performed on the target object and the duration of those fixations. However, the majority of research on adaptive dwell times has been validated within the context of text entry on virtual keyboards. While this is a critical application, it represents a highly structured and repetitive task. The applicability and effectiveness of these adaptation strategies in more dynamic and less predictable interaction scenarios remain largely unexplored. Furthermore, past works that utilize gaze-only features to predict user intent for target selection often result in highly biased systems. This bias emerges because such models are overly dependent on the specific gaze patterns and behavioral strategies reflected on the training data, which limits their ability to generalize to new users or different scenarios. Building on these observations, this work presents a novel adaptive dwell strategy that integrates visual and behavioral context to mitigate the user-specific biases of gaze-only models. This approach is validated on a UI navigation and a visual search task, demonstrating its broader applicability. 2.3 Cognitive Load Cognitive load estimation and the parameters that drive cognitive load in an environment, has been a well-researched topic over the years. Specifically, past work [ 10 ] has shown the analogous relationship between the visual complexity of an environment, such as the number of elements shown to the user, and the induced cognitive load, hindering task performance when the user is overwhelmed with visual elements. Reis et al. [ 36 ] also supports the notion of having a reduced graphical interface with presenting a positive effect on the cognitive demands of the system, thus increasing task performance. In [ 42 ], the authors investigated the effects of a visually cluttered scene on object recognition and conscious perception performed by a human, through the concept of Visual Crowding. VRST ’25, November 12–14, 2025, Montreal, QC, Canada George Ramiotis and Katerina Mania Table 1: Description of collected features Feature Name Description Units Objects The number of objects in the scene Numerical ObjectsFOV The number of objects in the Field of View (FOV) of the user Numerical ObjectDensity The density of the objects in the scene objects m−2 DynamicObjects The number of dynamic objects Numerical NearestNeighborDistance The distance of the nearest neighbor to the target m InteractFreq The frequency of gaze interactions interactions min−1 GazeSpeedVariance The variance of gaze speeds ◦2s−2 HeadRotVel The rotational velocity of the head ◦s−2 AngularSize The angular size of the target ◦ LabelComp The complexity of the target label CEFR score Results state that the eccentricity of a target, i.e. how easily discernible a target is in a crowd of objects, depends on how densely spaced the surrounding objects are. Thus, in a densely populated environment, targets are becoming less recognizable. Furthermore, when moving objects are present in the environment, the target is harder to be discerned from the distractions. This phenomenon also occurs in VR environments, where in presence of a densely populated environment the human eye displays a more random distribution in space, thus reporting a higher cognitive load, as demonstrated in [ 15 ]. Another factor that is attributed to high cognitive load present in an immersive environment, is the frequency of errors and the miss-interaction rate performed by a user [ 40 ], indicating a high interaction frequency to complete the task. Gaze-based metrics have also contributed greatly in cognitive load estimation, where individual eye movements can exhibit more chaotic patterns in presence of a cognitive intensive environment. More specifically, Pillai et al. [ 33 ] leveraged gaze features such as the entropy in eye movements, the nearest neighbor index, the entropy in gaze transitions and pupil size to detect cognitive load levels in a driving simulator. Pupil size and generally, pupil response [ 4 , 5 , 18 , 41 , 44 ] along with blink rate, saccadic movements and fixations [ 3 , 19 , 38 ], have been widely used in eye tracking research to detect cognitive load. Results from these studies display a common pattern, where erratic eye movements and increased entropy in gaze positions relate to increased cognitive load in both non-immersive and immersive environments. In general UI navigation tasks in XR environments, text labels often accompany UI buttons to indicate the function that will be executed upon activation. In prior research [ 1 , 3 ], manipulating string length has been proven to modulate different levels of cognitive load. Furthermore, the cognitive processing difficulty of a word in a sentence has been documented to be determined by the uncertainty that word causes in a sentence [ 11 ]. Thus, it can be concluded that the frequency and the length of a word based on task context can increase the cognitive processing demands of a word, leading to increased cognitive load. 3 Methodology In this section we will provide a detailed description of the apparatus and the procedure used for data collection, processing of contextual and behavioral features and training the classifier for adapting dwell thresholds. 3.1 Apparatus The experiment was conducted on a Microsoft HoloLens 2 headmounted display (HMD), which provided gaze data via its integrated eye tracker and displayed content on its 2k per-eye waveguide display (52°diagonal FOV). Experimental scenes were developed in the Unity Engine using the Mixed Reality Toolkit (MRTK3) to interface with the eye tracker. The corresponding machine learning model was trained and executed on a desktop PC (Intel Core i9-10900 CPU, 32 GB RAM, NVIDIA GeForce RTX 3070 GPU). During the experiment, the HoloLens 2 application streamed extracted features to the PC, which performed real-time inference and transmitted the resulting adaptive dwell times back to the HMD via a TCP socket. 3.2 Data Collection We conducted a data collection experiment with 20 participants (15 male, 5 female), in the age range of 18-44. Most of the participants had no experience with VR/AR headsets (n=9), with some reporting little (n=5) and average (n=2) experience with XR headsets, while four reported expert knowledge. Eight participants reported corrected-to-normal vision. In the experiment, participants performed gaze-based dwell interactions in a general UI navigation task, interacting with buttons; and a visual search task, where the user needed to find the appropriate target from a pool of objects randomly placed in the virtual space and activate it via dwell. After each dwell interaction the user was presented with three dwell choices: Same, Shorter and Longer, and were asked to provide feedback on the duration of their interactions. If the system provided enough time for the interaction the ’Same’ option was chosen. If the interaction could be performed quicker then the ’Shorter’ option was chosen and the ’Longer’ option when there was not CONTEXT-GAD: A Context-Aware Gaze Adaptive Dwell model for Gaze-based Selections in XR Environments VRST ’25, November 12–14, 2025, Montreal, QC, Canada Figure 2: An illustration of the visual search task, displaying several spheres with their unique numerical identifiers. The image highlights three distinct interaction states: ’Selected’ (green sphere), ’Highlighted’ (red sphere), and ’Default state’ (blue spheres). Each sphere features colored dots along its perimeter enough time to understand the button’s functions or a false activation of a button was occurred. To determine the optimal default dwell threshold, we conducted a pilot study with 5 participants (2 experts, 3 novices). Participants completed both a navigation and a visual search task and were asked to evaluate which of two undisclosed dwell thresholds (1.0s or 1.5s) was more suitable for each interaction. The majority of participants preferred the 1.5s threshold for its perceived balance between efficiency and ease of use across both tasks, which we then adopted as the default. Throughout the experiment, participants provided feedback on the chosen threshold during all interactions. 3.2.1 Navigation Task. In the navigation task inspired by [ 37 ], the participants were presented with a UI menu (Figure 1) featuring gaze interactable buttons. The participants could interact with each button to spawn a different menu. Subsequently, the menu featured more interactable buttons that activated nested menus and panels with general information about the scene. The participants were asked to explore the interactable buttons and understand their functions. The task finished when the user has interacted with all buttons. This task aims to emulate general UI navigation tasks in XR interfaces that feature a more strict structure but with the cognitive demands generated by multiple buttons with nested menus, submenus and panels. This task also featured dynamic objects that performed an animation when interacted with. 3.2.2 Visual Search Task. In the visual search task inspired by [ 35 ], the participants needed to fixate on a white cross that appeared in front of them, indicating the scene’s point of reference. Then, ten spheres spawned at random locations in front of the user (Figure 2). For each user the spheres spawned at different locations to minimize bias. Spheres spawned at a maximum horizontal angle of 36 ◦ and a vertical angle of 30 . 39 ◦ from the point of reference. Figure 3: Hierarchical classifier structure for adaptive dwell times The angular size of a sphere could be at maximum 10 . 55 ◦ and at minimum 6 . 12 ◦ . Sphere positions could permit obstruction from other spheres to increase complexity. Each sphere featured colored dots contrasting the color of the sphere, and were positioned at its 2D perimeter. Each sphere featured a random number of dots ranging from five to fourteen to add levels of complexity. Further, the dots were positioned at random positions at the perimeter of the sphere. An instruction was presented at the user, pinpointing a unique target sphere for interaction based on the number of dots on the perimeter. After the sphere was activated, new random sphere locations were chosen with a different number of spheres, dots and a new target, repeating the process. The task was finished until the user completed ten trials. 3.3 Feature Extraction We designed a feature schema based on the visual context of the scene and the cognitive demands induced by the task. This feature set (Table 1) was based on the reviewed literature on the properties of cognitive load. For the visual context-related features we accessed Unity’s API to calculate the number of objects in the scene and in the FOV of the user, with the criteria being that the object was subject to physics (i.e collisions and kinematic movement) and whether the objects were placed within the viewport of the HMD, respectively. From the identified objects we calculated the density of the scene. Further, we tracked the number of dynamic objects in the scene by the presence of either an Animator or RadialView component (MRTK). When the user starts dwelling on a target, the nearest neighbor to that target is identified and the distance between the target and the nearest neighbor is computed. Both the density of the scene and the distance to the nearest neighbor contribute to the crowding factor of the scene, thus increasing cognitive load. The angular size of the target is also calculated, since smaller target sizes exhibit higher cognitive demands. VRST ’25, November 12–14, 2025, Montreal, QC, Canada George Ramiotis and Katerina Mania (a) Original multiclass problem (b) Hierarchical first level (Same vs Not Same) (c) Hierarchical second level (Shorter vs Longer) Figure 4: t-SNE Projection of Features Table 2: Evaluation metrics for the first level machine learning architectures Architecture Accuracy Precision Recall F1-scores AUC Baseline 57.38 - - - - SVM-RBF 65.77 67.05 65.77 62.55 67.66 XGBOOST 68.82 68.69 68.82 68.74 73.31 MLP-2 65.77 65.47 65.77 64.44 66.67 Random Forest 70.72 70.48 70.72 70.36 73.98 To capture the inherent unpredictable user behavior imposed by a cognitive intensive scene, we considered the variance of gaze angular speed as a feature. We calculate the variance from samples of the last second on the integrated eye tracker with a 30 𝐻𝑧 sampling rate. Moreover, we keep track of the frequency of interactions in the last minute as an indicative measure of cognitive load. We compute the rotational speed of the head, since higher rotational speed can indicate a more complex scene, thus increasing cognitive efforts to identify the intended target. Finally, text labels of target objects could indicate higher cognitive effort in understanding the functions of the target, based on the uncertainty of the label related to the context of the scene. We capture the uncertainty of a text label based on the frequency of the words extracted from a public database with word frequencies of the english vocabulary. We then compute the mean log frequencies of all words and map the value to a complexity score using the Common European Framework of Reference (CEFR) [ 27 ] levels, which provide a standardized method for classifying language proficiency from beginner (A1) to advanced (C2). This method assumes that texts with higher average word frequency are less complex and correspond to a lower CEFR level, while those with lower average word frequency (more rare words) are more complex and correspond to a higher CEFR level. The resulting dataset from twenty participants consists of a feature vector with a shape (1385 samples, 10 features) for the class labels: ’Same’, ’Shorter’, ’Longer’. 3.4 Classification As a preliminary step to model development, t-SNE plots were generated to visualize the feature space for the original multiclass problem and the two levels of our hierarchical classifier. Figure Table 3: Evaluation metrics for the second level machine learning architectures Architecture Accuracy Precision Recall F1-scores AUC Baseline 52.25 - - - - SVM-RBF 83.44 84.27 83.44 83.41 87.21 XGBOOST 82.78 83.74 82.78 82.73 90.33 MLP-2 83.44 84.27 83.44 83.41 87.04 Random Forest 85.43 86.14 85.43 85.41 88.42 4a shows the distribution of the three classes in the original problem, revealing a highly overlapping and non-separable structure. In contrast, the t-SNE plots for the first and second levels of the hierarchical classifier (Figures 4b and 4c) show a much clearer separation between the binary classes. To classify the extracted gaze features, a two-level hierarchical classifier (Figure 3) was implemented, leveraging the ensemble learning capabilities of the Random Forest algorithm. At the initial level, features associated with ’Shorter’ and ’Longer’ categories were aggregated into a unified representation signifying an adaptive dwell threshold, while features derived from ’Same’ feature vector characterized the non-adaptive dwell threshold. This first-level classifier, a binary Random Forest configured with 𝑛_𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑜𝑟𝑠 = 200 and a balanced 𝑐𝑙𝑎𝑠𝑠_𝑤𝑒𝑖𝑔ℎ𝑡 , was implemented to discriminate between these adaptive and non-adaptive states based on the inferred cognitive load, thereby predicting whether the user required an alternate dwell threshold or if the system should maintain the default dwell time. The second-level classifier was conditionally activated only when the first stage determined the necessity for an alternate dwell time, and its training dataset consequently comprised features exclusively from the ’Shorter’ and ’Longer’ categories. This subsequent classifier also utilized a Random Forest model, configured with identical hyperparameters to its preceding level. Prior to classification, a crucial pre-processing step involved the application of the Local Outlier Factor (LOF) algorithm to the extracted features, effectively identifying and removing anomalous data points to enhance model robustness. Subsequently, all features underwent Standard Scaling, a critical normalization procedure that ensures consistent data representation. This scaling was applied uniformly across the feature sets designated for both the initial CONTEXT-GAD: A Context-Aware Gaze Adaptive Dwell model for Gaze-based Selections in XR Environments VRST ’25, November 12–14, 2025, Montreal, QC, Canada Figure 5: Quantitative results for each scene with each system (a) First level classifier (b) Second level classifier Figure 6: Confusion matrices of our classifier from all levels ’Same’ versus ’Not Same’ classification problem and the subsequent ’Shorter’ versus ’Longer’ adaptive gaze problem, thereby standardizing feature distributions for optimal classifier performance. In our investigation into classifying contextual and behavioral features, the adoption of a hierarchical classifier, comprising two sequential binary classifiers, offers distinct advantages over a monolithic multiclass approach. This architecture inherently decomposes the complex three-class problem (’Same’, ’Shorter’, ’Longer’) into more manageable sub-problems: an initial distinction between ’Same’ and ’Not Same’ patterns, followed by a granular classification of ’Not Same’ instances into ’Shorter’ or ’Longer’. This decomposition not only simplifies the learning task for each individual classifier, leading to enhanced predictive performance by allowing each stage to focus on specific decision boundaries, but also significantly improves model interpretability by pinpointing at which level a miss-classification occurs. 4 User Studies This study follows Helsinki Declaration principles (1975/2000). Informed consent was obtained, and ethical safeguards were implemented despite no institutional approval requirement. 4.1 Participants We conducted a user study with 17 participants (12 male, 5 female), in the age range of 18-54. Seven participants reported limited experience with VR/AR, seven participants reported average experience, and three participants were experts in the field. Four participants reported corrected-to-normal vision. 4.2 Procedure The user study incorporated the navigation and visual search tasks previously used in the data collection experiment. Our system was evaluated against a comparable static dwell interaction system, which employed the same threshold as the ’Same’ category. We implemented a within-subjects design, wherein participants completed both tasks using both systems. To prevent bias and balance potential order effects, a Latin square design was applied to the sequence of scene presentation and system exposure. Participant awareness of the system being used was eliminated through the consistent use of generic labels, ’System A’ and ’System B’, which alternated for each participant. For the navigation task, each participant was presented with a unique sequence of UI buttons, all of which were of a consistent predetermined length. Participants were required to follow and interact with each button in the given order, and the task concluded upon the successful completion of their specific sequence. During the task, we measured task completion time and the number of miss-interactions to calculate the error rate. Additionally, after experiencing each scene and system, participants completed two questionnaires: the System Usability Scale (SUS) to rate the usability and intuitiveness of both the adaptive and static dwell systems on 5-point Likert scale (1= Strongly Disagree, 5= Strongly Agree), and the NASA Task Load Index (NASA-TLX) to assess their perceived interaction workload on a 10-point Likert scale (1= Very Low, 10= Very High). 5 Results In this section, we present the results of our study, beginning with the task performance metrics such as task completion time and missinteraction rate. We then report on the results of the subjective VRST ’25, November 12–14, 2025, Montreal, QC, Canada George Ramiotis and Katerina Mania (a) First level classifier (b) Second level classifier Figure 7: ROC curves of our classifier from all levels questionnaires, including the statistical analysis of the System Usability Scale (SUS) [ 20 ] and NASA-Task Load Index (TLX) [ 14 ] data, to provide a comprehensive understanding of the user experience. 5.1 Quantitative The performance of our system was evaluated on standard machine learning metrics (Tables 2, 3) such as accuracy, precision, recall, F1scores and AUC scores for both levels of the hierarchical classifier. Additionally, we compare the performance of our Random Forest model to other architectures, including: Support Vector Machines (SVM), eXtreme Gradient Boosting (XGBOOST) and a Multi-Layer Perceptron (MLP) with two layers along with the baseline accuracy (majority class). Overall, Random Forest achieved the highest accuracy and F1-scores on all levels of the hierarchical classifier, displaying a 70.7% accuracy in predicting whether the user needs the default dwell time or not based on the cognitive demands of the scene. In the second level, the binary classifier is able to predict if the user needs a shorter or longer dwell time with a 85.43% accuracy. Thus, the hierarchical classifier is able to achieve a strict accuracy of 60.41%. Training a Random Forest model in the original multiclass problem with three classes results in an accuracy of 58.55%. Thus, dividing the three class problem into two binary problems yields higher classification accuracy. Further, we plot the confusion matrices (Figure 6) and the ROC curves (Figure 7) of our classifier to ensure that no overfitting is present. Further analysis of the confusion matrices reveals the practical impact of the classification errors. In the first level, a false negative (failing to adjust dwell time when needed) is a more critical error than a false positive (unnecessarily changing dwell time). The model’s confusion matrix shows a conservative bias towards generating more false positives, which minimizes the risk of user frustration from a non-responsive system. In the second level, the distribution of false positives and false negatives is more balanced, indicating the model’s robust ability to distinguish between shorter and longer dwell times with minimal bias, crucial for accurate adaptation to a user’s cognitive state. Task efficiency was measured in terms of task completion time and miss-interaction rate. To determine the appropriate statistical tests, we first assessed the normality of our data using the ShapiroWilk test. The results indicated that the distributions for task completion time for the Static and Adaptive systems in the navigation task were approximately normal ( 𝑝= 0 . 43 and 𝑝= 0 . 09, respectively), as was the Static system in the visual search task ( 𝑝= 0 . 30). However, the data for the Adaptive system in the visual search task was not normally distributed ( 𝑊= 0 . 796 , 𝑝 = 0 . 002). Similarly, the miss-interaction rate data for all conditions was not normally distributed (Static system in navigation: 𝑊= 0 . 450 , 𝑝 < 0 . 001; Adaptive system in navigation: 𝑊= 0 . 262 , 𝑝 < 0 . 001; Static system in visual search: 𝑊= 0 . 913 , 𝑝 = 0 . 11; Adaptive system in visual search: 𝑊= 0 . 889 , 𝑝 = 0 . 04). Given these findings, we performed a non-parametric analysis using Friedman’s test, followed by posthoc Wilcoxon signed-rank tests with a Bonferroni correction. For task completion time, a Friedman test revealed a statistically significant overall effect across all conditions ( 𝜒2( 3 )= 43 . 80 , 𝑝 < 0 . 001). Post-hoc comparisons showed a significant reduction in task completion time for the Adaptive system in the navigation task ( 𝑊= 17 . 00 , 𝑝𝑐𝑜𝑟𝑟𝑒𝑐𝑡𝑒𝑑 = 0 . 006), but no significant difference was found in the visual search task (𝑊=45.00, 𝑝𝑐𝑜𝑟𝑟𝑒𝑐𝑡𝑒𝑑 =0.29). Regarding miss-interaction rate, the Friedman test also indicated a significant overall effect ( 𝜒2( 3 )= 43 . 17 , 𝑝 < 0 . 001). Post-hoc analysis showed no significant difference between the Static and Adaptive systems in the navigation task ( 𝑊= 2 . 00 , 𝑝𝑐𝑜𝑟𝑟𝑒𝑐𝑡𝑒𝑑 = 0 . 51). However, the Adaptive system resulted in a significantly lower miss-interaction rate compared to the Static system in the more demanding visual search task ( 𝑊= 10 . 00 , 𝑝𝑐𝑜𝑟𝑟𝑒𝑐𝑡𝑒𝑑 = 0 . 009). Overall, the Adaptive system demonstrates a nuanced advantage. While it significantly improves task completion speed in less cognitively demanding scenarios like the navigation task, its most pronounced benefit is in significantly reducing miss-interaction rates under higher task demands, as observed in the visual search task, without significantly increasing task completion time. CONTEXT-GAD: A Context-Aware Gaze Adaptive Dwell model for Gaze-based Selections in XR Environments VRST ’25, November 12–14, 2025, Montreal, QC, Canada 5.2 Qualitative User experience was evaluated using the System Usability Scale (SUS) and NASA Task Load Index (TLX). As with the quantitative data, normality was assessed using the Shapiro-Wilk test. The results for the SUS scores indicated that the distributions for the Static system in navigation ( 𝑊= 0 . 939 , 𝑝 = 0 . 30) and Adaptive system in visual search (𝑊=0.900, 𝑝 =0.06) were approximately normal, while the Adaptive system in navigation ( 𝑊= 0 . 798 , 𝑝 = 0 . 002) and Static system in visual search ( 𝑊= 0 . 887 , 𝑝 = 0 . 04) were not. The TLX scores were similarly non-normal across conditions (Static system in navigation: 𝑊= 0 . 897 , 𝑝 = 0 . 06; Adaptive system in navigation: 𝑊= 0 . 749 , 𝑝 < 0 . 001; Static system in visual search: 𝑊= 0 . 948 , 𝑝 = 0 . 43; Adaptive system in visual search: 𝑊= 0 . 837 , 𝑝 = 0 . 007). Given these findings, a non-parametric Friedman test was conducted, followed by post-hoc Wilcoxon signedrank tests with a Bonferroni correction. For the SUS scores, the Friedman test revealed a statistically significant overall difference among the four experimental conditions ( 𝜒2( 3 )= 8 . 519 , 𝑝 = 0 . 03). Post-hoc analysis revealed that there was no significant difference in usability between the Static and Adaptive systems in the navigation task ( 𝑝𝑐𝑜𝑟𝑟𝑒𝑐𝑡𝑒𝑑 = 1 . 00), while the Adaptive system was rated as significantly more usable in the more demanding visual search task ( 𝑝𝑐𝑜𝑟𝑟𝑒𝑐𝑡𝑒𝑑 = 0 . 02). This pattern suggests that the adaptive features of the system offered a tangible advantage to users specifically when faced with a task of higher complexity. For the NASA-TLX workload scores, a significant overall difference was also found by the Friedman test ( 𝜒2( 3 )= 13 . 098 , 𝑝 = 0 . 004). However, the post-hoc tests clarified that there was no significant difference in workload between the Static and Adaptive systems in either the navigation task ( 𝑝𝑐𝑜𝑟𝑟𝑒𝑐𝑡𝑒𝑑 = 1 . 00) or the visual search task ( 𝑝𝑐𝑜𝑟𝑟𝑒𝑐𝑡𝑒𝑑 = 0 . 14). This indicates that the overall significance was driven by the inherent difference in difficulty between the two scenes themselves, rather than an effect of the system type. Critically, this finding implies that the significant usability improvements provided by the adaptive system were achieved without imposing any additional perceived cognitive load on the user. 6 Limitations & Future Work The apparatus used in this study introduces several inherent constraints that are important to acknowledge. The experiment was conducted exclusively on the Microsoft HoloLens 2 head-mounted display. As such, the findings are specific to the characteristics of this device, including its waveguide display technology, integrated eye tracker, and a 52°diagonal field of view (FOV). This constrained FOV may influence user gaze behavior and could affect the generalizability of our model’s performance to devices with wider fields of view. Another valid consideration is that the tasks used in this study were relatively simple, potentially not fully reflecting the complexity of real-world XR applications. While the experiments were conducted using a UI navigation task and a visual search task, these were chosen as core tasks of more complex interactions. For example, an emergency responder in a training simulation might perform a visual search for a tool while simultaneously using a UI to navigate a building’s floor plan. The central premise of this work was to develop a model on these core tasks with the intent that it could generalize to such more complex, composite scenarios. Future research should therefore focus on evaluating the model’s performance on these combined, more complex tasks to validate this generalizability. Additionally, future work can extend the model’s applicability by incorporating other core tasks that were not part of this study, such as object manipulation or text entry, to assess its adaptability to a wider range of user intentions and cognitive demands. 7 Conclusion This work introduced a novel context-aware adaptive gaze dwell system, designed to overcome static dwell threshold limitations in XR by dynamically adapting thresholds based on predicted cognitive load. Quantitative evaluation of our system demonstrated a nuanced advantage for the Adaptive system. It significantly reduced task completion time in the navigation task while its most significant impact was on the miss-interaction rate. In the demanding visual search task, the Adaptive system significantly lowered missinteraction rates without significantly increasing task completion time, directly addressing the Midas Touch problem. Qualitative measures supported these benefits. System Usability Scale (SUS) scores revealed the Adaptive system was perceived as substantially more usable in the visual search task. Critically, NASA-TLX analyses clarified that these usability improvements were achieved without imposing any additional perceived cognitive load on the user, as no significant difference in workload was found between the two systems. The experimental design involved testing the system within a navigation task and a visual search task. Future work could incorporate novel and more demanding scenarios, to further validate our system. While in the literature review we argued against an adaptive dwell system using only gaze features, we expect that such a system can be fused with our proposed methodology by scaling our adaptive dwell times to lower or higher thresholds based on individual gaze characteristics such as fixation duration, saccade velocity and number of blinks. This will result in a personalized system that adapts dwell times based on both the cognitive demands of the scene and the individual physiological characteristics of the user. In summary, the proposed adaptive gaze dwell system effectively addresses the Midas Touch Problem, enhancing hands-free XR interactions. By dynamically adjusting the dwell threshold, it improves task completion efficiency and, critically, significantly boosts interaction accuracy without additional perceived workload and frustration in complex visual search tasks. Acknowledgments This work has received financial support by the Horizon Europe Research & Innovation Programme under Grant agreement N.101092612 (Social and hUman ceNtered XR - SUN project). Views and opinions expressed in this work are those of the author(s) only and do not necessarily reflect those of the European Union. Neither the European Union nor the European Commission can be held responsible for them.