Grey-Box Adversarial Patch for Velocity Deception in Monocular ORB-SLAM3

Grey-Box Adversarial Patch for Velocity Deception in Monocular ORB-SLAM3 Jiheon Kim College of Space and Aeronautics Gyeongsang National University Jinju-si, Republic of Korea Email: [email protected] Abstract—We propose a grey-box physical-style adversarial patch that deceives the velocity estimation of classical monocular ORB-SLAM3 without modifying its source code. Instead of relying on differentiable surrogate models or loop-closure corruption, we explicitly target the ORB feature matching layer and bias it towards a larger fraction of almost-stationary but high-confidence correspondences between consecutive frames. To this end, we define a fitness score that rewards ORB matches with low Hamming distance and small pixel displacement under a simulated forward motion (zoom-in), and optimize the patch using a lightweight evolutionary algorithm (EA). On the EuRoC MAV V1_01 sequence with unmodified C++ ORB-SLAM3, our final patch increases ATE (RMSE) from 0.0876 m to 1.7717 m (about 20×), shortens the reconstructed trajectory length by 29%, and induces long intervals of near-zero estimated velocity with spiky artifacts. We release a reproducible pipeline based on OpenCV-Python patch evolution and C++ ORB-SLAM3 evaluation, and discuss system-level implications for feedback control and simple defenses based on match–motion consistency. Index Terms—Visual SLAM security, adversarial patch, ORBSLAM3, visual odometry, evolutionary optimization. I. INTRODUCTION Despite the rapid progress of learned dense SLAM and deep visual odometry, classical feature-based VO/SLAM pipelines remain a backbone for many robotic systems. Systems such as ORB-SLAM3 [1] still provide state-of-the-art accuracy and robustness in real-world deployments thanks to FAST corners [4], BRIEF-style binary descriptors [3], and RANSACbased geometric estimation [7]. Adversarial robustness of visual perception has mostly been studied for deep end-to-end models, often in the context of image classification [11], [15]–[17] or learned VO/SLAM. Recent work has begun to explore adversarial attacks on visual SLAM and VO [8]–[10], but these efforts typically focus on neural components, differentiable renderers, or loop-closure and map corruption. In contrast, classical VO-only settings, where control decisions are made from incremental motion estimates without explicit loops, remain under-explored from a security viewpoint. Key insight. If inter-frame feature matching is biased such that many accepted correspondences are high-confidence yet almost stationary in the image plane, then a geometric VO backend will tend to estimate very small translations and, consequently, low velocities. Crucially, this can be achieved without directly attacking the backend optimizer or gradientdescent machinery—it is sufficient to manipulate the matching statistics seen by a standard RANSAC-based motion estimator. Contribution. We make the following contributions: •We formulate a grey-box adversarial patch attack against monocular ORB-SLAM3, targeting its ORB descriptor and Hamming-distance matching layer. The attack does not require gradients, access to internal states, or any modification of the C++ implementation. •We design a fitness score that explicitly rewards lowdisplacement, low-Hamming-distance ORB matches under a synthetic zoom-in transformation, and optimize the patch using a simple evolutionary algorithm. •We demonstrate, on EuRoC MAV V1_01 [5] with unmodified ORB-SLAM3, that the final patch induces severe velocity collapse and large trajectory errors while keeping the tracker nominally running. •We analyze system-level hazards from a feedback-control perspective and discuss lightweight defenses based on match–motion consistency checks and sensor crossvalidation. II. RELATED WORK A. Classical Feature-Based VO/SLAM ORB [2] combines FAST [4] keypoint detection with oriented BRIEF-like binary descriptors [3] to provide efficient, rotation-robust features whose similarity can be measured by Hamming distance. ORB-SLAM3 [1] extends ORB-SLAM2 with visual-inertial fusion, multi-map support, and fisheye cameras, and remains a widely used baseline in robotics. EuRoC MAV [5] is a standard dataset for evaluating VO/SLAM and VIO systems, containing stereo and IMU measurements from a quadrotor in indoor environments. Trajectory evaluation commonly relies on Umeyama alignment [6] and ATE metrics, with RANSAC [7] used to robustify geometric estimation. B. Adversarial Patches and Physical Attacks Adversarial patches [11], [15]–[17] demonstrate that small, localized patterns can reliably fool deep networks in the physical world. Subsequent work has extended patches to universal and cross-modal settings [12], and to camera-based perception stacks in autonomous driving. Attack Scope (Grey-box) Patch Overlay (Pfinal) ORB-SLAM3 Frontend Velocity Deception (vest ≈0) PID Controller (e(t)↑) Control Hazard (u(t)→amax ) Raw Image Attacked Img False Matches State Est. Cmd Fig. 1. System overview of the proposed kinematic-illusion attack. The diagram illustrates the flow from the adversarial patch overlay to the control hazard. For VO/SLAM, Nemcovsky et al. [9] design physical passive patches that degrade learned monocular visual odometry, while Chen et al. [8] introduce AoR, an unnoticeable patch attack on visual SLAM with temporal consistency. Chawla et al. [10] study adversarial attacks on monocular pose estimation networks. Our work is complementary: we do not attack a learned VO module, but rather the classical ORB matching layer of ORB-SLAM3, leaving its code untouched. C. SLAM Security and Control-Level Vulnerabilities Beyond perception, Dirty Road [18] shows that delay-based vulnerabilities in autonomous driving control stacks can lead to unsafe behavior even when perception remains nominal. Scaramuzza and Fraundorfer [19] highlight the sensitivity of VO pipelines to feature distributions and motion assumptions. Our attack bridges these perspectives by showing that a simple patch can induce persistent velocity collapse in a feature-based VO system, which in turn can saturate a downstream PID controller. III. THREAT MODEL Target System: We target monocular ORB-SLAM3 [1] running with default parameters on EuRoC V1_01. The attack assumes no modification of the ORB-SLAM3 C++ code, configuration, or compiler flags. Adversary Capabilities: The adversary can place or overlay a 2D patch within the camera field of view. In this work we evaluate a digital overlay, but the design is compatible with a printed physical patch. We assume grey-box knowledge: the adversary knows that ORB+BRIEF descriptors and brute-force Hamming matching with cross-check are used, but has no access to gradients, internal states, or real-time control signals. Adversary Goal: The primary goal is to systematically underestimate the platform’s translational velocity by biasing the statistics of accepted ORB matches toward small pixel displacements with low Hamming distances. We do not explicitly attempt to cause tracking failure or map divergence; instead, we aim for long intervals where vest ≈0while the platform is actually moving. Out of Scope: We do not model: (i) stereo or visualinertial configurations; (ii) realistic physical-print effects and expectation-over-transformation tuning; (iii) end-to-end closed-loop hardware experiments. We restrict ourselves to off-line evaluation on EuRoC frames with digital overlay, and conceptual analysis of control hazards. IV. ATTACK METHOD A. ORB Matching as an Attack Surface In monocular ORB-SLAM3, each incoming frame is processed by detecting FAST corners and computing 256-bit ORB descriptors. Brute-force (BF) matching with Hamming distance and cross-check selects candidate correspondences between consecutive frames. After basic ratio tests and outlier rejection, a motion model, followed by RANSAC-based pose estimation, uses these matches to estimate the relative camera motion. Let Mt={mi}denote the set of accepted matches between frames tand t+1. For each match mwe can compute: •Hamming distance dH(m)between the binary descriptors. •Pixel displacement ∆p(m) = ∥pt+1 −pt∥2between keypoint locations. Intuitively, matches with small ∆pcorrespond to little apparent motion. If a significant fraction of accepted matches accumulate near ∆p≈0while maintaining low dH, the backend is encouraged to infer small translations. B. Virtual Forward Motion (Zoom-In) Simulation We emulate forward motion without a 3D simulator by applying a simple zoom-in transform to the clean frame. Given an original image I, we resize it by a scale factor α > 1, then centrally crop back to the original resolution using bilinear interpolation. This yields a pair (It, It+1)where It+1 approximates a forward motion of the camera. The same transform is applied to images containing the patch. This allows us to evaluate the match statistics induced by a candidate patch under a simple yet repeatable motion model. C. Fitness Score: Rewarding Stationary False Matches For a given candidate patch Pand transformation (It, It+1), we compute ORB keypoints and descriptors with OpenCV, perform BF matching with Hamming distance and crosscheck, and collect the set Mt(P). Fig. 2. Fitness score evolution over generations. Best fitness value per generation for the evolutionary algorithm optimizing Pfinal. We then define the fitness score as F(P) = X m∈Mt(P) wd(dH(m)) wp(∆p(m)),(1) where wdrewards small Hamming distance (e.g., an exponentially decaying function) and wprewards small pixel displacement (e.g., a window around ∆p < 3px). Effectively, F counts how many stationary-looking, high-confidence matches the patch induces. This fitness is differentiable in neither the ORB extraction nor the patch pixels, but it can be evaluated cheaply for any candidate patch using OpenCV. D. Evolutionary Optimization of the Patch We parameterize the patch as an RGB image of size Hp×Wp. To encourage high-contrast, corner-rich patterns, we initialize each individual on a white canvas with multiple black rectangles of random sizes and locations. Our evolutionary algorithm maintains a population of 20 individuals and iterates for 100 generations: •Selection: We retain the top 10% individuals (elite) and select parents from the top 50% according to F. •Crossover: We form offspring by vertically or horizontally concatenating patches from two parents. •Mutation: With probability 0.05, we replace random 10× 10 pixel blocks with new random RGB values. •Evaluation: Each candidate is overlaid at a fixed location (bottom-center, covering 30% of image height) and scored via Fusing the virtual zoom-in transformation. Fig. 2 shows the evolution of the best fitness over generations. We observe a typical EA pattern with occasional large jumps, indicating discovery of substantially better local patterns. E. Deployment to ORB-SLAM3 The final patch Pfinal is placed in the bottom-center of each EuRoC frame at 30% of the image height and rendered TABLE I EFFECT OF PFINAL ON MONOCULAR ORB-SLAM3 (EUROC V1_01). Metric Baseline Attacked Change ATE (RMSE) [m] 0.0876 1.7717 +20.2× Scale s2.069 1.768 −14.5% Trajectory length [m] 28.97 20.51 −29.2% # stationary matches – 124 – with nearest-neighbor scaling. We generate an attacked image sequence by digitally overlaying Pfinal using Python, then feed the original and attacked sequences into the unmodified ORBSLAM3 binary. From ORB-SLAM3, we collect pose logs for the baseline and attacked runs. We align both trajectories to the ground truth using Umeyama’s method [6], then compute ATE (RMSE), scale, trajectory length, and velocity profiles. V. EXPERIMENTAL SETUP Dataset: We use the EuRoC MAV V1_01 sequence [5], which provides monocular images and accurate ground-truth poses for a MAV flying in a machine hall. Software: Patch evolution, image overlay, and visualization are implemented in Python 3.10 using OpenCV, NumPy, Matplotlib, and related libraries. ORB-SLAM3 is built from the official C++ repository [1] with default monocular settings. Metrics: After trajectory alignment, we report: •Absolute Trajectory Error (ATE, RMSE), including global scale. •Estimated scale factor s. •Reconstructed trajectory length. •Velocity magnitude time series. •Number of stationary matches with dH<64 and ∆p < 3px in representative frames. Reproducibility: The full pipeline consists of: 1) evolve.py →generates Pfinal and fitness log. 2) create_attacked_images.py →overlays the patch onto EuRoC frames. 3) ORB-SLAM3 execution (baseline/attacked) →pose logs. 4) align_and_plot.py,plot_velocity.py → trajectory and velocity plots. 5) visualize_attack_mechanism.py →visualizes false matches. VI. RESULTS A. Quantitative Impact on EuRoC V1_01 Table I summarizes the impact of the patch on ORBSLAM3. The attack increases ATE by over an order of magnitude and shortens the reconstructed trajectory by nearly one third, indicating a systematic underestimation of motion. The global scale factor also changes moderately, but the dominant effect is the shape and length of the trajectory. Fig. 3. Aligned top-down trajectories on EuRoC V1_01. Ground truth (blue), baseline ORB-SLAM3 (green dashed), and attacked trajectory with Pfinal (red) after Umeyama alignment. B. Trajectory Deformation Fig. 3 compares top-down trajectories for ground truth, baseline ORB-SLAM3, and the attacked run after Umeyama alignment. The baseline closely tracks ground truth, while the attacked trajectory remains confined to a much smaller region near the starting point, reflecting motion underestimation. C. Velocity Collapse Fig. 4 plots the velocity magnitude over time for ground truth, baseline, and attacked runs. The attacked estimate shows long intervals where vest is nearly zero, punctuated by occasional spikes. From a control perspective, such persistent underestimation of velocity can be more dangerous than a catastrophic tracking failure, since downstream controllers may continue to trust the bogus state estimates. D. False Stationary Matches Fig. 5 visualizes ORB matches satisfying dH<64 and ∆p<3px under the attack. The correspondences concentrate densely around the patch region, forming a cluster of almost stationary matches that anchor the motion estimator. Fig. 6 shows the final learned patch. Although it consists of simple high-contrast rectangles, it successfully generates many stable, easy-to-match keypoints. VII. ABLATION AND SENSITIVITY We briefly summarize ablation experiments carried out in simulation. Patch Size: We vary the patch height from 10% to 30% of the image height. Larger patches systematically yield stronger attacks, with the default 30% setting providing the most stable velocity collapse. Patch Position: We test left, center, and right placements in the lower part of the image. The bottom-center placement proves most effective and stable, likely due to typical camera motion and scene layout. Number of Generations: Training for 100 generations improves the fitness by about 15% compared to 50 generations and extends the duration of velocity-collapse intervals. Beyond 100 generations, we observe diminishing returns. VIII. CONTROL-LEVEL IMPLICATIONS Our attack does not tamper with the control code itself. However, consider a simple longitudinal PID controller that attempts to track a target velocity vtarget using the estimated velocity vest from VO: e(t)=vtarget(t)−vest(t), u(t) = PID(e(t)),(2) where u(t)is a commanded acceleration subject to saturation |u(t)| ≤ amax. Under our attack, vest(t)≈0while the vehicle continues to move. Thus e(t)remains large and positive, driving the controller towards u(t)→amax. If the real vehicle is already moving at moderate speed, this can significantly extend the required braking distance and increase collision risk. Importantly, this hazard arises even if the trajectory remains approximately consistent and loop closures succeed: the controller is misled by a biased kinematic estimate rather than an obvious localization failure. IX. LIMITATIONS AND DEFENSES A. Limitations Our study has several limitations: •We only consider digital overlays on a single EuRoC sequence with a monocular configuration. Physical printing effects, lighting changes, and motion blur are not modeled. •The attack is tuned for a particular camera viewpoint and patch placement. Generalization across trajectories and cameras is left to future work. •We focus on ORB-SLAM3; other feature-based systems (e.g., VINS-Mono [13], DynaSLAM [14]) may react differently. B. Lightweight Defenses Despite the simplicity of the attack, several lightweight defenses could mitigate it: Match–Motion Consistency Monitoring: Flag situations where the fraction of small-displacement matches (e.g., ∆p < 3px) increases sharply while the estimated direction diversity decreases. Such anomalies can indicate artificially stationary patterns. Descriptor Sampling Jitter: Randomizing ORB/BRIEF sampling patterns across frames can reduce the stability of patch-induced binary descriptors, making them harder to exploit. Cross-Sensor Validation: When IMU or wheel odometry is available, reject pure VO solutions that imply persistent near-zero velocity despite inertial cues indicating motion. Match Hygiene: Strengthen ratio tests, enforce crosschecking, and penalize very dense local clusters of matches belonging to a small image region (e.g., around a patch-sized area). Fig. 4. Velocity magnitude comparison over time. Ground truth (blue), baseline ORB-SLAM3 (green), and attacked estimate (red). Under the patch attack, ORB-SLAM3 reports near-zero velocity for long intervals with sporadic spikes, despite continuous motion in the ground truth. Fig. 5. Mechanism of the kinematic illusion: dense stationary ORB matches. Fig. 6. Final evolved adversarial patch Pfinal. X. ETHICS AND RESPONSIBLE DISCLOSURE All experiments are conducted offline on a public dataset. Our primary goal is to highlight a realistic, interpretable t Value vtarget vest ≈0 Large Error e(t) u(t)→amax Gap Attack Start Fig. 7. PID hazard: velocity collapse causes saturated control. vulnerability in classical VO/SLAM stacks and to motivate the development of robust defenses and cross-checks. Any released code will include appropriate warnings and is intended solely for research and defense evaluation. XI. CONCLUSION We have shown that a simple grey-box adversarial patch, optimized via a lightweight evolutionary algorithm to maximize stationary, low-Hamming-distance ORB matches, can induce severe velocity underestimation in unmodified monocular ORB-SLAM3. On EuRoC V1_01, our patch increases ATE by about 20×, shortens the reconstructed trajectory by 29%, and produces extended intervals of near-zero estimated velocity. The attack is transparent and reproducible, and highlights the need for match–motion consistency checks and sensor crossvalidation even in classical, non-learning-based VO systems. In future work, we plan to extend the attack to physical patches, stereo and visual-inertial configurations, and to explore joint attacker–defender optimization schemes. REFERENCES [1] C. Campos, R. Elvira, J. J. G. Rodriguez, J. M. M. Montiel, and J. D. Tardós, “ORB-SLAM3: An accurate open-source library for visual, visual-inertial and multi-map SLAM,” IEEE Trans. Robot., vol. 37, no. 6, pp. 1874–1890, 2021. [2] E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, “ORB: An efficient alternative to SIFT or SURF,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), 2011, pp. 2564–2571. [3] M. Calonder, V. Lepetit, C. Strecha, and P. Fua, “BRIEF: Binary robust independent elementary features,” in Proc. ECCV, 2010, pp. 778–792. [4] E. Rosten and T. Drummond, “Machine learning for high-speed corner detection,” in Proc. ECCV, 2006, pp. 430–443. [5] M. Burri et al., “The EuRoC micro aerial vehicle datasets,” Int. J. Robot. Res., vol. 35, no. 10, pp. 1157–1163, 2016. [6] S. Umeyama, “Least-squares estimation of transformation parameters between two point patterns,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 13, no. 4, pp. 376–380, 1991. [7] M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM, vol. 24, no. 6, pp. 381–395, 1981. [8] B. Chen, W. Wang, P. Sikorski, and T. Zhu, “Adversary is on the road: Attacks on visual SLAM using unnoticeable adversarial patch,” in Proc. USENIX Security, 2024. [9] Y. Nemcovsky, M. Jacoby, A. M. Bronstein, and C. Baskin, “Physical passive patch adversarial attacks on visual odometry systems,” in Proc. ACCV, 2022, pp. 518–534. [10] H. Chawla, A. Varma, E. Arani, and B. Zonooz, “Adversarial attacks on monocular pose estimation,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots and Syst. (IROS), 2022, pp. 12500–12505. [11] T. B. Brown, D. Mané, A. Roy, M. Abadi, and J. Gilmer, “Adversarial patch,” in NeurIPS Workshop on Machine Learning and Computer Security, 2017. [Online]. Available: arXiv:1712.09665. [12] X. Wei, Y. Huang, Y. Sun, and J. Yu, “Unified adversarial patch for cross-modal attacks in the physical world,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2023. [13] T. Qin, P. Li, and S. Shen, “VINS-Mono: A robust and versatile monocular visual-inertial state estimator,” IEEE Trans. Robot., vol. 34, no. 4, pp. 1004–1020, 2018. [14] B. Bescos, J. M. Facil, J. Civera, and J. Neira, “DynaSLAM: Tracking, mapping and inpainting in dynamic scenes,” IEEE Robot. Autom. Lett., vol. 3, no. 4, pp. 4076–4083, 2018. [15] J. Su, D. V. Vargas, and K. Sakurai, “One pixel attack for fooling deep neural networks,” IEEE Trans. Evol. Comput., vol. 23, no. 5, pp. 828– 841, 2019. [16] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2015. [17] K. Eykholt et al., “Robust physical-world attacks on deep learning visual classification,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2018, pp. 1625–1634. [18] T. Sato, J. Shen, N. Wang, Y. Jia, X. Lin, and Q. A. Chen, “Dirty road can attack: Security of deep learning based automated lane centering under physical-world attack,” in Proc. USENIX Security, 2021. [19] D. Scaramuzza and F. Fraundorfer, “Visual odometry: Part I – The first 30 years and fundamentals,” IEEE Robot. Autom. Mag., vol. 18, no. 4, pp. 80–92, 2011.