Vision-Based Real-Time Estimation of Surgical Tissues Deformation from Video

Vision-Based Real-Time Estimation of Surgical Tissues Deformation from Video Gabriele Furnari Department of Sciences and Methods of Engineering / University of Modena and Reggio Emilia Italy [email protected] Jasper Hofman Orsi Innotech Orsi Academy Belgium Pieter De Backer Orsi Innotech Orsi Academy Belgium Federica Ferraguti Department of Sciences and Methods of Engineering University of Modena and Reggio Emilia Italy Abstract—Accurate modeling of organ deformation during surgery is crucial for intraoperative guidance, augmented reality overlays, and robotic assistance. We present a vision-based framework for estimating non-rigid deformation of kidney and tumor structures directly from stereo surgical video. Our method integrates depth reconstruction, tool-organ interaction analysis, and dense 3D motion estimation to deform preoperative meshes of the organs. Both local tool deformation and global organ motion are computed, while a restorative process models elastic recovery of tissues. We demonstrate the ability to continuously update a planned surgical cutting path to reflect intraoperative deformations, supporting adaptive and safe guide and/or execution. Index Terms—Tissue deformation, surgical robotics, surgical navigation I. INTRODUCTION Soft tissue deformation during surgery is a major challenge for robotic systems and surgical navigation. In particular, kidney surgery (e.g., laparoscopic or robot-assisted partial nephrectomy) involves significant non-rigid motion of organs and tumors due to tool interaction, organ manipulation, and intraoperative changes in pressure. To maintain accuracy, surgical guidance systems must adapt to these deformations in real time. Several recent works have addressed this challenge from different perspectives. [1]–[5]. While these methods demonstrate significant progress in organ deformation tracking, they also have limitations: monocular or point-cloud based approaches suffer from depth ambiguity and noise, learning-based methods depend on large training datasets, and registration-focused pipelines often assume rigid or quasi-rigid motion. Crucially, few approaches directly deform preoperative meshes and focus mostly on surface-level tracking without leveraging preoperative CT geometry. In contrast, preoperative CT-derived meshes of the kidney and tumor provide anatomically accurate and stable geometry, forming a reliable basis for surgical planning. By continuously deforming these meshes according to intraoperative observations, preoperative planning information (e.g., tumor margins and cutting paths) can be updated in real time. In this work, we present a video-based approach to estimate organ deformation directly from surgical RGBD sequences. Our contributions are: •A real-time pipeline for detecting tool–organ interactions and transferring deformation to preoperative CT meshes. •Integration of both tool-induced local deformation and organ-wide non-contact deformation, using RGBD optical flow and mesh projection. •Elastic recovery modeling to simulate tissue relaxation, and continuous update of surgical cutting paths defined on the CT meshes. II. METHODS A. Deformation Estimation Surgical video frames are processed using deep learningbased methods for segmentation of organs and instruments. Depth estimation produces frame-wise point clouds. Contact regions are identified by spatial proximity and overlap of tool and organ. From this moment the deformation process starts. We establish pixel-to-mesh correspondences by rasterizing the organ and tool meshes onto the image plane using the calibrated stereo camera parameters. During rasterization, each pixel covered by a triangular mesh face is expressed in terms of the face vertices via barycentric coordinates. This provides a mapping from 2D image pixels to 3D mesh vertices. Optical flow is then computed in the RGBD domain between consecutive frames, exploiting both intensity and depth information. Each pixel thus provides a 3D displacement vector (∆u, ∆v, ∆z). Using the barycentric weights obtained during rasterization, this displacement is distributed to the corresponding mesh vertices using barycentric interpolation. By accumulating contributions across all pixels, we obtain a consistent 3D displacement field defined directly on the mesh vertices. 2025 I-RIM Conference October 17-19, Rome, Italy ISBN: 9788894580570 10.5281/zenodo.17629750 121 This mechanism is applied to both tool and organ surfaces: •Tool-induced deformation: Tool motion, transferred through barycentric mapping, displaces vertices in the contact region of the organ mesh. •Organ-wise deformation: Optical flow of organ surface regions is similarly transferred, ensuring global mesh deformation. When tool interaction is not detected anymore, a restorative elasticity model gradually returns the deformed mesh regions toward their anatomical configuration. B. Cutting Path Deformation The surgical cutting path is preoperatively defined on the kidney mesh as the boundary of the tumor region with additional safety margins. During surgery, as the kidney and tumor meshes deform under tool interaction, the cutting path evolves consistently with the underlying surface geometry. Specifically, each vertex vi∈ C is displaced according to the deformation field of the organ mesh obtained via the RGBD optical flow transfer. This ensures that the surgical path remains anatomically valid and continuously updated in real time. As a result, intraoperative guidance can rely on a dynamically evolving cutting trajectory that reflects the actual state of the tissue, bridging the gap between preoperative planning and intraoperative execution. Fig. 1. Proposed pipeline: stereo video →segmentation and depth →tool– organ interaction →deformation transfer →optical flow →updated organs mesh and cutting path →mesh restoration. III. RESULTS AND APPLICATIONS Experiments on surgical video sequences demonstrate realistic and temporally coherent deformation of kidney and tumor meshes. The surgical cutting path defined on the preoperative model is dynamically updated to reflect intraoperative changes, unabling robot motion planning with updated tissue geometry and augmented reality overlays aligned to the true organ state as guide for the surgeon. In addition to qualitative visualization, we assessed the accuracy of the deformation using image-based metrics. The deformed mesh was projected onto the image plane via rasterization, enabling direct comparison with the observed video frames. •Silhouette error: We compared the projected organ contour against the ground-truth segmentation masks. The average silhouette discrepancy was 0.0001. •Flow consistency: Optical flow was reprojected from the mesh back into the image plane and compared with the observed RGB optical flow. The error was consistently below 0.5pixels, showing strong temporal coherence between mesh deformation and actual image motion. •Depth error: The projected depth values of the mesh were compared with stereo-reconstructed depth maps. The mean absolute depth difference was 2mm. These results demonstrate that the proposed method achieves both accurate and temporally consistent modeling of intraoperative deformation, suitable for surgical guidance applications. IV. CONCLUSION We present a video-based method for estimating organ deformation during surgery. By combining tool–organ interaction analysis, optical flow, our approach enables realtime tracking of kidney and tumor deformation. Future work includes learning-based appraoch and integration into autonomous robotic systems. ACKNOWLEDGMENT The research leading to these results has received funding from the Italian Ministry of University and Research under the D.D. 1236 (August, 1 2023), Fondo italiano per la scienza - Bando FIS 2 (TRAMIS, CUP E53C24003820001) REFERENCES [1] E. Wang, Y. Liu, P. Tu, Z.A. Taylor, and X. Chen, “Video-Based Soft Tissue Deformation Tracking for Laparoscopic Augmented RealityBased Navigation in Kidney Surgery,” *IEEE Trans. on Medical Imaging*, vol. 43, no. 12, pp. 4161-4173, Dec. 2024. :contentReference[oaicite:5]index=5 [2] Z. Liu et al., “Surface Deformation Tracking in Monocular Laparoscopic Video,” *Medical Image Analysis*, 2023. :contentReference[oaicite:6]index=6 [3] P. Henrich, J. Liu, J. Ge, S. Schmidgall, L. Shepard, A. Ezzat Ghazi, F. Mathis-Ullrich, A. Krieger, “Tracking Tumors under Deformation from Partial Point Clouds using Occupancy Networks,” arXiv preprint arXiv:2411.02619, Nov. 2024. :contentReference[oaicite:7]index=7 [4] M. Pfeiffer, C. Riediger, J. Weitz, S. Speidel, “Learning Soft Tissue Behavior of Organs for Surgical Navigation with Convolutional Neural Networks,” arXiv preprint arXiv:1904.00722, 2019. :contentReference[oaicite:8]index=8 [5] A. Acar et al., “Towards Navigation in Endoscopic Kidney Surgery Based on Preoperative CT Segmentation and Video,” *[Conference / Journal]*, 2023. :contentReference[oaicite:9]index=9 122