Grasping moving objects through human-like motions

Grasping moving objects through human-like motions Marco Baracca1,∗, Gianmarco Cei1, Giulia Pagnanelli1, Matteo Bianchi1. Abstract—Dynamic grasping of objects in motion is a key challenge in robotics manipulation for enabling efficient robotic manipulation in both daily-living and industrial-like scenarios. However, to go toward a better human-robot interaction, the behaviour of the manipulator also has to be taken into account, ensuring, for example, the human-likeness in the motion produced for an increased perceived safety. To this aim, in this work, we present a method which exploits a human-like motion planning algorithm to grasp moving objects. We tested the framework by grasping objects from a conveyor belt showing promising results both in terms of success rate and humanlikeness. Index Terms—Human-like motion generation, Dynamic grasping, Motion planning and control I. INTRODUCTION Grasping is one of the basic actions which permit us to perform complex tasks and interact with the environment. For this reason, object manipulation with robotic systems generates huge interest in research groups worldwide [1], [2]. Impactaware manipulation is one of these hot topics in this research branch in the last period. In fact, in the real world, there are several situations where humans need to handle dynamic tasks. Among these, the dynamic grasping of moving objects has shown great interest in the robotic community for a large range of applications not only in real life but also in industrial setups where objects can be moved in different places of a warehouse using conveyor belts. Furthermore, handling relative motions between the robot and the target during grasping could also be beneficial for mobile robots, which could perform the task without stopping. Historically, one of the most used techniques to solve this problem is visual servo control [3], where the information provided by a camera is used as feedback to control robot motion and reach the desired target. Over the years, several other approaches were developed to handle more complex scenarios [4], [5]. These approaches target only the task of object grasping without giving any guarantees on the motion behavior of the robotic arm. However, to build a system capable of working beside humans it is necessary to guarantee a predictable and acceptable behavior. In this sense, generating humanlike motion can sensibly improve the human-robot interaction [6]. However, dealing with the problem of ensuring humanlikeness in combination with the time constraint given by the task makes the problem not trivial. Furthermore, we live in a world designed for humans: under this regard, the human inspiration, for what concerns both the end-effector design and the primitives used for grasping, should also be considered. In this work, we exploit a human-like motion planning algorithm, based on the information extracted through functional This work was supported by European Union’s Horizon 2020 Research and Innovation Program under Grant Agreement No. 101017274 (DARKO); the Italian Ministry of Education and Research (MIUR) in the framework of the CrossLab and FoReLab project (Departments of Excellence); 1 are with Centro di Ricerca “Enrico Piaggio” and with Dipartimento di Ingegneria dell’Informazione, Universita‘ di Pisa; *Corresponding Author: [email protected] Fig. 1. Experimental Setup used for testing. Principal Component Analysis (fPCA) from human motion examples [7], as a base to build a framework capable of grasping moving objects. Leveraging on the low computational time (less than 0.01s in generic scenarios), the proposed algorithm can recompute a new trajectory to correct the reaching motion with respect to the motion state estimation of the object. To validate this approach, we proposed as testing scenario the grasping of objects placed on a conveyor belt moving at constant unknown velocity. II. PROPOSED FRAMEWORK The proposed framework is composed of two main blocks: a perception block, which provides information about the motion state of the object to the system, and a motion control block, which is in charge of controlling the motion of the manipulator given the target information. The perception part is based on the RGB-D camera. The point cloud is pre-processed to extract the part related to the object. Then, we obtained the position of the object by computing the centroid of the point cloud, and we applied the Principal Component Analysis for computing the principal direction of the object and properly placing the gripper. The computed centroid is then fed into a Kalman Filter to estimate the velocity of the target. For motion control, the core is the fPCA-based human-like motion planning algorithm. The idea is to exploit fPCA to extract the main characteristics of human upper limb motions and embed them in a motion planning algorithm. Here, we briefly recall the basis of this approach while we refer the interested reader to [7] for more details about it. Taking into account the single DoF, the fPCA permits the reconstruction of a generic motion x(t)as a weighted sum of functional Principal Components (fPCs) previously extracted analysing a dataset of recorded movement as x(t)≈¯x+S0(t) + smax X i=1 αiSi(t),(1) 2025 I-RIM Conference October 17-19, Rome, Italy ISBN: 9788894580570 10.5281/zenodo.17629808 179 Fig. 2. Snapshot of one of the trials performed during the validation tests. where ¯xdenotes the average pose of the hand, S0(t)is the average trajectory observed over all trajectories in the dataset, αiis the weight associated to the i-th basis element Si(t)and smax is the number of components used. The peculiarity of the basis of functions obtained through fPCA is that is ordered in terms of the explained variance that each element accounts for. In this way, the minimum number of element can be used to achieve a certain level of representation of the original dataset. Starting from this result, if we have a set of constraints for the desired trajectory (for example initial and final position and velocity), we can define an equation system to find the coefficients ¯xand αias    1S1(t0)S2(t0)S3(t0) 1S1(tf)S3(tf)S3(tf) 0˙ S1(t0)˙ S2(t0)˙ S3(t0) 0˙ S1(tf)˙ S2(tf)˙ S3(tf)       ¯x α1 α2 α3   =   x(t0)−S0(t0) x(tf)−S0(tf) ˙x(t0)−˙ S0(t0) ˙x(tf)−˙ S0(tf)    .(2) With the obtained weights, the desired trajectory can be computed by exploiting the weighted sum defined in the fPCA formulation to compute the desired motion as x(t) = ¯x+S0(t) + 3 X i=1 αiSi(t).(3) This framework is integrated with a replanning strategy which, taking into account the position and velocity estimated by the Kalman filter, adapts the planned trajectory to reach the target. In a nutshell, if during the motion the distance between the predicted final position of the object and the final point of the trajectory is higher than a certain threshold, the replanning is triggered, computing a new trajectory starting from the actual pose of the manipulator and ending in the predicted final pose of the object. III. EXPERIMENTAL VALIDATION The experimental setup comprises a 7 DoF Franka Emika manipulator equipped with a SoftHand as gripper, a conveyor belt and an RGB-D camera for the perception of the object (Fig. 1). The relative pose between the camera and the robot was calibrated through a custom procedure exploring the AprilTag library. The experimental validation consists of evaluating the success rate in grasping moving objects. To do this, we tested different objects and different velocities of the conveyor belt. In Fig. 2 we reported a set of snapshot of one of the trials performed. The grasp strategy implemented with the SoftHand is a top grasp choosing the orientation of the hand in a way that the principal dimension of the object identified through the PCA lies between the thumb and the index finger. We performed 117 trials obtaining an overall success rate of 76%. We also evaluated the success rate obtained at different velocities of the conveyor belt. We used three different velocities (0.16m/s,0.26m/s and 0.38m/s). As expected, the success rate decreased with the increasing of the velocity, obtaining respectively 95%, 91% and 52% for the tested velocities. This is due to the assumption of not knowing the velocity of the conveyor belt. In this scenario, the estimation of the Kalman filter has to converge to the right velocity estimation. Higher velocities give less time to the filter to converge and, consequently, the planner has less margin to adjust the trajectory and intercept the target. However, we tested as comparison the same framework without the replanning strategy, obtaining a success rate of 81% with the low velocity and 31% with the medium velocity (the higher velocity was not tested extensively due to not succeeding in any grasp in the preliminary setup of the validation scenario). These results prove the effectiveness of the replanning strategy proposed. In terms of the human-likeness of the motion produced, we evaluated the jerk of the motion of the manipulator. Across the several tests, we obtained a mean jerk of 0.4·10−3m/s3, which is consistent with other results obtained in the literature regarding the human-likeness. IV. CONCLUSION AND FUTURE WORKS In this work, we presented a framework for grasping objects in motion exploiting human-like motions. We tested it in grasping objects from a conveyor belt, showing promising results both in terms of success rate and human-likeness. For future steps, the implementation of more complex grasping strategies for handling different types of objects is crucial to increase the robustness of the system. REFERENCES [1] A. Bicchi and V. Kumar, “Robotic grasping and contact: A review,” in Proceedings 2000 ICRA. Millennium conference. IEEE international conference on robotics and automation. Symposia proceedings (Cat. No. 00CH37065), vol. 1. IEEE, 2000, pp. 348–353. [2] J. Bohg, A. Morales, T. Asfour, and D. Kragic, “Data-driven grasp synthesis—a survey,” IEEE Transactions on robotics, vol. 30, no. 2, pp. 289–309, 2013. [3] S. Hutchinson, G. D. Hager, and P. I. Corke, “A tutorial on visual servo control,” IEEE transactions on robotics and automation, vol. 12, no. 5, pp. 651–670, 1996. [4] N. Marturi, M. Kopicki, A. Rastegarpanah, V. Rajasekaran, M. Adjigble, R. Stolkin, A. Leonardis, and Y. Bekiroglu, “Dynamic grasp and trajectory planning for moving objects,” Autonomous Robots, vol. 43, pp. 1241– 1256, 2019. [5] S. S. M. Salehian, M. Khoramshahi, and A. Billard, “A dynamical system approach for softly catching a flying object: Theory and experiment,” IEEE Transactions on Robotics, vol. 32, no. 2, pp. 462–471, 2016. [6] A. M. Zanchettin, L. Bascetta, and P. Rocco, “Acceptability of robotic manipulators in shared working environments through human-like redundancy resolution,” Applied ergonomics, vol. 44, no. 6, pp. 982–989, 2013. [7] M. Baracca, G. Averta, and M. Bianchi, “A general approach for generating artificial human-like motions from functional components of human upper limb movements,” Control Engineering Practice, vol. 148, p. 105968, 2024. 180