AdvGrasp: Adversarial Attacks on Robotic Grasping from a Physical Perspective

1 Introduction

Grasping serves as a fundamental mechanism for robots to interact with the physical world [Bicchi and Kumar, 2000; Lin et al., 2022], enabling a wide range of applications from industrial automation [Cutkosky and others, 1989] to domestic services [Matheus and Dollar, 2010]. As a critical component of many safety-critical systems, even minor errors in robotic grasping can lead to severe and unpredictable consequences. Recent studies have shown that robotic grasping systems are vulnerable to various threats [Yaacoub et al., 2022], including adversarial attacks [Szegedy et al., 2014], where imperceptible perturbations to input data can cause erroneous decisions. Understanding and addressing these threats, including adversarial attacks, is essential for improving the robustness and reliability of robotic grasping systems.

Despite these developments, research on adversarial attacks in robotic grasping remains relatively limited. Alharthi and Brand˜ao [2024] extended adversarial attack techniques from image-based classification neural networks to imagebased grasp quality networks, generating adversarial examples that reduced predicted grasp quality. However, these attacks focus solely on exploiting vulnerabilities in grasp quality networks without addressing the physical grasping process directly. Fundamentally, this approach targets a neural network rather than the physical act of grasping. One limitation is that not all robotic grasping systems rely on neural networks for grasp quality evaluation, which limits the applicability of such attacks. Moreover, even in systems that do use these networks, their instability under adversarial conditions highlights the unreliability of their predictions. Namely, successfully attacking a grasp quality network does not necessarily translate to disrupting the physical grasp itself, see Fig. 1. This highlights the importance of developing adversarial attack strategies that directly target the grasping process, accounting for the physical interactions and challenges inherent in robotic grasping.

∗Yunbo Zhao and Keke Tang are co-corresponding authors.

Indeed, robotic grasping is a fundamentally complex physical process rather than a straightforward end-to-end task. It requires intricate interactions involving forces, torques, and friction, which are difficult to model solely with neural networks. Furthermore, a successful grasp involves multiple stages: overcoming gravity to lift the object and maintaining stability throughout transportation, often under the influence of external disturbances such as vibrations, collisions, or unexpected forces. These challenges underscore the limitations of adversarial attacks that fail to consider the physical realities of grasping and instead focus exclusively on neural network vulnerabilities. This raises a critical question: can adversarial attacks on robotic grasping be systematically formulated from a physical perspective, capturing these essential interactions and processes?

To address these challenges, we devise AdvGrasp, a systematic adversarial attack framework for robotic grasping from a physical perspective. Specifically, AdvGrasp focuses on two critical aspects of grasp performance: lift capability, which evaluates the gripper's ability to overcome gravity, and grasp stability, which measures its capacity to resist external disturbances. By subtly deforming the object's shape to increase the gravitational torque and reduce the stability margin in the wrench space, our method disrupts the grasp wrench equilibrium, reducing the physical effectiveness of grasping. To validate our approach, we introduce AdvGrasp-20, a comprehensive benchmark featuring 20 groups of objects with diverse shapes and multiple grasp configurations generated by both traditional and neural network-based methods for common two- and three-finger robotic grippers. Extensive experiments across various settings on this benchmark demonstrate the effectiveness of AdvGrasp. Besides, we also validate its performance in real-world scenarios to further confirm its practical applicability in physical robotic systems.

Overall, our contribution is summarized as follows:

• We present AdvGrasp-20, a benchmark with diverse object shapes and grasp configurations to standardize the evaluation of adversarial attacks on grasping.
• We propose a physical-aware adversarial attack framework for robotic grasping that systematically targets two critical metrics: lift capability and grasp stability.
• We validate the proposed framework through extensive experiments and real-world evaluations, demonstrating its effectiveness in degrading grasp performance.

2 RelatedWork

2.1 Security Issues in Robotics

As robotic systems are applied more broadly in diverse applications [Chen et al., 2015; Siciliano et al., 2008], their interconnected modules, such as communication networks, perception systems, and control mechanisms, have increasingly become targets for various attacks [Yaacoub et al., 2022]. Network security threats, such as data injection and denialof-service (DoS), have been shown to compromise localization and disrupt operations [Guerrero-Higueras et al., 2018]. Similarly, tampering with sensor data or control signals in human-robot collaboration can lead to unsafe behaviors or task failures [Meng and Weitschat, 2021; Amaya-Mej´ıa et al., 2022]. Operating systems like the Robot Operating System (ROS) are also vulnerable. Dieber et al. [2017] identified key weaknesses in ROS, while Mazzeo and Staffa [2020] proposed methods to mitigate privileged-access attacks. Although robotic security has been widely studied, adversarial attacks specifically targeting robotic grasping remain largely unexplored. This paper aims to bridge this gap by systematically investigating such attacks.

2.2 Adversarial Attacks on Robotic Grasping

Adversarial attacks aim to generate perturbed examples that mislead neural networks into making incorrect predictions. Since the introduction of adversarial examples by Szegedy et al. [2014], these attacks have been extensively studied in computer vision [Goodfellow et al., 2015; Dong et al., 2018; Carlini and Wagner, 2017; Moosavi-Dezfooli et al., 2016; Tang et al., 2024c] and extended to domains such as 3D point cloud perception [Xiang et al., 2019; Tang et al., 2022; Tang et al., 2023; Tang et al., 2024a; Tang et al., 2024b; Tang et al., 2024d; Tang et al., 2025a; Tang et al., 2025b; Wang et al., 2025], natural language processing [Morris et al., 2020], and audio processing [Zheng et al., 2021]. Recently, robotic grasping has also been investigated within the context of adversarial attacks.

Alharthi and Brand˜ao [2024] extended adversarial attack techniques from image-based classification neural networks to image-based grasp quality networks. Their work demonstrated adversarial examples in robotic grasping by modifying the intensity of a single pixel or physically placing a barely visible spherical object. However, this approach focuses on attacking the grasp quality network itself, rather than addressing the grasping process as a whole. Grasping is inherently a complex physical interaction involving factors such as forces, torques, and friction. Therefore, effective adversarial attacks on robotic grasping must account for these physical characteristics to accurately capture the challenges of the task.

Wang et al. [2019] incorporated certain physical properties of grasping in their study, but their objective was to generate objects that are universally difficult to grasp from any angle. This objective significantly increases the complexity of optimization and often leads to adversarial objects with unrealistic and conspicuous deformations. In contrast, our work focuses on attacking specific grasps of an object by systematically investigating adversarial attacks from a physical perspective, while ensuring that the generated perturbations remain realistic and physically plausible.

3 Problem Statement

3.1 Preliminaries

Formally, a 3D object O is represented as a triangle mesh defined by a set of triangular faces T (O) and vertices V(O). A successful grasp on O is denoted as GSP = {{ci}, {fi}}, where {ci} are the contact points, and {fi} are the forces exerted by the robotic fingers.

At each contact point ci, the force fi can be decomposed into two components:

where f ⊥ i is the normal component, and f t i is the tangential component. According to the Coulomb's law [Mason et al., 1989], the tangential force is constrained by the normal force:

where µ is the coefficient of friction.

3.2 Stable Condition of AGrasp

Wrench. A wrench w ∈ R1×6 is a vector that encapsulates both force and torque, defined as:

where f ∈ R1×3 represents the force vector, and τ ∈ R1×3 denotes the torque vector acting on the object.

Grasp Wrench Equilibrium. For a grasp GSP on an object O to be stable, the wrenches applied by the robotic gripper must counteract the external wrench wexternal, which may result from a single or multiple forces. This grasp wrench equilibrium is expressed as:

where wi is the wrench exerted by the gripper at contact point ci. In a coordinate system centered at the object's centroid z, wi is defined as:wi = (fi, (ci -z) × fi + τi) , (2)

For soft finger contact models [Mahler et al., 2018], the rotational torque τi satisfies:

where γ is the torsional friction coefficient.

This condition ensures that the external wrench is fully balanced by the combined wrenches applied at the contact points by the robotic gripper, thereby maintaining grasp stability.

3.3 Adversarial Attacks on Robotic Grasping

The goal of adversarial attacks on robotic grasping is to degrade the performance of a given grasp configuration GSP on an object O by perturbing it, producing an adversarial version O′ . These attacks can be approached from two core aspects of grasp functionality:

• Lift Capability: The gripper's ability to overcome gravity and successfully lift the object.
• Grasp Stability: The gripper's ability to maintain a secure hold under external disturbances.

By degrading these two core aspects, adversarial attacks effectively compromise robotic grasping performance.

4 Method

In this section, we introduce AdvGrasp, a systematic approach for generating adversarial objects that compromise the lift capability and grasp stability of robotic grasping. We first describe attack methods targeting these two grasp metrics individually, followed by the integrated framework and its implementation. Please refer to Fig. 2 for a demonstration.

4.1 Adversarial Attack on Lift Capability

Lift Capability of A Grasp. The lift capability of a grasp reflects the robotic gripper's ability to counteract gravitational forces and achieve wrench equilibrium. As outlined in Section 3.2, this equilibrium requires the combined wrenches at all contact points to balance the gravitational wrench.

The general condition for wrench equilibrium described in Eqn. 1 can be specified for gravity as:

where m is the mass of the object, and g is the gravitational acceleration vector.

To quantify lift capability, we adopt the metric as formulated in [Ferrari et al., 1992]:

where the set G(wgravity) contains the normal forces F ⊥ needed to balance the gravitational wrench wgravity.

Adversarial Strategy for Lift Capability. We aim to reduce the lift capability LC by perturbing the object's geometry. This optimization is formalized as:

where O′ represents the adversarially modified object. By altering the geometry of O, the gravitational torque is increased, necessitating a greater gripper force to maintain wrench equilibrium. This effectively reduces the object's lift capability LC, making it more challenging for the gripper to lift the object.

4.2 Adversarial Attack on Grasp Stability

Grasp Stability of A Grasp. Grasp stability measures the gripper's ability to resist external disturbances and maintain secure contact with the object throughout the manipulation process. As formulated in Ferrari et al. [1992], the stability score is expressed as:

where ConvexHull({Wi}) represents the convex hull of the wrenches {Wi} applied at all contact points, and Disp2s computes the distance between the origin and the convex hull.

Adversarial Strategy for Grasp Stability. We reduce the stability score GS by introducing deformations near the contact regions of the object, altering the normal vectors ni at these points. The optimization objective is:

By introducing targeted deformations to the object's shape, the surface normals ni are altered, which adjusts the wrench space and reduces the stability margin. This effectively decreases the grasp's resistance to external disturbances, lowering the stability score GS.

4.3 Unified AdvGrasp Framework

Our method provides a unified framework that simultaneously addresses lift capability and grasp stability, ensuring comprehensive adversarial modifications. The optimization objective is defined as:

where Lap(O) ensures smooth geometric deformations by penalizing irregularities in the object shape:

Here, V(O) represents the vertices of O, Neigh(v) denotes the neighboring vertices of v, and λ1 and λ2 are hyperparameters that balance the contributions of grasp stability, lift capability, and deformation regularization, respectively.

4.4 Implementation of AdvGrasp

AdvGrasp generates adversarial objects O′ for robotic grasping through the following three steps:

Bounding Box Initialization. An axis-aligned bounding box is constructed for the object O, providing spatial constraints for defining deformation regions.

Control Point Placement. Control points are evenly distributed across the bounding box surfaces, creating a structured framework for guiding subsequent deformations.

Iterative Shape Deformation and Optimization. Using the control points as anchors, iterative perturbations are applied to the object's geometry following [Ju et al., 2023]. In each iteration, the resolution of control points is progressively increased, allowing for finer adjustments.

The adversarial object O′ is refined to maximize degradation of grasp performance, while ensuring physical plausibility and maintaining imperceptibility of the modifications.

5 AdvGrasp-20 Benchmark

To advance research in the emerging field of adversarial attacks on robotic grasping, we establish a comprehensive benchmark, AdvGrasp-20, designed to facilitate fair comparisons and promote further development in this area.

Object Selection. We select 20 representative objects with diverse shapes from the GraspNet-1Billion dataset [Fang et al., 2020; Fang et al., 2023], including common items such as CRACKER BOX, TOMATO SOUP CAN, and MUSTARD BOTTLE. Please refer to Fig. 3 for a visual demonstration.

Grasp Generation. For each object, we generate 5 grasps for a two-finger gripper using the analytic Dex-Net 2.0 framework [Mahler et al., 2017] and 5 grasps using the deep learning-based GraspNet. In addition, we generate 5 grasps for a three finger gripper using deep learning-based GenDexGrasp [Li et al., 2022].

Post-processing. To ensure the generated grasps are feasible, we further perform post-processing. Specifically, we validate the grasps in the PyBullet simulator using the Robotiq 2-finger 2F-85 gripper and the Robotiq 3-finger gripper. Only the grasps that successfully lift the objects are retained as final grasps, while unsuccessful ones are filtered out.

By including diverse objects and grasp strategies, AdvGrasp-20 provides a robust platform for systematically evaluating adversarial attacks on various grasping configurations. This benchmark not only enables fair comparisons but also fosters further innovation in the field of robotic grasping under adversarial conditions.

6 Experimental Results

6.1 Experimental Setup

Implementation. We implement our framework in Python using the PyBullet simulator [Coumans and Bai, 2016] as the simulation environment. All simulations employ the soft finger contact model [Mahler et al., 2018], with the friction coefficient set to µ = 0.6 and the torsional friction coefficient γ = 0.3. To manipulate the object's shape, each face's four edges are evenly divided into a square grid based on the cage size, initially set to 0.04. The vertices of these grids serve as control points, which act as anchors for shape manipulation. The optimization process utilizes a simulated annealing algorithm, with an initial temperature T0 = 1000, a minimum temperature Tmin = 10-5 , a cooling rate α = 0.98, and a perturbation scale ϵ = 0.05 × cage size for control points. After completing each optimization cycle, the cage size is halved, and new control points are generated for further optimization. This process is repeated for a total of five cycles. To balance the contributions of lift capability, grasp stability, and shape regularization in AdvGrasp, we set the weighting parameters to λ1 = 10000 and λ2 = 50.

Our Attack Solutions. We consider three attack solutions for evaluation: adversarial attacks targeting only lift capability (ALC), adversarial attacks targeting only grasp stability (AGS), and the unified approach, AdvGrasp, which simultaneously considers both aspects.

Evaluation Metrics. Unlike the binary success criteria in image classification adversarial attacks, evaluating adversarial attacks on robotic grasping from a physical perspective is inherently more complex. To address this, we define a grasp failure as a scenario where the object undergoes significant slippage, explicitly characterized by a displacement exceeding 0.02 m or a rotation surpassing 10◦ . Based on this definition, we establish three metrics to evaluate the performance of adversarial attacks on robotic grasping:

• Minimal Grasp Force (MinGF): The minimal force required by the gripper to lift a 1 kg object. Specifically, we start the grasp force at 50 N and decrease it by 0.2 N per step until significant slippage occurs.
• Maximal Lifting Mass (MaxLM): The maximum weight the gripper can lift when each finger applies a maximum force of 50 N. Specifically, we start the object's weight at 1 kg and increase it by 0.1 kg per step until significant slippage occurs.
• Maximal External Disturbance (MaxED): The maximum external force a grasp can withstand while holding a 1 kg object, with each finger of the gripper applying a maximum force of 50 N. We use Fibonacci sphere sampling [Larkins et al., 2012] to generate 50 directions on a unit sphere. For each direction, a force is applied over a time interval of 0.008 s, starting at 1 N and increasing by 1 N per step. The force is incremented only if no significant slippage occurs in all 50 directions.

Among these metrics, MinGF and MaxLM primarily evaluate the grasp configuration's ability to counteract gravity, while MaxED focuses on its resistance to external forces. Together, these metrics form a comprehensive framework for assessing the physical robustness of robotic grasp configurations under adversarial conditions.

6.2 Main Results and Analyses

Attack Performance on Counteracting Gravity. As shown in Tab. 1, for objects of the same type and mass (1 kg), three-finger grasps generally achieve a lower MinGF compared to two-finger grasps, indicating that three-finger grippers require less force to counteract gravity for the same object. Additionally, under the same condition where each finger can apply a maximum force of 50 N, three-finger grippers can often counteract gravity for objects with greater mass.

After applying our method, the effectiveness of the original grasp configurations significantly decreases, as reflected by increased MinGF and decreased MaxLM. For most objects, ALC proves more effective in making counteracting gravity more difficult, as it introduces gravity-related factors into the optimization. In contrast, AGS is less effective in this regard. Attack Performance on Resisting External Forces. We further evaluate the ability of these grasps to resist external forces under attack. As shown in Tab. 2, three-finger grasps generally tolerate higher disturbances compared to two-finger grasps, indicating greater robustness to external perturbations. However, after applying our attacks, particularly the unified AdvGrasp, the stability of these grasps is significantly compromised, making them more vulnerable to disruption. These results validate the effectiveness of our methods.

Visualization. Fig. 4 provides a visual comparison between the original objects with their corresponding grasps and the adversarial objects with their respective counterparts. It shows that the overall shape of the objects is preserved. At the same time, localized deformations occur near the grasp contact points, including changes in surface normals and slight shifts in the center of mass. These perturbations disrupt the grasp's wrench equilibrium, validating the effectiveness of our method. This demonstrates how AdvGrasp generates imperceptible yet impactful adversarial perturbations to effectively compromise robotic grasping performance.

Analysis on Shape Deformation. We quantitatively analyze the impact of AdvGrasp on the object's geometry. As illustrated in Fig. 5, the perturbations remain minimal, aligning to some extent with the imperceptibility requirements of adversarial attacks. These results underscore the effectiveness of AdvGrasp in achieving adversarial objectives while preserving geometric plausibility.

6.3 Additional Analyses

Physical Results. To validate the effectiveness of our method in real-world scenarios, we select the DABAO SOD and TOMATO SOUP CAN for physical experiments. For each object, we choose a specific grasp configuration and generate adversarial versions using AdvGrasp. We 3D-print these objects and modify them by hollowing them out, splitting them into two halves, and inserting lead balls to increase their mass to 1 kg, addressing the lightweight nature of 3D-printed materials. Grasping experiments are conducted under controlled conditions using a Robotiq 2-finger 2F-85 gripper mounted on a robotic arm. When a maximum grasping force of 30 N is applied to the DABAO SOD and its adversarial version, the original model is successfully lifted, while the adversarial model exhibits slippage and subsequently drops. Similarly, when a 20 N grasping force is applied to the TOMATO SOUP CAN and its adversarial version, the original model is successfully lifted, while the adversarial model fails and is dropped. These experiments highlight the impact of physical properties on grasp quality and confirm the effectiveness of AdvGrasp in generating adversarial objects that compromise grasp stability in real-world conditions.

DNN-based Metric vs. Physical Metric. To validate the importance of physical metrics in grasp evaluation, we assess all grasps in the AdvGrasp-20 benchmark, as well as adversarial grasps generated by AdvGrasp, using both the DNNbased PointNetGPD [Liang et al., 2019] and physical metric. Specifically, we apply the 2-class classification approach from PointNetGPD to determine whether a grasp is good or bad. For the physical metric, a grasp is considered bad if the actual grasp position deviates from the expected position by more than 0.03 m or if the angle deviation exceeds 10 degrees during simulation experiments. As shown in Fig. 7, the physics-based metric identifies most adversarial grasps generated by AdvGrasp as bad. However, PointNetGPD, which relies solely on local geometric properties, fails to accurately assess these cases. These findings underscore the indispensable role of physical metrics in grasp evaluation.

7 Conclusion

In this paper, we have proposed a framework for adversarial attacks on robotic grasping, targeting lift capability and grasp stability. Through shape deformation to increase gravitational torque and reduce stability margin in the wrench space, our method systematically degrades grasp performance. Extensive experiments in simulated and real-world environments validate the effectiveness of our approach. Future work will consider dynamic settings and focus on defense design.

Acknowledgements

This work was supported in part by the Dreams Foundation of Jianghuai Advance Technology Center (2023ZM01G003), the National Natural Science Foundation of China (62472117), the Guangdong Basic and Applied Basic Research Foundation (2025A1515010157), and the Science and Technology Projects in Guangzhou (2025A03J0137).