Enhancing the Transferability of Adversarial Examples with Random Patch

1 Introduction

Many works [Szegedy et al., 2014; Kurakin et al., 2017a] have shown that deep neural networks (DNNs) are vulnerable to adversarial examples, i.e., input images perturbed by imperceptible noise can lead to inaccurate predictions of DNNs. The existence of adversarial examples raises concerns in security-sensitive applications, e.g., self-driving cars and face recognition. Studying adversarial attacks is helpful to evaluate and improve the robustness of different models [Goodfellow et al., 2015] to avoid future risks. Adversarial attacks are classified into white-box attacks and black-box attacks. Black-box attacks are more challenging and more practical in real-world applications. Compared to query-based black-box attacks [Ilyas et al., 2018; Shi et al., 2019], transfer-based black-box attacks [Liu et al., 2017; Zhou et al., 2018] are more realistic and flexible, which exploit the transferability of adversarial examples to execute black-box attacks.

∗Corresponding author.

Many works attempt to enhance transferability by improving the optimization process [Dong et al., 2018] or by exploiting data augmentation [Xie et al., 2019] or by disrupting feature maps of intermediate layers [Huang et al., 2019]. However, these methods are easily trapped into model-specific local optimum by indiscriminately perturbing features without considering the intrinsic features of objects in images. Some works tend to disrupt inherent features of objects to improve the transferability of adversarial examples. [Wang et al., 2021b] (FIA) adopt random pixel dropping transformations and aggregate the transformed images to find these objectrelated features. Although pixel-wise transformations can neutralize some model-specific non-semantic details, these model-specific features cannot be filtered out well due to the correlation between adjacent pixels in natural images. Simultaneously, pixel dropping introduces extra pixel-wise noises that interfere with the captured object-related features, weakening the transferability of adversarial examples.

This paper proposes a Random Patch Attack (RPA), which effectively obtains inherent critical features of objects by random patch transformations, thereby significantly enhancing the transferability of adversarial examples. Specifically, we introduce random patch transformations to benign images such that the transformed images can variate the non-object noise while preserving object-related spatial structure and texture information. The regions of important object-related features are stressed by computing and aggregating the gradients of the feature maps of the transformed images at the intermediate layer. In contrast, those model-specific features are suppressed. The aggregated gradients guide adversarial perturbations distort important features, making adversarial examples more transferable. Since natural images are usually composed of smooth patches [Mahendran and Vedaldi, 2015] and model's discriminative regions are generally concentrated on several patches of them, the patch-wise transformation can better preserve object-related areas and variate patch-wise model-specific details. Different models focus on different discriminative regions during recognition [Dong et al., 2020], thus aggregating many transformed images can better neutralize model-specific information.

As shown in Figure 1, the adversarial example crafted by the proposed RPA significantly distracts models, making models focus on entirely different areas from attention on the clean image. In contrast, the models' attention on the adversarial example generated by FIA partially overlaps with that on the clean image, thus illustrating the effectiveness of patch-wise random transformations for capturing objectrelated critical features. We follow the previous works to evaluate the performance of the proposed RPA against normally trained models and defense models. The experimental results demonstrate that the proposed RPA craft adversarial examples of remarkable transferability compared to the stateof-the-art transfer-based attacks. Besides, we also evaluate the performance of the proposed RPA against vision transformers, which achieve the state-of-the-art performance on recognition tasks and are shown strong robustness against adversarial examples [Mahmood et al., 2021]. The experimental results show that our attack reaches an average black-box success rate of 62.6% against vision transformers, thus further demonstrating the superior performance of the proposed RPA. Code is available at: https://github.com/alwaysfoggy/ RPA. Our main contributions are summarized as follows:

• We propose a Random Patch Attack by patch-wise random transformations to perturbs common discriminative regions of different models, significantly improving the transferability of adversarial examples.
• The random patch transformation operation effectively highlights important object-related features, thus guiding adversarial examples towards the more transferable direction.
• Extensive experiments demonstrate that the proposed RPA generates adversarial examples of excellent trans-

ferability compared to state-of-the-art transfer-based attack methods.

2 RelatedWork

Following the pioneering work in [Szegedy et al., 2014], many attacks have been proposed to show the vulnerability of convolutional neural networks (CNNs) to adversarial examples. FGSM [Goodfellow et al., 2015] and BIM [Kurakin et al., 2017a], as the first generation of gradient-based attack methods, are the basis for many subsequent works, most of which focus on enhancing the transferability of adversarial examples to execute the black-box attack. [Dong et al., 2018] integrates momentum term into iterative gradient-based attacks (MIM) to boost the transferability of adversarial examples and significantly improve the attack ability of black-box attacks. [Xie et al., 2019] applies image transformation to the inputs at each iteration (DIM) to improve transferability. [Wu et al., 2021] proposes adversarial transformations to improve the robustness of synthesized adversarial examples (ATTA) to boost transferability. [Wang et al., 2021a] proposes a novel input transformation method (Admix) to enhance the transferability of adversarial examples. [Dong et al., 2020] proposes translation-invariant attack (TIM) by optimizing perturbations over an ensemble of translated images, resulting in higher transferability against defense models. [Gao et al., 2020] proposes a patch-wise iterative algorithm (PIM) to enhance the transferability of adversarial examples.

Unlike the above works perturbing the output layer, some studies focus on maximizing internal feature perturbation to improve transferability. [Ganeshan et al., 2019] disrupts features at each layer of the network to generate perturbations, resulting in highly damaged deep features to obtain higher transferability. [Huang et al., 2019] aims to fine-tune an existing adversarial example for greater black-box transferability by increasing its perturbation on a pre-specified layer of the source model. [Wang et al., 2021b] proposes a Feature Importance-aware Attack (FIA) to significantly improve the transferability of adversarial examples by disrupting the essential object-aware features. The proposed RPA belongs to the same category as FIA, which considers the importance of features rather than the indiscriminate modification of all features as in previous work.

3 Methodology

Given a classification model f(x) : x ∈ X → y ∈ Y that outputs a label y as the prediction for an input x, we aim to craft an adversarial example x∗ which is visually indistinguishable from x but is misclassified by the classifier, i.e., f(x∗ ) ̸= y. The generation of adversarial examples can be formulated as the following optimization problem,

where the loss function J(·, ·) is often the cross-entropy loss, and the ℓp-norm is adopted to measure the distance between x and x∗ . This work uses p = ∞. However, optimizing Eq.1 needs to calculate the gradient of the loss function, which is not allowed in the black-box setting. Many methods [Dong et al., 2018; Gao et al., 2020] resort to the transferability of adversarial examples, i.e., generating adversarial examples on the source model and exploiting transferable adversarial examples attacks the target black-box model.

The key to generating adversarial examples of high transferability is to explore which features are more accessible to transfer across models. Classifiers tend to use any available signal to maximize classification accuracy [Ilyas et al., 2019]. In addition to learning intrinsic features of objects, a model also retains model-specific features. Different models may result in various model-specific features. Therefore, distorting object-related characteristics that dominate discriminative regions of different models can avoid adversarial examples falling into model-specific local optimum and enhance transferability across different models. [Wang et al., 2021b] adopt random pixel dropping to transform the input and introduce aggregate gradient to aggregate the transformed images, thereby finding and distorting object-related features to generate adversarial examples of high transferability. However, natural images are generally made of smooth patches [Mahendran and Vedaldi, 2015] and discriminative regions usually contain clustered pixels instead of scattered ones, only performing pixel-wise transformations can not filter out model-specific details well. Simultaneously, dropping pixels will introduce extra pixel-wise noises that impede obtaining important object-related features.

3.1 Patch-wise RandomTransformation

This paper introduces the patch-wise random transformation to effectively stress the important intrinsic features of objects and suppress model-specific elements. Specifically, we first generate a mask M of the same size as x, which is a matrix with all elements of 1. Then we divide M into regular non-overlapping patches. The patch size is n2 ; we suggest n < 10, since small-size patches can eliminate the redundancy of model-specific features while trying to preserve the semantic features of an image. Then we randomly sample a subset of patches with the probability pm and modify the subset following a uniform distribution. Let f denote the source model and fk(x) represent the feature maps from the k-th layer. We calculate the gradient ∆ w.r.t. fk(x) after integrat-

Algorithm 1 Random Patch Attack

Input: A classifier f; an original image x and ground-truth label y;

Parameter: The intermediate layer k; modify probability pm; the ensemble number N; the patch size n; the size of perturbation ϵ; the number of iteration T ; maskM.

Output: An adversarial example x∗ .

• 1: Let ∆ = 0, g0 = 0, µ = 1, α = ϵ/T;
• 2: for i = 0 to N -1 do
• 3: DivideM into non-overlapping n2 patches P ;
• 4: Random sample P with pm: Ppm = Rand(P , pm);
• 5: Modify Ppm ∼ U[0, 1);
• 6: Obtain the masked image: xM =x ⊙M;
• 7: Get the gradient of feature maps from the k-th layer:

• 8: ∆ =∆+∆i;
• 9: end for
• 10: Obtain the aggregate gradient: ∆k =∆/ ∥ ∆ ∥2;
• 11: Get loss function by Equation 3;
• 12: for t = 0 to T -1 do
• 13: Update the momentumgt+1:

• 14: Update x∗t+1 by applying the gradient sign:

15: end for

16: return x∗T .

ing the mask into x. After N iterations, we finally adopt the aggregate gradient ∆k to aggregate the above gradients as

where xM denotes the masked image, l(·, ·) denotes the logit output w.r.t. y, and C is obtained by ℓ2-norm on the corresponding summation term. Figure 2 shows the process of patch-wise random transformation, which weakens the nonobject regions that may cause local optimum and highlights the areas of object-related features that can guide the generation of adversarial examples towards the more transferable direction.

3.2 Attack Algorithm

This paper proposes a random patch attack (RPA) to distort object-related critical features obtained by the above patchwise random transformations, generating adversarial examples of high transferability. We utilize the obtained aggregate gradient ∆k to form the following loss function to steer the generation of x∗ .

The higher intensity in ∆k represents regions of objectrelated critical features. Increasing the lower intensity and declining the higher intensity guide adversarial examples towards the more transferable direction. Therefore, the RPA method solves the following optimization problem instead of Equation 1.

We adopt the momentum optimization method [Dong et al., 2018] to solve Equation 4. The details of the proposed RPA are shown in Algorithm 1.

4 Experiments

4.1 Setup

Datasets. We follow the previous works [Gao et al., 2020; Wang et al., 2021b] to conduct our experiments on the ImageNet-compatible dataset 1 , containing 1000 images used for the NIPS 2017 adversarial competition.

Models. We consider fourteen classification models as target models to evaluate the performance of different attacks, six of which are normally trained models-Vgg-16 [Simonyan and Zisserman, 2015], Resnet-v1-50 (Res-50) and Resnet-v1-152 (Res-152) [He et al., 2016], Inceptionv3 (Inc-v3) [Szegedy et al., 2016], Inception-v4 (Inc-v4) and Inception-Resnet-v2 (IncRes-v2) [Szegedy et al., 2017], four of which are defense models-Inc-v3adv and IncResv2adv [Kurakin et al., 2017b], Inc-v3ens3 and IncRes-v2ens [Tram`er et al., 2018], and four of which are state-of-theart vision transformers-PiT-S [Heo et al., 2021], CaiT-S24 (CaiT-S) [Touvron et al., 2021b], DeiT-B [Touvron et al., 2021a], Swin-B [Liu et al., 2021].

Baseline Attacks. We compare the proposed RPA to diverse state-of-the-art attack methods-MIM [Dong et al., 2018], DIM [Xie et al., 2019], TIM [Dong et al., 2020], PIM [Gao et al., 2020], Admix [Wang et al., 2021a], FIA [Wang et al., 2021b], and combinations of these methods-TIDIM [Dong et al., 2020], PIDIM [Gao et al., 2020], PIDITIM [Gao et al., 2020], PIDIM-FIA [Wang et al., 2021b], PIDITIM-FIA [Wang et al., 2021b].

Implementation Details. For the settings of parameters, we set the maximum perturbation to be ϵ = 16, the number of iteration as T = 10 and the step size as α = ϵ/T = 1.6 among all experiments. For the baseline attacks, we adopt the default settings. For MIM, the decay factor µ = 1.0. For DIM, the transform probability is 0.7. For TIM, the kernel size is 15. For PIM, settings vary with target models. When attacking normally trained models, the amplification factor β = 10, the project factor γ = 16, and project kernel size kω = 3. while its combined versions replace β = 2.5, γ = 2. When attacking defense models, PIM remove the momentum term, PIM and PIDITIM set β = 10, γ = 16, kω = 7. For Admix, the number of admixed copies m1 = 5, sampled images m2 = 3, and the strength of sampled image η = 0.2.

1 https://github.com/cleverhans-lab/cleverhans/tree/master/ cleverhans v3.1.0/examples/nips17 adversarial competition/dataset

For FIA, the ensemble number N = 30, when attacking normally trained models, the drop probability pd = 0.3; when attacking defense models, pd = 0.1. For the proposed RPA, the patch size alternately sets n = 1, 3, 5, 7, the ensemble number N is set to 60, the modify probability pm is set to 0.3, 0.2 and 0.2 when attacking normally trained models, defense models and vision transformers, respectively. The chosen feature layer and extra parameters of combined versions follow the settings in [Wang et al., 2021b] for a fair comparison. We set random seed in our experiments to guarantee that the proposed method is reproducible, the random seed is 1234.

4.2 Comparison of Transferability

This section compares the transferability between the proposed RPA and the baseline attacks quantitatively. We evaluate the success rate of different attacks against normally trained models, defense models, and visual transformers, respectively.

Attacking Normally Trained Models

Table 1 reports the black-box attack success rates of the proposed RPA and the other state-of-the-art transfer-based attacks against normally trained models. We craft adversarial examples on Vgg-16, Res-152, and Inc-v3, respectively. As shown in Table 1, our attacks outperform the other attacks, with an average improvement of 2.9% compared to the best results of other methods. When our RPA and PIDIM-RPA generate adversarial examples on Res-152 to attack IncResv2, transferability can be increased by 8.3% and 5.2%, respectively. Notably, combining our RPA with PIDIM can further improve the transferability, resulting in a 4.4% increase in transferability on average. In particular, the black-box attack success rate of PIDIM-RPA can achieve an average of about 92.5%. When the source model is Vgg-16, PIDIMRPA attacking Res-50 reaches 99.1%, close to the white-box attack success rate.

Attacking Defense Models

Table 2 (left) reports the black-box attack success rates of the proposed RPA and baseline attacks against defense models. We choose Vgg-16, Res-152, Inc-v3, and IncRes-v2 as the source model, respectively. Other methods with poor performance are not listed in Table 2 due to the space limit. The transferability of Table 2 (left) is consistently lower than that of Table 1 since defense models have stronger robustness to adversarial examples. However, our methods still significantly outperform the other state-of-the-art transferbased attacks, improving the black-box attack success rate by 4.7% on average. As shown in Table 2 (left), PIDIMRPA and PIDITIM-RPA further improve the transferability by 10.8% on average, the black-box attack success rate achieving 73.9% on average. In particular, when the source model is Vgg-16, the success rate of PIDIM-RPA against all defense models achieves 88.0% on average, especially the success rate against Inc-v3adv reaches 93.2%. When IncResv2 is the source model, PIDITIM-RPA against all defense models results in an average 11.4% increase in success rate compared to PIDITIM-FIA.

Attacking Vision Transformers

Table 2 (right) reports the black-box attack success rates of different attacks against state-of-the-art vision transformers (ViTs). Previous works [Mahmood et al., 2021] demonstrate that ViTs possess better adversarial robustness when compared with CNNs, and the transferability between CNNs and ViTs is, in general, remarkably low. Therefore, attacking ViTs are more arduous by the generated adversarial examples on CNNs. From the right four columns of Table 2, we can observe that the proposed RPA significantly outperforms other methods, improving the black-box attack success rate by 4.6% on average. The combination PIDIM-RPA further enhances the transferability by 8.2% on average, achieving an average black-box attack success rate of 62.6%. When the source model is Vgg-16, our PIDIM-RPA obtains the highest attack success rate of 87.8% against PiT-S.

4.3 Effect of Parameters

In the proposed RPA, the modify probability pm, the ensemble number N of aggregate gradient and the patch size n2 play a critical role in affecting the black-box attack success rate. We adopt Res-152 as the source model for generating adversarial examples by the proposed RPA with different pm and N. pm ranges from 0.1 to 0.5 with a granularity of 0.1, and N changes from 10 to 70 with a step of 10. We then evaluate the transferability of the generated adversarial examples by attacking Vgg-16, Inc-v3, IncRes-v2adv and Swin-B with the results shown in Figure 3.

In general, a larger ensemble number does increase the success rate. However, when the ensemble number is large enough, the success rate goes saturate and may even decrease. We thus determine the ensemble number N = 60. When attacking normally trained models, an optimal modify probability pm is between 0.3 and 0.4. While an optimal pm is between 0.1 to 0.3 for attacking defense models and vision transformers. We thus suggest the modify probability pm = 0.3 for attacking normally trained models and pm = 0.3 for attacking defense models and vision transformers. To study the effect of n2 on success rate, we adopt IncRes-v2 as the source model to craft adversarial examples by P1, P2, P3 and P4, respectively. P1 sets n = 1, P2 alternately uses n = 1, 3, P3 alternately uses n = 1, 3, 5 and P4 alternately sets n = 1, 3, 5, 7. As shown in Figure 4, P4 leads to a higher success rate, we thus suggest the setting of P4.

5 Conclusion

In this paper, we propose a novel Random Patch Attack (RPA) that generates adversarial examples of superior transferability. The proposed RPA effectively captures the raw image's important object-related features and eliminates the redundancy of model-specific features by the patch-wise random transformation. Perturbing these features steers the adversarial examples towards a more transferable direction. We conduct extensive experiments that demonstrate the excellent transferability of adversarial examples generated by the proposed RPA compared to state-of-the-art transfer-based blackbox attacks. Our attack strategy can serve as a benchmark for evaluating the effectiveness of different defense methods in the future and various models' robustness.

Acknowledgments

This work is supported by the National Natural Science Foundation of China under Grant (No. 61876019, 62072037, U1936218), Beijing Municipal Science and Technology Project Z211100004121009.