GenSeg: On Generating Unified Adversary for Segmentation

1 Introduction

Deep neural networks have excelled in diverse domains, however, their vulnerability to quasi-imperceptible adversarial perturbations remains a challenge [Goodfellow et al., 2014b; Szegedy et al., 2014]. Efforts to address this issue have predominantly focused on image classification tasks [Dong et al., 2018; Ilyas et al., 2018; Zhang et al., 2022a], leaving a gap in understanding adversarial robustness in segmentation models. Given the broader applications of segmentation tasks (such as in autonomous driving and medical image analysis), there is an urgent need to develop effective strategies of adversarial attack and defense for them.

* Corresponding author. Email: qubit@ustc.edu.cn

Code is available at: https://github.com/YXZhang979/GenSeg

Predominant adversarial attacks on segmentation tasks, to date, have faced two main challenges: 1) Transferability: Dense-prediction tasks are more complicated than classification tasks, leading to the difficulty in designing a strong attack. Although progress has been made [Hendrik Metzen et al., 2017; Xie et al., 2017; Xu et al., 2021], existing methods still exhibit limited transferability across different models due to their instance-specific strategies, relying too much on specific source models. 2) Efficiency: Current instancespecific attacks [Gu et al., 2022; Agnihotri and Keuper, 2023; Rony et al., 2023] typically employ an iterative gradientbased paradigm, which is time-consuming. However, insufficient iterations to reduce time consumption will decrease the effectiveness and further reduce attack transferability. Therefore, enhancing attack transferability and efficiency remains challenging at the same time. Additionally, existing approaches primarily focus on adversarial attacks in semantic segmentation (SS) [Bar et al., 2020; Rossolini et al., 2023], neglecting other segmentation tasks such as instance segmentation (IS) and panoptic segmentation (PS). However, these tasks are typically vulnerable to adversarial attacks, as illustrated in Figure 1.

The above challenges prompt two intriguing questions: 1)

How can we improve both attack transferability and efficiency of adversarial attacks at the same time? 2) Is it possible to devise a unified attack paradigm that is applicable across diverse segmentation tasks?

To address these two questions, we propose a pipeline to generate unified adversaries for segmentation tasks based on a generative model [Goodfellow et al., 2014a], termed GenSeg. For the first question, contrasting to the iterative instance-specific methods, GenSeg crafts an adversary in a single forward pass, showcasing its high efficiency. To enhance attack transferability, we introduce an intermediatelevel attack catering to segmentation tasks for optimizing the generator. Such an attack contains two objectives, i.e., a mutual agreement loss and a prototype obfuscating loss. The former aims to deviate the adversarial features from the benign features. As our goal is to deceive as many pixels as possible, we incorporate a dynamic weight strategy to punish the insufficient-perturbed pixels. The latter focuses on damaging the intra-class and inter-class prototype relationships [Zhang et al., 2023b], which contributes to a stronger ambiguity in the latent space.

As to the second question, given the shared structure in different segmentation tasks (involving a backbone for feature extraction and a head network for prediction), we introduce a category-and-mask view to unify segmentation tasks and make it a uniform paradigm applicable to attack these tasks, including SS, IS, and PS.

To evaluate the effectiveness of GenSeg, we conduct comprehensive experiments on SS, IS, and PS, respectively. We employ 9 datasets and 15 models in total to validate our method. Experimental results show that GenSeg performs a stronger cross-model black-box attack transferability compared with state-of-the-art methods. To our best knowledge, GenSeg first extensively explores cross-domain and crosstask attacks in segmentation tasks, and sets a baseline.

We briefly summarize our contributions as follows:

• We propose GenSeg, a unified generative paradigm to attack diverse segmentation tasks, including semantic segmentation, instance segmentation, and panoptic segmentation.
• We design an intermediate-level objective for segmentation, including a mutual agreement attack to distance the feature space and a prototype obfuscating attack to disrupt the intra-class and inter-class relationships.
• Extensive experiments show GenSeg's superior attack capability on three tasks, encompassing 9 datasets and 15 models in total. GenSeg also pioneers the exploration of cross-domain and cross-task attacks in segmentation.

2 RelatedWork

2.1 Adversarial Attack in Classification

The pioneered work [Goodfellow et al., 2014b] highlights the vulnerability of neural networks, leading to numerous approaches using iterative gradient-based optimization for generating instance-specific adversaries [Madry et al., 2017; Dong et al., 2018; Xie et al., 2019; Shi et al., 2021]. However, these methods are time-consuming and lack transferability across models or domains due to their reliance on the output score of a specific source model [Liu et al., 2016; Naseer et al., 2021; Shi et al., 2022; Multimedia, 2023]. In contrast, universal perturbations aim to deceive models using global patterns, yet their effectiveness is relatively limited [Moosavi-Dezfooli et al., 2017; Li et al., 2022]. More recently, generative perturbations have been proven to be more effective than directly optimizing the universal perturbations [Feng et al., 2023; Yang et al., 2022; Naseer et al., 2021]. Our method follows the concept of generative perturbations, crafting adversaries in a single forward pass. Unlike previous approaches, we focus on designing a generator tailored to generating adversaries for segmentation tasks, targeting the deception of as many pixels as possible.

2.2 Adversarial Attack in Segmentation

Current adversarial attacks in segmentation primarily focus on SS. [Arnab et al., 2018] transfers the attacks in classification to assess the adversarial robustness on segmentation models. [Hendrik Metzen et al., 2017] optimizes a universal perturbation to fool the majority of test images in SS, while the performance is unsatisfactory. For deceiving as many pixels as possible, approaches like DAG [Xie et al., 2017], SegPGD [Gu et al., 2022], and CosPGD [Agnihotri and Keuper, 2023] pay more attention to attacking benign pixels. [Rony et al., 2023] proposes a whitebox attack to reduce the perturbation budget. In addition, there are a few works [Zhang et al., 2022b; Li et al., 2018; Lang et al., 2022] on adversarial attacks in IS. However, most of these approaches treat IS as a detect-then-segment method, while one-stage and query-based approaches have become the mainstream paradigm. Furthermore, only [Daza et al., 2022] delves into adversarial attack exploration in PS, to the best of our knowledge.

Despite their advancements, current methods are still limited by 1) their focus on a single task, 2) the poor transferability to other target models, and 3) low efficiency with the iterative strategies. To address these issues, we design GenSeg, a unified attack framework applicable to multiple segmentation tasks, motivated by MaskFormer series [Cheng et al., 2021; Cheng et al., 2022]. Moreover, GenSeg also performs higher attack transferability and efficiency compared with iterative instance-specific attacks.

3 Methodology

3.1 General Formulation of Segmentation

We start by providing a unified formulation for segmentation tasks in a category-and-mask view. Despite the apparent diversity of SS, IS, and PS, current solutions share a common structure, a backbone for feature extraction and a head network for prediction. Given an image I ∈ RH×W×3 , we derive the intermediate feature F using the backbone. Subsequently, F is input into the head network to predict an instance mask set ˆ Sc = {ˆ yc i } ˆNc i=1 for each category c ∈ C, approximating the ground-truth Sc = {yc i }Nc i=1 . Here, yc i ∈ {0, 1}H×W denotes the binary mask of the i-th instance for category c and Nc is the total mask count for category c.

Note that, C varies across different tasks. As to the image segmentation tasks, C typically comprises two subsets: stuff and thing, which are denoted as C St and C Th , respectively, and C St ∩ C Th = ∅. Stuff segmentation involves segmenting uncountable regions that do not correspond to individual objects, e.g., sky, grass, and wall. Thing segmentation focuses on identifying each separate object, e.g., car, bus, and person.

In SS, we do not need to distinguish different thing objects. That is, we treat the countable thing equal to the uncountable stuff. Therefore, the segmented category C = C St ∪ C Th and Nc = {0, 1} for each category c ∈ C. Given the ground-truth mask set Sc = {yc i }Nc i=1 for c ∈ C, we have ∑c∈C ∑Nc i=1 ||yc i ||1 = H ×W.

In IS, we only care about thing segmentation. Thus C = C Th . As IS aims to distinguish intra-class objects, we have Nc ∈ N for c ∈ C. Note that, the different masks may have an intersection, since the initial masks are typically encoded as a polygon. Thus we cannot determine which one is greater between ∑c∈C ∑Nc i=1 ||yc i ||1 and H×W. However, this does not influence the unified attack objective.

In PS, we need to not only segment stuff category, but also identify different objects in thing category. Accordingly, we have C = C St ∪ C Th , Nc = {0, 1} for c ∈ C St , and Nc ∈ N for c ∈ C Th . It is worth noting that the masks are not allowed to have intersections in PS, which is different from IS. Thus we draw that ∑c∈C ∑Nc i=1 ||yc i ||1 = H ×W.

Despite their differences, we are able to unify them into this category-and-mask formulation. This formulation enables the development of a unified objective in the generative network, providing a seamless approach to attack various segmentation tasks.

3.2 Overview of GenSeg

The overall procedure of GenSeg is illustrated in Figure 2. Such a pipeline contains two key networks, i.e., a generator Gθ and a discriminator Dϕ, which are parameterized by θ and ϕ, respectively. Given a clean image I, we first create an augmented copy ˜ I, aiming to enhance attack transferability [Naseer et al., 2021]. After that, I and ˜ I are input into Gθ to produce unrestricted adversaries. As the perturbation should be imperceptible, we strictly constrain the adversary into an ϵ-ball with l∞-norm. This is guaranteed by a differentiable projecting operation, which is formulated as:

where ϵ is the perturbation budget and W is a smoothing operator with fixed weights [Feng et al., 2023]. Such a projection tightly bounds the output of Gθ within l∞-norm. Meanwhile, with the smooth operation W, the generator is guided to avoid redundant high frequencies during training, making Gθ converge to a reasonable solution. Clip is a clipping operation.

Subsequently, we put adversary I ′ , augmented adversary ˜ I ′ , and clean image I into Dϕ for feature extraction. Accordingly, we obtain the feature F ′ , ˜ F ′ , and F ∈ RH×W×D , respectively, where D is the dimension of the latent space. Then the features are employed to calculate an intermediatelevel objective for optimizing Gθ, which is formulated as:

where λ1 and λ2 are two balanced weights, and we set λ1 = λ2 = 0.5 empirically. The mutual agreement loss LMA aims to distance the feature of the adversary and its benign state, and the prototype obfuscating loss LPO further leverages the intra-class and inter-class prototype relationships to enhance the intermediate-level attack. We will present the details of both losses in the following subsections.

Accordingly, during the training period, we employ Ltotal to optimize the parameter θ of the generator, which guides Gθ to produce an adversary whose feature is deviated from the expected one. During inference, we solely apply Gθ and a clip operation to craft an adversary in a single forward pass.

3.3 Mutual Agreement Attack

Intuitively, if the adversarial feature F ′ is totally different from the benign feature F, it would mislead the subsequent classifier to conduct false predictions. Given this, our attack design is to pull the distribution of adversarial features away from the benign distribution. To achieve this goal in the dense-prediction task, we take Kullback Leibler (KL) divergence [Kullback and Leibler, 1951] to measure the pixel-wise mutual agreement between F ′ and F , which is calculated by:

where j represents the 2D spatial location, d stands for the dth dimension of the latent space, and σ denotes a softmax operation for normalization. In short, KL divergence measures the pixel-level differences between F and F ′ . It is worth noting that, KL divergence is asymmetric, i.e., DKL(F ′ j ||Fj) ̸= DKL(Fj ||F ′ j ), and the value is not normalized. Thus we employ Jensen Shannon (JS) divergence [Goodfellow et al., 2014a] to achieve both symmetry and normalization, which is formulated by:

where ¯ Fj = Fj+F ′ j 2 denotes the average of Fj and F ′ j .

Inspired by [Gu et al., 2022], a strong attack can deceive as many pixels as possible in segmentation tasks. Therefore, paying equal attention to all pixels is less effective. In particular, the pixels in F ′ with a higher similarity to F are supposed to be highlighted. Accordingly, we introduce a dynamic weight to each pixel, calculated by: Wj = Fj·F ′ j ||Fj ||·||F ′ j || , and we focus on reducing the original mutual agreement loss between F and F ′ by:

If F ′ j is close to Fj, we have DJS(F ′ j ||Fj) → 0. Thus we assign a large weight Wj to punish this high alignment, which contributes to the deception of more pixels.

Besides, to enhance attack transferability, we employ an augmented copy, which guides the generator to produce an adversary that is robust to the input transformation. Similarly, we reduce the feature similarity between F and ˜ F by the augmented mutual agreement loss:

where ˜Wj = Fj· ˜ F ′ j ||Fj||·|| ˜ F ′ j || is a dynamic weight.

Finally, the total mutual agreement loss is calculated by:

3.4 Prototype Obfuscating Attack

With segmentation masks, the intermediate feature map offers a greater interpretability compared with classification tasks. This allows us to improve intermediate-level attacks guided by the ground-truth masks. Specifically, we employ a global descriptor known as a prototype to represent the average feature of each category. The prototype is widely used in numerous segmentation tasks [Wang et al., 2019; Zhang et al., 2023a; Xu et al., 2022]. Damaging the prototype relationship for each category will enhance the intermediatelevel attack capability.

On the one hand, the intra-class features share a high similarity, which is the foundation for utilizing a prototype to represent each specific category. Given an image I and its intermediate feature F, we divide F into category-specific regions based on the ground-truth mask set Sc = {yc i }Nc i=1 for ∀c ∈ Cseg, where Cseg = {c|N c ≥ 1, ∀c ∈ C} denotes the set of categories that appears in the ground-truth mask. Then, the prototype pc ∈ Rd of category c is calculated by a masked average pooling operation:

where ⊙ and 1 denote a Hadamard product and an indicator function, respectively. i represents the i-th mask of category c, and j is the 2D spatial index. Besides, τ(yc i ) stands for the dimension expansion of yc i (from RH×W to RH×W×D ), which aims to align the dimension of F .

After obtaining the prototype for each segmented category, we make all the pixel-wise representations deviated from their original cluster center. Therefore, the intra-class attack objective of category c can be formulated as:

where cosSim(·, ·) denotes cosine similarity. The total intraclass objective considering all categories is computed by:

Our attack goal is to minimize Lintra, thereby reducing the similarity between the pixel-wise representation and its corresponding prototype. This approach dilates the feature space of each segmented category, consequently misleading the head network to conduct false predictions.

On the other hand, the inter-class feature spaces is supposed to be discernible. In particular, a substantial margin between feature clusters of distinct categories enhances the separation performance, resulting in a distinct decision boundary. Hence, our attack is aimed at blurring the decision boundary, and the objective for inter-class attack is formulated as:

where K = 2(|Cseg| 2 ) denotes the number of pair-wise prototype permutations.

After obtaining the above two attack objectives, we formulate the overall prototype obfuscating loss by:

4 Experiments

Dataset. To evaluate the attack effectiveness of GenSeg, we employ the commonly used Pascal VOC (20 classes) [Everingham et al., 2010], Cityscapes (19 classes) [Cordts et al., 2016], and ADE20K (150 classes) [Zhou et al., 2017] for SS. For IS, we use the widely used CityScapes (8 things), COCO (80 things) [Lin et al., 2014], and ADE20K (100 things). As to PS, we adopt COCO (80 things and 53 stuff ), Cityscapes (8 things and 11 stuff ), and ADE20K (100 things and 50 stuff ). Please see Appendix B for detailed descriptions of all datasets used in experiments.

Evaluation metrics. For SS, we use the metric of mean Intersection-over-Union (mIoU) for evaluation. For IS, we report the standard average precision (AP) metric. For PS, we employ the panoptic quality (PQ) metric. The drop score of each metric correlates directly with the attack capability.

4.1 Implementation Detail

We utilize the widely-adopted ResNet-based model [He et al., 2016] in [Naseer et al., 2019] for generator Gθ, producing adversaries that match the input size. The discriminator Dϕ employs a pre-trained ResNet50 to generate a multi-scale hybrid feature map. During training, Dϕ remains frozen while optimizing Gθ. We use the Adam optimizer with a learning rate of 5e-3 (β1 = .5, β2 = .999) for 100 epochs. For each segmentation task, we train an individual generator on each separate dataset, allowing the evaluation of both intra-domain and cross-domain attacks. We set the perturbation budget ϵ to the typical value of 8/255. Additionally, our smooth operator W is a differentiable Gaussian kernel specified in [Naseer et al., 2021]. After training, the generator creates adversaries without any augmentation. Subsequently, we conduct attacks across various models and domains to thoroughly evaluate the effectiveness of GenSeg.

4.2 Attack Scenarios and Results

We assess untargeted black-box attacks of GenSeg in the following two scenarios: 1) Cross-model Intra-domain: The attacker lacks access to the target model, while both the source and target models are trained within the same domain. 2) Cross-model Cross-domain: The attacker is unaware of the target model's architecture. Moreover, the source and target models are trained on different domains. This scenario more closely reflects real-world conditions.

Cross-model Intra-domain Attack

In this scenario, GenSeg is trained on the same dataset as the target models. Then we assess the attack transferability of GenSeg in comparison to other transfer-based attacks.

Semantic Segmentation: We evaluate the black-box attack capability of GenSeg by comparing it with several top-performing attacks, i.e., DAG [Xie et al., 2017], SegPGD [Gu et al., 2022], CosPGD [Agnihotri and Keuper, 2023], and Prox [Rony et al., 2023]. Since most attacks are iterative instance-specific methods, we designate three source models, i.e., PSPNet-R50 (PSP-R50) [Zhao et al., 2017; He et al., 2016], DeepLabv3-R101 (DLv3-R101) [Chen et al., 2017], and Mask2FormerSwin-S (M2F-Swin-S) [Cheng et al., 2022; Liu et al., 2021] for these attacks to generate adversaries and conduct transfer-based attacks. As to the victim target models, we use PSPNet-R50 (PSP-R50), PSPNet-R101 (PSP-R101), DeepLabv3-R50 (DLv3-R50), DeepLabv3-R101 (DLv3R101), Mask2Former-R101 (M2F-R101), Mask2FormerSwin-T (M2F-Swin-T), and Mask2Former-Swin-S (M2FSwin-S) for comprehensive evaluations. Both source and target models are well-trained within the same domain.

To ensure a fair comparison across these attacks, we reimplement them using the setting of the step size 3/255 and the perturbation budget 8/255. Besides, we set attack iteration to 5, striking a balance between attack capability and efficiency. Note that, unless otherwise specified, this setting generally applies to the iterative attacks mentioned later in this paper.

The results on Pascal VOC are summarized in Table 1. GenSeg demonstrates a strong attack capability across multiple SS models, showcasing its effective transferability. Although existing adversaries successfully deceive the source model (results in red in the table), their transferability to other target models remains limited. This limitation stems from instance-specific attacks heavily relying on the decision boundary of the specific source model. In contrast, GenSeg operates independently of the source model and employs an intermediate-level attack, which helps to enhance its attack transferability. Furthermore, the efficiency of GenSeg stands out as it generates adversaries in a single forward pass, presenting a more streamlined approach compared with the iterative attacks. For more details, please refer to Appendix F.

We also conduct experiments on Cityscapes and ADE20K, respectively. For detailed results, please refer to Appendix C. The results show that the adversaries generated by GenSeg effectively deceive a range of top-performing SS models, exhibiting the highlighted attack transferability of GenSeg.

Instance Segmentation: We explore the attack efficacy of GenSeg on IS, compared with the popular methods MIM [Dong et al., 2018], Improved PGD (I-PGD) [Zhang et al., 2022b], and DIM [Xie et al., 2019]. As to the selection of source and target models, we choose Mask-RCNN (MRCNN) [He et al., 2017], Yolact [Bolya et al., 2019], PolarMask [Xie et al., 2020], and Mask2Former (M2F) [Cheng et al., 2022]. These models represent four diverse solutions for IS: top-down, one-stage, contour-based, and query-based, respectively. As their structures are totally different, we use the same backbone ResNet50 for each model.

The results on Cityscapes are presented in Table 2, and the results on COCO and ADE20K are provided in Appendix C. Similar to SS, although iterative attacks own high efficacy in white-box attack cases, their transferability to different target models remains deficient. Note that, this deficiency is more pronounced in comparison to SS, as the huge divergence among different IS models exacerbates the challenge. In contrast, GenSeg demonstrates robust transferability across diverse models as it is detached from any source model. Moreover, the intermediate-level attack is applicable to all solutions. Once the feature space is corrupted, all the subsequent head networks will be misled.

Panoptic Segmentation: We also explore the attack transferability of GenSeg on PS. To compare GenSeg with other top-performing iterative attacks (PGD [Madry et al., 2017] and AutoPGD [Daza et al., 2022]), we select four end-to-end PS models, i.e., DETR [Carion et al., 2020], MaskFormer (MF) [Cheng et al., 2021], Mask2Former (M2F) [Cheng et al., 2022], and kMax-DeepLab (kMDL) [Yu et al., 2022]. The backbone is selected to be ResNet50 for each model.

The results on Cityscapes are shown in Table 3, and the results of COCO and ADE20K are depicted in Appendix C. GenSeg performs a good transferability to attack a diversity of models across different datasets, which is superior to existing instance-specific approaches.

Cross-model Cross-domain Attack

This scenario is more practical in the real world, as we have no access to either the deployed model structure or the target domain. Considering that we have no knowledge about the target domain and the guidance of the ground-truth mask, current iterative instance-specific attacks are unable to generate adversaries for unknown domains, as the misalignment of output dimension and ground-truth mask missing. However, as only one input image is required for GenSeg, it can be technically applied to generate adversaries for any unknown target domain.

We evaluate the cross-model cross-domain attack capability of GenSeg. For SS, we leverage GenSeg trained on source domains to generate adversaries to attack victim models that are trained on different target domains. We utilize Pascal VOC, Cityscapes, and ADE20K datasets to explore the crossdomain attack transferability. As to the victim target models, we select PSPNet-R50 (PSP-R50) [Zhao et al., 2017; He et al., 2016] and DeepLabv3-R50 (DLv3-R50) [Li et al., 2018] to test their mIoU results. The results of mIoU drop are reported in Table 4. As evident from the table, in the crossdomain attack scenario, GenSeg induces mIoU drops ranging from 28.4% to 45.7% for the target models. While this effectiveness is comparatively lower than that in the intra-domain attack, the substantial absolute value of mIoU drop remains noteworthy. This indicates that GenSeg possesses the capability to transfer and attack victim models trained on unknown domains without overly relying on the source domain.

Moreover, the settings and results of IS and PS are detailedly demonstrated in Appendix D. It can be observed that the adversaries produced by GenSeg can decrease the performances of victim models trained on other domains by 23%- 47%, which is sufficient to bring potential risks to deceive the unknown deployed models trained on unknown domains.

4.3 Ablation Study

We conduct thorough ablation experiments to determine the optimal setting for GenSeg and analyze the potential reasons. Utilizing GenSeg trained on Cityscapes with SS, IS, and PS masks, we conduct attacks on corresponding target victim models. Specifically, we employ each corresponding GenSeg to attack PSPNet for SS, Yolact for IS, and Mask2Former for PS. The attack efficacy is measured in terms of the metrics drops in mIoU, AP, and PQ scores, respectively.

Loss combination: Table 5 illustrates the attack capability of GenSeg when trained on various loss combinations. Basically, the amalgamation of Lori MA, Laug MA, Lintra, and Linter shows the most effective attack across all segmentation tasks, namely Group (6). By contrast, Groups (1) and (2) indicate the efficacy of the augmentation in enhancing attack transferability; Groups (3) and (4) highlight the advantageous use of Lintra and Linter, respectively, as they obscure the representation space in a category-specific manner, significantly bolstering the overall attack capability; Group (5) underscores the pivotal role of Lori MA in ensuring a strong attack.

(b) with �푀

Table 6: Ablation results on augmentation-based training of GenSeg.

To further analyze the impact of each loss, we illustrate the variations of representation space in Figure 3. These prototypes are obtained based on PSPNet with the guidance of ground-truth masks on Cityscapes for SS. As shown in the figure, the intra-class and inter-class prototypes of benign features are compacted and separate, respectively. With the introduction of the mutual agreement loss, the feature space tends to be obfuscated without any regular pattern. However, with the usage of Lintra, the prototypes within each specific category tend to be uncompacted, which leads to an ambiguous decision boundary. On the other hand, Linter mixes the prototypes of different categories together, significantly reducing the margin of the decision boundary. Therefore, the mutual agreement loss brings a random ambiguity to the latent space, while the intra-class and inter-class losses give a directional perturbation to damage the category relationships. We take the advantages of them to formulate the total objective for GenSeg with a stronger effectiveness. Note that, Lintra and Linter act as auxiliary objectives to further enhance attack ability. As to the reasons, we believe that the attack directions of Lintra and Linter are related to the prototype space of Dϕ, whereas the omnidirectional attack LMA is model-agnostic. However, none of these attacks are domainagnostic, leading to relatively inferior results of cross-domain attacks.

Augmentation: We investigate the efficacy of the augmentation-based training strategy. To guarantee pixelwise alignment for segmentation tasks, we employ augmentations without spatial transformation, including Gamma Transformation, Pepper Noise, and Gaussian Noise, compared with the standard training without any augmentation. As demonstrated in Table 6, Gaussian Noise augmentation achieves the best results. The augmentations can make GenSeg robust to slight input transformations and focus specifically on perturbing the latent space, which significantly enhances the attack effectiveness.

Mutual agreement: According to Eq. (6), we use JS divergence to distinguish between the features of clean images and adversaries. We assess alternative metrics like cosine similarity and Euclidean distance as mutual agreement measures. Table 7 displays the results, indicating JS divergence as the superior metric. Compared with Euclidean distance, JS divergence's normalized range of 0-1 aids in optimizing the total loss. Furthermore, compared with cosine similarity, JS divergence avoids redundancy in attacks, as the prototype obfuscating loss also employs cosine similarity for intra-class and inter-class attacks. Moreover, JS divergence, following the softmax operation, highlights the salient features. Targeting these features can further deceive the segmentation model, as they encode crucial image patterns.

Dynamic weight: We explore the results of using dynamic weight in the mutual agreement loss, compared with the equal weight on all pixels, and the results are depicted in Table 8. As shown in the table, the dynamic weight strategy causes a larger metrics drop compared with the equal weight assignment. By assigning larger weights to the inadequateperturbative representations, we are able to lead numerous pixel-wise representations apart from their benign states. This will impose obstacles on the following head network and indirectly impact the final predictions.

Discriminator: We use well-trained ResNet50 and Swin-S as discriminators for feature extraction. For each model, we extract intermediate features from the output of stages 2, 3, 4, and a hybrid output [Xie et al., 2021] that incorporates them together, respectively, as the output of Dϕ. Bilinear interpolation is then applied to match the output size to the input image. Table 9 shows the results, highlighting the superior attack based on the hybrid feature. Moreover, the results of ResNet50 are close to Swin-S. On the one hand, the mutual agreement loss does not heavily rely on the feature quality, as it concentrates more on expanding mutual distances. On the other hand, the prototype establishment depends on semantic features, and the hybrid feature leverages multi-scale information, enhancing the prototype obfuscating attack. Considering the efficiency, we apply ResNet50 as Dϕ for feature extraction.

4.4 Generalized Cross-task Attack

The above attacks necessitate training GenSeg for each task and each dataset separately. However, our ultimate goal is to train a single GenSeg capable of generating adversaries across all segmentation tasks. Unlike MaskFormer, achieving this objective with GenSeg is technically feasible, as it only requires outputting the common feature map and employing a unified mask formulation for optimization.

Considering PS as a fusion task of SS and IS, we explore whether GenSeg trained on PS can generate adversaries to deceive SS and IS models. Specifically, we train GenSeg on Cityscapes with panoptic masks. Subsequently, GenSeg generates adversaries to attack PSPNet-R50 and DeepLabv3R50 in SS, as well as Yolact and PolarMask in IS. To simulate a general scenario, we employ Pascal VOC and ADE20K datasets for SS, as well as COCO and ADE20K for IS. The results, shown in Table 10, reveal that the attacks cause approximately 16%-26% metric drops. As segmentation models share similar semantic features, attacks targeting the latent space affect all subsequent head networks, despite variations in models, domains, and tasks. Of course, there is still ample room for enhancing attack efficacy in the cross-task manner.

5 Conclusion

In this paper, we introduced GenSeg, a unified adversarial attack paradigm tailored for multiple segmentation tasks. GenSeg employs mutual agreement loss and prototype obfuscating loss, showcasing robust attack transferability across diverse models, domains, and tasks. Moreover, the effective adversary is generated with a single forward pass. Extensive experiments demonstrate that the black-box attack capability of GenSeg outperforms the top-performing iterative instancespecific attacks across 3 tasks, 9 datasets and 15 models in total.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 62172385), the Innovation Program for Quantum Science and Technology (No. 2021ZD0302900), the Anhui Provincial Department of Science and Technology under Grant 202103a05020009, and the Laboratory for Advanced Computing and Intelligence Engineering Fund.