Detecting Adversarial Faces Using Only Real Face Self-Perturbations

1 Introduction

1 Deep neural networks have been widely used in many tasks [Ou et al., 2017; Shi et al., 2020; Zhang et al., 2021], and have achieved remarkable success in facial recognition systems (FRS) [Deng et al., 2019] with a wide range of real-world applications such as online payment [Avarikioti et al., 2019] and financial management [Dumbre et al.,

∗Corresponding author

1 Code at https://github.com/cc13qq/SAPD

2016]. However, deep neural networks are known to be vulnerable to adversarial attacks [Goodfellow et al., 2014; Kurakin et al., 2016; Madry et al., 2017], making commercial FRSs unreliable.

Some research attempts to defend against adversarial attacks by adversarial example detection techniques [Feinman et al., 2017; Ma et al., 2018; Tian et al., 2021] which filter out adversarial samples before feeding them into protected systems. However, they tend to overfit to known attacks and do not generalize well to unseen advanced attacks [Wu et al., 2021; Yang et al., 2021]. In specific, GAN-based adversarial attacks such as AdvMakeup [Yin et al., 2021] and AMT-GAN [Hu et al., 2022] are recently developed to generate more natural adversarial faces (adv-faces) by adding completely different noise patterns to face images in contrast to traditional gradient-based attacks such as FGSM [Goodfellow et al., 2014], and are able to circumvent detection. Therefore, a plug-and-play method of detecting adv-faces with improved generalization performance over both unseen gradient-based attacks and unseen GAN-based attacks is highly demanded.

Due to the insight that adversarial samples are all modified from real samples [Ma et al., 2018], one direct way to encourage models to learn generic representations for detecting adv-faces is to train models with synthetic data [Shiohara and Yamasaki, 2022], forming a decision boundary wrapping the synthetic subspace. To achieve this, we investigate the intrinsic generality of adv-faces generated by gradient-based attacks and GAN-based attacks respectively, and propose two assumptions: (1) Gradient-based attacks have a resemblance in noise pattern on account of the consistency of the basic attack algorithm which uses gradients to construct adversarial examples. (2) GAN-based attacks aim to simultaneously change the prediction of FRSs while preserving visual quality. In the meantime, FRSs pay the most attention to highfrequency regions [Li et al., 2020], which induces GANs to modify these regions. Therefore, GAN-made adv-faces have manipulated clues like abnormal color aberrations in highfrequency regions. If universal noise patterns that cover all kinds of adversarial noises are attainable, a set of pseudo advfaces could be produced, and we can train a model with them to learn generic representations for adv-faces.

Based on the above assumptions, we propose three kinds of real face self-perturbations synthesized with pseudo noise patterns which summarize adversarial noises of gradientbased attacks and GAN-based attacks, and a max-poolingbased classifier focusing on capturing abnormal local color aberration. Unlike previous methods, this data-augmentationlike method is proposed from a novel and more intrinsic perspective by investigating the generality of adversarial noise patterns, to detect all recent adv-faces from unseen attacks, protecting FRSs without access to them. Trained on only real faces and their self-perturbations, as shown in Figure 1, our model is able to detect all the recent adv-faces. This new framework simultaneously ensures detection performance on unseen adversarial attacks and portability in use, without any access to protected systems.

Although we are trying to make self-perturbed faces as general as possible, the trained network still overfits to indomain data and might fail to classify adv-faces far away from the distribution of self-perturbed faces. Regarding advfaces as out-of-distribution (OOD) data from a shifted data manifold [Deng et al., 2021], we incorporate a regularization term to narrow the model's decision boundary of real face class during training, and naturally introduce a novel cascaded system for adv-face detection, which consists of training data self-perturbations, decision boundary regularization, and max-pooling-based binary classifier. We evaluate our approach on LFW and CelebA-HQ datasets with eight gradientbased attack methods and two GAN-based attack models. Experiment results validate our assumptions and demonstrate the reliability of our detection performance on adv-faces from unseen adversarial attack methods.

Our contributions are summarized in four thrusts:

• By investigating the intrinsic similarities among varying adv-faces, we propose two assumptions that (1) different adversarial noises have resemblances, and (2) color aberrations exist in high-frequency regions. We empirically validate the assumptions which in turn indicate universal noise patterns are attainable for all the recent adv-faces.
• Based on our assumptions, we propose three kinds of real face self-perturbations for gradient-based adv-faces and GAN-based adv-faces. We are the first to train an adv-face detector using only real faces and their self-perturbations, agnostic to victim FRSs and agnostic to adv-faces. This enables us to learn generalizable representations of adv-faces without overfitting to any specific one.
• We then naturally introduce a novel cascaded system for adv-face detection, which consists of training data selfperturbations, decision boundary regularization, and a maxpooling-based binary classifier focusing on abnormal local color aberrations.
• Evaluations are conducted on LFW and CelebA-HQ datasets with eight gradient-based attacks and two GANbased attacks, demonstrating the consistently improved performance of our system on adv-face detection.

2 RelatedWork

Adversarial attacks Adversarial attacks [Goodfellow et al., 2014] aim to disturb a target system by adding subtle noise to input samples, while maintaining imperceptibility from human eyes. In contrast to previous works

[Madry et al., 2017; Kurakin et al., 2016], DIM [Wu et al., 2021] and TIM [Dong et al., 2019] are improved attack algorithms with enhanced black-box attack accuracy and breaching several defense techniques. Focusing on breaking through FRSs, TIPIM [Yang et al., 2021] generates adversarial masks for faces to conduct targeting attacks while remaining visually identical to the original version for human beings. Recently, GAN-based attack models [Yin et al., 2021; Hu et al., 2022] are presented to generate adversarial patches and imperceptible adv-faces, bringing new challenges to advfaces detection. In this paper, we proposed a simple yet effective detection method against all the above attacks while being blind to them during training.

Adversarial example detection One of the technical solutions for protecting DNNs from adversarial attacks is adversarial example detection [Feinman et al., 2017], aiming to filter out adversarial inputs before the protected system functions. Remedying the limitations of the previous method, LID [Ma et al., 2018] is proposed for evaluating the proximity of an input to the manifold of normal examples. Incorporated with wavelet transform, SID [Tian et al., 2021] is able to transform the decision boundary and can be collaboratively used with other classifiers. While all of these methods focus on utilizing features subtracted by FRS backbone and show effective performance in detecting adversarial examples, they require modifying or visiting the protected system and do not generalize well to new-type attacks like GAN-based AMTGAN [Hu et al., 2022] and AdvMakeup [Yin et al., 2021]. To address the problems above, our method is proposed to detect all recent adv-faces from unseen attacks, protecting FRSs without access to them.

Out-of-distribution (OOD) detection OOD detection techniques [Girish et al., 2021] have been widely used in image classification tasks [Yu et al., 2019], trying to recognize examples from an unknown distribution. ODIN [Liang et al., 2017] uses temperature scaling and tiny perturbations to the inputs to separate the in-distribution (ID) and OOD images. Lee et al. [Lee et al., 2018] uses the Mahalanobis distance to evaluate the dissimilarity between ID and OOD samples. By sampling and synthesizing virtual outliers from the lowlikelihood regions, VOS [Du et al., 2022] adaptively regularizes the decision boundary during training. Sun et al. [Sun et al., 2021] introduce a simple and effective post hoc OOD detection approach utilizing activation truncation. Regarding adv-faces as OOD data, we leverage an uncertainty regularization term to narrow the decision boundary of real face class during the training phase to boost the accuracy of adv-faces detection.

3 Methodology

We propose a plug-and-play cascaded system for advfaces detection method which consists of training data selfperturbations, decision boundary regularization, and a maxpooling-based binary classifier focusing on abnormal local color aberrations, agnostic to unseen adversarial attacks, and agnostic to victim FRSs. Training pipeline as shown in Figure 2. We propose to synthesize diverse self-perturbed faces Algorithm 1 Self-perturbation for gradient-based attack Input: A empty perturbation matrix ηp ∈ RH×W×3 with the same shape of real face image xr . Parameter: Max perturbation magnitude ϵ, pattern mode. Output: Self-perturbed face image xp . 1: A random direction matrix R = {⃗rij} ∈ RH×W×3 . 2: for ⃗ηij in ηp do 3: Select random noise value α. 4: if pattern mode is 'point-wise' then 5: ⃗ηij := α · ⃗rij. 6: else if pattern mode is 'block-wise' then 7: Select a random neighborhood Aij of ⃗ηij . 8: for ⃗ηijk in Aij do 9: ⃗ηijk := α · ⃗rij. 10: end for 11: end if 12: end for 13: Clip perturbation ηp using Equation 2. 14: Generate self-perturbed face xp using Equation 3. 15: return xp by adding three noise patterns to real face images, summarizing adv-faces generated from gradient-based and GANbased attacks. A convolutional neural network is learned via real faces and self-perturbed faces, regularized by uncertainty loss, to distinguish real and adversarial faces. As no attack method is observed during training, the resulting network is not biased to any attack, yielding generic and discriminative representations for detecting unseen adv-faces. Therefore, in the testing phase, the embeddings are sent to the learned MaxPooling Classifier to accomplish prediction.

3.1 Real Face Self-Perturbations

Self-perturbation for gradient-based attack

Given a real face image xr , adversarial attack generates an adversarial image xa by adding a perturbation image η to xr [Goodfellow et al., 2014]:

where ||η|| ≤ ϵ, and ϵ called perturbation magnitude.

We observe that a binary classifier trained on real faces and adv-faces generated by FGSM is able to classify a part of attack images generated from other attack methods as shown in Table 2. This generalization in classification means that attack noises have intrinsic similarities. And it means once we master all the similarities we master the attacks, even for unseen attacks. This leads to non-trivial designs for our selfperturbation image ηp .

The noise value of neighbor points in ηp may be the same or different. From this perspective, we introduce point-wise and block-wise noise patterns for gradient-based attacks. As presented in Algorithm 1, we perturb each point in the pointwise pattern and each block in the block-wise pattern in a stochastic direction, where blocks are random neighborhoods of a set of scattered points. The generated perturbation image ηp is constrained in l∞ norm, and clipped according to ϵ,

Algorithm 2 Self-perturbation for GAN-based attack

Input: Real face image xr . A empty perturbation matrix ηp = {ηij} with the same shape of xr .

Parameter: Max perturbation magnitude ϵ, face landmarks, high-frequency threshold γ, and empty high-frequency pixel set H.

Output: Self-perturbed face image xp .

Function: value(x) measures pixel value of x, neighborhood(x) is a random neighborhood of x.

• 1: Obtain gradient image of xr using Sobel operator.
• 2: Obtain high-frequency convex hull from gradient image according to landmarks.
• 3: for xij in convex hull do
• 4: if value(xij) < γ then
• 5: ηij join to H.
• 6: end if
• 7: end for
• 8: Random select a subset Hs of H.
• 9: for ηH ij in Hs do
• 10: Generate a random gradient color patch P.
• 11: neighborhood(ηH ij ) := P.
• 12: end for
• 13: Clip perturbation ηp using Equation 2.
• 14: Generate self-perturbed face xp using Equation 3.
• 15: return xp

And self-perturbed face is calculated as

Self-perturbation for GAN-based attack

Spectrums of real and fake images distribute differently in high frequency [Luo et al., 2021], such as eyes, nose, and mouth. This distribution difference should become more intense because GAN-made adv-faces aim to change the classification results of FRSs meanwhile maintain perceptual invariance. This leads to color aberration and boundary abnormality in high-frequency regions on account of that FRSs pay the most attention to these regions. Focusing on producing natural and imperceptible attack noise, these abnormal color aberrations should have gradient or blurred boundaries. In such a perspective, we use gradient color patches to act as self-perturbation.

As presented in Algorithm 2, we first obtain the gradient image of a real image using Sobel operator [Duda and Hart, 1973] and convex hull of the high-frequency area according to facial landmarks. The pixels in the convex hull are selected as high-frequency pixels if their values surpass a threshold γ. Then we generate a few gradient color patches through a series of affine transformations. Finally, the pseudo adv-faces are generated by adding gradient color patches to a random set of high-frequency pixels.

After crafting a part of real faces to self-perturbed faces as negative samples, we label other real faces as positive samples and train a well-designed backbone network such as XceptionNet [Chollet, 2017] in a binary classification manner.

3.2 Max-Pooling Classifier

In view of abnormal speckles in GAN-based adv-faces are always tiny and are not easy to be observed, we proposed MaxPooling Classifier (MPC) to capture abnormal local color aberrations. MPC produces classification scores and predicts whether input images are real or adversarial.

A typical backbone network includes a feature extractor f(·; φ) composed of several convolutional blocks, and a classifier g(·; ϕ) comprising an average pooling layer, an activation layer, and a fully connected layer [Chollet, 2017]. To detect adv-faces with abnormal local color aberrations, a simple derivation is to divide an image into several rectangle areas, and if abnormal color aberration occurs in any of the rectangle areas we classify this image as adv-face.

Input an image x to feature extractor f(·;φ), it produces a feature map M(x) ∈ RN×N×d composed of N × N features, where d is the embedding size, and each feature is corresponding to a rectangle area i.e. the receptive field. Features are sent to the activation function and fully connected layer respectively and produce N × N logits. We take the max-pooling of these logits as the final logit and obtain prediction score Scls ∈ R2 by softmax operation. Classification of input image x is computed by argmax function:

where Gcls(x) = 1 indicates real face and Gcls(x) = 0 indicates adv-face.

3.3 Decision Boundary Regularization

Although we are trying to make self-perturbed faces as general as possible, the trained network still overfits to ID data and might fail to classify faces far away from the distribution of negative samples. Regarding adv-faces as OOD data, we incorporate a regularization term [Du et al., 2022] to our work to enhance adv-face detection performance.

Assuming the feature representation of real faces forms a multivariate Gaussian distribution, we sample virtual outliers V ⊂ Rm from the ε-likelihood region of the estimated classconditional distribution:

where ̂µ and ̂Σ are the estimated mean and covariance using latent features of real faces.

The uncertainty loss regularizes the model to produce a low OOD score for ID data and a high OOD score for the synthesized outliers, and narrows the decision boundary of real face class to boost the performance of adv-face detection:

where Dr represents distribution of real face, f(·; θ) is a linear transformation function, and ϕ(·) is a nonlinear MLP function. The learning process shapes the uncertainty surface, which predicts a high probability for ID data and a low probability for virtual outliers v.

The training objective combines the real-adv classification loss and the regularization term:

where D represents the distribution of training data. β is the weight of the regularization Luncertainty, and Lcls is the CrossEntropy classification loss [Zhang and Sabuncu, 2018].

4 Experiment

To validate the detection performance of our approach on adv-faces generated by various adversarial attack methods, we conduct extensive empirical studies on two datasets in this section. We validate our assumptions by investigating the intrinsic similarity of adv-faces and compare our method to baselines from three research streams. After that, we analyze our approach through a series of ablation studies.

General setup Duringmodel training, real faces are used as positive samples, and only self-perturbations of real faces are used as negative samples. Any adv-faces are agnostic. During testing, all the negative samples are adv-faces. In detail, half of the real faces in the training phase are labeled 1 as the positive samples, and others are self-perturbed and labeled 0 as the negative samples. In the testing phase, all of adv-faces are labeled 0. For evaluating the performance of detectors, we choose the widely used AUC score as the main metric.

Datasets Face images in this work are sampled from LFW [Gary et al., 2007] and CelebA-HQ [Karras et al., 2017] datasets. LFW contains 13,233 face images of 5,749 subjects. Subjects with at least two face images take part in advface producing. The first image of each subject is regarded as a reference and the others are sampled to produce advfaces which includes 7,484 images. CelebA-HQ is a highresolution subset of CelebA, containing 30,000 images.

Attack methods We employ eight recent gradient-based adversarial attack methods FGSM [Goodfellow et al., 2014], BIM [Kurakin et al., 2016], PGD [Madry et al., 2017], RFGSM [Tram`er et al., 2018], MIM [Dong et al., 2018], DIM [Wu et al., 2021], TIM [Dong et al., 2019] and TIPIM [Yang et al., 2021], and two GAN-based attack methods: AdvMakeup [Yin et al., 2021] and AMT-GAN [Hu et al., 2022] as attackers, and ArcFace [Deng et al., 2019] as victim FRS to produce adv-faces on gradient-based attacks. All the gradient-based attacks are applied on LFW and CelebA-HQ datasets to generate adv-faces, while AdvMakeup and AMTGAN are on the CelebA-HQ dataset.

Implementation details We modify an ImageNet-pretrained XceptionNet [Deng et al., 2009; Chollet, 2017] as the backbone network in our method. We set N = 7 to produce a 7 × 7 feature map in the last convolution layer and choose ReLU as an activation function. We utilize DLIB [Sagonas et al., 2016] for face extraction and alignment, Torchattacks [Kim, 2020] for generating adv-faces, and OpenOOD [Girish et al., 2021] for network training. All face images are aligned and resized to 256 × 256 before training and testing. The perturbation magnitude ϵ in self-perturbations and adv-faces producing is set to 5/255, a small value. Threshold γ in the convex hull of gradient image in Algorithm 2 is set to 50. The regularization loss weight β is set to 0.1. Training epochs are set to 5 and convergence is witnessed.

Baselines Previous adversarial example detection methods [Ma et al., 2018; Tian et al., 2021] focus on specific tasks or gradient-based attacks and hence can hardly be effectively extended to GAN-based adv-faces, while face forgery detection methods [Luo et al., 2021; He et al., 2021] are used to detect GAN-made fake faces. On the other hand, some OOD detection methods [Liang et al., 2017; Lee et al., 2018; Sun et al., 2021] only rely on output features and logits of the backbone network and keep agnosticism to unknown attacks, closing to our setting. On account of this, we compare our method to various methods in three problem settings. All baselines listed in Table 1 include methods of adversarial example detection, OOD detection, and face forgery detection. It is worth noting that only our method does not require either pre-computed adv-faces or access to FRS.

4.1 Assumption Validation

Our assumption for gradient-based noise patterns stands on the visual similarities between adversarial noises. However, the human eye is sometimes unreliable because we cannot observe tiny differences between pixels. To verify the reliability of the hypotheses, we train a simple XceptionNet with real faces from the CelebA-HQ dataset and adv-faces generated by one attack algorithm and test detection accuracy on 1,000 adv-faces per attack. As shown in Table 2, a detector for a specific gradient-based attack is able to generalize to other attacks except those based on GANs. but the success rate on some attacks such as TIM is much lower. We also calculate the Fr´echet Inception Distance of various attacks. What the results demonstrate is that attack noises are to some extent similar to each other, and the way of extracting a universal noise pattern is feasible. The result also indicates that noise patterns of GAN-based attacks are not close to gradient-based attack noise patterns and need to be specifically treated.

4.2 Main Results

Gradient-based adv-face detection We first compare the detection performance of our method to other detectors on gradient-based attacks. Baselines include 2 detectors proposed for adversarial example detection [Ma et al., 2018; Tian et al., 2021] and 3 detectors for OOD detection [Liang et al., 2017; Lee et al., 2018; Sun et al., 2021]. The detectors are trained on real images, along with adv-faces images (LID and SID) or pseudo adv-faces images (Ours). We compute the classification AUC for all methods on a dataset comprising of 1k real images and 1k adversarial face images per attack type in LFW and CelebA-HQ datasets. As shown in Table 3, trained on the known attack (FGSM), previous adversarial example detection methods is difficult to detect unknown attacks. In a contrast, our method reaches a high level of detecting gradient-based adv-faces and almost approaches saturation performance on the LFW dataset. This is likely because the network learns a generic representation for advfaces. The result that the AUC score on LFW is higher than which on CelebA-HQ may contribute to the fact that CelebAHQ is a more complicated dataset with high resolution and diversified background so self-perturbation works much harder on it. Thus we speculate that a more complex and more diversified data environment will increase the difficulty of adv-face detection tasks.

GAN-based adv-face detection To investigate the advantage of our method on detecting GAN-based samples, we make comparisons with SOTA methods of face forgery detection [He et al., 2021; Luo et al., 2021] and OOD detection [Liang et al., 2017; Lee et al., 2018; Sun et al., 2021]. We train our model with only real faces and self-perturbed faces from CelebA-HQ and test both our approaches and baseline models on GAN-based adv-faces. As shown in Table 4, the performance of our method exceeds which of other algorithms and models. This result verifies our assumption that GAN-made adv-faces have manipulated clues like an abnormal color aberration in high-frequency regions. The observation that detection performance on Adv-Makeup is higher than which on AMT-GAN is likely because abnormal local aberrations of Adv-Makeup are more obvious than that of AMT-GAN. Besides, the result shows that the detection of face forgery can not generalize to adv-faces although both deepfake faces and adv-faces are generated by GANs. This is possible because a huge difference exists between the fingerprints of deepfake GANs and that of adv-face GANs.

4.3 Ablation

Component ablation As argued, self-perturbation makes the detector learn a more generic representation of adv-faces, Max-Pooling Classifier captures abnormal local color aberrations, and the regularization term helps to boost adv-faces detection. We then conduct an ablation study to verify the effectiveness of each component. As result shown in Table 5, detectors trained without self-perturbation totally failed in detecting adv-faces from TIPIM and GAN-based attacks, indicating the indispensability of self-perturbation in detecting advanced adversarial attacks. There is also an obvious gap between using and not using Max-Pooling Classifier, especially on GAN-based adv-faces. Narrowing the decision boundary, the regularization term helps to filter out a small number of adversarial samples which are not similar enough to selfperturbed faces, further improving detection performance. As for some gradient-based adv-faces such as DIM, detection accuracy is high enough regardless of using Max-Pooling Classifier and a regularization term. It is likely because selfperturbation is similar enough to attack noise patterns so that the representations learned by networks are generic enough.

Complementarity of self-perturbations To explore the necessity and complementarity of self-perturbations, we train a simple XceptionNet on self-perturbed faces generated from CelebA-HQ using only a single self-perturbation. The results are shown in Table 6. As we can see, training a detector only relying on one of the self-perturbations is insufficient for detecting all unseen adv-faces. Due to the differences in generation procedures, self-perturbations are complementary to each other in the detection tasks.

4.4 Analysis of Our Approach

Functionality of self-perturbations To explore the functionality of self-perturbations for gradient-based attacks, we extract noise features and face features of adversarial, selfperturbed, and real faces and visualize them using 2D t-SNE projection. As Figure 3 shows, self-perturbation is very close to attack noise and far from zero-vector (real faces). By separating real faces from self-perturbed faces, the trained network is able to distinguish between real and adv-faces indirectly. Although some adv-faces generated by TIM are not covered by self-perturbations, a model trained on decision boundary regularization still separates them from real faces.

Impact of hyper-parameter Another experiment is about the choice of ϵ. We train a simple XceptionNet on FGSM adv-faces at ϵ = 1/255 to ϵ = 15/255 and test on adv-faces at different ϵ. The result shows that detectors trained on a smaller ϵ are able to detect adv-faces generated with a larger ϵ. But an extremely small ϵ may cause failure in model training. In practice, we do not need to select an extremely small value for ϵ because the attack success rate is too low to take into account as shown in the Appendix. In consideration of this, we choose ϵ = 5/255 (attack success rate lower than 1%) for producing and detecting adv-faces.

5 Conclusion

In this paper, we investigate the intrinsic similarities of recent adv-faces and heuristically design three kinds of real face self-perturbations close to attack noise pattern. Regarding adv-faces as OOD data, we propose an FRS-agnostic and attack-agnostic cascaded system for adv-faces detection, which includes real face self-perturbations, decision boundary regularization, and Max-Pooling Classifier focusing on abnormal local color aberrations. Trained on only real faces and self-perturbed real faces, our model learned generic representations for adv-faces. Comprehensive analysis validates the proposed assumptions that noises of adv-faces have intrinsic similarities and exist in high-frequency areas, and extensive experiments demonstrate the improved effectiveness of our method compared with several recent baselines. We also observe that the latent representation learned by the detector is hard to generalize to other domains. For future work, we will try to solve this matter.

Acknowledgements

This work was supported in part by the Natural Science Foundation of China under Grant 61972169, in part by the National key research and development program of China(2019QY(Y)0202, 2022YFB2601802), in part by the Major Scientific and Technological Project of Hubei Province (2022BAA046, 2022BAA042), in part by the Research Programme on Applied Fundamentals and Frontier Technologies of Wuhan(2020010601012182) and the Knowledge Innovation Program of Wuhan-Basic Research, in part by China Postdoctoral Science Foundation 2022M711251.