On the Effects of Fairness to Adversarial Vulnerability

1 Introduction

Data-driven learning systems have become instrumental for decision-making in a variety of consequential contests, including assisting in legal decisions, [Jayatilake and Ganegoda, 2021], lending, [Stevens et al., 2020], hiring, [Schumann et al., 2020], and providing personalized recommendations. [Burke, 2003]. Consequentially, fairness has emerged as a critical requirement for adoption and usage of these systems. Various notions of fairness drawing from legal and philosophical doctrine have been proposed to ensure that the models' errors do not affect specific groups [Mehrabi et al., 2021].

In general, fair models attempt at constraining their hypothesis space so that errors in reported outcomes are uniformly distributed across different protected groups [Mehrabi et al., 2021]. When these fairness constraints are enforced in learning systems, a commonly observed behavior is an overall degradation of the model accuracy. Thus, a growing body of research has been focusing on striking the right balance between fairness and accuracy [Rodolfa et al., 2021]. This paper shows that fairness may have another important consequence on the deployed models: a reduction of the model robustness. Given the susceptibility of deep learning models to adversarial attacks in security-sensitive applications, this is an important yet underexplored issue.

Contribution. This paper shows that enforcing fairness may negatively affect the robustness of a model. Specifically, the paper (1) analyzes when and why fairness and robustness may be misaligned in their objectives, (2) provides an understanding on the relationship between fair, robust, and "natural" (e.g., non-fair non-robust) models, and (3) identifies the distance to the decision boundary as a key aspect linking fairness and robustness. Moreover, (4) the paper shows how the distance to the decision boundary can explain the increase of adversarial vulnerability of fair models, providing validation over a variety of tasks and architectures, and verifying the presence of the fairness/robustness dichotomy for multiple techniques aimed at achieving fairness and measuring robustness. Finally, (5) building from the reported theoretical observations, the paper also proposes a simple, yet effective, strategy to find a good tradeoff between accuracy, fairness, and robustness.

The results show that, without careful considerations, inducing a desired equity property may create significant security challenges. These results should not be read as an endorsement to avoid constructing fairer or safer models; rather as a call for additional research to achieve appropriate tradeoffs.

Relation with previous work. We discuss related work in Appendix A 1 and highlight here the distinguishing features of this study in the context of robustness and fairness research. The intersection of fairness and robustness has received limited attention thus far, with only a handful of studies examining this area. For instance, [Xu et al., 2021a] recently showed that adversarially robust models can exhibit significant accuracy disparity among different classes, as opposed to their standard counterparts. To address this issue, they proposed a Fair-Robust-Learning framework for adversarial defense. Meanwhile, [Khani and Liang, 2020] analyzed the impact of noise in features on disparities in error rates when learning regression models. While previous studies highlighted how adversarial training can disproportionately harm certain protected groups [Xu et al., 2021a; Nanda et al., 2021], we demonstrate that enforcing fairness comes at the expense of reduced robustness. As a result, the proposed analysis requires a distinct approach from those proposed in earlier studies.

1 Please refer to [Tran et al., 2022c] for the appendices.

2 Problem Settings

Consider a multi-class classification problem, whose input is a dataset D consisting of n data points (Xi↪ Ai↪ Yi), each of which drawn i.i.d. from an unknown distribution ⇧ and where Xi 2 X is a feature vector, Ai 2 A is a protected attribute, and Yi 2 Y = [C] is a label, with C being the number of possible class labels. For example, consider the case of a classifier to predict the age range of an individual. The features Xi may describe the pixels associated with the individual headshot and their demographics, the protected attribute Ai may describe the individual gender or ethnicity, and Yi represents the age range. The goal is to learn a classifier f✓ : X ! Y, where ✓ is a vector of real-valued parameters. The model quality is assessed in terms of a non-negative loss function ` : Y⇥Y ! R+, and the training aims at minimizing the empirical risk function:

For a group a 2 A, notation Da is used to denote the subset of D containing exclusively samples i with Ai = a. Importantly, the paper assumes that the attribute A is not part of the model input during inference. The paper focuses on learning classifiers that satisfy group fairness (to be defined shortly) and on analyzing the robustness impact of fairness.

3 Preliminaries

Fairness and fair learning. This paper considers a classifier f satisfying accuracy parity [Zhao and Gordon, 2019], a group fairness notion commonly adopted in machine learning requiring model misclassification rates to be conditionally independent of the protected attribute. That is, 8(X↪ A↪ Y ) ⇠ ⇧ and 8a 2 A,

where ↵ denotes the allowed fairness violation. In practice, the above is expressed as a difference of empirical expectations of the group and population misclassification rates. That is, 8a 2 A:

Several approaches have been proposed in the literature to encourage the satisfaction of accuracy parity. They can be summarized in methods that use penalty terms into the empirical risk loss function to capture the fairness violations, and those which minimize the maximum group loss. The core of the paper focuses on the first set of methods; the analysis for the second set is presented in Appendix C.

Penalty-based methods. In this category, the model loss function (Equation (1)) is augmented with penalty fairness constraint terms [Agarwal et al., 2018; Tran et al., 2021b] as:

where L✓(Da) = 1◁♣Da♣ P(X↪A↪Y )2Da `(f✓(X)↪ Y ) is the empirical risk loss of protected group a 2 A. In addition, λ > 0 is the fairness penalty parameter that enforces a tradeoff between fairness and accuracy.

Robustness and robust learning. Following robust learning conventions, the robustness of a model f is measured in terms of the robust error:

which measures the sensitivity of the model errors to small input perturbations k⌧kp  ✏ in `p norms, with p often considered in ¶0↪ 1↪ 2↪1♦. Robust errors can be decomposed into two components [Zhang et al., 2019]:

where the first denotes the natural error and the second the boundary error. The natural error measures the standard model performance when exposed to unperturbed samples (X↪ A↪ Y ):

whose empirical version is defined in Equation (1) with a 0/1 loss function. The boundary error measures the probability that the model predictions change on perturbed samples (X + k⌧ kp↪ A↪ Y ):

The concept of boundary error inherently involves a decision boundary as well as a distance between an input sample and the decision boundary. In the context of linear classifiers, this boundary is typified by a hyperplane. The distance from a sample X to the decision boundary in a classifier f✓ is expressed as:

Note that samples close to the decision boundary will be less tolerant to noise than those lying far from it. The analysis in this paper regarding the impact of fairness on robustness is based on this concept. In particular, the results show that imposing fairness constraints may reduce the distance to the decision boundary of the samples (X↪ A↪ Y ) ⇠ ⇧.

4 Real-World Implications

Prior diving into the analysis, we provide an example showing how robustness errors can be exacerbated when a image classifier is trained to satisfy fairness. When perturbations (either due to noise or by malicious adversaries) are introduced in the model inputs, they may cause harmful effects as they lead the classifier to misclassify targeted inputs.

Figure 1 shows UTKFace dataset examples processed by classifiers trained either on the standard empirical risk loss from Equation (1) (top) or the fair empirical risk loss from Equation (3) (bottom). Although both inputs receive identical

`1 noise perturbations, the fair network is much more brittle, inducing errors in the classifier's outputs.

It is important to note that while this paper uses datasets such as UTKFace (see details in Appendix D) to demonstrate the effects of fairness to robustness, the very task of predicting gender, race, or other characteristics from a person face is flawed and raises deep ethical concerns [Raji et al., 2020].

5 Why Fairness Weakens Robustness?

This section presents the main results of the paper. It will show that fairness affects model robustness because the learned decision boundary is pulled in opposite directions by fair and robust models. To render the analysis tractable, the theoretical discussion focuses on linear classifiers, and more specifically on learning a mixture of Gaussians with a linear classifiers. In addition, Section 6 will show that a similar phenomenon occurs in large non-linear models.

5.1 Optimal Models for Mixtures of Gaussians

Consider a binary classification setting (i.e., Y = ¶-1↪ 1♦) with data drawn from a mixture of Gaussian distributions, so that Pr (X ♣ Y = -1) / N(θ↪ 1) and Pr (X ♣ Y = 1) / N(¯θ↪K2 ), with θ < ¯θ and different variances (K > 1). These non-restrictive assumptions help simplifying and clarifying exposition, but the appendix generalizes the above to higher-dimensional cases. An illustration of this setting is reported in Figure 2 (top) where the data distributions are highlighted with black dashed curves.

The following analysis poses no restrictions on the relative subgroup sizes ♣D1♣ and ♣D-1♣ and focuses on the balanced data setting, in which data samples from different protected groups are equally likely. The paper studies a family of parametric classifiers ¶f✓♦✓ with ✓ 2 [θ↪ ¯θ] ✓ R, where f✓(X) = 1¶X > ✓♦ denotes the classification output of the classifier. The optimal models with respect to the natural, fair, and robust losses can be specified as follows:

• ÷ Optimal natural model (f ?✓ ). It is the Bayes classifier which minimizes the natural classification error as defined in Equation (1). In Figure 2 (top), this classifier is represented

by vertical blue lines.

• ÷ Optimal fair model (f✓f ). Intuitively, this classifier is ✓f(1) as defined in Equation (3). Formally speaking, this classifier minimizes a lexicographic function whose first component is Pa2A ♣(L✓(Da) - L✓(D))♣ and second component is L✓(D). In Figure 2 (top), this classifier is represented by vertical red lines.
• ÷ Optimal robust model (f✓(✏) r ). This classifier minimizes the robust classification error in Equation (5), for a given ✏. In Figure 2 (top), it is depicted by vertical green lines.

5.2 Relationships Between the Optimal Models

The next result characterizes the positional relationship among the three optimal models mentioned above, which can be observed in Figure 2.

Theorem 1. For any ✏ 2 [0↪ (θ-¯θ)◁2] and K 2 (1↪ BK], where BK =min {exp ((θ-¯θ-2✏)2◁2) ↪ (θ-¯θ)◁✏ -1 ↪

Besides, ✓(✏) r is an increasing function of ✏ over [0↪ (θ-¯θ)◁2].

The result follows from the observation that the optimal natural model f ?✓ can be expressed as

the fair classifier f✓f as:

and the robust classifier f✓(✏) r as

We note that [Xu et al., 2021a] derives expressions for the optimal natural and robust models, which are used to investigate the natural error gap between the two classes. Importantly, however, Theorem 1 above provides a unique comparison among the three classifiers analyzed, highlighting the difficulty in achieving both robustness and fairness simultaneously, as fairness and robustness pull the optimal classifier in opposing directions. While [Xu et al., 2021a] focuses on the unfairness resulting from robust training, the remainder of this section examines the cost of adversarial robustness in fair training. Specifically, we measure this reduced robustness cost analyzing robust and boundary errors.

From the relations highlighted in Theorem 1, it follows,

1. fair classifiers achieve the largest robust errors while robust classifiers result in the least error;
2. fair classifiers achieve the largest boundary errors while robust classifiers result in the smallest boundary error, as expressed by the following Corollaries.

where ¯BK = min nexp ⇣(θ-¯θ-2✏)2 2 ⌘↪ φ-1⇣θ-¯θ ✏ -2⌘o and φ-1 is the inverse function associated with φ : [1↪+1) 7! [2↪ +1) such that φ(x) = x + 1◁x.

These results further highlight the impossibility of achieving fairness and robustness simultaneously in this classification task. Fairness and robustness are pulling the classifier in opposite directions.

5.3 The Role of the Decision Boundary

Building on the previous results, this section provides the key theoretical intuitions to explain why fairness increases adversarial vulnerability. It identifies the average distance to the decision boundary as the central aspect linking fairness and robustness, which is formalized in Theorem 2.

Theorem 2. For any ✏ 2 [0↪ (θ-¯θ)◁2] and K 2 (1↪ BK],

In addition, the fair model minimizes the average distance to its decision boundary over all valid classifiers, i.e.,

Theorem 2 indicates that, among the three considered optimal models, the fair model has the smallest average distance to the decision boundary while the robust model has the largest distance. The result above is exemplified in Figure 2. The bottom plots show the losses associated with the optimal natural, fair, and robust models for two choices of K (left and right) while the top plots show the optimal decision boundaries associated with each of the three models - notice they correspond to the minima of their relative losses.

Observe that class Y = 1 has a higher classification error than class Y = -1 under the natural (and thus unfair) classifier f ?✓ . This is intuitive since the conditional distribution Pr (X ♣ Y =1) has much higher variance than Pr (X ♣ Y =-1). Hence, to balance the classification errors, the fair classifier pushes the decision boundary towards the mean of class Y =-1. This increases the error of class Y =-1 while decreasing the error of class Y =1. In contrast, the robust classifier pushes the decision boundary far away from the dense input region, i.e., the mean of the data associated with class Y = -1.

There are a few points worth emphasizing. First, robustness and fairness pull the decision boundary into two opposite directions. Second, the fair model f✓f results in predictions with higher robust errors, when compared to the optimal natural model f ?✓ , and it also increases adversarial vulnerability as the variance K increases. The variance K regulates the difference in the standard deviation of the underlying distributions associated with the protected groups and thus controls the overall distance to the decision boundary. In summary, fairness can reduce the average samples distance of the training samples to the decision boundary which, in turn, makes the model less tolerant to adversarial noise.

This section concludes with another important result. The previous relationships continue to hold even when the optimality conditions of the fair classifier are relaxed, i.e., when λ is taking values different from1. Moreover, the fairness constraints always reduce the distance to the decision boundary among protected groups and this reduction is proportional to the strength of the fairness constraints (or the tightness of the required fairness bound ↵).

Theorem 3. Consider the fair classifier f✓f (λ) that optimizes Eq. (3). It follows that, for any λ 2 ⇣ K-1 K+1 ↪ +1⌘,

which means that the fair classifier f✓f (λ) coincides with the optimal fair classifier f✓f , when the fairness penalty λ is large. while for any λ 2 [0↪ K-1◁K+1],

Moreover, the parameter ✓f (λ) associated with the fair classifier and the average distance to its decision boundary E ⇥∆ (X↪ f✓f (λ) )⇤ are both decreasing as λ increases.

Informally speaking, Theorem 3 states that applying fairness constraint with large enough penalty λ will push the decision boundary towards the negative class (group with smallest variance). As a result, the average distance to the decision boundary of all samples will be reduced.

While this analysis applies to the setting considered in this section, the results are empirically validated on large nonlinear models. For example, Figure 3 compares the performance of a fair CNN model (bottom plots) with λ = 1▷0 against a natural (non-fair) CNN classifier (top plots). The left plots report the task accuracy by each subgroup (denoting races) and average distance to decision boundary (right) of each subgroup. Note how the fair classifier reduces the disparities in task accuracy experienced by the various subgroups. This effect, however, also reduces the overall average distance to the decision boundary. As a consequence, fair models will be more vulnerable to adversarial perturbations.

The next sections focus on assessing these theoretical intuitions onto general non-linear classifiers in a variety of settings and on devising a possible mitigation strategy to balance a good tradeoff between fairness and robustness.

6 Beyond the Linear Case

This section empirically validates the theoretical insights discussed earlier, extending them to more complex architectures, datasets, and loss functions. Our experiments focus on exploring the interplay between fairness, robustness, and error rates in relation to decision boundary proximity. For nonlinear models f✓, calculating this proximity becomes a computational challenge. Hence, we employ a widely-used proxy metric that quantifies the difference between the two highest order statistics of the softmax output [Wang and Loog, 2022].

Datasets. The experiments of this section focus on three vision datasets: UTK-Face [Zhang et al., 2017], FMNIST [Xiao et al., 2017] and CIFAR-10 [Krizhevsky et al., 2009]. The adopted protected groups and labels in the UTK-Face datasets are ethnicity (White/Black/Indian/Asian/Others) or age (nine age bins), resulting in two distinct tasks. For FMNIST and CIFAR, the experiments use their standard labels and assume that labels are also protected groups, mirroring the setting of previous work [Tran et al., 2022a; Ma et al., 2022]. A complete description of the dataset and settings is found in Appendix E.

Settings. The experiments consider several deep neural network architectures, including CNN, ResNet 50 [He et al., 2016] and VGG-13 [Simonyan and Zisserman, 2014]. The former uses 3 convolutional layers followed by 3 fully connected layers. Models trained on the UTK-Face data use a learning rate of 1e-3 and 70 epochs. Those trained on FMNIST and CIFAR, use a learning rate of 1e-1 and 200 epochs, as suggested in previous work [Xu et al., 2021a]. The experiments analyze penalty-based fairness method, RFGSM attacks [Tram`er et al., 2017], and the VGG-13 network, unless specified otherwise. Additional experiments using group-loss focused method (see Appendix C), additional network architectures, and adversarial attacks are reported in Appendix E.

Fairness impacts on the decision boundary. As shown by Theorem 3, fairness reduces the average distance of the testing samples to the decision boundary. This section illustrates how this result carries over to larger non-linear models. Figure 4 reports results obtained by executing the penalty-based fair models on the UTK-Face datasets for ethnicity (top) and age (middle) classification and on CIFAR (bottom). A clear trend emerges: As more fairness is enforced (larger λ values), the natural errors (left plots) increase, while the fairness violations (center plots) decrease. Importantly, and in agreement with the theoretical results, the experiments report a sharp reduction to the average distance to the decision boundary (right plots). This behavior renders fair models more vulnerable to adversarial attacks, as will be highlighted shortly. Similar results are also observed for the group-loss based models and other architectures (see Appendix E).

Boundary errors increase as fairness decreases. This section highlights the key consequence of the sharp reduction to the average distance to the decision boundary: the increase of the vulnerability to adversarial attacks. Figure 5 (top) reports the natural errors (left), boundary errors (middle), and fairness violations (right) for a VGG-13 model trained on UTKFace dataset on the ethnicity task using a standard cross-entropy (CE) loss. Once again, other architectures 2 and datasets are reported in the appendix and the results follow the same trends as those reported here. Error rates and fairness violations are evaluated for fair classifiers across different fairness coefficients, denoted as λ. We also report boundary errors for classifiers that meet various fairness levels (λ in ¶0↪ 1↪ 2♦) and robustness strengths (✏), as defined in Equation (4).

Notice how, compared to natural models, fair models incur much higher natural and boundary errors. In particular, the fairness models have boundary errors that are up to 9% larger than their natural counterparts. These observations match the theoretical analysis and highlight a significant increase in vulnerability to adversarial examples by the fair models, even for moderate selections of the fairness violation parameters λ.

2 With the caveat that VGG-13 could not be used for FMNIST since the 28x28 pixel resolution of FMNIST is smaller than that required by some VGG filters.

6.1 Enforcing Both Fairness and Robustness

This section considers an additional experiment that highlights the potential negative impact of fairness on robustness. The experiment involves a classifier attempting to achieve both fairness and robustness. similar to [Xu et al., 2021a], which incorporates two fairness components to align per-class natural/robust accuracy per class with overall natural/robust accuracy (see Equation (9) in [Xu et al., 2021a]), our approach adds both robustness and fairness regularization terms to the standard classification objective function. The resulting model aims at solving the following regularized empirical risk problem:

using stochastic gradient descent. The second component aims at increasing the robustness of the classifier under a margin perturbation ✏⇤, following the PGD training [Madry et al., 2017] with perturbation norm p = 1. It works by first generating adversarial samples Xi + ⌧ , where k⌧k1  ✏⇤, and then the learning progress aims at minimizing the loss between the model prediction for that adversarial samples and the ground-truth `(f✓(Xi + ⌧)↪ Yi). The larger the margin perturbation ✏⇤, the more robust the resulting classifier. The third component implements a penalty-based fairness strategy [Fioretto et al., 2020; Tran et al., 2022b; Agarwal et al., 2018], which promotes fairness by penalizing the difference among each groups' average loss and the overall's average loss.

The experiments vary the margin perturbation ✏⇤ (robustness) and the penalty value λ (fairness). Figure 6 reports the (natural) error (left) and fairness violations (right) for different levels of the margin perturbation ✏⇤ on the UTK-Face (ethnicity) dataset. As expected, enforcing larger margin perturbations ✏⇤ increases model robustness, but at the cost of significantly increasing the natural errors. Increasing the fairness parameter λ decreases the fairness violation.

Figure 7 reports the robust errors under different levels of adversarial attacks, specified by the perturbation value ✏. Notice how the level of defense ✏⇤ correlates with higher robustness (smaller robust errors) for all fairness parameters λ tested. These results show the challenge to achieving simultaneously robustness, fairness, and accuracy. They also suggest that the incorporation of both robustness and fairness, as proposed in [Xu et al., 2021a], may not effectively reduce the trade-off between accuracy, robustness, and fairness.

7 AMitigating Solution

While previous sections established the inherent tradeoff between fairness and robustness, we now introduce a theoretically-grounded solution to mitigate this conflict. Note that in standard (unbounded) loss functions, misclassified samples far from the decision boundary incur much higher losses than those near it. Given that the decision boundary is a pivotal factor connecting fairness and robustness, we propose using a bounded loss function [Goh et al., 2016; Collobert et al., 2006]:

and referred to as Ramp loss, with domain (0↪ 1]. Our strategy incorporates this bounded loss function into a fair classifier, as outlined in Equation (3). The benefits are evident in Figure 5, where it shows decreased natural (left) and boundary errors (middle) for fair classifiers with λ > 0.

Figure 8, further illustrates the strenghts of the proposed strategy. The results depict the same setting used in the previous section and compare a fair classifier trained using the ramp loss with one trained using a 0/1-loss (which is also bounded but not differentiable), a log-loss , and an exponential loss (both unbounded) (top). The results show that the fair classifier trained using a ramp-loss is the least impacted by misclassified samples, resulting in lower robust errors compared to unbounded losses. It can be observed in the bottom subplot, where its associated loss is the closest, among all differentiable losses, to the local minima. Further analysis reported in Appendix E illustrates the strength of the proposed solution.

8 Conclusions

This paper was motivated by two key challenges brought by the the adoption of modern machine learning systems in consequential domains: fairness and robustness. The paper observed and analyzed the relationship between these two important machine-learning properties and showed that fairness increases vulnerability to adversarial examples. Through a theoretical analysis on linear models, this work provided a new understanding of why such tension arises and identified the distance to the decision boundary as a key explanation factor linking fairness and robustness. These theoretical findings were validated on non-linear models through extensive experiments on a variety of vision tasks. Finally, building from this new understanding, the paper proposed a simple, yet effective, strategy to find a better balance between accuracy, fairness and robustness.

Overall, our results show that, without a careful consideration, inducing a desired equity property on a learning task may create significant security challenges. We stress that this should not be read as an endorsement to satisfy a single property, but as a call for additional research at the intersection of fairness and robustness in order to design appropriate tradeoffs and hope that our results could stimulate such a needed discussion.

Acknowledgements

This research is partially supported by NSF grants 2227082, 2232054 and NSF CAREER Award 2143706. Fioretto is also supported by an Amazon Research Award and a Google Research Scholar Award. Its views and conclusions are those of the authors only.