FairCognizer: A Model for Accurate Predictions with Inherent Fairness Evaluation (Extended Abstract)

1 Introduction

Machine learning (ML) systems are increasingly pervasive, playing a crucial role in diverse applications like predictive maintenance, autonomous driving, and extending into sensitive domains such as judicial and medical fields [Dressel and Farid2018, Chen et al.2019, Cohen et al.2020, Ghassemi et al.2014]. This broad impact underscores the importance of fair predictions, especially given the biases in historical data [Falletti2023,Mittelstadt et al.2016,Fabris et al.2022].

Examples like biased skin condition diagnostic tools leading to misdiagnoses for minorities [Seyyed-Kalantari et al.2021, Wen et al.2022] and unfair recidivism risk prediction algorithms [Dressel and Farid2018] highlight the deep implications of this issue. Regulatory efforts like the EU AI Act aim to address these concerns.

∗ Published at the European Conference on Artificial Intelligence ECAI-2024 [Bendoukha et al.2024]

Several works [Chouldechova2016, Kamiran and Calders2011, Feldman et al.2015, Fish et al.2016, Zafar et al.2017,Bendoukha et al.2025] show that ensuring fairness in supervised learning is often framed as a trade-off between considering a fair representation or an accurate one, in terms of proximity to ground-truth observations. An accurate classifier learns from historical records, generalizing observed statistical patterns to unseen data. However, if these patterns involve many discriminatory records, the classifier will adopt this biased behaviour. Achieving fair training often requires learning an alternative representation of data, generated via pre-processing techniques to remove biases [Kamiran and Calders2011, Chouldechova2016, He et al.2019, Xu et al.2018]. This alternative representation does not perfectly mirror reality. Consequently, this distributional drift will inevitably degrade the utility of a classifier trained on this alternative fair representation.

This work introduces FairCognizer [Bendoukha et al.2024], a dual-objective classifier learning both accurate and fair representations, providing an unfairness risk assessment for each prediction. Our contributions are threefold: (1) the FairCognizer framework for learning dual labels, (2) a novel sample-level unfairness risk measure to identify "vulnerable" records, and (3) experimental validation on Adult and Compas demonstrating maintained accuracy (86% and 68%) with fairness insights, at a 45% model size increase. This extended abstract of the original paper essentially presents the first contribution.

2 Background

We briefly review the essential concepts for our approach.

Multi-output learning Unlike traditional single-label classification, multi-output classification predicts multiple outputs simultaneously from the same input features. The learning process requires data samples to be labeled accordingly. That is, D = {(x1, y(1) 1 , . . . , y(k) 1 ), . . . , (xn, y(1) n , . . . , y(k) n )} and the optimization is performed on the loss of every output and the corresponding label, as such, at each learning iteration t :

For convex loss functions, a common approach for optimization is to use Stochastic Gradient Descent (SGD). This iterative algorithm processes the data in batches (B ⊂ D) and updates the model parameters (θ) through gradient descent. The learning rate (η) controls the step size of these updates, guiding the parameters towards a minimum of the loss function, such that:

Fairness in Machine Learning Group fairness assesses whether a model's predictions ˆ Y are independent of a sensitive attribute S (e.g., gender or race).

We review key fairness metrics at the data and classifier levels, assuming S is binary with values {s0, s1}.

Classifier unfairness

These metrics assess how data biases affect a model's outputs.

• Statistical Parity Difference (SPD) [Chouldechova2016] measures the gap in positive prediction rates:

A dummy classifier predicting all positives yields zero SPD but poor utility.

• Equal Opportunity Difference (EOD) [Hardt et al.2016] compares true positive rates across groups:

Other metrics (e.g., FPD and FND) examine disparities in false-positive and false-negative rates across groups.

3 RelatedWork

Accuracy vs fairness trade-off Fairness-aware training often leads to reduced predictive performance, with many studies [Kamiran and Calders2011, Liu and Vicente2022, Fish et al.2016, Xu et al.2018,Wang et al.2021] highlighting an inherent trade-off between accuracy and fairness. However, Wick et al. [Wick et al.2019] suggest this trade-off may be avoidable under certain conditions. Kamiran and Calders [Kamiran and Calders2011] show a linear drop in accuracy from fairness constraints, while Liu et al. [Liu and Vicente2022] use Pareto fronts to illustrate the trade-off. Fish et al. [Fish et al.2016] improve fairness by shifting decision boundaries based on prediction confidence. Wang et al. [Wang et al.2021] explore fairness in multi-task learning. Overall, there is a growing consensus that improving fairness often compromises accuracy.

Delivering fairness insights along with prediction Recent works on individual fairness [Yadav et al.2024, Gajane and Pechenizkiy2018] and explainable AI (XAI) [Jain et al.2020, Alikhademi et al.2021] explore combining fairness with optimal accuracy. In individual fairness, this is framed as finding the largest neighborhood around an input x∗ where predictions remain consistent:

Yadav et al. [Yadav et al.2024] define fairness certificates per input based on distances in key features. In XAI, fairness is assessed via feature attribution-for example, Jain et al. [Jain et al.2020] use Shapley values to quantify the effect of sensitive features. Maughan et al. [Maughan and Near2020] introduce prediction-sensitivity, which computes the gradient of the model output with respect to sensitive inputs, with its norm indicating local unfairness.

In this work, the main challenge is developing classifiers aware of their inherent fairness risk for each input. This risk requires a clear definition and incorporation into the training data while ensuring that classifiers will still learn properly.

4 Towards a Dual Label Fair Learning

We assume a single binary sensitive attribute S ∈ {s0, s1} and a set of non-sensitive attributes denoted X (as presented in Figure 1). Subsequently, we consider a first binary classification task with Y = Ybin ∈ {0, 1}. Ybin corresponds to the default classification task of the trained model. Then, we propose to extract a second fairness class label Yfair, that represents a fair-aware assignation of outcomes with respect to groups S = s0 or S = s1.

4.1 Generating Yfair

We formulate the problem of finding fair-aware class labels, as finding an optimal vector Yfair that maximizes the correlation with the default labels Ybin, and minimizes the correlation with the sensitive attribute vector S (as presented in Figure 1).

We use the Pearson correlation coefficient to generate Yfair as it captures linear dependence and is zero for independent variables, enabling influence modeling without inherent bias. Its linearity also supports optimization tasks [He et al.2020]. We recall that Pearson's correlation satisfies:

Where Cov(X, Y ) is the covariance of X and Y , and σX, σY are respectively standard deviations of X and Y . Correlation values range from -1 to 1. A correlation of both 1 and -1 indicates a deterministic functional relationship between the two variables Y = f(X). A positive correlation indicates that X and Y follow the same variation (increasing f, when the correlation is equal to 1). Conversely, a negative correlation indicates opposite variations of X and Y (decreasing f when the correlation is equal to -1).

We leverage the transitive property of Pearson's correlation 1 . Since Ybin reflects the non-sensitive labels (X), maximizing its correlation with the fair prediction (Yfair) helps maintain the relationship between Yfair and X. This leads to a high-performing classifier trained on data containing sensitive attributes (S), non-sensitive attributes (X), and fair predictions (Yfair). Importantly, minimizing the correlation between Yfair and the sensitive attribute (S ) reduces bias in the fair predictions. This results in data with less disparate treatment based on the sensitive attribute compared to the original data. These requirements are expressed using the function Fλ defined as:

Where λ is a trade-off parameter that reflects the relation between both terms of Equation 2. Figures 2a and 2b show Fλ with λ = 150 and λ = 300, respectively. These two values of lambda induce different variations of the function. For example, a variation in the x-axis (i.e., in the correlation of S and Yfair) has a larger impact on Fλ when λ = 300.

We deduce from the previous observation that higher values of λ result in highly fair-aware class label Yfair. But, they sacrifice the accuracy of the induced classifier (trained on the (X, S, Yfair) records) due to the loss of useful correlations. Conversely, lower values of λ prioritize the first term and, therefore, produce a highly accurate classifier with slightly improved fairness.

4.2 Optimization strategy

The Fλ function can be categorized as a pseudo-Boolean function according to the PBO definition. A pseudo-Boolean function f is a function that maps a set of binary variables

1Ybin acts as a statistical proxy of the non-sensitive labels X.

(0 or 1) to real numbers such as f : {0, 1}n → R. Several approaches to solving non-linear PBO problems are investigated, including the use of constrained integer programming methods. These methods aim to minimize an objective function subject to constraints on the function variables.

In our case, we introduce hard constraints 2 to limit the search space to the binary space of dimension |D|. We use the Constraint Optimization BY Linear Approximation (COBYLA) solver [Powell1994] which is particularly suited for non-linear cost functions with hard constraints. Since this solver does not handle equality constraints, we define the constraints of the boolean solution as two inequality constraints satisfying:

where ϵ = 10-5 characterizes the constraints on solutions within narrow intervals around the values of 0 or 1. Finally, the solver is run with a maximum of 10k iterations with Ybin given as the initial guess.

4.3 Learning the dual label

Once the fair class label Yfair are generated, the learning objective becomes the mapping: X, S → (Ybin, Yfair). Indeed, the classifier makes two predictions for each data point (x):

• ˆ ybin: this prediction focuses solely on accurately matching the default label for the given input (x).
• ˆ yfair: this prediction represents a fairer and more ethical statistical outcome with respect to the sensitive attribute (S).

This essentially converts the original binary classification task into a multi-output classification. Figure 3 depicts a neural network architecture based on our proposed framework for dual labeling. Applied to the Adult dataset, the figure illustrates how the network achieves this task with two separate branches. Ideally, both predictions, ˆ ybin and ˆ yfair, would be the same. However, in cases where these predictions differ, the classifier identifies these discrepancies and acts as a perprediction unfairness alert system, prompting further human evaluation of the specific data point.

2 Hard constraints must be satisfied by all variables, while soft constraints only impose penalties on variables that fail to meet them.

4.4 FairCognizer implementation

To evaluate the performance of our framework, we conduct various experiments on two datasets: Adult (25k samples) and Compas (5k samples). For our analysis, we focus on the binary groups male and female within these datasets.

Training analysis

First, we compute the Yfair vector using the COBYLA solver implementation of the SciPy package [Virtanen et al.2020]. We generate Yfair for our training subsets, i.e., 34 of both datasets, for the binary groups.

Second, for validation, we train a three-layer MLP classifier on the records (X, S, Yfair) for 10 values of λ, and compare their predictive capabilities and fairness metrics (EOD and SPD) with a baseline classifier trained on the original data. Figure 4 depicts the obtained results.

It shows greater fairness improvement using Yfair on the Adult dataset compared to Compas. Indeed, our method mainly removes the disparate treatment (direct discrimination), but does not act on the disparate impact (indirect correlation) contained in the dataset. The predominant source of unfairness in the Adult dataset is disparate treatment, whereas in Compas, the primary origin of unfairness is disparate impact 3 . Finally, we train larger FairCognizer classifiers (from 2500 to nearly 3800 trainable parameters) on datarecords (X , S, (Ybin, Yfair)). We measure their predictive performances and fairness with respect to the initial labels, the fair ones, and the dual-label. Table 1 presents the obtained results on both outputs of the FairCognizer classifier. We note from Table 1 that FairCognizer achieves optimal accuracy for the default class label. This means it prioritizes fairness for the second prediction, ˆ yfair, without compromising accuracy on the original prediction.

Classification interpretation

The four possible predictions made by the dual-label classifier on test data are examined, especially data-records for which the classifier's prediction is (1, 0) or (0, 1) (ˆ ybin ̸= ˆ yfair). We measure the prediction inconsistency rate (PIR). That is, we compute the probability: P(⌈h(x)⌋ ̸= ⌈h(xS )⌋), where h is a classifier trained on the samples (X, S → Ybin) and xS is the sample x where the binary value of S is flipped.

3 For reference, the BER in Adult for gender as the sensitive attribute S is 0.183, compared to 0.087 in Compas.

We observe that for the subset of data-records with dual predictions ˆ ybin ̸= ˆ yfair, the prediction inconsistency rate is significantly higher compared to data-records for which ˆ ybin = ˆ yfair .

Table 2 shows that the dual-label model can identify unfair predictions, even if they are accurate. These predictions occur for data points similar to the ones from the opposite sensitive group (S), but with different labels. By similar, we refer to close data-records with respect to non-sensitive attributes X. Indeed, flipping the sensitive attribute value for such a point is likely to flip the model prediction (as indicated by the high PIR values on these points).

5 Conclusion

In this work, we explore a novel paradigm of fairness-aware learning that can be succinctly described as follows: If a classifier cannot simultaneously achieve optimal accuracy and group fairness, it can still provide valuable per-prediction insights about fairness risks. We provide two different hybrid pre-processing and in-processing approaches to implement this paradigm. Our study introduces a nuanced analysis of unfairness that encompasses both the classifier and the individual data records. Specifically, individuals exhibit varying degrees of vulnerability to the group-unfairness of a classifier. This nuanced perspective enables a targeted approach for enhancing fairness by directing efforts towards the most susceptible subset of data records affected by classifier unfairness.

Acknowledgments

This work was supported by the France ANR project ANR22-CE39- 0002 EQUIHID