FEAMOE: Fair, Explainable and Adaptive Mixture of Experts

1 Introduction

The field of responsible artificial intelligence has several desiderata that are motivated by regulations such as the General Data Protected Regulation [Butterworth, 2018]. These include: ensuring that an AI model is non-discriminatory and transparent; individuals subject to model decisions should have access to explanations that point a path towards recourse; and models should adapt to any changes in the characteristics of the data post-deployment so as to maintain their quality and trustworthiness.

Most approaches towards the mitigation of any form of bias assume a static classifier. A practitioner decides on some definition of fairness, trains a model that attempts to enforce this notion of fairness and then deploys the model. Many of the fairness definitions are based on model outcomes or on error rates (the gap between true and/or false positive rates) that are associated with different subgroups specified by a protected attribute. The goal is to reduce the difference between these error rates across relevant subgroups. For example, average odds difference [Bellamy et al., 2018] is a measure signifying equalized odds and is given by the sum of the differences in both true positive and false positive rates between two groups, scaled by a factor of 0.5. Equality of opportunity and demographic parity [Barocas et al., 2019] are also popular definitions of fairness. Recently, fairness in terms of a gap of recourse has been proposed, where recourse is defined as the ability to obtain a positive outcome from the model [Sharma et al., 2020a]. While the suitability of a fairness measure is application dependent [Mehrabi et al., 2019; Barocas et al., 2019], demographic parity and equalized odds remain the most popularly used, and the need for recourse gap-based fairness is being increasingly recognized [Karimi et al., 2020].

However, static models can encounter drift once deployed, as the statistical properties of real data often change over time. This can lead to deteriorating performance. Model drift can occur when the properties of the target variable change (concept drift) or when the input data distribution changes, or both. The performance of models has largely been measured through accuracy-based metrics such as misclassification rates, F-score or AUC. [Stanley, 2003]. However, a model trained in the past and found to be fair at training time may act unfairly for data in the present. Addressing drift with respect to fairness in addition to accuracy has remained largely unexplored though it is an important aspect of trustworthy AI in practice.

Explainability of individual model outcomes is another principal concern for trustworthy ML. Among many methods of explanations in terms of feature attribution, [Bhatt et al., 2020], the SHAP approach based on Shapley values is particularly popular as it enjoys several axiomatic guarantees [Lundberg and Lee, 2017a]. While computation of SHAP values is fast for linear and tree-based models, it can be very slow for neural networks and several other model types, especially when the data has a large numbers of features or when a large number of explanations are required [Molnar, 2019]. This poses a barrier to deployments that demand fast explanations in real-time, production settings.

In this paper, we address these fairness, data/model drift, and explainability concerns by proposing FEAMOE: Fair, Explainable and Adaptive Mixture of Experts, an incrementally grown mixture of experts (MOE) with fairness constraints. In the standard mixture of experts setup, each expert is a machine learning model, and so is the gating network. The gating network learns to assign an input-dependent weight gu(x) to the uth expert for input x, and the final output of the model is a weighted combination of the outputs of each expert. Hence, each expert contributes differently for every data point towards the final outcome, which is a key difference from standard ensembles.

Many types of MOE's exist in the literature [Yuksel et al., 2012] - the architecture is not standard. For FEAMOE, we chose this family, with some novel modifications described later, for three main reasons: 1) Suitable regularization penalties that promote fairness can be readily incorporated into the loss function. 2) Online learning is possible, so changes in the data can be tracked. Crucially, since localized changes in data distribution post-deployment may impact only one or a few experts, the other experts may not need to be adjusted, making the experts localized and only loosely coupled. This allows for handling drift and avoiding catastrophic forgetting, which is a prime concern in widely used neural network models [Robins, 1995]. 3) Simpler models can be used to fit a more complex problem in the mixture of experts, as each model needs to fit well in only a limited part of the input space. In particular, even linear models, which provide very fast SHAP explanations, can be used. The overall mixture of experts, even with such simple base models (the "experts") often has predictive power that is comparable to a single complex model such as a neural network, as shown by our experiments as well as in many previous studies [Yuksel et al., 2012].

A motivating toy example of why FEAMOE is needed and how it works is shown in Figure 1. Consider a linear binary classifier (1a) that has perfect accuracy. The colors represent the ground truth labels, and green is the positive (desired) class label. The circles are the privileged group and diamonds are the underprivileged group. As can be seen in the figure, more diamonds receive a negative outcome and more circles receive a positive outcome. Consider new data that arrives for predictions. This classifier (1b) not only misclassifies individuals but also gives more underprivileged individuals that were actually in the positive class a negative outcome, hence inducing bias with respect to equalized odds. There is drift with respect to accuracy and fairness. A more complex model (1c) such as a neural network, if retrained, may handle some of these concerns but would be less explainable.

FEAMOE can address these imperative concerns, as shown in 1d. Trained in an online manner, a new linear model is added (i.e., an expert) once the new data arrives. The gating network dictates which region each expert operates in (shown by the blue and pink colors), and FEAMOE is able to adapt automatically with respect to accuracy and fairness. This dynamic framework enables the overall model to be fairer, adjust to drift, maintain accuracy, while also remaining explainable since the decision boundary is locally linear.

We show how three fairness constraints-demographic parity, equalized odds, and burden-based fairness-can be incorporated into the mixture of experts training procedure in order to encourage fitting fairer models (according to these measures). We use these three popular fairness measures as illustrative examples to demonstrate the effectiveness of FEAMOE, but our method can be adapted to incorporate other fairness constraints as well. We then describe a new algorithm for training to account for drift, where the drift in question can be due to accuracy or fairness. We show experimentally that by using a set of logistic regression experts, the accuracy of the mixture is comparable to using a complex model like a neural network. Additionally, we show we can efficiently compute Shapley value explanations when explanations for every individual expert can be computed quickly. To the best of our knowledge, this is the first work that addresses the problem of drift with respect to fairness in a large-scale real world dataset. We then introduce a framework that can flexibly adapt to drifts in both fairness and accuracy with the added benefit of delivering explanations quickly, while comparing to the less explainable neural network model class trained in online mode.

The key contributions of this work are: a mixture of experts framework that can incorporate multiple fairness constraints, a method to handle drift, where drift can be with respect to accuracy or fairness, empirical evidence of the presence of drift with respect to fairness in a real-world, large-scale dataset, a theoretical proof that FEAMOE leads to the generation of fast explanations given a suitable choice of experts, and extensive experimentation on three datasets to show that our method has predictive performance similar to neural networks while being fairer, handling different types of drift, and generating faster explanations.

2 RelatedWork

The mixture of experts (MOE) [Jacobs et al., 1991; ?] represent a class of co-operative ensemble models; detailed surveys on their design and use can be found in [Yuksel et al., 2012] and [Masoudnia and Ebrahimpour, 2014]. Very recently, the deep learning community has started recognizing and leveraging several advantageous properties that MOE's have for efficient design of complex, multi-purpose learners [Riquelme et al., 2021]. This paper contributes to this expanding literature by proposing a new algorithm to train this model class to account for both fairness and drift, and by also adding an explainability module.

Fairness in machine learning is a growing field of research [Hacker, 2018]. Mitigating biases in models can be done through pre-processing, in-processing, or post-processing techniques. A description of these techniques can be found in [Bellamy et al., 2018]. In-processing techniques for fairness have been gaining traction [Zhang et al., 2018; Mehrabi et al., 2019; ?]. However, there is limited work on investigating the usefulness of ensemble models in dealing with biases. [Grgi´cHlaˇca et al., 2017] show that an ensemble of fair classifiers is guaranteed to be fair for several different measures of fairness, an ensemble of unfair classifiers can still achieve fair outcomes, and an ensemble of classifiers can achieve better accuracy-fairness trade-offs than a single classifier. However, they neither provide experimental evidence nor discuss specific methods to incorporate fairness into ensemble learning. [Madras et al., 2017] develop a method to learn to defer in the case of unfair predictions. [Bhaskaruni et al., 2019] use an AdaBoost framework to build a fairer model. [Nejdl, ] use adaptive random forest classifiers to account for fairness in online learning.

Accounting for drift is a widely explored problem, and is now appearing in commercial products as well (e.g. model monitoring is a key part of MLOPs) as ML solutions get deployed in business environments. Details on many such approaches can be found in [Gama et al., 2014; Lu et al., 2018]. Among these approaches, the one that comes closest to ours is [Stanley, 2003] which uses a committee of decision trees to account for drift. However, ensuring fairness in the presence of drift remains an open problem. [Biswas and Mukherjee, 2020] is a very recent work on achieving a fairer model by building a set of classifiers in the presence of prior distribution shifts. The method is built for a shift between the training and test distributions, and not for online learning. [Zhang and Ntoutsi, 2019; Zhang et al., 2021] provide online learning methods for tree-based models. We show experimentally (in the supplementary material 1 ) that FEAMOE can work comparably or better to adapt for drift. Recently, some researchers have studied fairness in online learning [Ntoutsi et al., 2020; Bechavod et al., 2020]. Innovations include the notion of cumulative fairness monitoring to account for discriminatory outcomes from the beginning of the stream until a time point.

There are many ways to explain a machine learning model [Burkart and Huber, 2021; Molnar, 2019]. In this paper, we focus on Shapley values-based explanations, which are widely used in practical applications [Bhatt et al., 2020]. [Lundberg et al., 2018] propose the computation of Shapley values for tree ensembles, which is a faster way to get Shapley values than through the more broadly applicable method, KernelShap [Lundberg and Lee, 2017a]. We show that in FEAMOE, the Shap values for the overall model are just a data-dependent linear combination of the values from individual experts. Thus the mixture approach does not add any significant complexity to the computation of feature attribution scores.

1 https://drive.google.com/file/d/1l2qz50Flvj4VAEvrRrH4Gdy3QAmCRnY/view?usp=sharing

3 Theory

We first summarize the original mixture of experts framework and then describe the addition of fairness constraints. Then, we introduce the algorithm to detect and mitigate data drift when the data input is sequential (online learning). Thereafter, we show how using the proposed mixture of experts architecture leads to computing faster Shapley value explanations for the overall non-linear model.

Mixture of Experts (MoE) [Jacobs et al., 1991] is a technique where multiple experts (learners) can be used to softly divide the problem space into regions. A gating network decides which expert to weigh most heavily for each input region. Learning thus consists of the following: 1) learning the parameters of individual learners and 2) learning the parameters of the gating network. Both the gating network and every expert have access to the input x. The gating network has one output gi for every expert i. The output vector is the weighted (by the gating network outputs) mean of the expert outputs: y(x) = ∑mi=1 gi(x)yi(x). Consistent with [Jacobs et al., 1991], the error associated with training the mixture of experts for case j for an accurate prediction is given by:

,where yji is the output vector of expert i on case j, gj i is the proportional contribution of expert i to the combined output vector, and dj is the desired output vector.

3.1 Fairness Constraints

In this paper, we incorporate three diverse fairness definitions into the mixture of experts framework: demographic parity only depends on the model outcome, equalized odds is conditioned on the ground-truth label, and burden-based fairness depends on the distance of the input to the boundary. These three popular definitions have been chosen as illustrative metrics; our approach can be readily extended to several other fairness metrics as well.

For simplicity, we consider a binary classification setting with a binary protected attribute (our approach readily extends to multi-class and multi-protected attribute problems, where a protected attribute is a feature such as race or gender). Let yj i = 1 be the positive outcome. Let A = 0 and A = 1 represent the underprivileged and privileged protected attribute groups, respectively. For a given dataset D, let Dad represent all individuals that belong to the protected attribute group a and original class label d.

Statistical parity difference (SPD), which is a measure of demographic parity, measures the difference between the probability of getting a positive outcome between protected attribute groups [Bellamy et al., 2018; Sharma et al., 2020b]. Let D0 be the set of individuals in the underprivileged group and D1 be the set of individuals in the privileged group. Inspired by [Slack et al., 2020], the associated penalty for demographic parity for case j is:

The idea behind this term is that individuals belonging to the underprivileged group predicted as getting a negative outcome are assigned a higher penalty. Similarly, individuals belonging to the privileged group predicted to have a positive outcome are assigned a higher penalty, thereby encouraging an SPD value closer to zero.

Average odds difference (AOD), which is a measure of equalized odds, measures the difference in true and false rates between protected attribute groups. Details on the measure can be found in [Bellamy et al., 2018; Sharma et al., 2020b]. The associated penalty for equalized odds is:

This term encourages the true and false positive rate gaps between groups to reduce by conditioning the indicator function on the ground truth label in addition to the protected attribute (as was in the demographic parity formulation).

Burden for a protected attribute group is a measure of the ability to obtain recourse for individuals in that group. As shown in [Sharma et al., 2020a], burden-based fairness can be calculated as:

,where d(x, B) represents the distance to the boundary for a given x that is classified as being in the negative class. Then, the associated penalty for burden based fairness is:

The overall loss for case j is then given by:

Note that there may be a trade-off between accuracy and fairness measures when the true positive outcome rates differ among the different population segments [Sharma et al., 2020a; Bellamy et al., 2018], and the practitioner needs to decide the relative importance of the different constraints for the given application.

3.2 Data Drift and the FEAMOE Algorithm

Data Drift means that the statistical properties of the data, embodied in the underlying joint distribution of independent and

Algorithm 1 Learning FEAMOE

Inputs: data X, labels Y Hyperparameters: k, ∆λ1, ∆λ2, ∆λ3 s = 1 # Select k points {x}kl=1 ⊂ X X(s) = X \ {x}kl=1 # Learn the first expert, ms # Initialize ws, weights of ms for j in {1, ..., k} do # Take gradient steps to minimize MoE loss: wjs = wj-1 s - ν∇Ejacc end for λ1 = 0, λ2 = 0, and λ3 = 0 while X(s) is not empty do s+=1 λ1 = λ1 + ∆λ1, λ2 = λ2 + ∆λ2, λ3 = λ3 + ∆λ3 # Select k points {x}kl=1 ⊂ X(s) X(s) = X(s) \ {x}kl=1 # Learn the subsequent expert, ms Initialize ws for j in {1, ..., s ∗ k} do # Take gradient steps to minimize Equation 5 for l in {1, .., s} do wjs = wj-1 s -ν∇Ejacc -λ1∇ESPD-λ2∇EjAOD - λ3∇EjBurden end for end for end while

dependent variables, can change over time, often in unforeseen ways. The change could be in the class priors, the class conditional distributions (concept drift), in the distribution of the independent variables etc. Drift can cause the model to become less accurate as time passes. However, drift can also cause other properties associated with the model to change, such as fairness. We develop an algorithm that can handle drift with respect to both accuracy and fairness.

Consider an online learning setup where input data points are observed sequentially. The algorithm to learn FEAMOE (Algorithm 1) is as follows: start with a single model. Begin to train with data points (using stochastic gradient descent) and train the current model for a certain number of data points k using only Eacc (equation 2). After k points, introduce a new logistic regression model and train the mixture of experts with a softmax gating function using the loss in Equation 5. Simultaneously, introduce the fairness penalties. Then, continue training for the next k points, and then add another expert. As more experts are added, gradually increase the hyperparameters (λ' values) associated with the three fairness losses. This process is continued until available data is seen.

The motivation behind this training scheme is two-fold: in beginning with the accuracy penalty for the first expert, we ensure that the fairness measures do not interfere with training an accurate classifier, since high weights on the fairness terms would result in a less accurate classifier (as shown in experiments). Then, we slowly increase the weights on the fairness penalties with the goal that for individuals that are classified unfairly with respect to these group fairness measures, another expert takes over in this data regime to train for these individuals over time. This is because the mixture of experts framework allows some or all of the experts to learn on different regions of the data. Secondly, the algorithm allows us to account for drift, both with respect to the accuracy of the classifier and the fairness, since our framework allows for fairness constraints. If there is a change in the statistical properties of the data that impacts any of the loss terms, the mixture of experts adapts to this change over time through the addition of experts.

3.3 Fast Shapley Value Explanations

A prominent class of feature attribution methods is based on Shapley values from cooperative game theory [Shapley, 1953]. Details about Shapley value explanations can be found in [Lundberg and Lee, 2017b], [Sundararajan et al., 2017], and [Aas et al., 2019]. While computing Shapley values for a linear model is fast, doing so for nonlinear models with methods like KernelShap [Lundberg and Lee, 2017a] requires approximations and methods that cause the overall computation to become slow [Molnar, 2019; Aas et al., 2019]. Another method, TreeShap, [Lundberg et al., 2018] works only for tree models. Though the mixture of experts model proposed is non-linear, as the individual experts are linear, the theorem below shows how to compute them for the whole model quickly and efficiently.

Consider a mixture of experts model with m experts. Let ϕj(m(x)) be the Shapley value associated with expert m for feature j for an input instance x.

Theorem 1. For a mixture of experts model, the Shapley value for a given instance x and feature j for the model prediction is given by:

The proof is provided in the supplementary material. This result shows that the Shapley value for a given feature and input instance for the mixture of linear experts is a linear combination of the Shapley values of the feature and that input instance from every expert, weighted by the gating network's assigned weights for that input. This means that so long as the Shap values for individual models can be quickly computed (as is the case for linear/logistic regression, decision trees, XGBoost), the FEAMOE system-level Shap computation is also very quick. In this paper, we illustrate FEAMOE using logistic regression experts, so this desirable property holds, even though the mixture model is able to construct non-linear models of arbitrary complexity by including as many logistic regression-based experts as needed.

4 Experiments and Results

Experiments are performed using the mixture of experts with logistic regression experts and a softmax gating function. We implement the logistic regression models using scikit learn with default parameters. We show that using logistic regression experts within the MOE produces accuracies similar to using appropriately sized neural networks while allowing for the generation of faster explanations. All neural networks are multilayer perceptrons. There are two sets of experiments, highlighting different aspects of FEAMOE:

(a) Fairness Study. We use two classification datasets that are very well studied in the fairness community: UCI Adult [Kohavi, 1996] and COMPAS [ProPublica, 2016]. Gender is considered as the protected attribute for UCI Adult. A two layer multilayer perceptron with 30 hidden units in each layer was trained for the UCI Adult dataset. Additional or larger hidden layers, or ensemble methods such as xgboost do not provide extra benefit for these two tabular datasets, and hence are omitted for comparison purposes. Experiments on the COMPAS dataset are in the supplementary material.

(b) Drift Study. The large HMDA (Home Mortgage Disclosure Act) dataset [Bureau, 2020] reflects data frommultiple years, with the underlying data statistics varying considerably over the years. Gender is the protected attribute.

A five layer multilayer perceptron with 50 hidden units in each layer is trained for the HMDA dataset as the baseline neural network. Experiments on a synthetic streaming version of the UCI Adult dataset are in the supplementary material.

First, we use UCI Adult to demonstrate the effects of incorporating the proposed fairness constraints on the mixture of experts models. Similar results for the COMPAS dataset are provided in the supplementary material. We then show that the HMDA dataset demonstrates drift with respect to both fairness and accuracy, and that FEAMOE can adapt to such drifts. Comparisons are made to neural networks (both with and without fairness constraints), which is the state-of-theart model class for accuracy-based performance across these datasets in all experiments. Experiments on faster Shapley value explanations are in the supplementary material.

4.1 Fairness Constraints

Experiments are performed on UCI Adult in seven different regimes based on model types and fairness constraints. Details of these regimes are in Table 1. Experts are added every 4000 data points for the UCI Adult dataset. Hyperparameters associated with the fairness constraints are incremented in levels of 0.02 per expert for the UCI Adult dataset. The parameters are found using grid search and vary based on dataset size and extent of prevalent bias (details in supplementary material). The results are averaged across five runs. We report the accuracy and the absolute value of the three

(a) Accuracy

(b) Demographic Parity

fairness measures. We provide comparisons to other methods for bias reduction [Calmon et al., 2017; Sharma et al., 2020a; Agarwal et al., 2018] in the supplementary material.

The results are shown in Figure 2. The accuracy across different model types remains similar for the UCI Adult dataset, but using just a neural network with fairness constraints works poorly, as shown in Figure 2a. As seen in 2b,c,d, the fairness measures also work well even in isolation from each other That is, in trying to improve based on just one measure, the other measures also improve. In this regard, the burden-based fairness measure (FEAMOE3) has the best effect; just using burden-based fairness alone helps significantly improve the other fairness measures while maintaining reasonable accuracy. The fair neural network (FairNN) performs worse for demographic parity and equalized odds compared to FEAMOE. We hypothesize that this happens because our learning process slowly induces fairness with every expert.

4.2 Real-World Drift: The HMDADataset

We first demonstrate that the HMDA dataset exhibits drift across years, and then show FEAMOE's effectiveness in handling it. The HMDA dataset has millions of records of individuals spanning several years. It contains consumer characteristics; the target variable indicates whether a consumer received a mortgage. While this dataset considered as a whole has been previously shown to exhibit bias, there is no investigation into how such bias varies across the years. First, to quantify drift in this dataset in both fairness and accuracy, we trained one neural network per year from 2007 to 2017, each on 100,000 random samples in that year, and tested each of these networks on data from the years 2016-2017 (Once trained, these models, which we call fixed neural networks, cannot be updated). The results are shown in Figure 3 by the blue points. In general, the farther the training data is away from the test year the more the accuracy and fairness measures degrade (i.e., accuracy decreases and fairness differences increase). Also training a single model on a dataset of the same size but sampled uniformly over all the previous years does not help either as the data is non-stationary.

We now study how fairness aware neural networks with online updates compare with FEAMOE in their ability to handle drift. Note that for our setting, we cannot use certain models such as popular ensemble approaches (XGBoost etc), that need multiple passes through new data in batch mode after the initial model is built and deployed. For FEAMOE, we train models for each year separately and consider each of those as one expert. Then, we add on experts based on a single pass on new data (from the 2016-2017 years). We compare this to neural networks (with fairness constraints) that are also trained on each of the past years in batch mode, and then updated online with a single pass on the new data. The FEAMOE results are shown in Figure 3 by the green points, and the trainable neural networks results by orange points. FEAMOE is noticeably better at maintaining good accuracy and keeping lower bias across all bias metrics, irrespective of how old the original model was, even when compared to adaptive neural networks. The loosely coupled architecture and adaptive model complexity is key to FEAMOE's success in handling drift. Also, using the FEAMOE architecture provides for much faster Shapley value explanations compared to the trainable neural network.

Additional experiments and findings are reported in the supplementary material. Some key information includes a comparison to other methods, further details on hyperparameters, experiments on fast Shapley value explanations, experiments on the COMPAS dataset for the fairness study, and on the synthetic streaming version of the UCI Adult dataset.

5 Conclusion and FutureWork

We propose FEAMOE: a novel mixture of experts architecture and learning framework that can better maintain the fairness of a model in the face of data drift. We show how three fairness constraints can be incorporated into this framework. We prove that by using this mixture of experts, Shapley value explanations can be computed efficiently even though the overall model is non-linear. Experiments are performed on three datasets to demonstrate the various properties and effectiveness of FEAMOE. In particular, we identified a large-scale, real-world dataset that induces drift with respect to fairness over time in non-adaptive models, and show that our framework can adequately address this challenge. We would now like to extend it to incorporate other potential forms of drift, such as those that cause changes in adversarial robustness.