FairTCD: Dual-Teacher Temporal Contrastive Distillation for Twofold Fair Dynamic Graph Embedding

1 Introduction

Dynamic graph embedding has become an important technique in modeling temporal relational data and has found widespread applications in recommendation systems, social network analysis, and related domains [Barros et al., 2021; Anand and Maurya, 2024; Pham et al., 2025]. Despite their success, these embeddings often inherit and even amplify bias from graph data. To address this problem, research on fair graph embedding has attracted considerable attention [Dong et al., 2023; Chen et al., 2024a].

∗Corresponding authors.

The fairness in graph embedding is typically defined as attribute fairness and structural fairness. Attribute fairness aims to balance the model's performance among sensitive demographic groups (e.g., gender, race), while structural fairness aims to balance the model's performance among high-degree and low-degree node groups. To achieve attribute fairness, most studies introduce an adversarial network to obscure sensitive attribute information in node representations [Khajehnejad et al., 2020; Dai and Wang, 2021; Zhang et al., 2024]. To achieve structural fairness, some studies propose debiasing functions or contrastive learning to reduce the gap between representations of high-degree nodes and low-degree nodes [Liu et al., 2023; Wang et al., 2022].

However, existing fairness studies typically optimize only one type of fairness, ignoring the co-existing twofold bias. As shown in Figure 1(a), when only attribute fairness along the horizontal axis is considered, structural bias along the vertical axis cannot be addressed. In contrast, focusing solely on structural fairness along the vertical axis also fails to mitigate attribute bias. This issue is further exacerbated in dynamic graphs, where evolving topologies continuously reshape node degrees and neighborhood structures, leading to twofold bias that may drift across snapshots. This non-stationary twofold bias renders static and single-objective debiasing strategies ineffective. Therefore, to address these issues, this study aims to achieve both attribute and structural fairness (i.e., twofold fairness) in dynamic graph embedding.

Naively optimizing attribute and structural fairness simultaneously raises three fundamental challenges. (i) Coupling relationship between two fairness objectives. In practice, sensitive attribute groups and structural groups often exhibit substantial overlap, leading to interdependencies between the two fairness objectives. For example, the statistical results in Figure 1(b) show that nodes within the same sensitive attribute group can belong to different structural groups. (ii) Optimization conflicts induced by the coupling relationship. Drawing on the empirical results [Mansoury et al., 2020], Figure (a) shows low-degree male users and highdegree female users exhibit opposite bias directions under the structural and attribute fairness objectives, which leads the two fairness objectives may impose conflicting gradient signals on the same node and hinder model convergence. Moreover, the statistical result of the third snapshot in Figure 1(b) shows that 54% of male users are low-degree nodes and 49% of female users are high-degree nodes, indicating that such conflicts are widespread rather than isolated phenomena. (iii) Conflicts drift in dynamic graphs. As bias distributions evolve in dynamic graphs, the optimization conflicts exhibit temporal drift. As shown in Figure 1(b), the proportions of these groups change over time, reflecting the non-stationary fairness signals and the evolving coupling relationship between the two fairness objectives.

To address these challenges, this study proposes a novel Fair Dual-Teacher Temporal Contrastive Distillation (FairTCD) framework for twofold fair dynamic graph embedding. To tackle the first challenge, we construct a dual-teacher adversarial Graph Neural Network (GNN) framework, in which attribute and structural fairness are learnt by two independent teachers via adversarial debiasing. This design separates the optimization processes and gradient flows of the two fairness objectives, decoupling the coupling relationship between two fairness objectives. To tackle the second challenge, we employ contrastive distillation to align structural and attribute fairness representations of the same node, thereby reconciling conflicting gradient directions induced by competing fairness objectives. To tackle the third challenge, we introduce temporal contrastive distillation (TCD), which extends contrastive distillation to the temporal dimension and generalizes fairness representations from snapshot-level to timeaware modeling, enabling adaptation to the conflict drift between twofold fairness objectives over time. Finally, a student model distills knowledge from both teachers for achieving twofold fairness. In summary, the primary contributions of this study are listed below.

• A new problem and formulation. We formally define the research problem and challenges of twofold fairness in dynamic graph embedding, in which attribute and structural fairness are coupled and conflict over time.
• A novel and effective framework. We propose
• FairTCD, a novel dynamic graph embedding framework that decouples attribute and structural objectives and reconciles their conflicts to achieve twofold fairness.
• Superior Twofold Fairness. Experiment results show that FairTCD significantly reduces both attribute and structural biases, while maintaining competitive embedding performance.

2 ProblemDefinition

In this section, we formally define the dynamic graphs, twofold fairness metrics, and the problem of twofold fair dynamic graph embedding.

Definition (Dynamic Graph and Twofold Fairness Metrics). A dynamic graph is defined as G = {Gt}Tt=1 , where each snapshot Gt = (Vt, Et,Xt) consists of nodes, edges, and node attributes. Let V = ⋃Tt=1 Vt denote the set of all nodes across time. We adopt Equal Opportunity (EO) and Statistical Parity (SP) to define structural and attribute fairness. Given groups {G1, . . . , Gk}, the EO-based and the SP-based fairness metrics are defined as:

where ˆ y and y denote the predicted and true labels, P is the probability, and {G1, . . . ,Gk} is either high and low degree structural groups or sensitive attribute groups. In practice, the disparities of EO and SP (i.e., DEO and DSP) across different groups are used as the corresponding fairness metrics. Following prior multi-objective optimization studies [Guerreiro et al., 2021], we further adopt Hypervolume (HV) as a unified twofold fairness metric, which is defined as:

where f = [DEOatt,DEOstr,DSPatt,DSPstr] denotes the four-dimensional fairness objective vector consisting of the DEO and DSP for both attribute and structural fairness, r = (1, 1, 1, 1) is the reference point, and u ⪯ f indicates the Pareto-dominant region with respect to f . Larger HV implies better twofold fairness.

Problem (Twofold Fair Dynamic Graph Embedding). The objective is to learn a mapping function fθ : {Gt}Tt=1 ↦→ { ˜H(T ) DF ∈ Rd×V } that produces a d-dimensional embedding ˜H(T ) DF for the T-th snapshot, such that this embedding jointly optimizes downstream task performance, structural fairness, and the attribute fairness.

3 Methodology

Figure 2 presents the FairTCD framework. First, the dualteacher fairness model learns structural and attribute fairness features respectively. Then, a TCD module aligns their crosssnapshot representations to reconcile dynamic conflicts while preserving fairness and temporal features. Finally, a twofold fairness student model inherits both fairness objectives.

3.1 Dual-Teacher Fairness Model

Since the structural and attribute fairness teachers are built upon the same adversarial debiasing paradigm, we present a unified group-adversarial fairness teacher. In the following, we define ∗ ∈ {SF, AF} to index the instantiated structural or attribute fairness teacher.

Snapshot-wise Graph Encoder. Given a dynamic graph sequence {Gt}Tt=1 , the fairness teacher aims to learn node representations that are informative for downstream tasks while being invariant to group membership. At each snapshot, a GNN encoder is first employed to capture local structural and feature-based dependencies. As our primary focus lies in fairness modeling rather than architectural design, we adopt the widely used GCN [Kipf and Welling, 2017] as the graph encoder:

where H(1:T ) ∗ ∈ R|Vt|×d denotes the node embeddings from snapshot 1 to snapshot T .

Adversarial FairnessConstraint. To remove bias encoded in the learned representations, the fairness teacher incorporates an adversarial discriminator that attempts to infer sensitive group membership from node embeddings, while the GNN encoder learns to suppress sensitive information.

Let Y(t) = {0, 1}|Vt|×2 denote the group labels at snapshot t, where |Vt| denotes the number of nodes in the t-th snapshot. For structural fairness, Y(t) denotes high-degree and low-degree group . For attribute fairness, Y(t) corresponds to sensitive attributes. At each snapshot, the discriminator maps node embeddings to group membership probabilities via a Multi-Layer Perceptron (MLP):

The discriminator is trained to maximize its ability to infer group membership from node embeddings, quantified by the cross-entropy loss:

where 1 denotes the all-ones vector and ⊙ denotes element-wise product. By aggregating over all snapshots, we obtain the overall adversarial objective:

where θG and θD denote parameter set of GCN encoder and discriminator of teacher, L(1:T) adv,o denotes the adversarial loss from snapshot 1 to T, and ϕ(·) denotes mean pooling. Through this minimax optimization, the discriminator continuously amplifies bias-related signals, while the encoder is forced to eliminate group-discriminative information. As a result, the encoder generates fairness representations H′(1:T) ∗ .

Dual-Teacher Fairness Optimization. While adversarial debiasing effectively mitigates bias within each snapshot, two fairness objectives in dynamic graphs are inherently nonstationary and may evolve and conflict over time. To address this temporal conflict between two fairness objectives, we further introduce a TCD module to guide the evolution of fairness representations:

where ˜H(T) ∗ denotes the final fairness feature of teacher model at snapshot T. The complete training objective of a fairness teacher is thus defined as:

where LT CD is the TCD loss that enforces consistency in the evolution of fairness representations across snapshots. The details of TCD module will be explained in Section 3.2.

3.2 Temporal Contrastive Distillation

Since structural and attribute fairness objectives impose opposite gradient signals on certain nodes, this optimization conflict makes direct joint distillation ineffective. To address this, we propose TCD to reconcile the dynamic conflicts by aligning the representations of the same node from both teachers across snapshots while preserving their diversity.

First, the structural and attribute fairness representations, denoted as H′(1:T) SF and H′(1:T) AF , are obtained by instantiating the unified fairness representation H′(1:T) ∗ with different group definitions. Then, we construct positive pairs and negative pairs in the t-th snapshot. The positive pairs are composed of structural and attribute fairness embedding from the same node. The negative pairs consist of the embeddings of different nodes from different teachers. Specifically, the positive pair at snapshot t is defined as Pt = {(uS , uA)}, where uS and uA denote the embedding of node u from the structural and attribute teacher, respectively. The negative pair is defined as Nt = {(uS , vA) | u ̸= v, v ∈ Vt}. Let h′SF t,u ∈ H′(1:T) SF and h′AF t,v ∈ H′(1:T) AF denote the structural fairness embedding of node u and attribute fairness embedding of node v at snapshot t, respectively. We average the contrastive distillation loss across all nodes and snapshots to obtain LT CD, which is defined as:

where g(ξ) = 1 T |V| ∑Tt=1 ∑u∈V ξ, sim(·) denotes cosine similarity, and τ is the scaling parameter. The LT CD loss aligns embeddings produced by the attribute and structural teachers to reconcile conflicts. Theoretical proof is presented in Appendix A.

Temporal Encoder. By minimizing the structural and attribute teacher loss (Eq. 9), we can induce contrasted structural fairness feature ̂H(1:T) SF and contrasted attribute fairness feature ̂H(1:T) AF for each snapshot. Then, a temporal encoder is designed to aggregate dynamic fairness features. To reduce computational cost, we adopt a decay factor to encode fairness features of historical snapshots for obtaining final fairness features ˜H(T) SF and ˜H(T) AF :

where ̂H(T-d) SF and ̂H(T-d) AF denote the structural fairness feature and the attribute fairness feature of the (T - d)-th snapshot obtained after updating via the two teacher losses, and γ denotes the decay factor.

3.3 Twofold Fairness Student Model

The goal of the student model is to jointly achieve twofold fairness by distilling complementary fairness knowledge from two teacher models, while preserving downstream performance. The student model is supervised by both hard label loss and soft label loss. Specifically, the student model first learns dynamic graph representations using a GCN with temporal decay to integrate evolving information, which is defined as:

where GCNDF denotes the GCN of student model.

Hard Label Loss. Hard labels provide ground-truth supervision for the downstream task. In this study, we employ link prediction as the downstream task and adopt the Bayesian Personalized Ranking (BPR) loss as the student model's hard-label loss:

where |I| denotes the size of the set of sample pairs formed by a positive edge e+ and negative edge e-, Se+ denotes the scores of positive edges, Se- denotes the scores of negative edges, and σ indicates the sigmoid function. The scores of the edges between two nodes are calculated by the dot product of the two node embeddings.

Soft Label Loss. To align the features of the student model with the two teacher models, we employ the mean-squared error (MSE) between their features as the soft-label distillation loss for the student model, which is defined as:

where || · ||2F denotes squared Frobenius norm.

Student Twofold Fairness Optimization. The overall objective of the student model combines the hard-label loss for the downstream task with two soft-label distillation terms, formulated as:

where α and β denote loss weights. The pseudo code of FairTCD is presented in Appendix D.

3.4 Time Complexity

FairTCD performs two primary operations. The first operation computes pairwise contrastive losses over all n nodes in a snapshot, incurring O(n2 ) time. The second operation performs a graph convolution on the adjacency matrix to propagate and linearly transform node features. In graphs where the total number of edges grows linearly with the number of nodes, each graph convolution layer has a cost proportional to the number of edges and therefore runs in O(n) time. Therefore, the total time complexity is O(n2 + n), which reduces to O(n2 ), omitting lower-order terms. In addition, the scalability of FairTCD is analyzed in Appendix E.

4 Experiments

This section reports experimental results addressing five research questions (RQs). RQ1: Does FairTCD improve attribute and structural fairness while preserving embedding capability? RQ2: Is TCD more effective than existing alignment methods? RQ3: How does FairTCD balance the performance-fairness trade-off? RQ4: Does TCD effectively reconcile the dynamic conflicts? RQ5: How does hyperparameter selection influence performance and fairness? The source code of FairTCD is available at https://github.com/FairTCD.

4.1 Experiment Setting

Datasets. We evaluate the FairTCD on three real-world datasets: MovieLens-1M (ML-1M) [Harper and Konstan, 2015], Amazon Electronics (ELEC) [Wan et al., 2020], and ModCloth (CLOTH) [Wan et al., 2020]. We divide the ML1M dataset of 677 users annotated with gender and 3135 movies from September 2001 to January 2003 into five snapshots. We split the ELEC dataset of 4060 gender labeled users and 940 products from January 2012 to January 2013 into five snapshots. Moreover, we split the ModCloth dataset of 11537 users annotated with body shape information and 1017 clothing items from September 2016 to August 2018 into eight snapshots. Dataset statistics are summarized in Table 2.

Baseline methods. We compare against eight recent fair graph embedding baseline methods, categorized as graph embedding (LightGCN [He et al., 2020]), attribute fairness graph embedding (FairGNN [Dai and Wang, 2021], FairSAD [Zhu et al., 2024b], FairSIN [Yang et al., 2024], Fairgap [Chen et al., 2024b], FairGKD [Zhu et al., 2024a]), structural fairness graph embedding (DegFairGNN [Liu et al., 2023], GRADE [Wang et al., 2022]), hybrid fairness graph embedding (LibraGNN [Luo et al., 2025b]), and dynamic attribute fairness graph embedding (FTGNN-M [Song et al., 2025]). As none of the baselines jointly address attribute and structural fairness, we equip each with adversarial debiasing for twofold fairness and apply a decay factor to ensure fair comparison in dynamic graphs.

Evaluation metrics. We evaluate link prediction performance using HR@k and NDCG@k and measure structural and attribute fairness using DEO and DSP (i.e., DEOatt, DEOstr, DSPatt, and DSPstr). We further adopt HV as a unified twofold fairness metric.

Implementation details. The experiments were conducted on Ubuntu 22.04.5 with an Intel Xeon Platinum 8336C CPU, 125 GB RAM, and an NVIDIA GeForce RTX 3090 GPU. FairTCD was implemented in Python 3.11 using PyTorch 2.1, with ReLU activation, dropout 0.5, and Adam (learning rate 1e-3), trained for 150 epochs with 64-dimensional hidden and output layers. The results are averaged over five runs with mean and standard deviation reported.

4.2 RQ1: Performance and Fairness Comparison

Table 1 reports the performance and twofold fairness results on three dynamic graph benchmarks. Overall, FairTCD maintains competitive performance while substantially reducing both attribute and structural bias across all datasets. Compared with the performance-oriented baseline LightGCN, FairTCD achieves a comparable performance while improving the HV by 23.12%-53.33%. In contrast, existing fairness methods that do not explicitly reconcile the conflict between two fairness tend to improve one fairness dimension at the expense of the other, resulting in lower HV.

Although LibraGNN incorporates a decay factor module to enable temporal modeling, its static preference control remains limited in coordinating the coupling between dynamically evolving dual fairness objectives, resulting in a 4.53%-9.76% lower HV compared with FairTCD. Moreover, although FTGNN-M is equipped with an additional structural fairness module, it fails to reconcile the dynamic coupling between attribute and structural fairness objectives over time, causing gradient components induced by different fairness constraints to conflict across snapshots and consequently amplify biases along both dimensions. In summary, by explicitly modeling and reconciling snapshot-wise dynamic conflicts through the TCD mechanism, FairTCD achieves the highest HV scores, demonstrating its advantage in jointly balancing the dynamic twofold fairness objectives.

4.3 RQ2: Ablation study for FairTCD

Table 3 investigates the contribution of the TCD module by replacing it with three alternative designs, namely simple cosine alignment (SCA), canonical correlation analysis (CCA), and an ordinary multi-objective scalarization with learned weights (MOS). When TCD is substituted by SCA or CCA, the model retains comparable performance but reduced HV (computed based on DSPatt and DSPstr) by 2.19%-6.58%, indicating that static or projection-based alignment is insufficient to reconcile the complex and temporal shifting conflicts. In contrast, TCD explicitly treats the structural and attribute fairness representations of the same node as positive pairs and enforces snapshot-wise alignment through contrastive objectives. This design preserves node-level consistency between the two fairness teachers and suppresses conflicting gradient components induced by competing fairness objectives as the graph evolves over time. By extending contrastive distillation to the temporal dimension, TCD further captures the non-stationary nature of fairness conflicts, enabling adaptation to shifting coupling patterns between structural and attribute biases. Furthermore, removing TCD and adopting a simple MOS to directly distill dual teachers leads to unstable fairness trade-offs, which validates the limitations of scalar-based multi-objective optimization under conflicting fairness objectives. The additional ablation results on the ML-1M dataset are presented in Appendix F.

4.4 RQ3: Pareto Front Trade-off Analysis

Figure 3 presents the Pareto fronts obtained by FairTCD, FairGNN, GRADE, and DegFairGNN on the ELEC and ML1M datasets, where each Pareto solution is selected from 2000 randomly sampled hyperparameter configurations and simultaneously considers HR and two fairness objectives. A solution is Pareto-dominant if it achieves higher recommendation performance while incurring lower attribute and structural fairness gaps. As shown in Figure 3, FairTCD consistently achieves a Pareto front that dominates those of the other three methods, indicating a more favorable performance-fairness trade-off across a densely explored hyperparameter space. In contrast, although baseline methods are additionally equipped with debiasing modules for the other fairness dimension, they still lack explicit mechanisms to reconcile competing fairness objectives, leading to constrained and suboptimal trade-off regions even under extensive hyperparameter sampling. By dynamically reconciling multiobjective conflicts, FairTCD expands the feasible solution space and attains Pareto superior solutions.

4.5 RQ4: Dual-Teacher Knowledge Alignment

To validate the effectiveness of the TCD in reconciling dynamic conflicts, Figure 4(a) first presents the L2-distance distributions between the student model and the two fairness teacher models when removing the TCD. The results indicate that the L2-distance between the student model embeddings and the teacher model embeddings consistently remains at a high value, exhibiting significant fluctuations across different snapshots. It indicates that the student model fails to distill twofold fairness knowledge from attribute and structural teachers and the dynamic conflicts consistently persist across different snapshots. By introducing TCD, as shown in Figure 4(b), the L2-distance becomes smaller and more concentrated across snapshots. This is because representations produced by the two fairness teachers for the same node are explicitly treated as positive pairs and aligned at each snapshot, which continuously suppresses opposing gradient components as the graph evolves. These results demonstrate that TCD effectively reconciles dynamically shifting conflicts between attribute and structural fairness.

4.6 RQ5: Hyperparameter Sensitivity Analysis

We conduct a hyperparameter analysis on the ELEC dataset by varying the number of snapshots and the temporal decay factor, as shown in Figure 5, with performance evaluated using HR and NDCG at k = 30, and attribute and structural fairness measured by DEOatt and DEOstr. (1) Number of snapshots: Figure 5(a) shows the performance peaks at four snapshots, reflecting a balance between capturing shortterm dynamics and avoiding data sparsity. As the number of snapshots increases, both performance and fairness slightly decline because low-degree nodes experience sparse interactions while high-degree nodes remain relatively active. (2) Decay factor: Figure 5(b) indicates that larger decay factors, which emphasize long-term history, lead to mismatches with current preferences and bias accumulation, resulting in degraded performance and fairness. Additional hyperparameter analysis on the ML-1M dataset is presented in Appendix C.

5 RelatedWork

5.1 Attribute Fairness Graph Embedding

Traditional attribute fairness methods mainly mitigate sensitive attribute bias through pre-processing and in-processing strategies. Pre-processing approaches rebalance random walks to ensure fair exposure, such as Fairwalk, which modifies Node2Vec transition probabilities to balance visitation across sensitive groups [Rahman et al., 2019; Grover and Leskovec, 2016]. In-processing methods typically adopt adversarial debiasing to remove sensitive information from representations [Luo et al., 2025a; Li et al., 2024a; Zhang et al., 2024]. For instance, FairGNN couples a GCN encoder with an adversarial classifier to eliminate gender information while handling missing sensitive labels [Dai and Wang, 2021]. In addition, contrastive learning-based approaches have been proposed to disentangle sensitive and non-sensitive factors [Zhang et al., 2025; Ling et al., 2023]. FDGNN employs disentangled contrastive learning to suppress sensitive information in node representations [Zhang et al., 2025], while FairGKD and FairSIN leverage knowledge distillation and fairness-facilitating neighbor features, respectively, to further mitigate attribute bias [Zhu et al., 2024a; Yang et al., 2024]. In the attribute fairness dynamic graph embedding, FTGNN-M [Song et al., 2025] monitored attribute fairness in real time, identifying sensitive embedding channels responsible for bias, and adaptively adjusting the corresponding model parameters.

5.2 Structural Fairness Graph Embedding

Structural fairness in node embeddings has recently received considerable attention due to biases between highand low-degree nodes. RawlsGCN [Kang et al., 2022] employs Rawlsian pre- and in-processing to balance predictive accuracy across degrees without altering model architecture. GRADE [Wang et al., 2022] applies graph contrastive learning to mitigate representation bias between nodes of varying degree. DegFairGNN [Liu et al., 2023] introduces a learnable debiasing function to refine low- and high-degree neighborhoods and eliminate degree bias. FairDGE [Li et al., 2024b] extends fairness to dynamic graphs by applying dual debiasing for fairness over time. In addition, LibraGNN [Luo et al., 2025b] jointly considers hybrid fairness via multi-teacher distillation and Pareto optimization, but it relies on static fairness teachers and preference control, making it unable to handle the temporal evolution of coupling relationship of twofold fairness objectives in dynamic graphs, thereby cannot achieving the twofold fairness in dynamic graph settings. However, prior studies only mitigate either attribute or structural bias in isolation, failing to consider the coupling relationship between them and ensure twofold fairness in dynamic graphs.

6 Conclusion

This study addressed the key problem of twofold fairness in dynamic graph embedding. To decouple the relationship between twofold fairness objectives, we proposed dualteacher fairness model to learn each fairness feature and separate the optimization processes. To reconcile the dynamic conflicts, a TCD was introduced to induce consistency between the two fairness representations while preserving diversity and temporal semantics. Finally, a student model optimized downstream performance and aligned with both teachers, achieving twofold fairness. Experiments on three benchmarks showed FairTCD improves the twofold fairness while keeping embedding performance. In future work, we will explore reinforcement learning to adaptively learn distillation weights, enabling the model to adaptively adjust to the dynamic conflicts between fairness objectives.

Acknowledgments

This work was supported by National Natural Science Foundation of China (Grant No.62472220, No.62502204), the Natural Science Fund of Jiangsu Province (Grant No. BK20251381), the Open Project Program of State Key Laboratory of Novel Software Technology, Nanjing University (KFKT2025B14), the Fundamental Research Funds for the Central Universities (NS2025041), Collaborative Innovation Center of Novel Software Technology and Industrialization. This work was partially supported by High Performance Computing Platform of Nanjing University of Aeronautics and Astronautics.

Contribution Statement

Yuxuan Gu and Yicong Li made equal contributions to this work.