Disentangled Graph-Enhanced Large Language Models for Fair Learning

1 Introduction

Large Language Models (LLMs) have achieved significant advancements in various natural language processing tasks, such as text classification [Zhang et al., 2025b], sentiment analysis [Zhang et al., 2024b], language translation [Gong et al., 2024], and dialog systems [Yi et al., 2024]. These successes can be primarily attributed to the Transformer architecture, which effectively models long-range dependencies in sequential data, combined with extensive training on massive textual datasets [Mienye, 2025]. Transformerbased models are highly dependent on attention mechanisms to capture semantic and contextual nuances from textual input, enabling them to generalize effectively to various downstream tasks [Wang et al., 2025c]. However, despite their proven effectiveness on textual information, LLMs face notable challenges when applied to graph-structured data [Li et al., 2025]. Unlike linear and sequential text, graphs represent non-linear relational structures characterized by nodes and edges, lacking inherent ordering and spatial locality [Osman and Barukub, 2020]. This fundamental difference impedes the direct application of LLMs, as their architectures are optimized for textual rather than structural dependencies [Osman and Barukub, 2020; Wang et al., 2025d]. Addressing this limitation is crucial, as graph data play an increasingly important role in numerous real-world domains, such as social network analysis [Campbell et al., 2013], drug discovery [Bonner et al., 2022], recommendation systems [Yang et al., 2025], and fraud detection [Pourhabibi et al., 2020]. In these scenarios, effectively using the rich structural relationships embedded in graph data is essential for making accurate, reliable, and context-aware predictions. Therefore, addressing the current limitations of LLMs in handling graph-structured data represents an important and timely research challenge.

∗Corresponding author.

Building upon this motivation, considerable research has sought to bridge the gap between graph data and LLMs, exploring innovative strategies for effective integration. For instance, GaLM [Xie et al., 2023] leverages graph-aware language modeling to jointly incorporate structural graph information and textual semantics, enabling language models to better capture contextual relationships within graphstructured data. Similarly, GraphTranslator [Zhang et al., 2024a] introduces a translation-based framework that converts graph-structured inputs into natural language prompts through structural linearization, allowing LLMs to process and reason over structured graph data. Despite these important contributions, existing approaches largely overlook critical fairness considerations, inadvertently perpetuating biases during the integration of graph data with textual representations. Specifically, beyond biases already present within the graph data itself, during alignment between graph embeddings and their corresponding textual descriptions, training objectives may unintentionally amplify biases by reinforcing correlations between sensitive attributes and task-relevant semantic information. Additionally, another significant source of bias emerges in the serialization step, whereby complex graph structures are transformed into linear token sequences suitable for LLM input. This serialization process can inadvertently distort the underlying connectivity patterns, disproportionately emphasizing nodes or edges associated with sensitive attributes, thereby introducing structural biases into downstream predictions.

Addressing these fairness-related limitations is essential, as enabling LLMs to process graph-structured data in a fair and unbiased manner is of significant practical importance, yet there are still several key challenges to achieving fair graphenhanced LLMs: i) In graph-enhanced LLM pipelines, sensitive attributes and task-relevant relational features are frequently co-encoded during graph-to-text representation learning. This entanglement makes it difficult to prevent protected signals from being embedded into language representations, which can cause downstream LLM predictions to depend on sensitive information. As a result, reliable fairness control and bias mitigation become substantially more challenging. ii) Biases can arise from multiple interconnected sources, notably during the alignment of structural and textual modalities. When graph representations are synchronized with textual descriptions, biased correlations inherent in either domain can propagate and reinforce each other, amplifying stereotypical associations. iii) A substantial modality gap exists between graph-based models and language models due to fundamental differences in data representation and processing mechanisms. LLMs operate on token sequences and are optimized for natural language understanding and generation, rather than directly modeling relational structures. Bridging this gap while suppressing bias transmission requires carefully designed mechanisms that preserve structural semantics without introducing or amplifying unfair correlations during graph-to-text conversion. These interconnected challenges underscore the necessity of developing fairness-aware approaches explicitly tailored for integrating graph data with LLMs.

To tackle the aforementioned challenges, we propose Disentangled Graph-Enhanced Large Language Model (FairGEnt), a novel disentangled graph-enhanced large language model explicitly designed to promote fairness in LLM-based graph learning. To the best of our knowledge, FairGEnt is the first work that explicitly addresses fairness in graphenhanced large language models by jointly disentangling and aligning sensitive-related structural information and taskrelevant representations, thereby effectively mitigating bias transmission across modalities. Specifically, FairGEnt first disentangles sensitive-related and task-relevant structural factors in both graph and textual representations, effectively mitigating bias caused by correlated sensitive attributes. It then employs a fairness-aware alignment module to achieve unbiased integration between the disentangled graph embeddings and textual semantics. In addition, FairGEnt incorporates a fair graph-enhanced instruction-tuning strategy, enabling large language models to better understand complex structural contexts while preventing bias amplification. The key contributions of this work are summarized as follows:

• We study a new fairness problem in graph-enhanced LLMs: how to mitigate the implicit entanglement between sensitive attributes and task-relevant relational information during graph-to-text representation learning, paving the way for achieving fair node prediction with LLM-based graph learning.
• We propose FairGEnt, a disentangled graph-enhanced LLM explicitly designed to promote fairness in graph-
• LLM integration. It systematically separates sensitive related and invariant attributes, leverages fairness-aware alignment and instruction-tuning strategies, enabling accurate and equitable node classification.
• Through extensive evaluations in multiple real-world datasets, we demonstrate that FairGEnt significantly improves both fairness and node classification accuracy compared to existing state-of-the-art methods.

2 RelatedWork

2.1 Graph-enhanced LLMs

LLMs have exhibited impressive performance in diverse natural language processing tasks [Lopes et al., 2020; Azher et al., 2024], yet their ability to directly process and reason over graph data remains inherently limited due to their sequential nature. Recent research has sought to bridge this gap by developing graph-enhanced frameworks that integrate graph data into language modeling. For example, GaLM [Xie et al., 2023] jointly models graph structural information and textual semantics through graph-aware language modeling, enabling language models to better capture contextual dependencies within graph data. Similarly, GraphLLM [Chai et al., 2025] introduces a specialized graph transformer architecture, incorporating learnable structural queries and positional encodings to encode graph representations that guide prefixtuning strategies within LLMs. HiGPT [Tang et al., 2024] advances hierarchical graph encoding, generating structureaware prompts through multi-granularity encoding to improve language model reasoning over complex graphs. Moreover, GraphTranslator [Zhang et al., 2024a] employs contrastive pre-training to facilitate semantic alignment between graph structures and textual contexts, significantly enriching LLM representations. Although these approaches extend the capabilities of LLMs to structured data, they largely neglect fairness considerations, leading to biases that exacerbate discrimination across sensitive attributes.

2.2 Fairness in LLMs

Fairness has gained considerable attention in natural language processing, as pretrained LLMs inherently absorb and replicate societal biases from large-scale textual datasets [Wang et al., 2025a]. These biases manifest prominently through stereotypical associations within contextual embeddings, uneven outcomes in masked language modeling tasks [Radaideh et al., 2025], and skewed reasoning patterns observed during chain-of-thought prompting [Wei et al., 2022]. To mitigate such biases, existing strategies include adversarial training to enforce demographic invariance [Chinta et al., 2024], counterfactual data augmentation for robust representation learning [Zmigrod et al., 2019], and carefully designed fairnessoriented prompting techniques [Chu et al., 2024]. However, these fairness-enhancing approaches primarily focus on textual information alone, neglecting structural biases embedded within graph-structured data and thus limiting their effectiveness in multi-modal or cross-domain scenarios that integrate graph and textual data.

Different from these works, our approach explicitly addresses the fairness challenges that arise when training LLMs on graph data. Specifically, we propose a fairness-aware graph learning model to guide LLM training by disentangling sensitive contextual information from task-related features, thus enabling the model to make fair predictions without compromising task performance. Through this disentanglement, our method effectively reduces biases originating from sensitive attributes embedded within the graph, providing a direct solution to critical limitations identified in previous research.

3 Notations

Let G = (V, E,X) represent an undirected attributed graph, where V = {v1, v2, . . . , vn} denotes the node set with |V| = n, and E ⊆ V ×V denotes the set of edges. We also let ΦG(·) and ΦL(·) represent the graph encoder and the language model encoder, respectively, mapping input data into corresponding latent representations. Specifically, we define two types of input data: X ∈ Rn×d denotes the node feature data and P denotes the textual data provided to the LLM. The resulting embedding representations are denoted by HG for the graph model and HL for the LLM. Each node vi ∈ V is associated with a binary sensitive attribute si ∈ {0, 1}, collectively denoted as S ∈ {0, 1}n×1 . The nodes with si = 0 constitute the deprived group Sd (e.g., female), whereas nodes with si = 1 form the favored group Sf (e.g., male). The adjacency matrix is represented by A ∈ {0, 1}n×n , where Aij = 1 indicates an edge between the nodes vi and vj, and 0 otherwise. Furthermore, we use yi ∈ {0, 1} to indicate the binary ground-truth label for each node vi, and ˆ yG i to denote its predicted label from the graph model. Lastly, the predictions obtained from the language model are denoted as ˆ yL i .

4 Methodology

4.1 The ProposedModel - FairGEnt

In this section, we introduce the novel graph-enhanced LLM, FairGEnt, which systematically ensures fairness-aware representation learning and enhances downstream reasoning performance by disentangling sensitive attributes from taskrelevant information. The FairGEnt framework consists of three core components: i) Fairness-guided Graph-enhanced Encoder Module, ii) Fair Graph-enhanced Alignment Module, and iii) Fair Graph-enhanced Instruction Tuning strategy. Specifically, we first propose the Fairness-guided graphenhanced Encoder, which explicitly separates sensitive attributes from task-relevant attributes within both graph and textual data, producing sensitive-invariant embeddings to suppress potential bias leakage. We then introduce the fair graph-enhanced alignment module, which ensures fairnessaware integration between invariant embeddings and LLM representations. Finally, the Fair Graph-enhanced Instruction Tuning strategy utilizes aligned invariant representations to generate LLM-compatible graph tokens, further enhancing fairness-aware predictive performance through selfsupervised and task instruction tuning.

4.2 Fairness-guided disentangled Encoder Module

We begin by introducing FairGEnt's disentanglement mechanism, which employs a fairness-guided encoder composed of two key components: (i) graph disentanglement, which separates sensitive-related information from sensitive-invariant structural representations within graph embeddings; and (ii) textual graph disentanglement, which isolates sensitive semantic content from task-relevant textual representations obtained from an LLM. In the subsequent discussion, we elaborate on the graph disentanglement component in detail.

Graph disentanglement. To achieve graph structural disentanglement, we introduce a Disentangled Graph Variational Autoencoder (DG-VAE) that explicitly partitions the graph structure into two separate latent variables: sensitive-related (Esen ) and sensitive-invariant (Einv ). Leveraging variational Bayesian inference, our approach captures sensitive and nonsensitive information from the input graph by modeling Esen and Einv as latent variables within a Bayesian network. Specifically, our framework comprises two inference networks, parameterized by ϕ and θ, approximating posterior distributions defined as: qϕ(Esen | A,X, S) and qθ(Einv | A,X), respectively. The sensitive-related encoder explicitly utilizes sensitive attributes (S) to capture structural features directly linked to sensitive factors, whereas the invariant encoder intentionally excludes S to ensure the resulting latent representations are insensitive to sensitive attributes. To further strengthen the disentanglement beyond the explicit disentanglement architecture, we introduce an additional independence constraint by minimizing the Hirschfeld-GebeleinR´enyi (HGR) maximal correlation between Esen and Einv , as detail below:

where f G sen and f G inv are learnable functions with zero mean and unit variance. This term is optimized in a min-max fashion: adversarial networks are trained to maximize the correlation estimate, while the encoders are updated to minimize it, thereby encouraging Esen and Einv to be maximally independent.

Building on this, the final loss function of the structural graph disentanglement component is defined as follows:

where the first three terms constitute the Evidence Lower Bound (ELBO), maximizing the likelihood of reconstructing the graph structure and node attributes while regularizing the variational posterior distributions toward their priors. λHGR is a hyperparameter balancing the strength of the independence constraint. In summary, this module explicitly disentangles graph structures into sensitive-related and sensitiveinvariant latent representations. The disentanglement is further enhanced by applying the HGR-based independence constraint, effectively minimizing correlation between these representations. By clearly isolating sensitive structural information, this module provides explicit guidance for training LLMs on graph-based tasks.

Textual graph disentanglement. In our previous disentanglement stage, we successfully separated sensitive attributes from sensitive-invariant information within graph embeddings. Nevertheless, accurate alignment between graph structures and textual semantics necessitates a similar disentanglement within textual embeddings generated by LLMs. To achieve this, we introduce a dedicated textual graph disentanglement approach, explicitly decomposing textual embeddings into sensitive and sensitive-invariant components, guided by the previously disentangled graph representations, enhancing fairness-aware alignment.

Given a graph G, we sample local subgraphs and serialize them into structured textual prompts using GraphML [Brandes et al., 2013], preserving node features and relational structures in structured textual form. Each textual prompt Pn is fed into a pretrained LLM encoder ΦL(·) (i.e., BERT [Koroteev, 2021]) to obtain subgraph-level embeddings: HLn = ΦL(Pn). However, embeddings from LLM encoders often entangle task-relevant information with sensitive attribute information, which can lead to bias.

To control and disentangle these factors, we design two lightweight projection networks to decompose HLn into complementary subspaces. Specifically, we introduce two lightweight multilayer perceptron projection heads, f L sen (·) and f L inv (·), to decompose the latent representation as follows:

The embedding zsen n is intended to capture sensitive-related factors, while zinv n encodes sensitive-invariant and taskrelevant information that is invariant to sensitive attributes.

To ensure effective disentanglement, we impose an explicit independence constraint between the sensitive-related and sensitive-invariant textual embeddings, with the goal of reducing statistical dependence and preventing bias leakage during graph-text alignment. To limit information leakage between these two textual subspaces, we introduce a distance covariance-based regularization term as a principled dependence measure. Specifically, given a mini-batch of B samples, each sample n is associated with a pair of textual embeddings (zsen n , zinv n ), where zsen n captures sensitive-related information and zinv n represents sensitive-invariant content. We collect these embeddings as Zsen = {zsen n }Bn=1 and Zinv = {zinv n }Bn=1 , and measure their dependence using distance correlation, defined as:

where dCov(·, ·) denotes the distance covariance computed from pairwise Euclidean distance matrices after doublecentering operations. Minimizing Ltextual ensures statistical independence between sensitive-related and invariant textual representations, effectively suppressing both both linear and non-linear dependencies.

4.3 Fair Graph-enhanced Alignment Module

Building upon the disentangled embeddings from the previous step, aligning with invariant attributes is critical to prevent downstream tasks from inheriting biases related to sensitive attributes such as gender or ethnicity, thus promoting fairness. We next propose the Fair Graph-enhanced Alignment Module, designed explicitly to align only sensitive-invariant attributes across textual graph embeddings and raw graph representations. Furthermore, invariant attributes encapsulate structural and functional knowledge less susceptible to domain-specific biases, enhancing the robustness and generalizability of learned representations. This alignment is implemented through: (1) lightweight projection into a shared latent space and (2) leveraging neighborhood contexts for enhanced structural coherence.

Given node-level textual graph invariant embeddings zinv n ∈ Rdz and graph data sensitive-invariant representations Einv n ∈ Rde , we project both into a common alignment space to facilitate cross-modal matching. Formally, we employ lightweight linear transformations defined as follows:

where ϕZ(·) and ϕE(·) are lightweight linear projection layers mapping inputs into a shared latent space of dimensionality d.

After projecting the sensitive-invariant attributes into a shared latent space, a remaining challenge is the potential degradation of alignment quality when relying exclusively on isolated node-level information. To address this issue, we enhance the representation of each node by aggregating information from its immediate neighborhood. This graphenhanced contextual representation explicitly incorporates local graph topology, yielding embeddings that are structurally coherent and more informative for alignment. We define the neighborhood-enhanced textual representation of node n as:

where N(n) is the set of neighboring nodes derived from the adjacency matrix of the raw graph. This contextualization ensures that our alignment is robust against local perturbations and captures structural coherence effectively.

To effectively align sensitive-invariant embeddings across modalities, we adopt a multi-component contrastive learning framework. We design three distinct yet complementary similarity measures to facilitate alignment:

Node-level alignment captures the immediate semantic alignment between node representations:

where τ is a temperature hyperparameter controlling the distribution sharpness.

Node-to-neighborhood alignment explicitly incorporates contextual information, aligning node embeddings with aggregated neighborhood representations from the opposing modality:

Intra-modal textual consistency alignment ensures internal consistency within textual embeddings and their neighborhood contexts, stabilizing embedding distributions:

We unify these similarity matrices into a joint alignment loss:

where the target vector y = (0, 1, . . . , B - 1) indicates correct node-to-node correspondences within each batch, and λ balances the influence of each component.

To optimize FairGEnt in a unified manner, we combine the alignment loss with the disentanglement losses from both textual and structural disentanglement modules, forming the overall pre-training objective. The total loss function as follow:

where α modulates the strength of disentanglement and β controls the alignment intensity.

4.4 Fair Graph-enhanced Instruction Tuning

Following the fair graph-enhanced alignment established in the previous module, it remains critical to ensure that LLMs not only align structurally across modalities, but also interpret and reason about these invariant graph structures without introducing biases related to sensitive attributes. To achieve this goal, we utilized the previously aligned sensitive-invariant representations to construct a fairness-aware instruction tuning. Specifically, this paradigm empowers the LLMwith robust graph reasoning capabilities while restricting the input graph tokens to those derived solely from sensitive-invariant attributes, thereby ensuring fairness.

To effectively integrate previously grounded invariant embeddings ˆEinv ∈ RN×d into a pretrained LLM, it is necessary to convert these embeddings into graph tokens compatible with the LLM's input format. In order to efficiently achieve this, we use a lightweight linear projection function: X G n = LP (ˆEninv ), X G n ∈ RdLLM, where LP (·) is designed to map invariant graph embeddings into the embedding space of the pretrained LLM encoder ΦL(·), thus ensuring minimal computational overhead and compatibility with downstream tasks. The graph tokens are organized sequentially by performing h-hop neighbor sampling around a central node, resulting in an ordered token sequence structured as follows: X G(v) = {xGu1 , xGu2 , . . . , xGum }, ui ∈Mh(v), where Mh(v) denotes the nodes sampled from the h-hop neighborhood around node v. This ordered sequence maintains clear locality information essential for graph reasoning tasks.

To enable the pretrained LLM to effectively interpret and reason about invariant graph tokens without reliance on explicitly labeled data, we further propose a structure-aware self-supervised graph matching task. Specifically, given the invariant graph tokens generated in the previous step, we sample a h-hop subgraph S(v) around each central node v using random neighbor sampling. We then construct a sequential ordering of the invariant graph tokens X G(v) according to the sampled subgraph structure. Corresponding textual descriptions, such as titles or node attributes, are collected for each node within this subgraph. To form the self-supervised training task, we randomly shuffle these textual descriptions and instruct the LLM to recover their original order, guided solely by the provided invariant graph tokens. This setup allows the model to inherently capture structural relationships embedded in the tokens, facilitating robust graph reasoning capabilities in the absence of direct supervision.

Given shuffled texts T ′ = tπ(1), . . . , tπ(m), where π denotes a permutation, we aim to maximize conditional likelihood:

In particular, during this stage, we freeze the parameters of both the LLMΦL(·) and the invariant graph encoder, training solely the lightweight projector LP (·). This strategy ensures efficiency and prevents overfitting to specific graph data.

For targeted downstream tasks such as node classification and link prediction, we introduce an instruction fine-tuning stage. Initialized with the pretrained projector LP (·) from first stage, this stage involves providing the LLM with invariant graph tokens X G(v), a concise textual description of the central node, and a clearly defined query specifying the intended task. The LLM is then expected to generate task-specific responses, including predicted labels or structured explanations. We employ the standard negative loglikelihood loss for instruction tuning:

where yi denotes the tokens of the target sequence and l is the length of the response. For computational efficiency, we freeze both the pretrained LLM encoder and the graph encoder, optimizing only the lightweight projection module. Key hyperparameters, such as neighborhood size h, token sequence length, and temperature parameter τ, should be carefully tuned for optimal performance. Overall, by exclusively leveraging sensitive-invariant attribute representations, our proposed instruction tuning equips the LLM to perform fair, robust, and interpretable predictions and classifications on graph-structured data, effectively mitigating biases associated with sensitive attributes.

5 Experiments

5.1 Datasets

We evaluate FairGEnt on three widely used real-world datasets, namely German [Asuncion and Newman, 2007], Bail [Agarwal et al., 2021], and Credit [Yeh and Lien, 2009]. The German dataset consists of bank clients represented as nodes, where edges capture similarities between individuals and node labels indicate credit risk. The Bail dataset is constructed from criminal justice records, where nodes denote defendants and labels correspond to bail outcomes. The Credit dataset is derived from credit card records, where nodes represent individuals and labels indicate default risk. For all datasets, node features describe demographic and behavioral attributes, while gender, race, and age are used as the sensitive attributes, respectively. The detailed statistics of these datasets are reported in Table 2.

5.2 Baselines

We compare the proposed method with several state-of-theart baselines, categorized into two groups: i) Vanilla Language Model, which aim to improve LLM reasoning by injecting graph information through different integration strategies. Specifically, Vanilla-LLM, a standard LLMwithout explicit integration of graph data and fairness considerations; ii) Graph-enhanced Language Models: GaLM [Xie et al., 2023] utilizes graph-attention mechanisms to jointly learn graph and text embeddings; LLaGA [Chen et al., 2024] reorganizes graph nodes into structured token sequences, enabling pretrained LLMs to directly reason over graph data; GraphTranslator [Zhang et al., 2024a] transforms structured graph data into textual representations through a translation-based framework, thereby bridging graph inputs and language modeling; GraphLLM [Chai et al., 2025] employs vector quantization to tokenize graph structures, enabling LLMs to directly leverage graph information; and GraphICL [Sun et al., 2025] integrates graph-structured information into language models by leveraging in-context learning, enabling effective structural reasoning without explicit fine-tuning or parameter updates.

5.3 EvaluationMetrics

We evaluate our framework from two perspectives: predictive performance and fairness. For predictive performance, we adopt Accuracy and Macro-F1 [Sokolova and Lapalme, 2009], where higher values indicate better classification performance. For fairness evaluation, we employ two widely used group fairness metrics: Statistical Parity Difference (SPD) [Dwork et al., 2012] and Equal Opportunity Difference (EOD) [Hardt et al., 2016]. These metrics quantify disparities in model predictions across different sensitive groups, with values closer to zero indicating improved fairness.

5.4 Experiment Results

Comparison Study. We compare FairGEnt against six baseline models on three datasets. The averaged results are summarized in Table 1. The experimental outcomes demonstrate that FairGEnt consistently achieves superior performance compared to baseline methods across most evaluation metrics. Specifically, FairGEnt significantly outperforms all baselines in fairness metrics, indicating its strong capability in mitigating bias propagation within graph-enhanced LLMs. The notable fairness improvements can be primarily attributed to three key factors: (i) explicitly disentangling sensitive attributes from task-relevant information in graph and textual embeddings, thereby reducing biased correlations during representation learning; (ii) employing a fairness-guided alignment module that integrates disentangled embeddings with LLM representations, effectively minimizing residual bias leakage; and (iii) refining the fairness-aware representation through a graph-enhanced instruction tuning process, which promotes unbiased downstream reasoning. Moreover, FairGEnt maintains competitive predictive performance comparable to or surpassing the baselines, highlighting the framework's ability to simultaneously ensure fairness and retain utility performance. Overall, these results confirm the effectiveness of FairGEnt in delivering fair and robust predictions within graph-structured language modeling tasks.

Ablation Study. We conducted ablation studies to investigate the contributions of individual modules within FairGEnt. Specifically, we constructed two variants: FairGEnt-NA and FairGEnt-NF. FairGEnt-NA integrates graph-enhanced embeddings with LLM representations without applying explicit disentanglement of sensitive attributes, thereby prioritizing task-relevant information and predictive performance but omitting fairness constraints. In contrast, FairGEnt-NF incorporates sensitive attribute disentanglement but excludes the fairness-guided alignment module, thus lacking explicit fairness constraints during representation integration. Results are summarized in Figure 1. Compared to FairGEnt, FairGEnt-NA achieves higher predictive performance but significantly compromised fairness, as sensitive attributes remain entangled with task-relevant information, leading to substantial bias propagation. In contrary, FairGEnt-NF exhibits decreased predictive performance due to potential loss of task-relevant information during disentanglement; however, fairness metrics notably improve relative to FairGEntNA, highlighting the effectiveness of sensitive attribute disentanglement. Finally, our proposed FairGEnt demonstrates optimal fairness; at the same time, its prediction performance is better than FairGEnt-NF, validating that combining sensitive attribute disentanglement and fairness-guided alignment effectively balances performance and fairness.

Hyperparameter Sensitivity Analysis. We further conduct a hyperparameter sensitivity analysis to examine how FairGEnt balances fairness preservation and predictive utility under different strengths of disentanglement and cross-modal alignment. In the overall pre-training objective, α controls the contribution of the graph and text disentanglement losses, while β regulates the invariant graph-text alignment objective. Therefore, these two parameters directly determine how strongly FairGEnt suppresses sensitive information and how effectively it transfers sensitive-invariant knowledge across modalities. We vary α and β over the exponential range {e-4 , e-3 , . . . , e3 , e4 } on the Citeseer dataset, and report the results in Figure 2. The results show that parameters play a meaningful role in shaping the fairness-utility trade-off. As α increases, the model places greater emphasis on separating sensitive-related factors from invariant representations, leading to lower SPD and EOD. However, overly large values may also remove part of the task-relevant signal that is statistically entangled with sensitive attributes, resulting in a mild decline in Accuracy and Macro-F1. Similarly, increasing β improves fairness by encouraging stronger alignment between invariant graph and textual subspaces, thereby reducing bias propagation during cross-modal integration. Nevertheless, excessive alignment strength can constrain representation flexibility and slightly affect predictive performance. Overall, the best results are achieved when α and β are set to moderate values, suggesting that FairGEnt benefits most from a balanced optimization strategy, sufficiently strong disentanglement to suppress sensitive leakage, together with stable invariant alignment to preserve graph-text semantic consistency.

6 Conclusion

Given the inherent limitations of existing LLMs in effectively and fairly processing graph-structured data, this paper presents an initial exploration into fairness-aware integration of graph knowledge and language modeling. Specifically, we propose FairGEnt, a disentangled graph-enhanced large language model designed explicitly to mitigate biases from structural and textual sources. FairGEnt disentangles sensitive-related and sensitive-invariant factors, thereby preventing protected attributes from adversely influencing downstream predictions. Additionally, it incorporates fairnessaware alignment and instruction-tuning strategies to enable unbiased integration and enhance graph-structure comprehension. Extensive empirical evaluations demonstrate that FairGEnt consistently outperforms baseline methods in fairness and predictive accuracy, confirming its effectiveness as a practical and generalizable approach to fair graph-enhanced language modeling.