Learning Causally Disentangled Representations for Fair Personality Detection

1 Introduction

Personality refers to an individual's psychological constitution, including unique aspects of cognition, emotions, attitudes, and values. With the blossoming of social media, users generate massive posts daily that reveal their psychological activities, providing new opportunities to infer personality traits automatically [Fang et al., 2023]. Successfully inferring traits from such content can enhance massive applications, such as recommendation system [Yang et al., 2022], dialogue system [Chawla et al., 2023], and human-computer interaction design [Chien et al., 2022].

∗Corresponding authors.

In previous research works, Deep Neural Networks (DNNs) are the default choice because they can reveal the most predictive hidden patterns from posts [Lynn et al., 2020; Yang et al., 2021b; Zhu et al., 2022; Yang et al., 2023b]. Recently, Large Language Models (LLMs) have been rapidly applied in personality detection, such as psychological questionnaires based chain of thought [Yang et al., 2023c] and large model-based data augmentation [Hu et al., 2024]. Despite impressive improvements in personality detection, these well-trained DNNs may "cheat" us by inadvertently capturing and even amplifying unintended individual biases through author-generated content. As an example, Figure 1 illustrates how racial bias confounds the detection models. To further verify detection modal is not independent of the human individual attributes, we conducted an experimental investigation with unbalanced demographic attributes on the Pandora dataset. As shown in Table 1, the unbalanced group (Group I) always obtains a higher rate of false positives and false negatives. This demonstrates that although the text does not explicitly contain any individual attribute, well-trained models do incorrectly learn the spurious correlations between posts and traits.

While some statistical studies have explored the relationship between personality and demographics [Gjurkovi´c et al., 2021; Kerz et al., 2022], the research on eliminating individual confounders to improve the fairness of personality detection is still a wilderness. To achieve this goal, we face the following challenges: (1) Due to privacy concerns, people are reluctant to share personal information on the internet, making it labor-intensive and time-consuming to obtain usergenerated posts labeled with individual attributes. (2) Even with access to most attributes information, some unknown types of human biases are difficult to predefine in advance.

To solve the above challenges, we attempted to improve the personality detection model by applying causal intervention. we propose a novel Interventional Personality Detection Network (IPDN) to remove the confounding effect of individual attributes C through backdoor adjustment P(Y |do(X)) instead of the conventional likelihood P(Y |X) estimated by traditional DNNs-based methods. Specifically, unlike approximating the intervention via predefined confounders, we turn to learning implicit confounders in user-generated posts to achieve confounder-agnostic general debiasing. Our IPDN starts to decompose the causal feature and biased feature by disentanglement and reconstruction module. Then, a confounder builder cumulatively clusters bias features and approximates confounder prototypes as the iteration of training. Finally, the reconstruction network are reused to approximate backdoor adjustment on raw posts, ensuring that traits see each confounder equally before detection. In summary, our main contributions are as follows:

• To the best of our knowledge, this is the first effort to take individual biases into account for personality detection from the causal perspective. By considering individual attributes as confounding factors, we aim to look for spurious correlations in the model and mitigate unfair detection.
• Based on the causal theory of backdoor adjustment, we propose a novel interventional personality detection network (IPDN) that automatically learns confounders from post and cuts off the spurious correlations between posts and traits through deconfounded training with intervention.
• Extensive experiments demonstrate the effectiveness of IPDN and the fairness of personality detection is improved to a certain extent.

2 RelatedWork

2.1 Personality Detection

Early efforts heavily relied on psycholinguistic statistics features to detect personality, such as Linguistic Inquiry and Word Count (LIWC) [Tausczik and Pennebaker, 2010] Mairesse [Mehta et al., 2020], and MRC [Coltheart, 1981], considering the influence of personality traits on language use patterns. However, statistical analysis has limitations in effectively capturing the semantics of posts. With the rapid advancement of Deep Neural Networks (DNNs), a line of DNNs approaches like CNNs [Xue et al., 2018], LSTMs [Tandera et al., 2017] and Transformer [Yang et al., 2021a; Zhu et al., 2022] have demonstrated impressive results in this field. Meanwhile, another line of research focus on the structure of user-generated posts, such as hierarchical structure model (SN+Attn) [Lynn et al., 2020], tripartite graph networks (TrigNet) [Yang et al., 2021b], graph contrastive learning (CGTN) [Zhu et al., 2022], and dynamic graph networks (D-DGCN) [Yang et al., 2023b]. Recently, Large Language Models (LLMs) have been rapidly applied in personality detection, The PsyCoT model uses psychological questionnaires as Chain of Thought to simulate personality tests through multiple rounds of dialogue [Yang et al., 2023c]. TAE model tried to distill LLM knowledge to enhance the small model for personality detection [Hu et al., 2024]. However, the above methods mainly focus on obtaining a meaningful representation of user-generated posts, overlooking the influence of the inherent individual attributes of authors. Certainly, some researchers have investigated and confirmed the relationships between demographic variables and personality traits based on psychological theories, the de-biased personality detection remains unexplored to our knowledge.

2.2 Causal Inference

Causal inference is developed to estimate causal effect with covariate shift. Benefiting from the great potential of the causal tool to provide unbiased estimation solutions, it has been widely applied to diverse fields, such as recommendation [Wei et al., 2022], emotion recognition [Yang et al., 2023a] natural language inference [Feder et al., 2022] and pretrained language models [Zhou et al., 2023]. The most relevant work to us is the debiased text classification. CORSAIR [Qian et al., 2021] imagines the counterfactual docu'ment to distill and mitigate the label bias and keyword biases in text. CCD [Chen et al., 2023] applies Normalized Weighted Geometric Mean (NWGM) [Xu et al., 2015] approximate causal intervention, removing the psycholinguistic bias in fake news detection. In this work, we discover human bias in personality detection and propose a novel causal intervention-based debiasing framework to enhance detection performance.

3 Methodology

3.1 ProblemDefinition

Given a set of posts P = {p1,p2...pn} from a user, the goal of personality detection is to predict the categories of tdimensional personality traits, denoted as Y = {y1, y2, yt}.

The personality traits are categorized into the Myers-Briggs personality inventory (MBTI) [Myers, 1997] and the Big-Five indicators [Digman, 1990]. the MBTI taxonomy includes Introversion / Extraversion, Sensing / Ntuition, Thinking / Feeling, and Judging / Perceiving. While the Big-Five taxonomy includes Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism.

3.2 Causal View at Personality Detection

To answer the whys of the individual attribute affects personality detection, we formulate the Structural Causal Models (SCM) [Pearl, 2009] to illustrate causal relations in the detection process. As shown in Figure 2, it is a directed acyclic graph that consists of three variables (nodes): Node X, Y , and C denote the user-generated posts, the predicted probability of the personality traits, and the individual attributes, respectively. The individual attributes include demographic and some unknown attributes, such as cultural background, emotions. The directed edges are the causalities between two variables: cause → effect. X → Y . Traditional methods only consider that the personality traits could be inferred by user-generated posts. C → Y . Compared to the traditional method, we realistically add a confounder node C to portray the detection process. The author attribute directly affects the final detected probability, which is expected since detectors can learn to associate post with individual information to make unfair decisions about certain groups. C →X. Similarly, the individual attributes has a causal effect on posts and indirectly affects the detected probability. Such individual information are normally embedded in the user-generated posts, for instance, the wording and tone of the authors affect their posts.

Based on this causal graph, we can observe the presence of confounder C opens a backdoor path as show in Figure 2 (b): X ← C → Y introduces spurious correlation between the posts X and personality traits Y by learning the likelihood P(Y |X), thereby degrading the generalization performance. This process is formulated by bayes rule:

The confounders C introduce the observational bias via P(C|X), because C satisfy the backdoor criterion that all backdoor paths between X and Y are blocked by conditioning on C and C do not consist of any variables that are descendants of X.

3.3 Intervention via Backdoor Adjustment

Our goal is to develop a detection model that is unaffected by unobserved confounders C. One straightforward way is conducting a randomized controlled trial by collecting any attribute of any individual to ensure that traits are balanced across all confounders. In this case, the conditional probability P(Y |X) equals the causal probability P(Y |do(X)). However, implementing such solutions may not be practical. First, users are often unwilling to disclose personal information. Furthermore, the complexity of confounders makes constructing a balanced dataset challenging. We turn to a more elegant intervention on the input posts X by blocking the backdoor path between X and Y based on the backdoor adjustment (Figure 2 C). The backdoor adjustment performs a do-operator on X, effectively cutting off the causal path from C to X. As a result, the detection model will approximate the causal intervention P(Y |do(X)) instead of the spurious association P(Y |X) estimated by traditional methods:

where f(·) is nonlinear transformation that incorporate independent variables X and each confounder c. Through causal intervention, we provide a fair opportunity for X to observe each confounder c when detecting trait Y . After that, the spurious correlation from C to X is cut off, allowing the causal effect X → Y free from the effect of C.

3.4 IPDNArchitecture

Backdoor adjustment requires confounder C could be stratified. Unfortunately, there is no strict and widely accepted definition for the confounder that influence traits, let alone stratifying for them. To end this, we propose an Interventional Personality Detection Network (IPDN) to automatically learn confounders C and mitigate biases. As shown in Figure 3, the IPDN comprises three modules: 1) the feature disentanglement and reconstruction; 2) the confounder builder; 3) the deconfounded training with intervention.

Feature Disentanglement and Reconstruction

Given the post vector ui for each user, fc and fb are applied as the projection function to learn the causal and biased representations:

where θc and θb are learnable parameters of the two projection function, respectively. Neural network dynamics research has found that biased features are easier to learn than the desired knowledge in the early stages of training [Nam et al., 2020]. The Generalized Cross Entropy (GCE) loss is used [Zhang and Sabuncu, 2018] to amplify bias. Specifically, we forward the biased features ebi for detection:

where yb are personality traits predicted based on biased features, py(ui, θb) are probability assigned to the target traits y, and q ∈ (0, 1] is a hyperparameter that controls the degree of amplification. The gradient of the GCE loss up-weights the gradient of standard Cross Entropy (CE) loss when the sample has a high prediction probability toward target trait:

The GCE loss trains a biased projection function by giving more weight to "easier" samples with strong agreements between the predictions and the labels, which amplifies the "prejudice" of bias projection function. In this way we require no explicit labeling of confounders, but instead cheap and generalized confounders. While causal representations are enforced by a soft orthogonal constraint between output of causal and biased projection function:

where ‖·‖2F is the squared frobenius norm. To ensure that the separated features are within a reasonable range, given the input of causal features eci and biased features ebi , the IPDN should be able to reconstruct the basic features ui:

The reconstruction loss is defined as mean squared error between ui and ˆui:

Confounder Builder

Confounder builder is designed to stratify confounder prototypes from separated bias features for backdoor adjustment. We cluster the learned biased features to approximate confounder prototypes C = [c1, c2, ...,cK], where K is the number of confounder, and each cluster ck ∈ Rd represents a confounder prototype. In practice, instead of updating clusters of the entire training set, we incrementally update clusters as the mini-batch iterates. In m-th iteration, the confounder dictionary C m = [cm1 , cm2 , ..., cmK ] are dynamic updates by K-Means++ (ebi ), ebi ∈ batch{1, 2 · ··,m}, where ebi is the biased feature from the previous m batch data. This incremental updating process allows for efficient and scalable confounder refinement during the training process.

Deconfounded Training with Intervention

As described in Section 3.3, the backdoor adjustment is to obtain a personality detection model P(Y |do(X)) that is unaffected by unobserved confounders C. Unlike mainstream NWGM, which uses linear models to approximate conditional probabilities, we perform backdoor adjustment during training, i.e., any personality trait is viewed equally to all confounding stratum by the following training objective:

where ck is the confounder generated in each iteration of the training process, and given a decoupled causal feature eci , we reuse the reconstruction network fr as encoder and combine each confounding feature ck to forward K times. Each pattern of the unknown confounder is considered in the detection and the spurious correlation would not dominate. In general, our IPDN is end-to-end trained with a joint learning strategy, where a feature disentanglement task is to help learn confounders and an intervention personality detection task to pursue the causal effect collaboratively. The overall objective is to minimize both losses simultaneously,where λ is hyper-

Model Inference

Once the IPDN network is well-trained, given a test user, the personality traits are inferred through the fc, fr, and fd branch.

where cMk is the confounder feature aggregated in the last iteration M of training phase. By equalizing various confounders, unbiased personality inferences are ensured to a certain extent.

4 Experimental Settings

4.1 Datasets

Kaggle dataset is collected from PersonalityCafe platform where people openly discuss their MBTI in daily communication. The Kaggle dataset comprises 8675 anonymous users, with each user contributing between 45 to 50 posts. Pandora is consist of Reddit posts, which labeled with MBTI and demographics (age, gender, and location) for 9067 anonymous users. Essays is a well-known stream-ofconsciousness dataset [Pennebaker and King, 1999] consisting of 2468 anonymous volunteers with essays recorded their minds within a short time frame. Each user is marked with binary labels of Big-Five. These annotations are determined through a standardized self-report questionnaire. Following previous works [Yang et al., 2021b; Yang et al., 2023b], these datasets are randomly divided into 6:2:2 for training, validation, and testing respectively.

4.2 Baselines

For a comprehensive evaluations, we compare IPDN with the following 4 group baselines:1) Post semantics models: LIWC [Tighe et al., 2016] pioneerly extracts the LIWC psycholinguistic features and employs SVM for detection. RCNN [Xue et al., 2018] is a hierarchical CNN structure, which combines textual semantics and LIWC psychological features. BERTfinetune [Mehta et al., 2020; Ren et al., 2021] fine-tuned BERT to achieve the optimal configuration for personality detection.2) Post structure models: SN+Attn [Lynn et al., 2020] is a hierarchical network that utilizes GRU to learn user representations from word-level and sentencelevel sequences. TrigNet [Yang et al., 2021b] is a novel graph attention network that aggregates different posts from each user by inherent psychological structures. D-DGCN [Yang et al., 2023b] is a dynamic graph convolutional network that automatically learns the structure between user-generated posts. 3) LLMs-basedmodels: the "gpt-3.5-turbo-0301" version of ChatGPT is applied for personalty detection, the temperature is set to 0, making the outputs deterministic for the identical inputs. TAE [Hu et al., 2024] distills the LLM's knowledge to enhance the small model for personality detection. 4) Debiasing text classification: CORSAIR [Qian et al., 2021] is a counterfactual framework for debiasing text classification, It addresses dataset and keyword biases by generating two counterfactual documents during inference. In our implementation, we use demographic lexicon [Sap et al., 2014]

as keywords. NWGM [Chen et al., 2023] approximates the do-calculus for desired interventions at the feature level. In practice, the confounder dictionary x is also pre-defined demographic lexicon.

4.3 Implementation Details

We implement our IPDN in Pytorch 935 1.11.0 and train it on three NVIDIA GeForce RTX 2080 GPUs. We utilized the Adam optimizer (Kingma et al., 2017) and searched for the learning rate among {1e-2 , 1e-3 , 1e-4 }. IPDN are trained for 80 and 120 epochs in single-dataset and cross-dataset experiments, respectively. Early stopping strategy is employed for training. To initialize the post embeddings, we used the pre-trained language model BERT with the bert-base-cased architecture. The output dimensions of the mapping function are set to 200, 200, and 300 for the Kaggle, Pandora, and Essays dataset. The dimensions of the confounder prototype are the same as the output dimension of the mapping function, which is to facilitate feature-level computation. The size K of confounder dictionary C = [c1, c2, ...,cK] (i.e., the number of clusters) are set to 64, 128, and 64 for the three datasets, respectively. We search for the trade-off parameter λ are searched in (0, 1) for different datasets.

5 Experimental Analysis

5.1 Main Results

The overall results are shown in Table 2 and the major findings can be summarized as follows:

First, we can observe that IPDN outperforms all baselines. Compared with the state-of-the-art method (TAE), IPDN achieves 1.81%, 1.08%, and 1.30% improvements in average Macro-F1 on Kaggle, Pandora, and Essays datasets respectively. The results verify the superiority of our model in personality detection. We attribute this success to two-fold factors: 1) The users in these datasets have diverse individual attributes, and the generated posts come from uncontrolled real-world scenarios. In this situation, IPDN can effectively eliminate spurious correlations by causal intervention and get substantial gains. 2) Unknown confounders in personality detection are well captured by batch-based progressive confounder learning during model training.

Second, we also conducted a comparison between IPDN and mainstream debiasing methods (CORSAIR, NWGM). Both CORSAIR and NWGM utilize the same BERT model as the backbone network and rely on predefined demographic dictionaries for counterfactual inference and backdoor adjustment. CORSAIR and NWGM outperformed BERTfinetune in all datasets, indicating that explicitly modeling confounders in the network is effective. Furthermore, IPDN generally performs better than CORSAIR and NWGM. A reasonable explanation is that IPDN performs real intervention by treating confounder features as trainable parameters, while the predefined confounders in CORSAIR and NWGM may not be sufficiently representative.

Third, all baselines exhibit generally poor performance on the Sensing / Ntuition trait in both the Kaggle and Pandora datasets. This deviation arises from the imbalance in the distribution of this trait, leading to insufficient predictive capability for the fewer traits. Fortunately, our IPDN strives to find contextual stratification confounders and refines the causal representation, leading to some improvements in this dimension. Moreover, it is worth noting that although causal intervention brings gains for all traits across three datasets, the improvements on the Essays dataset are not as pronounced as Kaggle and Pandora. We conjecture that this could be due to a limited training sample, resulting in insignificant confounding effects and insufficient intervention.

5.2 Ablation Study

We conduct thorough ablation studies for our IPDN on three datasets to verify the effectiveness of different components.

As shown in Table 3, the performance drops by around 5.76%, 4.81% and 1.89% on Kaggle, Pandora, and Essays dataset after removing the causal intervention component (w/o CL) and feds the causality features eci to the detection layer fd(fr(eci ), which demonstrates the necessity of deconfounded training with intervention. Then, the performance continues to decline when the disentanglement component is further removed (w/o CL+FD) and uses the initial features ui for detection fd(fr(ui)), highlighting the causal representations obtained by disentanglement are advantageous for personality detection. Conversely, directly employing biased features fd(fr(ebi ) yields the lowest performance (r/w BF), further emphasizing that our designed disentanglement module effectively isolates some hidden confounders.

5.3 Validation of Out-of-Distribution Scenario

We devise a challenging yet practical Out-of-Distribution (OOD) test configuration through cross-dataset validation. This is makes sense as the posts provided to recognition systems in real-world scenarios are produced by diverse individuals with varying preferences and in different environmental contexts. Consequently, the distribution of such data is beyond our control. Specifically, we use Kaggle as the training set and allocate 20% of Pandora as the test set, and vice versa. As shown in Table 4, IPDN shows an overall improvement compared to the strong baseline D-DGCN in cross dataset experiment. Although D-DGCN learns a rich document representation through post structure, there are more implicit confounders in cross-dataset testing. In contrast, IPDN isolates and removes these confounders through causal intervention, focusing more on causal relationships related to the target personality.

5.4 Effect of Confounder Prototype

To demonstrate the rationality of the learned confounder prototype, the random confounder dictionary is used for comparison. Simultaneously, to find the optimal size K for the confounder C, we conduct experiments on all datasets by setting K to 16, 32, 64, 128, 256, and 512, respectively. The results in Figure 4 indicate that the random prototype significantly hurts performance, thus confirming the effectiveness of our learnable confounder prototype. For datasets containing varying degrees of harmful bias, selecting an appropriate size K can effectively assist in deconfounded training.

5.5 Case Study of Causal Intervention

In Figure 5, we select three representative test cases to show the performance of the model before and after the intervention. Those cases are collected wildly from the MBTI community on Reddit, where people are mostly MBTI-tested with personal information. In the first row, this is a daily post shared by an African-American vernacular girl, and the popular slang ("dogg") has led the detection model to form a stereotype. In the second row, highly educated individuals typically have clear and logical language expression, which also serves as a shortcut in prediction. Furthermore, the main users of online social networks are young people, whereas, for older people, IPDN has untangled the false correlation between personality traits and some old-fashioned vocabulary ("icebox"), thereby improving the performance of the model.

5.6 Ethics Statement

The current research requires a thoughtful examination of ethical issues. 1) Privacy protection: This study is built on anonymized public datasets and was ethically vetted to comply with ethical guidelines and norms. 2) Risk of misinformation and misuse: Detection systems may be subject to misuse, e.g., discrimination, fraud, or manipulation. It is important to take steps to ensure that the detection results are used correctly, e.g., employment selection, loan approval, or legal judgments. and to prevent them from being misused or misrepresented. 3) Psychological impacts and individual rights: When personality assessment results in conflict with an individual's self-identity or social identity, it may harm selfesteem, personal image, and social relationships. Attention should be paid to mental well-being and personal rights when employing these systems.

6 Conclusion

In this paper, we analyze and identify the individual bias in the user-generated posts for personality detection from the causality perspective. We propose an interventional personality detection network that approximate backdoor adjustment to eliminates the hidden bias. Our IPDN does not require predefined confounders and instead learns confounders from posts. Experiments on three real-world datasets verify that IPDN can effectively eliminate biases and improve personality detection. In future work, we are interested in addressing the personality trait label bias with causal inference theory.

Acknowledgments

This work was sponsored by Beijing Nova Program (20230484409), National Natural Science Foundation of China (62272322, 62272323,62206148), and Beijing Postdoctoral Research Foundation (2025-135).