Good Advisor for Source Localization: Using Large Language Model to Guide the Source Inference Process

1 Introduction

The wide usage of social media has brought both convenience and potential risks to everyone's lives [Meel and Vishwakarma, 2020]. One key issue that has gained significant attention from the government is the spread of rumors. Various fast-spreading rumors have led to significant economic losses [Depoux et al., 2020]. Therefore, it is crucial to identify the rumor sources to prevent further damage [Jiang et al., 2019].

In the field of rumor source localization, the widely employed methods are primarily based on graph theory algorithms [Wang et al., 2017; Hou et al., 2024a] and Graph Neural Network (GNN) algorithms [Dong et al., 2019; Hou et al., 2025], which mainly adopt the snapshot propagation cascades without textual information [Jiang et al., 2016; Jin and Wu, 2021]. In practice, these traditional methods have preliminarily demonstrated their effectiveness. However, with the rapid development of the internet, an increasing amount of textual comments are generated by social media users in the propagation process. The textual comments are not only the direct carrier for users to express their opinions [Yang et al., 2019] and emotions [Zhang et al., 2021], but also potentially serve as a critical channel for the propagation and diffusion of rumors. When confronting the everincreasing volume and significance of textual comments, the traditional methods cannot fully utilize this textual information. Therefore, how to leverage the potential of textual comments for effectively locating the source becomes a critical question.

In the field of text analysis, pre-trained language models like BERT [Devlin, 2018] are commonly used for their deep linguistic understanding ability. However, small language models like BERT have limitations in domain-specific background knowledge, particularly the lack of external background knowledge or experiences (e.g., historical facts), which led to limited accuracy. As a powerful tool, LLMs like GPT-4 can capture and analyze fine-grained segments (such as historical factual contexts) based on extensive knowledge bases and inference capabilities [Achiam et al., 2023]. Like the application of LLMs in the rumor detection field [Lai et al., 2024], an LLM can be used to address the rumor source localization problem by using snapshots containing textual comments as input. However, as can be seen from Fig. 1(a), the LLM fails to output the correct judgment of the real propagation source if directly using it as a predictor. This suggests that the LLMs are not suitable decoders for rumor source localization tasks, which may not fully comprehend the intricate propagation patterns and network structures embedded in cascade data. In summary, integrating textual comments and snapshots of propagation cascades to enhance the accuracy of rumor source localization is a challenge.

In this paper, we propose a novel framework for rumor source localization, Contrastive Rumor Source Localization via LLM (CRSLL), which uniquely integrates user comments and propagation dynamics for source identification. As illustrated in Fig. 1(b), unlike prior works that do not consider user comments data, we leverage LLMs with prompt engineering to analyze whether each comment indicates the user is a potential rumor source [Hu et al., 2024]. These generated comment analyses replace raw comments as the advisor for downstream localization tasks. To effectively embed the analysis information, we adopt contrastive learning to distinguish rumor-relevant comment patterns from non-rumor comment patterns, and introduce a differentiable GumbelSoftmax masking mechanism to filter out noise and retain discriminative features. In parallel, for propagation modeling, we construct cascade dynamics and user profile features and use a GNN to learn propagation-aware embeddings. Finally, a cross-modal attention mechanism fuses the comment and propagation signals, enabling the model to identify the sources more accurately and robustly. The major contributions are as follows:

• LLM Advisor for Source Localization: Instead of directly using an LLM as predictors for localization, we use LLMs as advisors and analyze whether comments could be potential sources via prompt reasoning. After integrating the comment, dynamics, and profiles mode, LLM provides indirect but more interpretable guidance for the small model and consistently outperforms direct LLM based predictions across multiple datasets.
• Effective Processing for Comment Analysis: We con-
• sider the quality of LLM-generated analysis, leveraging contrastive learning to enhance the representational ability of analysis embeddings, and implementing a differentiable masking technique to filter out invalid features, thereby improving the robustness.
• Complete Propagation Datasets in Real-World Scenarios: We expand propagation datasets containing user profiles, raw comments, and LLM-generated comment analyses. And the ablation study demonstrates the usefulness of these features in datasets.

2 RelatedWork

In the field of rumor propagation analysis, there are two basic and intertwined issues: how rumor propagates through networks and how to locate the source of rumor. Propagation models describe the mechanisms of rumor propagation and they are capable of providing simulated data for source localization. Conversely, source localization methods aim to reverse this process, leveraging observed propagation patterns to locate the sources of rumor. These two concepts exist in a mutually dependent relationship. Therefore, we conduct a comprehensive review of related work in propagation models and source localization methods.

2.1 PropagationModels

The study of information propagation in social networks began with simple epidemiological models such as the Susceptible-Infected (SI) model [Yang et al., 2020; Paluch et al., 2021; Zang et al., 2015], the Susceptible-InfectedRecovered (SIR)model [Zhu and Ying, 2014; Tang et al., 2018] and the Susceptible-Infected-Susceptible (SIS) model. These models provide a basic framework for understanding information spread and generate datasets for source localization tasks. However, they were primarily based on the assumption of homogeneity among individuals, where all individuals in the propagation models were assumed to have the same features such as infection and recovery rates. This homogeneity assumption does not accurately reflect the complexity and diversity observed in real-world scenarios. To overcome this limitation, some heterogeneous diffusion models such as the Heterogeneous SI (HSI) and Heterogeneous SIR (HSIR) were introduced [Karrer and Newman, 2010; Ellison, 2020]. These models simulate a more realistic information propagation process by considering differences between individuals. Additionally, there are some influence models such as the Independent-Cascade (IC) and Linear Threshold (LT) [Goldenberg et al., 2001; Granovetter, 1978] were introduced, which highlight the dynamics of mutual influence of information propagation. However, it is important to note that while these models are valuable tools for propagation simulating, they aren't based on real data and don't consider textual comments. Therefore, their applicability in real-world scenarios is limited.

2.2 Source LocalizationMethods

In real-world scenarios, snapshot data, which captures the state of the network at specific points in time, is easily obtainable. Consequently, there is a significant amount of research on snapshot based source localization. Wang et al. proposed the LPSI method. This method employs a label propagation technique, which is based on source prominence, to locate the source [Wang et al., 2017]. The GCNSI method proposes a GCN based model to locate multiple rumor sources without prior knowledge of the underlying propagation model [Dong et al., 2019]. Furthermore, methods like IVGD [Wang et al., 2022], MCGNN [Shu et al., 2021], and SL VAE [Ling et al., 2022] build dynamic propagation features prior to source inference. Hou et al. utilize an encoder-decoder framework to learn the influence matrix between any two users, which is then employed in the source inference process [Hou et al., 2023]. Furthermore, Huang et al. address the illposed problem of the source localization problem by proposing a two-stage optimization framework, the source localization denoising diffusion model (SL-Diff), which quantifies uncertainty in the propagation process to improve detection accuracy [Huang et al., 2023]. Unlike the above methods, our proposed CRSLL method goes beyond static snapshots and innovatively integrates the consideration of textual comments, leveraging the LLM to guide the localization of rumor sources.

3 Preliminaries

3.1 Propagation Cascades

We obtain K number of available experienced historical propagation cascades Ck=(Vk, Ek,Fk) (1 ≤ k ≤ K) from Twitter or Weibo platforms, where Vk is the participant user set with UID in a social media platform, Ek is the set of participant's directed propagation interaction (including comments or retweets from a user to another), and Fk is the feature set (i.e., user profiles) for each user, including user description, blue verification status, location, registration date, number of posts, fans list, and followings list.

3.2 Historical Relationship Network

Drawing from K historical cascades Ck=(Vk, Ek, Fk) in a social media platform, we construct the historical relationship network G=(V, E, F), which is a union graph by combining structural information of different cascades based on the same UIDs. Sincerely, we pick this idea from the field of diffusion inference [Ramezani et al., 2023], where it is widely used as an intuitive yet effective approach when the underlying network is unknown. Specifically, if different cascades share the same UID, it typically suggest that these cascades are not isolated incidents but rather part of an underlying relationship network, driven by shared interests or topics. By uniting these cascades, a complex historical relationship network emerges where users from various cascades engage with each other either directly or indirectly, based on shared interests or topics. Focusing on this distinct identified area, our research is to locate the sources from a new propagation within G.

3.3 ProblemDefinition

Having constructed the historical relationship network G=(V, E,F)in a social platform, as for a new propagation cascade C = {V,E} at a timestamp of a concerned topic spreading in the area G, we conveniently observe an available snapshot V , which only includes the UID of the participants. And we denote the original rumor sources set as R ⊂ C. The goal of our method is to predict a source set ˆR which can maximize the indicator like ˆR∩R ˆR∪R based on the historical prior knowledge G and a new snapshot V .

4 Method

In this part, the source localization framework incorporating LLM prompt engineering, called CRSLL, is proposed. As shown in Fig. 2, CRSLL innovatively includes five main components: propagation embedding construction, contrastive learning for comment embedding, feature selection with differentiable masking, attention fusion across modalities, and weighted binary classification. In detail, it works as follows: First, it takes traditional snapshot data and textual comments data as inputs, converting them into propagation and comment embeddings, respectively. Then, it utilizes contrastive learning to enhance the representational ability of comment embeddings and conducts feature selection through differentiable masking to improve the quality of high-dimensional comment embeddings, which contain redundant information. Lastly, CRSLL integrates the comment and propagation embeddings by attention mechanism and optimizes source inference with weighted binary classification loss to achieve the rumor source localization task.

4.1 Propagation Embedding Construction

For a new observed propagation snapshot V of the new concerned cascade C, historical relationships in G are used to extract a knowledge based snapshot subgraph S = {ˆ V , ˆE} or adjacency A. To perceive the future potential participants from the historical experience in G, first, we extract additional one-hop relationships of V from the focused area G.

where N G (v) is the neighbor set of user v in the historical network G. Then, a single snapshot V can be mapped onto G and is denoted as S = { ˆ V , ˆE, ˆ Y }. Here, ˆ Y (vj) = 1 indicates that a user vj has participated in the new cascade C of concerned topic, and ˆ Y (vj) = 0, otherwise. After obtaining a snapshot subgraph S, first, in the encoder phase, the propagation embedding including propagation dynamic features and user profiles is constructed to better solve the user-level-based source localization task.

Some unique propagation dynamic features were designed for each user and then combined with user profiles to differentiate each unique user. These features include seven explicit dynamic indicators (denoted as H1-H7), which are constructed to characterize the propagation dynamic features of an individual. Among them, the ratio of participated neighbors and non-participated neighbors of vj are shown in Eq. (3) and Eq. (4), respectively.

What's more, we also consider the normalized number of participated and non-participated neighbors of vj, which are shown in Eq. (5) and Eq. (6), respectively.

Here, features H1-H4 indicate that we are not solely focused on the dynamic ratio of neighbor users. Both the total number of participated and non-participated neighbors emphasize our concern for the precise count of neighbors' states, not just their proportions. For example, considering that a user only has one neighbor and the neighbor participates in the topic, then H1 is a relatively large feature indicator. However, indicator H3 of such a user is small. Therefore, both normalized numerical features and proportional features need to be considered. Moreover, the original state ˆ Y (vj) of each user in S, whether participated (H5) or non-participated (H6), collectively represents the essential property of the individual. Furthermore, we also pay attention to the degree centrality (H7) [Simmie et al., 2013] which can reflect the celebrity effect in social networks. After these seven propagation dynamic features are obtained, the propagation embedding H(vj) ∈ Rd can be obtained by concatenating the normalized user profile features in F(vj).

Since we have the propagation embedding of users in the topology scenarios, an intuitive strategy to aggregate user features is to use the GNN unit. However, in the aggregation process of single-layer GCN, we observe that celebrities with higher degrees, despite having a larger number of interacting neighbors, often contribute with relatively lower feature weights in the aggregation process, impacting both themselves and their neighboring nodes. This tendency highlights an application challenge of the single-layer GCN module in the source localization field, where the influence of highly connected nodes might be diminished in the aggregation process. Therefore, we propose a self-loop attention based GCN to revise the coefficient weight of propagation dynamic features and profiles H(vj) of each user during the process of information aggregation. In this way, celebrities can weaken the average of feature influence by its neighbors, so as to better reflect the real-world level of influence of their characteristics during the aggregation process. As shown in Eq. (7), we add a learnable diagonal matrix to personalize the element value on the diagonal of the matrix Λ ∈ R| ˆ V |∗| ˆ V | . The propagation embedding including propagation dynamics and user profiles can be updated as follows:

whereW is the learnable weights in the module. ˜ A = A+I, where I is an identity matrix. ˜D is the corresponding degree matrix of ˜ A, and σ is an activation function. The diagonal elements of this matrix are controlled by a multi-head attention mechanism. And a single-layer BP neural network ⃗a ∈ R14 is applied for each head of the attention mechanism. To make coefficients easily comparable across all users, we normalize the self-loop attention mechanism.

where WA is the learnable matrix in the attention module, and e(vj) is the self-loop attention coefficient of vj. Furthermore, by diagonalizing the normalized self-loop attention coefficients ϕ(vj) for each user, the matrix Λ can be obtained. Λ allows for the automated and dynamic adjustment of influence coefficients for each node, catering to its unique role in the network.

4.2 Contrastive Learning for Comment Embedding

The propagation embedding H of a propagation cascade C is constructed in the above section. Considering the fact that propagation is user-driven, integrating user profile information can enhance the quality of the embedding. However, comments play a crucial role in the dynamics of information spread, acting as indicators of user engagement and sentiment. Therefore, considering the factor of comments during the source inference process can further improve the detection performance. Considering the powerful expert inference capability of LLMs in analyzing external background knowledge or experience (i.e., historical facts) , which is beyond the capabilities of smaller models like BERT, we opt not to directly use pre-trained BERT for embedding conversion. Instead, we refine the comment modality based on LLM prompt engineering.

Prompt Engineering: Source Analysis Generation

System Prompt: This is a propagation cascade in a social network, involving user IDs and comments. You then need to analyze the reasons whether each comment corresponding to the uid is a source user (the first person to initiate the propagation) or not. Your answer must also be in JSON format.

Context Prompt: All UIDs and comments of a propagation cascade with JSON format.

After converting the source analysis from all comments of a cascade C to the embedding H∗ based on the pre-trained BERT, a contrastive learning mechanism between rumors and non-rumors is deployed to enhance the embedding quality by minimizing the distance between positive similar pairs (i.e., comments from other rumor cascades) and maximizing the distance between negative dissimilar pairs (i.e., comments from non-rumor cascades). The procedure is demonstrated as follows:

where H+ is the positive pair from the same batch but different fromH∗ , and H- is the negative pair from the non-rumor cascades, S* denotes the random sampling operator, which selects an instance from the set, sim(·, ·) is the cosine similarity evaluation measuring the closeness between two vectors. The contrastive revision effectively refines the differentiation of the embeddings between the rumor comment and non-rumor comment, ensuring that the overall representation quantity of H∗ .

4.3 Feature Selection with Differentiable Masking

Considering the high-dimensional embedding of the comment modality, i.e., 768 dimensions of H∗ , may introduce redundant or highly correlated features. This can lead to noise and inefficiencies in subsequent modal fusion processes, ultimately degrading the quality of the final representation. To mitigate this issue, we implement a GumbelSoftmax based feature masking process that enables the differentiable masking of invalid features. More precisely, the binary classification-based decision network Θ determines whether each feature should be masked or retained through a linear transformation that produces logits for each feature of the comment embedding H∗ :

For a differentiable approximation of discrete feature selection, we employ the Gumbel-Softmax technique:

where τ denotes the temperature parameter controlling the softness of the output, and 'hard=True' ensures a one-hot vector output during the forward pass while preserving differentiability during the backward pass through the use of a straight-through gradient estimator.

The decision network's output dictates whether features are masked or retained:

where ⊙ represents element-wise multiplication and w is a decay coefficient less than 1. By introducing Gumbel noise into the logits for feature selection and applying the softmax function, we achieve a continuous, differentiable approximation of the feature decision and masking process for H∗ .

4.4 Attention Fusion Across Modalities

After obtaining the propagation embedding H based on the self-loop attention mechanism and comment embedding H∗ based on contrastive learning and differentiable masking, a cross-attention mechanism is applied for H and H∗ to dynamically adjust the weight of each user.

where Q(H) is the query matrices applied to H, K(H) is the key matrices applied to H, and V(H)is the value matrices applied to H. d is the dimensionality. Furthermore, the two optimized embeddings are concatenated to assemble a more comprehensive representation containing rich propagation characteristics for further binary classification task.

4.5 Weighted Binary Classification

Further, a loss function is required to realize the parameters optimization of CRSLL. Without loss of generality, ˆR[:, 0] is denoted to be the predicted probability for the non-source classification and ˆR[:, 1] to be the probability for the source classification. And the loss L is used to train the complete process of CRSLL based on the |V |-nodes accumulated binary classification task.

5 Experiments

5.1 Experimental Setup

We used three datasets collected from two real-world social platforms, Weibo and Twitter, for source localization, namely Weibo [Ma et al., 2017], Twitter15, and Twitter16 [Liu et al., 2015; Ma et al., 2016]. Furthermore, we have crawled user profile information for each user based on the UID in the propagation cascade, achieving user profile alignment on the social platform [Hou et al., 2024b]. And the comments information in the cascades are integrated. The relevant information of the three datasets is shown in Tab. 1. To demonstrate the validity and novelty of the proposed localization methods, we consider TGASI [Hou et al., 2023], IVGD [Wang et al., 2022], SL VAE [Ling et al., 2022], GCSSI [Dong et al., 2022], MCGNN [Shu et al., 2021], GIN-SD [Cheng et al., 2024b], and HFSD [Cheng et al., 2024a] for comparison. What's more, we also use the common language model, including pre-trained BERT [Devlin et al., 2019], GPT-4o, and GPT-4 [OpenAI, 2022].

And to demonstrate the source prediction performance of all methods rigorously, the widely used standard F1score [Sokolova et al., 2006] is chosen as the evaluation metric [Wang et al., 2023; Hou et al., 2023].

where the Precision is the ratio of the ground-truth sources in the predictions. The Recall is the ratio of successful predictions in the ground-truth sources.

In our experiments, we employ a 10-fold cross-validation strategy to divide the training and test datasets. Further, CRSLL utilizes the training dataset for learning, and then the final result is output by averaging the prediction across each fold in the test dataset. For optimization, the Adam optimizer is used, configured with a learning rate of 0.0005 for all model parameters. In the loss function, both α and β are set to 0.5.

5.2 Overall Experimental Results

The source detection performance based on the real-world dataset is illustrated in Tab. 2. GPT-4 outperforms all other deep learning based comparison methods, exhibiting the best detection performance among all SOTA methods. This underscores the significant potential of LLMs in the field of propagation source localization when many kinds of social context information such as text are available.

Then, compared with the optimal baseline GPT-4, CRSLL exhibits an average improvement of 62.3% in real-world datasets. There are three key reasons for the significant improvement in real-world datasets: (1) The self-loop attentionbased mechanism refines the aggregation strategy for each user, improving the representation quality of propagation dynamics and profile. (2) The contrastive learning from nonrumor comments enhances the representation quality of rumor comments, and the learnable feature masking module further removes the redundant features of high-dimensional comment embedding. (3) Different modalities are dynamically adjusted and more complete information is considered for the source inference process.

5.3 Ablation Study for Datasets

To verify the effectiveness of the collected user profiles, comments, and comment analysis for the localization task, the ablation study for these features is conducted on CRSLL and the lower-cost LLM (i.e., GPT-4o). Due to the limited space, we only present the experiment results in the Twitter15 and Twitter16 datasets. We consider the combinations of topology (T), user profiles (U), comments (C), and comment analysis (A). As can be seen from Tab. 3, both CRSLL and GPT-4o are initially conducted using traditional information that solely includes structural topology for source inference. These models progressively incorporate additional propagation features, including user profiles and comments, to validate the performance on the source detection tasks. It can be observed that the lack of user profiles or comment information leads to a decrease in model performance for each method, underscoring the importance of these features for source localization. More importantly, we have discovered that LLMs are highly sensitive to textual information. The localization performance significantly drops (28.3% in Twitter15 and 32.5% in Twitter16) when comment information is lacking. This suggests that LLMs have limited capability in parsing structural topology information, they have a stronger ability for processing and analyzing textual content in source localization tasks.

5.4 Ablation Study for CSRLF

We further study the influence of designed components of CSRLF on the source detection performance to prove their contributions. The critical modules of CSRLF include the self-loop attention mechanism, contrastive learning, learnable feature masking, modal attention mechanism, and weighted binary classification loss. So five variant models of CSRLF are developed as follows.

• CSRLF S- removes the self-loop attention in Eq. (8).
• CSRLF C- removes the contrastive learning in Eq. (9).
• CSRLF M- removes the learnable feature masking in Eqs. (10)-(12).
• CSRLF A- removes modal attention in Eqs. (13)-(14).
• CSRLF-CE replaces the weighted binary classification loss with a standard cross-entropy loss.

Due to the limited space, we only present the experiment results in the Twitter15 and Twitter16 datasets. As can be seen from Tab. 4, it will lead to a performance decrease no matter removing or replacing any critical modules.

6 Conclusion

Comments in real-world propagation cascades contain rich information (such as background, emotions, etc.), which can provide a new perspective for locating the propagation source. However, models such as BERT have limitations in background knowledge when processing text information in social networks, resulting in limited performance. Therefore, we propose a contrastive rumor source localization via LLM, using the LLM to analyze whether a comment is the source of a rumor. On the one hand, we design contrastive learning to enhance the representational ability of comment analysis, and implement a differentiable masking technique to filter out invalid features. On the other hand, we introduce propagation dynamics and user profile features to construct propagation embeddings to jointly determine the propagation source. Experiments demonstrate the effectiveness of comments and user profiles in localization tasks.

Acknowledgments

This research was supported by the National Natural Science Foundation of China (Nos. U22B2036, 62271411,

62471403), the Technological Innovation Team of Shaanxi Province (No. 2025RS-CXTD-009), the International Cooperation Project of Shaanxi Province (No. 2025GH-YBXM017), the Fundamental Research Funds for the Central Universities (Nos. G2024WD0151, D5000240309), and the XPLORER PRIZE.