An LLM-enhanced Agent-based Simulation Tool for Information Propagation

1 Introduction

Information propagation describes how pieces of information, such as news, ideas, opinions, and advertisements, spread through social networks [Gomez-Rodriguez et al., 2013; Wu et al., 2019]. Simulating this phenomenon aids in understanding how information diffuses through these networks. Influence diffusion models [Kempe et al., 2003], extensively used to simulate information propagation, follow a 'contagion' pattern based on the network structure [GomezRodriguez et al., 2013], where influence spreads from one node to another through connecting edges. Agent-based modeling (ABM) [Macal, 2016] has been applied to simulate influence diffusion. By leveraging the advantages of ABM [Li et al., 2023; Li et al., 2016; Hu et al., 2019], the diffusion process is driven and captured by the behaviors of individual agents, allowing for a more accurate representation of the complex and dynamic nature of information diffusion.

Most existing influence diffusion models are probabilitybased, which presents several challenges. Firstly, the content of information plays a critical role in individuals' decisionmaking behaviors, but is often ignored by most probabilitybased models. Although some topic-aware influence diffusion models [Chen et al., 2015] consider how tagged content impacts these probabilistic models, they are typically constrained to predefined topics [Chen et al., 2023]. Secondly, probability-based models, including agent-based models, neglect individuals' ability to alter information [Wang et al., 2023]. Specifically, the content of a piece of received information can be altered according to an individual's prior knowledge or interests.

In recent years, the emergence of Large Language Models (LLMs) has shed light on filling some of the above gaps [Gao et al., 2023]. In this paper, we present an LLM-enhanced agent-based influence diffusion model, named LAID, along with a software simulation tool, named LAIDSim 12 for information propagation simulations. LAID is developed based on the conventional Independent Cascade model [Kempe et al., 2003], while leveraging ABM [Li et al., 2023] to capture the personal traits and individual behaviors of social network users. In LAID, users are represented as autonomous agents with distinctive profiles and complex social behaviors. Information content is considered a crucial factor influencing user agents' decision-making behaviors, specifically whether to be influenced by received messages. We particularly emphasize information alteration [Wang et al., 2023] as a key feature in the propagation process, referring to the ability of user agents to modify the content of initially spread information during peer-to-peer propagation. LLMs are utilized to realize this feature, serving as the 'brain' of user agents. The LLM generates the content of an influence message based on the user agent's profile and the content of its received message, facilitating further propagation to its neighbors.

LAIDSim is implemented based on the ABM framework Melodie [Yu and Hou, 2023]. This tool enables end-users to simulate information propagation with self-defined scenarios, visualize the propagation process over time, and generate analytical results at a global level. To demonstrate the capabilities of LAIDSim, we designed and conducted four scenarios within two experiments. These experiments showcase the advantages of LAID in representing language-level behaviors of social network users and highlight the ability of LAIDSim to simulate information propagation across various application settings.

2 The LLM-enhanced Agent-based Influence Diffusion Model

2.1 Preliminaries

In LAID, a social network is modelled as is a directed graph G = (V,E,M) with a set of user agents V =

1 https://youtu.be/AaxDqK96RZE

2 https://github.com/shaunahu/LAIDSim

{v1, v2, ..., vn}, a set of edges E = {eij|1≤i≤n,1≤j≤n,i̸=j}, and a set of influence messages M = {m1, m2, ..., mx}.

A use agent vi = (ri, Γi) represents a social network user with a profile ri. Γi = Γin i ∪ Γout i is the neighbor set of vi which refers to a collection of user agents who are connected with vi. Notably, Γin i = {vj|eji ∈ E} is the in-neighbors of vi, so that vi receives influence messages from these neighbors. While Γout i = {vj|eij ∈ E} is the out-neighbors of vi, so that vi sends influence messages to these users. An influence message mx = (cx, vi) is a piece of textual information spread from a user agent vi to its neighbors, where cx is the content of the message. Since language representation is not the focus of our study, we used BERT [Devlin et al., 2018], a pre-trained model, to learn user profile embedding ri of a user agent vi based on its textual profile ri, and content embedding ck of an influence message mk based on its content ck .

2.2 Overall Framework

Figure 1 shows the overall framework of LAID with an example. The yellow square represents a LLM-enhanced user agent. In the social network, each user agent has its unique user profile, a repository to store received influence messages, and access to an LLM. The diffusion process starts from a user agent with an initial influence message, spreading to its neighbors through network topology. Once an agent receives an influence message from its neighbor, it decides whether to be influenced by the message or not based on the similarities between itself and the sender of the message, as well as the content of the message. If it was influenced, it retrieves its personal profile and the received message, then sends a request to an LLM, asking it to generate a new message for dissemination. The diffusion process continuous until there is no more user agents can be influenced.

2.3 Initialisation

We adopted GPT-3.5-turbo to generate user profiles with the following rules: (1) a user profile is a summary of a social network user within 50 words; (2) the ages of these users follow a normalised Gaussian distribution; (3) the genders of these users are evenly distributed, half male and half female. At the beginning of the influence diffusion, k user agents are randomly selected to spread an influence message ms to their neighbors.

2.4 Influence Diffusion

Influence diffusion is developed based on the conventional IC model. In the IC model, each user agent has two possible states: active and inactive. An inactive user agent has a single chance to be activated by its active neighbors with an influence probability. The diffusion process stops if there were no more user agents could be influenced.

The influence probability pkij is the probability for a user agent vi to accept an influence message mk from its neighbor vj. This is calculated by the consideration of the content of the message mk and the relationship between two user agents:

In Equation 1, sim(mk, vi) is the cosine similarity between the content embedding ck of mk and the embedding ri of vi. It suggests how the content ck of an influence message mk matches a user agent vi:

Similarly, the similarity between two user agents vi and vj is measured by the cosine similarity between the user profile embeddings ri and rj, where ri and rj are the profiles of vi and vj :

In the influence diffusion process, we identify content influence as a critical factor in individuals accepting an influential message, denoted as sim(mk, vi). The influence parameter α in the proposed diffusion model is a trade-off between these two factors. A higher α indicates that the user agent is more likely to be influenced by social influence rather than content, while a lower α suggests that the user agent is more sensitive to the content of the information but pays little attention to its neighbors' decisions.

Once a user agent is influenced, it accepts the spreading influence message mk and further propagates to its neighbors. Specifically, LLM model is adopted for information alteration. That is, a user agent can alter the content of an existing influence message mk that it received from any of its in-neighbors vj ∈ Γin i , adding or changing words based on its profiles, and further diffuses the altered influence message m′k to its out-neighbors Γout i .

2.5 Information Alteration

GPT-3.5 Turbo is used in this stage for user agents to generate altered influence messages based on their profiles and the influence messages they have accepted. To evaluate the changes between two influence messages, we adopted alteration degree [Wang et al., 2023] to measure the extent of information alteration by a user agent.

3 Experiments

3.1 Experimental Settings

Parameters: We consider seed size k and influence parameter α as two parameters of our proposed model that can be adjusted from the front-end. The seed size k represents the number of initially active user agents. A higher k suggests a quicker diffusion but also entails higher costs. As explained in Subsection 2.4, the influence parameter α potentially impacts the influence probability by controlling the trade-off between social influence and content influence.

Scenarios: We designed four scenarios with varying initial influence message content and application settings for the proposed model to simulate. The initial influence message is also generated by the GPT-3.5-Turbo model. Table 1 lists these scenarios.

Initialization Setup: At the beginning of each scenario simulation, a synthetic network based on Erdos-Renyi model [Erd˝os et al., 2012] is randomly constructed with a size of 20 nodes, and a probability of 0.1 for these nodes to connect to each other.

3.2 Experiment 1: Influence Coverage Analysis

This experiment analyses the cumulative number of user agents in two states: active and inactive over a particular time period, i.e., 10 time steps. Figure 2 shows the simulation results. The slight differences of the numbers of active and inactive agents across the four scenarios suggest that user agents possess the capability to respond to information with different contents. In all four scenarios, the diffusion process adheres to a contagion pattern, indicating LAID's capacity to effectively simulate information propagation.

3.3 Experiment 2: Information Alteration Analysis

Figure 3 shows the simulation results of four scenarios monitored from time step 1. It can be seen that all four scenarios have been slightly altered at the first time step compared to the initial influence message. S4 shows the most significant alteration. This is because the initial influence message in S4

is an 'open' question. The alteration degree scenarios s2 to s4 decreases, implying that user agents generate information with a similar pattern to their received influence messages. The degree of alteration converges after a few time steps, indicating that the diffusion process stops, and no more user agents can be influenced to generate new influence messages.

4 Conclusion

In this paper, we presented an LLM-enhanced agent-based simulation tool, LAIDSim, for simulating information propagation in social networks, based on our proposed diffusion model LAID. We showcased the interpretability of the proposed model across four scenarios by demonstrating language-level interaction and inference of user agents. In the future, we plan to extend our work by improving the user agent model with more comprehensive user behaviors.

Contribution Statement

The first two authors contributed equally to this work and should be considered as joint first authors.