Simulating Misinformation Diffusion on Social Media Through CoNVaI: A Textual- and Agent-Based Diffusion Model

1 Introduction

Fake news and misinformation have interfered with fundamental foreign affairs or spread dangerous health-related advice [Cuan-Baltazar et al., 2020]. Detecting the diffusion of this content proves a significant challenge to mitigate its spread [Raza and Ding, 2022]. From current efforts, we notice a general absence of a holistic perspective, studying misinformation through separate components, from a local [Hu et al., 2024] to a global perspective [Caldarelli et al., 2020].

Evaluating detection models or mitigation strategies before implementation is also relevant to these efforts, which rely on understanding information diffusion processes. In these terms, propagation diffusion models are a powerful tool to study information cascades [Lotito et al., 2021]. While not the only approaches, epidemiology-based models are the most widespread [Muhammad and Kasahara, 2024], mainly focused on user behavior without considering shared content. Whenever textual content is included [Kumar et al., 2021; Milli, 2021], it is limited to user profiling, not as part of the communicative device. As such, texts with distinct characteristics (e.g. empty string, emotionally manipulative, or unintelligible) would have the same diffusion, not addressing why fake and real information differ [Vosoughi et al., 2018].

Contrary to these models, we consider content a relevant aspect of information diffusion modeling. In this area of epidemiology-based models, we base our approach on a stateof-the-art agent-based model [Serrano and Iglesias, 2016], that we modify and extend to propose the Textual Contentbased Neutral-Vaccinated-Infected (CoNVaI) model to simulate the diffusion of textual content on social media. We exploit textual characteristics from two perspectives: the novelty of the content; and the diffusion and behavior over time. We empirically validate and compare our approach to the base state-of-the-art agent-based model [Serrano and Iglesias, 2016] that we considered representative of similar epidemiology-based models that ignore textual content.

We also correct another standard limitation in evaluation processes: the lack of realistic evaluation environments. Current approaches rely on synthetic networks [Coates et al., 2021] or real topologies that do not match the information being propagated [Zehmakan et al., 2023], disregarding their impact on engagement [Karnstedt et al., 2011].

With this paper, we make the following three contributions: i) We propose the CoNVaI model 1 , where each agent is characterized based on a unique user and provided with a decision mechanism to determine when and how to disseminate information. ii) We consider the textual content of the information shared, mainly ignored in epidemiology-based models, from different perspectives. iii) We validate our model with data from real scenarios and compare it to a state-of-the-art model, highlighting the importance of modeling the content.

The paper is structured as follows: Section 2 reviews related work. Section 3 presents the fundamentals of CoNVaI. Section 4 covers the components of our model and its behavior. Experimental results are discussed in Section 5, and Section 6 details our findings and future work.

1 The code, supplementary material, and experimentation results are available in https://github.com/Kasdeyael/ABSS CoNVaI

2 RelatedWork

Many research efforts have been dedicated to studying information cascades and predicting their spread [Zhong et al., 2023]. These approaches exploit various characteristics, from network topologies to temporal dynamics or the content of the messages [Liu et al., 2023; Sun et al., 2023; Zhong et al., 2023], with a focus on deep learning. Related to these efforts, we have propagation diffusion models. Besides the potential prediction of the information cascades, the focus veers to modeling the users affected by the information and its diffusion, where the emphasis is placed on their decisionmaking abilities and behaviors [Coates et al., 2021].

Propagation models originate in deterministic compartmental epidemiological models for viral contagion [Kermack and McKendrick, 1927], which use ordinary differential equations to reflect transition rates. Infected individuals are introduced into a group, and the virus spreads to susceptible individuals until they get removed. Individuals are compartmentalized into Susceptible, Infected, or Removed, creating the SIR model. An early application to information propagation considers it an "intellectual epidemic" [Goffman and Newill, 1964], where the virus is the information. Other initial variations, such as the Daley-Kendall model [Daley and Kendall, 1964], adopted elements from information diffusion.

The limitations of epidemic models applied to information diffusion eventually become apparent, such as assuming homogeneous behaviors [Nekovee et al., 2007]. Some models include behaviors from social media, reflecting a belief system and a hesitancy stage [Xia et al., 2015]. The Emotionbased SIS (ESIS) [Wang et al., 2015] introduces the concepts of emotion within the information by categorizing them, making some emotions more effective for propagation. These models still present limitations, such as compartmentalization, to compensate for the lack of individual behavior [Zhang et al., 2018]. Other approaches have been inspired by physical phenomena, such as the Forest Fire Model [Kumar et al., 2021], influenced by a fire spreading in a forest, which also introduces user-based similarity leveraging shared topics. Still, these textual characteristics only model users' relationships without giving relevance to the information itself.

Agent-based simulation has been used to overcome limitations regarding user and topology homogeneity. Epidemiology-based models have been implemented [Serrano and Iglesias, 2016] while differentiating user behavior [Gausen et al., 2021]. Social theories have also been researched through skepticism and gullibility [Tambuscio et al., 2018], or user-based similarity [Li et al., 2019]. The Big Five Personality traits model has been proposed to study the effect of political beliefs [Coates et al., 2021] or to model agentbased trust [Muhammad and Kasahara, 2024]. Once again, the content is ignored in favor of the users, sometimes characterized based on psychological models, without considering specific social media behavior and personality traits may lack correlation [Azucar et al., 2018] or be time-dependent.

3 Preliminaries

This section defines and formalizes the fundamental components of our proposal.

3.1 Information Diffusion Fundamentals

To model information diffusion, we first introduce the formal definitions of the elements that shape it from the standpoint of our agent-based framework.

We adopt and extend the definition of a multi-agent system (MAS) provided by Centeno et al. [2009]. Thus, we define an Agent-based Simulation Diffusion System as follows:

Definition 1. An Agent-based Simulation Diffusion System is a tuple ⟨U, X , A, Φ, x0, φ, t⟩, where:

• U is a set of social agents, where |U| denotes the total number of social agents within the system.
• X is the environmental state space. As an attribute of X , we consider the set of conversations C the social agents create, where |C| denotes the number of conversations.
• A is the action space formed by the 3 actions that agents can perform. In our system, these are: starting a new conversation as anew, replying to a conversation as areply(c), or doing nothing as askip, where c ∈ C.
• Φ : X ×A|U| × X → [0..1] is the system's transition probability distribution, reflecting how X evolves with the agents' actions.
• x0 ∈ X establishes the initial state of the system.
• φ : U ×X ×A→ {0, 1} is the agent's capability function, which determines whether an agent can perform an action at a given environmental state.
• t reflects the time, discretized in steps, which represents the execution time of the system.

We have extended the MAS definition to consider conversations a part of X. We deem the time an explicit part of the system, enabling the agents to perform actions that affect the environment within each time unit.

Following the definition of an agent provided by Centeno et al. [2009], we define a social agent as follows:

Definition 2. A Social Agent is a tuple ⟨S, O, Uin, g, f, per, s0⟩, where:

• S defines the set of internal states of an agent.
• O is the set of observations the agent perceives from its environment. As part of O, the agent has a set of conversations C that they perceive.
• Uin is a subset of social agents such that the agent can read their conversations (their followees).
• g : O×S → S is the transition function of the agent's states.
• f : S → A is the decision function, representing the agent's diffusion model.
• per : C ×X →O is a perception function of the agents, allowing them to assign an observation in an environmental state. For an agent ui, O is composed of conversations Ci ⊆ C such that ui is already part of a conversation or ∃uj ∈ U where ui ∈ Uin(uj) and uj has participated in a conversation.
• s0 is the initial internal state of the agent.

We have extended the definition to reflect the social setting of the agent, introducing Uin, such that Uin(ui) = {uj ∈ U | (uj, ui) ∈ E}, where E is a set of connections between the agents such that E ⊆ {(x, y) | x, y ∈ U 2 and x ̸= y}. We also consider the conversations to be part of the observations of an agent, which affect the perception function. From this definition, social agents can participate in the conversations they perceive. These conversations are defined as follows:

Definition 3. A conversation is a tuple ⟨m0, p,M⟩, where:

• m0 is the initial message that starts a conversation.
• p is the textual content that is being discussed.
• M is a set of messages that reply to the conversation.

A conversation is a diffusion process that encapsulates other messages, allowing agents to maintain discussions about some information 2 p ∈ P . Messages are defined as:

Definition 4. A message is a tuple ⟨ui, tj, sk⟩, where:

• ui is the agent that sent the message.
• tj is the time step at which the message was sent.
• sk is the message's state, reflecting the agent's opinion. It can manifest their agreement or disagreement.

Through their actions, agents create conversations and messages to interact with each other. These interactions are determined by the actions A = {anew, areply(c), askip} they can take. Formally, their behavior involves:

• anew starts a conversation c about a content p with a message m0 and includes c in the conversations of the system C ← C ∪ {c}.
• areply(c) replies to a conversation c with a message mi, such that M ← M ∪ {mi}, where M ∈ c.
• askip does nothing. 3

3.2 Base AB-SIRModel

Most widespread information diffusion simulation models are designed based on epidemiological principles and employ minor to no textual features. Since our contribution focuses on these approaches, we chose a sophisticated MAS proposed by Serrano and Iglesias [2016] for our base model, which leverages epidemiological concepts for information diffusion. Thus, we consider it to represent these models, posing an option to simulate diffusion processes while being complex and validated with empirical data, guaranteeing it reflects reality.

This model, which we refer to as Agent-Based SIR (ABSIR) for this paper, extends the definition of social agent as:

Definition 5. A SIR-agent is a tuple ⟨S, O, Uin, g, f, per, s0⟩, where:

• O, Uin, f, s0 are defined as in Definition 2.
• S has an attribute that labels the agent as one of four states regarding c ∈ C, such that S = {Neu, Inf, V ac,Cu}. A Neutral (Neu) agent is unaware of c. Infected (Inf) and Vaccinated (V ac) agents

2 In this work, we focus only on textual information, leaving multimedia information such as images or videos.

3 It allows us to model and simulate asynchronous simulations.

Algorithm 1 AB-SIR Transition function g

1: ⟨m0, p,Mi⟩ ←ExtractConversation(oj) 2: sj ← si 3: for m= ⟨u, t, sk⟩ inMi do 4: Let U1,U2,U3,U4 ←U(0, 1) be random values 5: if sk = V ac ∧ si = Inf ∧ U1 ≤ PAD then 6: sj ← Cu 7: else if sk = V ac ∧ si = Neu ∧U2 ≤ PAD then 8: sj ← V ac 9: else if sk = Inf ∧ si = Neu then 10: if U3 ≤ PINF then 11: sj ← Inf 12: else if U4 ≤ PMD then 13: sj ← V ac 14: end if 15: end if 16: end for 17: return sj

spread it by agreeing or disagreeing, respectively. A Cured (Cu) agent was Infected stops spreading.

• the transition function g : O × S × PTR → S is also dependent on a set of transition parameters PTR = {PINF , PMD, PAD}, that determine the infection, vaccination and cure rates.
• per : C×X →O limits the conversations they perceive. For each agent ui such that ∃uj ∈ U, ui ∈ Uin(uj), and ∀⟨m0, p,M⟩ ∈ C, they perceive a conversation ⟨m0, p,Mk⟩ where Mk ← {m ∈ M | uj ∈ Mk}.

The g function is detailed in Algorithm 1. Let u1 = ⟨S, O, Uin, g, f, per, s0⟩ represent an AB-SIR agent, and let xj represent the current environmental state, where oj = per(xi) is its current observation space, and si is its current state. Transitions are probabilistic and depend on random values drawn from uniform distributions U and the configurable probabilities PTR. When reading a message sk = Inf from a perceived conversation (ExtractConversation(oj)), a Neutral agent might get infected with a probability PINF , or vaccinated with a probability (1 - PINF ) · PAD. An Infected agent might turn Cured with a probability PAD from reading a message sk = V ac. In contrast, a Neutral agent would turn Vaccinated with the same probability.

After updating the state sj according to the changes in the environment, an agent-SIR would determine its action based on f , which can be defined as follows:

An agent-SIR would start a conversation (anew) or reply to the one that updated its state (areply(c)) if sj ∈ {Inf, V ac}, otherwise they would do nothing (askip). This cycle would repeat until the maximum time limit for the simulation is reached at T, or the users stop spreading the information after a time X. For this last condition, let t denote the current time in the simulation and tl(ui) the last time the agent ui modified si, ∀c ∈ C. The simulation stops when ∀ui ∈ U, t - tl(ui) > X.

4 CoNVaI: Textual Content-Based Neutral-Vaccinated-Infected

To correct the shortcomings of most epidemiology-based models where the textual component is disregarded, we introduce the CoNVaI model following the fundamentals introduced in Section 3.1. We propose an extension of a SIRagent where the f decision and g transition functions rely on two additional components (compared to AB-SIR): i) user profiles with individual characteristics, which are common in state-of-the-art approaches; and ii) the textual content shared through two approaches: the novelty; and the influence and engagement.

4.1 User Characteristics

In terms of the characterization of the users, we determine their influence over others. Previous approaches exploit user similarity measures and metrics based on the user profile [Kumar et al., 2021; Milli, 2021]. To this end, we decided to employ the social-based information from their profiles for our study. We extract three of the most relevant measures: the follower count (followers), the number of listed posts (posts), and the verified status (verified). These values have previously been utilized to characterize users and indicate a user's influence. We also consider one that has not yet been fully explored: the number of followees (followees). When studied with the number of followers, this value has been positively associated with engagement [Peng and Lu, 2024].

Based on the previous information, we set the probability of a user u ∈ U to influence others with the formula:

where FINFL is a configurable parameter to set the relevance of Infl(u) to sway opinions, and Infl(u) is the influence of a user, calculated with the previous four factors as follows:

where:

• sc(X) represents a logarithmic scaling function that allows us to normalize the factor X as follows:

where α sets the middle value of the scaling function. For each metric, we select α based on the mean.

• The weights (0.4; 0.4; and 0.2) have been adjusted to assign higher importance to graded metrics since they provide more information about a user.

4.2 News Novelty

Regarding the shared textual content, we introduce the novelty of a piece of information p ∈ P . Following works where the novelty of a news piece has proven relevant to the spread of information [Photiou et al., 2021; Ulloa et al., 2023], we decided to model its effects using Gaussian distributions. Let ini ∈ T be the initial time when p was sent within a conversation c ∈ C, and t ∈ T be the current time. The novelty of p is obtained as:

where FNOV is a configurable parameter to set the relevance of the novelty Nov(p). We consider two processes for Nov(p). The first one would be for users to be cognizant of the information. It would take some time for any information to reach them. The second one would be the increase of the novelty, as the forgetting and the need to keep up with the information starts acting again. To model the novelty of each content p, we use the following formula:

where:

• tm refers to the time in minutes when the last information related to p was sent.
• Entr represents the Entropy obtained through the Kullback-Leibler Divergence (KLD) to evaluate the information gain. We consider an initial window of the six previous hours before the content was sent for the forgetting mechanism. We explored intervals from two to 24 hours to account for different speeds in information diffusion and time zones, and settled on six to favor the fast-paced news environment. Since KLD measures the probability of an event, we obtain the probability of each word as the relationship between its frequency in a text and the rest of the frequencies for the other words. We use the formula:

where sk = [sk1, sk2, . . . , skn] is the vector of the probabilities of the k words in the content p for a time t, and qk = [qk1, qk2, . . . , qkn] is the vector of the probabilities for t - 1. We perform a previous smoothing step, introducing a small ϵ when probabilities are zero to avoid indeterminations, and adjust the others accordingly. A higher entropy corresponds with new information, while a value of 0 would indicate the information has been seen before. To keep Entr(p) normalized, we have applied Equation 1, with α being the median of the values.

4.3 News Influence and Engagement

For the second textual dimension, we study the influence of news and its engagement over time with two variables. One is the cumulative engagement a piece of information has (News Influence), and the other is the distribution of that engagement over time (Engagement Over Time). Our model considers both, since two pieces of information could have the same total engagement with different temporal dynamics.

For these components, we used two regression models to predict engagement of p ∈ P, represented as ˆ yp = fr(p), where ˆ yp is the prediction and fr(p) represents the function used by the regression model. Since the predicted ˆ yp differs depending on the component, we cover them individually:

• News Influence. For this component, the prediction ˆ yp would be a scalar value ˆ yp ∈ R, as the prediction of the aggregated engagement of p. To use within the decision mechanism of our model, we apply Equation 1 to scale the value, producing:

• Engagement Over Time. This model would predict the timeline of the diffusion per time unit, and it is represented by a one-dimensional vector ˆ yp = (ˆ yp1 , ˆ yp2 , . . . , ˆ ypn ). For the model, we turn the prediction into a distribution as follows:

where ˆ y∗ p is a repeated array from the second element onward of ˆ yp = (ˆ yp1 , . . . , ˆ ypn ), to disregard the initial comment that started the conversation, for w times:

and L is the maximum output size, determined by the 99th percentile of the real diffusion during training. For each index i = 1, . . . ,M, where M = len(ˆ y∗ p ) -w + 1 in Equation 2, we have computed a rolling window and scaled the values, obtaining the one-dimensional vector Prpl(p) to reflect likelihood intervals for sent messages.

For these regressor models, we employ some standard algorithms for the News Influence component: Random Forest, AdaBoost, and Gradient Boosting, with the Scikit-learn library 4 . For the Engagement Over Time, we selected KNeighbors and Decision Trees since they support multiple outputs natively. In terms of the input, we explored a selection of characteristics through a set of tools: MultiAzterTest [Bengoetxea, 2021] and Empath [Fast et al., 2016], to explore syntactic and semantic measures (lexical diversity, readability, polysemic index, verbs in passive voice...), as well as the emotional dimensions, such as joy or anger. We considered three approaches: using only the characteristics from MultiAzterTest and Empath, using the texts directly with two bagof-words representations: frequency-based and TF-IDF, or combining both texts and characteristics. We select the more significant categories through Recursive Feature Elimination (RFE) since they cannot be generalized due to being dataset and social media-dependent [Aldous et al., 2019].

4.4 Runtime Behavior

We conclude the section by defining a ConVaI-agent, which extends Definition 2 as follows:

4 https://scikit-learn.org/

Algorithm 2 CoNVaI Transition function g

1: t ← current time 2: Let U1 ←U(0, 1) 3: if U1 ≤ Pread(ui, t) then 4: ⟨m0, p,Mi⟩ ←ExtractConversation(oj) 5: m= ⟨u, t, sk⟩ ←ExtractMessage(Mi) 6: if UnknownConversation(c) then 7: sj ← ReadSc(m0, p, t, PTX, PINF , PMD, Pusr) 8: else 9: sj ← ReadMs(m, si, p, t, PTX , PAD, POP I , Pusr) 10: end if 11: end if 12: return sj

Definition 6. A CoNVaI-agent is a tuple ⟨S, O, Uin, g, f, per, s0⟩, where:

• O, Uin, f, per, s0 are defined as in Definition 2.
• S considers three labels S = {Neu, Inf, V ac}, similar to the SIR-agent. In AB-SIR, Cu is introduced as a sink state to limit diffusion. A CoNVaI-agent only interacts in response to a message, and would merely stop engaging.
• the transition function g : O×S × t ×PTR ×PTX × Pusr → S is dependent on a set of transition parameters PTR, as well as the parameters from the textual PTX = {Pnov, Prpl, Pnw} and user characteristics Pusr, as well as the time t.

The transition function of a CoNVaI-agent is conditioned by a set of PTR = {PINF , PAD, PMD, PRD, POPI} where we consider a rate to read messages per time unit (PRD) and to share an opinion (POPI ), besides the parameters from ABSIR. We also include the previously defined textual PTX = {Pnov, Prpl, Pnw} and user-based characteristics Pusr.

The transition function g would behave as in Algorithm 2. Let u1 = ⟨S,O,Uin, g, f, per, s0⟩ represent a CoNVaIagent, and let xj represent the current environmental state, where oj = per(xi) is its observation space, and si is its state. After determining whether they can read a message based on a probability Pread(ui, t) = PRD/mr(ui, t) where mr(ui, t) reflect the messages a user ui has read at the current t, they extract a message (ExtractMessage(Mi)) from a conversation they perceive. Their actions depend on whether they know said information (UnknownConversation(c)).

If the information is new, their behavior follows Algorithm 3. They would reply to c based on whether the information is relevant, determined by PTX, and turn Infected or Vaccinated based on the poster's influence (Pusr), and the rates to infect (PINF ) or vaccinate (PMD). If the information is not new, their behavior follows Algorithm 4. In this case, agents determine whether p is relevant based on PTX and Pusr, and their state is conditioned to their previous si and that of the message. If they both agree, agents might feel validated and affirm their posture [Ballara, 2023]. If they disagree, they could change their mind and share the other agent's state or

Algorithm 3 ReadSc

Input: m0 = ⟨uk, t0, sk⟩, p, t, PTX = {Pnov, Prpl, Pnw} PINF , PMD, Pusr 1: sj ←Neu 2: Let U1 ←U(0, 1-Pnov(p, t)) 3: if U1 ≤ Prpl(p)[t] then 4: Replying() 5: Let U2,U3 ←U(0, 1-Pusr(uk) -Pnw(p)) 6: if U2 ≤ PINF then 7: sj ← Inf 8: else if U3 ≤ PMD then 9: sj ← V ac 10: end if 11: end if 12: return sj

trigger a confirmation bias or a "backfire effect," making the agent defend and reinforce their opinion [O'Boyle, 2022].

Finally, the diffusion function f is defined as follows:

Our model considers real-time dynamics, social behavior, and the message's content as part of the decision-making. The diffusion is now conditional on a desire to reply (Replying), contrary to AB-SIR, which assumes agents always engage.

5 Experimental Evaluation

After presenting the model, we proceed with the evaluation to establish its ability to reflect information diffusion processes.

5.1 Selected Dataset

For the empirical evaluation, we aim to recreate the observed scenarios of a news piece's diffusion on social media, so we prioritized realistic scenarios. After evaluating the most commonly used datasets, we selected PHEME-9 [Zubiaga et al., 2016]. As far as we know, this is the only readily available dataset that contains the information to recreate those scenarios: textual data of the shared news, temporal information, user characteristics and their topology, and the stance of the messages, which we use to evaluate diffusion. We avoided synthetic data, such as the network topology, because it would add noise or biases.

PHEME-9 centers around nine 2014/2015 events, with 66k tweets and retweets organized into 297 threads for 55k users, containing their ego-networks and the HTML of external links. We employed the Wayback Machine to extract unrecovered articles. We decided on an 80:20 ratio for the train and test partitions for the evaluation, choosing the Ottawa shooting event for testing. We employ the train partition to tune the News Influence and Engagement Over Time components. Since the information these components exploit for this tuning differs from the one for the evaluation, data leakage is not a concern.

Algorithm 4 ReadMs

Input: m = ⟨uk, ti, sk⟩, si, p, t, PTX = {Pnov, Prpl, Pnw} PAD, POPI , Pusr 1: sj ← si 2: Let U1 ←U(0, 1-Pnov(p, t) -Pusr(uk)) 3: if U1 ≤ Prpl(p)[t] then 4: Let U2,U3,U4 ←U(0, 1) 5: if si = sk ∧ U2 ≤ POPI then 6: Replying() 7: sj ← si 8: else if si ̸= sk then 9: if U3 ≤ PAD then 10: Replying() 11: sj ← sk 12: else if U4 ≤ POPI then 13: Replying() 14: sj ← si 15: end if 16: end if 17: end if 18: return sj

5.2 Metrics

For the evaluation, we follow the methodology used to validate AB-SIR, where the accumulated real diffusion is compared to the model's output [Serrano and Iglesias, 2016]. The message's stance is used to establish the user's state. We assume a retweet or comment supporting reflects their Infected state. If users are denying, they would be Vaccinated. We also assume underspecified comments are supporting.

We selected RMSE to express our results. Based on the stances of the users, we have both Vaccinated and Infected states. We generate two metrics, RMSEdeb and RMSEspr, for the users denying and spreading information, respectively. We use the formula RMSE =

• T reflects the total number of time units per test instance, which we set as one-minute intervals. T is set based on the lack of engagement after 60 minutes.
• ystatei is the number of users in the dataset with state ∈ {V ac, Inf} for RMSEdeb or RMSEspr, respectively.
• ˆ ystatei is the number of agents in the simulation, with state ∈ {V ac, Inf}, at each given time.

We linearly combine and normalize the metrics as follows:

where statemin and statemax are the minimum and maximum diffusion values, respectively. Similarly, we normalize RMSEspr and RMSEdeb with their cumulative values.

5.3 Experimental Setup

CoNVaI and AB-SIR were implemented with Repast Simphony 5 . It is extensible, scalable, open source, and allows for complex behavior. For the components of CoNVaI, we used the training partition with a validation set to tune the regressor models and choose the best-performing ones. We observed differences depending on the text sources when applying RFE. Most features focus on the tweets for the News Influence component, while it is evenly split for the Engagement Over Time. For the first component, only 20% relate to the emotional aspect, while that percentage goes over 60% for the second one. Some of the common characteristics, such as readability, A1-C1 incidence, or passive voice, relate to the linguistic differences between real and fake news [Kasseropoulos and Tjortjis, 2021; Zhou et al., 2020; Manikonda et al., 2022], indicating that there are similarities despite the differences in intent.

5 https://repast.github.io/

Regarding the test set, we start by setting the simulator to create the agents depending on the model and the connections based on the ego-networks from the dataset. The user who started the conversation introduces the content in t1 = 0. In each consecutive discretized step, agents that receive the information will choose how to act. The combinations for the adjustable parameters for each model are covered in the supplementary material and repeated for each test instance. We report the results for the best-performing combinations.

5.4 Experimental Results and Discussion

Table 1 covers the results for the test set. We include the average, standard deviation (SD), and median of NRMSEst and the normalized versions of NRMSEspr and NRMSEdeb. CoNVaI is more accurate at describing reality, where most differences are in lower orders of magnitude. The most striking variations are observed for NRMSEdeb. After analyzing the debunkers in the test set, we observed that the highest value for a test instance was under ten. These errors are smoothed in CoNVaI for NRMSEst, indicating that debunkers are not a big concern in the overall diffusion.

We observed some outliers when comparing the SD and averages. Selecting the 90th percentile of NRMSEst shows a significant decrease for CoNVaI and AB-SIR, although the errors remain orders of magnitude over CoNVaI. To determine the reason behind the performance of AB-SIR, we further adjusted the parameters for a representative set of test instances.

From these additional experiments, we observed some interesting trends. Firstly, run time increased by several hours per run. Secondly, although the simulation was closer to the real diffusion, information still reached most of the network. It showed a linear growth, only modifying the curve gradient, affecting the whole population without reflecting real diffusion: fast in the beginning, and slowly losing momentum. Agent-SIR behavior determines that, to control diffusion, we need Infected agents to turn Cured. It requires Vaccinated users, which are uncommon and rarely engage [Zubiaga et al., 2016; Vosoughi et al., 2018]. Even limiting infection rates, Vaccinated users will continue spreading. This would be consistent with epidemiological models since the population is expected to recover, but not with information diffusion.

From previous experiments with AB-SIR, we considered how it was validated through empirical evaluation since information would affect the entire network. We theorize that a part of this could be due to the population and the evaluation framework. Typically, simulations are compared to the diffusion in a real dataset, which means it is being compared to actual engagement. We might see the distinction between the possible states (Infected or Vaccinated), but those users have engaged somehow. Our study has considered other nonaffected users by employing ego-networks instead of synthetic [Serrano and Iglesias, 2016]. These evaluation scenarios would be biased by disregarding other users who decided not to engage. Another possible cause is how results are measured, comparing only spreaders [Gausen et al., 2021], ignoring the rest. Our experiments show that this benefits the ABSIR model without accurately reflecting diffusion. The statebased metric is also bounded by the population, limiting the evaluation scenario. Despite the popularity of epidemiological models, these effects have not been reported to the best of our knowledge. However, to study users' behavior, we need realistic scenarios with interacting and non-interacting users.

6 Conclusions and FutureWork

We propose CoNVaI in the scope of epidemiology-based models, which are the most widespread in simulating information diffusion. Our contribution shows how information diffusion can be approached holistically. From a local perspective, we incorporate the textual content of the messages, which had been mainly ignored. From a global perspective, agents interact with each other based on their ego-networks.

From our experiments, we determined that incorporating textual content positively affected our simulations. AB-SIR, a representative model, overestimates how many users the information reaches. It highlights the need for models focused on online diffusion with realistic evaluation scenarios. We have also shown how the evaluation is bounded by the total users and does not reflect engagement. This simplification could be problematic when applying social science concepts to user behaviors, such as gullible or skeptical users.

In future work, we plan to study unbounded evaluation methodologies centered around messages. We also plan to enhance CoNVaI by considering how often users continue conversing and incorporating in-depth user profiling.

Acknowledgements

This work has been partially funded by the Spanish Research Agency (Agencia Estatal de Investigaci´on), through the DeepInfo project PID2021-127777OB-C22 (MCIU/AEI/FEDER, UE) and the HOLISTIC ANALYSIS OF ORGANISED MISINFORMATION ACTIVITY IN SOCIAL NETWORKS project (PCI2022-135026-2).

Raquel Rodr´ıguez-Garc´ıa is supported by the UNED's FPI doctoral grant, Spain.