Generating Grounded Responses to Counter Misinformation via Learning Efficient Fine-Grained Critiques

1 Introduction

Misinformation spreads faster and farther than truthful information [Vosoughi et al., 2018], posing significant risks to public health, trust, and society. While professional fact-checkers and journalists provide reliable veracity assessments, their efforts are labor-intensive and often focus only on popular claims. Automated fact-checking [Guo et al., 2022; Wang and Shu, 2023; Zeng and Gao, 2024] have been proposed to identify misinformation at scale, but their focus has primarily been on veracity prediction rather than generating direct counter-responses in real-time. The widespread propagation of misinformation, especially on social media platforms, calls for the automated generation of factually grounded counter-responses in real-time and at scale.

∗Corresponding author.

The advancement of Large Language models (LLMs) offers the opportunity for automated generation of counterresponses at scale. Leveraging LLMs, [He et al., 2023] proposed to employ LLMs for counter-response generation, but their focus is on the language quality - relevant, fluent, polite and of refutation attitude. However, the critical issue of information factuality, or hallucination, remains largely overlooked. More broadly, studies on the mitigation of LLM hallucination are reported in the literature. RetrievalAugmented Generation (RAG) approaches augment the input context for LLM generation with retrieved knowledge [Lewis et al., 2020; Yu et al., 2023; Shi et al., 2023] to confine LLM generation and reduce hallucination. Originating from Chain of Thought (CoT) reasoning [Kojima et al., 2022; Zhang et al., 2023], various prompting strategies are proposed to enhance LLM reasoning capabilities to reduce hallucination. Recently, LLM self-feedback strategy [Madaan et al., 2024; Aky¨urek et al., 2023] is proposed to refine LLM generation, demonstrating strong performance to reduce hallucination. None of these studies directly address the issue of information factuality in LLM outputs. Importantly, all these approaches require additional LLM runs to generate reasoning steps or feedback, which is computationally expensive.

Recent research [Barr´on-Cedeno et al., 2020] shows that fact-check-worthy claims typically fall into two categories: potentially harmful claims attacking individuals, organizations, nations, or social groups and time-sensitive claims concerning prominent people, organizations, countries and events. Fact-checking such claims requires precise comparisons of key elements - such as numbers, entities, and topics - against factual evidence. For instance, as shown in Figure 1, a false claim contains key elements such as numbers (e.g. 122,000, Eighty percent), entities (e.g. Australia) and topics (e.g. the problem of homelessness) that must be verified. Here an element refers to an individual piece of information that can be assessed for qualities such as factuality, correctness or relevance.

In this paper, we propose MisMitiFact - Misinformation Mitigation grounded in Facts, an efficient framework for generating fact-grounded counter-responses to misinformation at scale. Unlike existing approaches that rely on expensive LLM inference for general feedback, our method introduces lightweight, fine-grained critique models that directly pinpoint factual errors about elements such as numbers, named entities, and topics. In our framework, critique models for factual errors about specific elements in the initial generation are trained using data within the readily available factchecking articles from fact-checking sites. Figure 1 shows how MisMitiFact works. The input x consists of the misinformation claim and the evidence. We collect an initial counterresponse, y, from the LLM, then obtain feedback from three critique models, call G = {cnum G , cne G , ctopic G }, which respectively evaluate the counter-response on numbers (cnum G ), entities (cne G ), and topics (ctopic G ). The LLM then takes the initial counter-response and the critiques into consideration and generates a refined counter-response, ˆ y.

To our best knowledge, we are the first to introduce a task for counter-response generation grounded in factual evidence. We further propose a simple critique feedback approach for LLMs to refine generation and ensure information factuality. We train lightweight, efficient critique models targeting element-wise information correctness, the models are trained on data sourced from readily available fact-checking sites and do not require human-annotated critiques or counterresponses. Experiments on two real-world datasets show that our system MisMitFact can generate grounded counterresponses of quality comparable to the state-of-the-art LLM self-feedback approach while using significantly smaller critique models. Additionally, it achieves ∼5x increase in critique generation throughput, making our system highly suitable for cost-effective, large-scale misinformation mitigation.

2 RelatedWork

Related work comes from three lines of research.

2.1 MisinformationMitigation

A line of research on misinformation mitigation focuses on limiting the dissemination of false information while promoting corrective or clarifying truthful information [Farajtabar et al., 2017; Saxena et al., 2020; Goindani and Neville, 2020; Xu et al., 2022; Xu et al., 2024]. These studies typically assume that counter responses containing truthful information are given. Another line of research focuses on automated factchecking - predicting the veracity of claims based on given claims and evidence [Guo et al., 2022; Wang and Shu, 2023; Zeng and Gao, 2024]. However, their focus is not on generating counter-responses to directly refute false claims.

Some recent studies [He et al., 2023] focus on counterresponse generation. Their system MisinfoCorrect employs LLMs to generate counter-responses to COVID-19 vaccine misinformation. While their work models politeness, refutation attitude, and fluency within responses, it does not address the factual correctness of counter-responses, which is a critical issue and the focus of our research.

2.2 Mitigation of LLMHallucination

A line of research focuses on improving the generation process and broadly includes two classes of studies. One class is based on RAG [Lewis et al., 2020; Yu et al., 2023; Shi et al., 2023], which aims to reduce LLM hallucinations by augmenting the LLM input context with retrieved external knowledge base. Another class leverages CoT prompting [Kojima et al., 2022; Zhang et al., 2023], which improves LLM reasoning capabilities to reduce hallucinations. While these methods enhance the overall quality of LLM generation, they do not directly address the information factuality.

Another line of research provides feedback to refine LLM generation and improve its quality [Schick et al., 2022; Aky¨urek et al., 2023; Lee et al., 2023; Yu et al., 2023; Madaan et al., 2024]. Early research using textual critique feedback prompts to improve LLM generation, where a separate critique model for generating critiques [Schick et al.,

2022; Aky¨urek et al., 2023] is trained from human-written critiques as training data. A recent study called SELFREFINE [Madaan et al., 2024] repeatedly calls an LLM to generate feedback on its own generation. This approach not only generates general feedback but also incurs repeated LLM calls, which are computationally expensive. In contrast, our approach introduces lightweight, fine-grained critique models that generate simple critique prompts targeting factual errors in key elements (e.g., numbers, entities, and topics). Our lightweight critique models generate simple, element-wise critiques that incur significantly less computation cost and our MisMitiFact achieves comparable textual quality to LLM-based self-feedback.

2.3 Factual Error Correction

Our general idea of identifying and correcting errors in texts is somewhat related to the NLP task of Factual Error Correction (FEC) for abstractive text summarization [Thorne and Vlachos, 2021; Lee et al., 2022]. FEC focuses on checking the factual correctness of summaries given source documents, a fundamentally different and much simpler problem setting that does not involve claims and their counter-responses.

3 ProblemDefinition

Our task is to generate counter-response text y to debunk a misinformation claim xc, given factual evidence text containing facts related to the claim, xe. Counter-response y is desired to have the following properties:

• Faithful to the evidence. Denote all statements in counter-response y as S = {s1, · · · , sN} and all verifiable statements in y as V = {v1, · · · , vN}. The response y is faithful to the evidence iff ∀si ∈ y, si |= xe and S - V = ∅. For a response to hold this property, all the statements in the response must be verifiable and be verified true (|=) according to xe.
• Refute the misinformation. For a successful refutation to the misinformation, the counter-response should satisfy the property of faithful to the evidence and also satisfy the following property: Define all false statements in the misinformation claim xc as F = {f1, · · · , fN}. Response y refutes the misinformation iff ∀fi ∈ xc, fi |̸= y. For a response to hold this property, all the false statements in the claim must be clarified as false (|̸=) by y.

For counter-responses failing to satisfy the property of faithful to the evidence, the response may contain non-factual information and is not acceptable when debunking misinformation. For counter-responses failing to satisfy the property of refute the misinformation, the responses will not clarify the false claim and thus never counter the misinformation. We aim for a framework that leverages LLMs to automatically generate counter-responses to refute false claims while correcting factual errors contradicting facts in evidence.

4 Methodology

We propose a prompt-learning approach to counter-response generation, focusing on critique feedback to prompt LLMs to correct errors within the initial generation. Inspired by observations that fact-check-worthy claims often contain false information about elements such as numbers, named entities and topics, we propose to train small critique models to pinpoint errors and generate critique feedback for elements (See Figure 1 for an example). We also aim not to incur extra human annotations to train the critique models.

4.1 MisMitiFact

Figure 2 shows our MisMitiFact framework, which can be summarized as follows: (1) A large language model with frozen weights, LLMgen, is used to generate an initial counter-response to a misinformation claim. (2) The initial output of LLMgen is critiqued by a set of three trained small critique models, LLMcritique. (3) The text output of the critics, along with the initial input to LLMgen, are used to prompt LLMgen to generate a refined output. In contrast to the existing critique-style feedback approach of [Aky¨urek et al., 2023], which relies on expensive human-written critiques, our approach trains lightweight, critique models on specific elements like numbers, named entities and topics. These models are trained using training data automatically generated from readily available fact-checking articles (Details in Section 4.2).

MisMitiFact consists of two phases: critique model training and counter-response generation. In the critique model training phase, given input x (consists of claim xc and evidence xe) and explanation yE, we generate training data comprising two types of instances to train LLMcritique: The factual instances, where the input x and the original explanation yE is paired with a positive indicator like "The entities are correct" to train the model to confirm factual accuracy. The counter-factual instances, where the factual elements number, name entity and topic in yE are replaced with false number, false named entity and off-topic, generating ynum E , yne E , and ytopic E . Each counter-factual instance is paired with input x and critiques based on templates (cnum E , cne E , and ctopic E ), indicating the introduced factual errors.

This generated structured training data ensures the LLMcritique models learn both to validate factually explanations and to detect counterfactual explanations using critiques. For example, with a false claim xc ="122,000 people sleeping rough." and collected evidence xe ="Official figures show that 122,494 people were experiencing homelessness, only 7,636 of those people were sleeping rough.", a journalist might write an explanation yE as "only 7,636 people were sleeping rough.". We will replace the number 7,636 in yE with other numbers provided in evidence xe, that is 122,494. Thus, the ynum E will be "only 122,494 people were sleeping rough." with a corresponding cnum E as "122,494 is not correct, the correct number is 7,636". The same idea also applies to (yne E , cne E ) and (ytopic E , ctopic E ).

At the inference stage, all models, including the trained critique model LLMcritique and the generation model LLMgen, are frozen. When generating the critique for the initially generated counter-response, the critique is given by all the critique models in LLMcritique as cnum G , cne G and ctopic G respectively. We concatenate all the critiques generated by LLMcritique as call G and feed into the prompt template of LLMgen. Prompt templates of LLMgen are at https://github.com/xxfwin/MisMitiFact.

4.2 Training Data Generation for Critique Models

We next describe how to automatically generate training data for critique models from readily available fact-checking articles. Fact-checking articles usually include four components: (1) a claim, (2) human-curated evidence gathered from credible sources to fact-check the claim, (3) a verdict ("true", "false" or "mixed"), and (4) human explanation to support the fact-check verdict for the claim. As we focus on the false claims, we only include the false claims.

To generate training data for critique models, we focus on using the original claim xc, the human-curated evidence xe and the explanation yE to generate two training components: a) factual instances: We pair the explanation yE with its corresponding claim xc and evidence xe, appending affirmative critiques (e.g., "The numbers/entities are correct" or "The explanation is on the topic of the claim"). These instances train the model to confirm factual accuracy when no factual errors exist. b) counter-factual instances: The yE will be modified to generate three types of false explanations - numerically replaced ynum E , named-entity replaced yne E , and off-topic ytopic E . For ynum E , we systematically replace number qy in an explanation yE with a different number qx from the evidence text xe and then create critique cnum E following template "qx is not correct, the correct number is qy ". Similarly, for yne E , we replace named entity qy in an explanation yE with a different named entity qx from the evidence text xe and then create critique cne E following template "qx is not correct, the correct text is qy ". For ytopic E , we prompt Google Gemini [Team et al., 2023] to generate off-topic messages based on the given evidence. The instruction is to generate off-topic messages that are different from the explanation but still related to the evidence. The training data for the topic critique model is generated by prompting gemini-pro using the prompt in Table 1. For each claim xc, we generate a maximum of 20 instances of numerically replaced and named-entity replaced explanations and generate 3 instances of off-topic explanations. Here, we utilize spaCy [Honnibal and Montani, 2017] to perform named entity recognition to extract all named entities or numbers.

Your task is to rewrite the explanation so that it is offtopic to the provided claim but still on the topic of the provided facts. The output of your response in a single plain json list, please return the rewritten explanation as "rewritten explanation" and reason as "reason" in the json list. Please also try to keep a similar length of the provided claim.

Here is the template of the reason: The claim is about <on topic part of claim>, but the explanation is not correct because it is about <off topic part of explanation>.

This is the claim: {claim}

This is the explanation: {explanation}

This is the facts: {evidence}

Table 1: Prompt for generating training data for topic critique model

5 Experiments

Experiments are conducted on a cluster where each node has 32 cores, 128G memory and is equipped with an NVIDIA Geforce RTX 3090. All deep neural networks are implemented using Transformers [Wolf et al., 2019] under the support of PyTorch [Paszke et al., 2019].

5.1 MisMitiFact and Baselines

For MisMitiFact implementation, we fine-tuned the T5-large [Raffel et al., 2020] for the critique models (LLMcitiqiue in Fig. 2), with a learning rate of 1e-5, 5

epochs, temperature of 1.0 and output length of 5-30 tokens. The overall training time of the three critique models is around 30 hours. Generate critiques are set to maximum 150 tokens. For the generation model of MisMitiFact (LLMgen in Fig. 2) we experimented with two popular open-source LLMs Vicuna-1.5 [Zheng et al., 2024] and LLaMA-2 [Touvron et al., 2023].

We compare MisMitiFact against 5 baseline models. For a fair comparison, we also use the same Vicuna and LLaMA models as the generation models for all baselines.

• MisinfoCorrect [He et al., 2023] is a recent model for misinformation counter-response generation model.
• MisinfoCorrect w/ Evidence is an extension of MisinfoCorrect, including evidence in the input context.
• SELF-REFINE [Madaan et al., 2024] is a recent selffeedback approach that uses an LLM for generation and calls the LLM again to generate feedback and refine its generation. For a fair comparison, we include evidence in the input context.
• Chain-of-Thought (CoT) is a popular prompt technique that aims to reduce hallucination in LLMs. We applied zero-shot CoT [Kojima et al., 2022] to instruct the LLMs to solve the task step by step.
• Plug-and-plug REtrieval FEEDback (REFEED) [Yu et al., 2023] is a RAG-based model, where the retrieved documents are fed back to the initially generated content, allowing the generation model to refine the content. For our task, the evidence is used as the retrieved documents.

Note that we have not included as baseline models for critic-style feedback to improve LLM generation discussed in Section 2, as they all require gold standard critiques, which are not available in our problem setting.

5.2 Experiment Setup

Datasets: Experiments were conducted on two datasets:

• PUBHEALTH [Kotonya and Toni, 2020] is a large factchecking dataset containing 11.8K claims on public health topics. For each claim, there is evidence curated by journalists from credible sources, veracity labels (true, false, unproven, mixture) and counter-responses (called "explanations") crafted by journalists. For our experiments, we use the 2153 claims with the "false" or "mixture" label.
• COVID-19 [He et al., 2023] includes false claims about the COVID-19 vaccine from Twitter (now X) and gold standard counter-responses by crowdsourcing. For our experiments, we crawled the source claims based on

tweet ID 1 and got 307 false claims. To construct relevant evidence about the claims, we crawled the "Facts about COVID-19 Vaccines" web page from the CDC website 2 . The same evidence on the topic of the COVID-19 vaccine is used for all claims.

We analysed the contents of the claim, evidence and explanation text to understand the characteristic differences between the two datasets. We found that the datasets share a similar count of numbers and entities in the claims, but the counts in the explanations differ a lot. In COVID-19, the explanations contain nearly no numbers. This might be because the majority of false claims can be explained without quoting numbers. From the contents of the evidence, it can be seen that the evidence in PUBHEALTH contains more numbers and named entities, meaning that it is effectively more information-dense than the COVID-19 Vaccine dataset. Table 2 includes statistics of the two datasets used. For all models, we use 80% of claims for training, 10% for development and 10% for testing.

Evaluation Metrics: To evaluate the quality of counterresponses, we employed widely used LLM-based automatic metrics, which are scalable and have demonstrated humancomparable performance [Liu et al., 2023; Min et al., 2023]. We adapted G-EVAL [Liu et al., 2023] based on OpenAI's GPT-4o-mini to evaluate four key dimensions: numerical accuracy, named entity accuracy, faithfulness, and refutation. G-EVAL is a framework for evaluating LLM outputs using metrics in a human-like evaluation criterion. We prompt GPT-4o-mini to produce scores on a scale of 1 to 5 for each of these dimensions, and scale the score to 0-1 to be consistent with other metrics. We also used FActScore [Min et al., 2023] to assess the factual precision. FActScore evaluates LLM outputs by breaking them down into atomic facts and verifying them against a knowledge source using LLMs. We use GPT-4o-mini as the base LLM, and the parameter γ is set as 10 to penalize counter-responses with <10 atomic facts. We use the generated counter-responses from the test set to extract atomic facts and use the corresponding evidence as the knowledge source to score the atomic facts. We further compute an Overall score by averaging the scores from G-EVAL and FActScore. This comprehensive metric reflects both the factual accuracy and the effectiveness of the counterresponses in refuting the misinformation.

In practice a false claim may have multiple valid counterresponses focusing on different aspects of the claim and varying in semantic content. We therefore focus on evaluating the factuality - factual consistency with the given evidence - and their refutation for the claim. Due to the cost limit of evaluation, we evaluate a total of 100 claims for each dataset, ensuring a representative assessment of our approach.

1 https://developer.twitter.com/en/docs/twitter-api

2 https://www.cdc.gov/coronavirus/2019ncov/vaccines/facts.html

5.3 Results

Table 3 shows the main experiment result. As can be seen, our MisMitiFact achieves robust performance, particularly in generating refutational counter-responses while maintaining faithfulness. Notably, MisMitiFact consistently matches or surpasses the best-performing baselines in numerical correctness, entity accuracy, faithfulness, and overall refutation quality. On PUBHEALTH, MisMitiFact achieves the highest average score (0.838) when using Vicuna as the LLMgen, outperforming all baselines except for MisinfoCorrect w/ Evidence, which achieves a comparable score of 0.844. When using LLaMA2 as the LLMgen, MisMitiFact maintains strong performance with an average score of 0.810, close to the best-performing baselines like SELFREFINE (0.827) and MisinfoCorrect w/ Evidence (0.813). On COVID-19, MisMitiFact achieves the highest average score (0.840) with Vicuna as the LLMgen, outperforming all baselines, including SELF-REFINE (0.828) and MisinfoCorrect w/ Evidence (0.830). With LLaMA2 as the LLMgen, MisMitiFact achieves an average score of 0.804, outperforming all baselines except SELF-REFINE, which achieves a comparable score of 0.806. While SELF-REFINE achieves competitive results in terms of FActScore, it is important to note that this approach leverages iterative refinement based on LLM feedback, which may introduce bias in favor of LLM evaluators. In summary, MisMitiFact demonstrates superior or comparable performance against the best-performing baselines across all datasets.

5.4 Accuracy and Efficiency of Critique Models

We apply G-EVAL to evaluate how accurately the critique models pinpoint the inconsistencies between the counterresponses and evidence on a scale of 1 to 5. Table 5 shows the accuracy of the critique models for identifying errors in numbers, entities and topics. It can be seen that overall the critique model shows robust performance, especially when LLMgen = LLaMA2 (with overall ratings of 4.18-4.41). Critique models show the strongest performance on numerical errors (ratings 4.71-4.96 for Vicuna and LLaMA2 on both datasets), but varying performance for topic errors (from ratings of 2.93 for Vicuna on COVID-19 to 4.12 for LLaMA2 on PUBHEALTH). The informal social media language and diverse topics in the COVID-19 dataset may explain this.

We assess the throughput of MisMitiFact critique models on a single machine with a single inference instance to evaluate their efficiency for large-scale applications. Throughput is measured in #critiques/feedbacks per second, a metric that measures how many critique/feedback requests a model can process in one second. MisMitiFact's critique models, based on T5-large (0.738B parameters), achieve a throughput of 0.925 critiques per second. In contrast, SELF-REFINE, which uses LLaMA2 (6.74B parameters) for feedback generation, achieves a throughput of only 0.165 critiques per second. MisMitiFact achieved a 5.6 times (0.925 vs. 0.165) improvement in critique/feedback throughput, and this significant difference in throughput highlights the efficiency of MisMitiFact. MisMitiFact critique models are not only significantly smaller in size (0.738B vs. 6.74B parameters) but also faster in processing critiques, making them more suitable for large-scale misinformation mitigation.

The high throughput of MisMitiFact can also be attributed to the simple, short critiques generated by MisMitiFact in contrast to the long narrative feedback generated by SELFREFINE. As shown in the example in Fig. 1, MisMitiFact generates critiques on numbers, facts and topics - in 3 sentences of a total of 28 tokens. In contrast, SELF-REFINE generates a feedback narrative like "Thank you for providing the claim and the facts. Here's my feedback on the original explanation: ... the explanation does not address the fact that males make up the majority of both the total count of people experiencing homelessness (57%) and those sleeping rough (66%)." - in 5 sentences of a total of 124 tokens.

5.5 Ablation Study

We evaluated several variations of MisMitiFact with different parts of critique models:

• MisMitiFact w/o number and named entity critique models (NNE) is the variation that removes the number and
• named entity critique models and only includes the topic critique model.
• MisMitiFact w/o topic critique model (T) is the variation that removes the topic critique model and only includes the number and named entity critique models. Number and named entity critique models are treated together since they both contribute to the faithfulness of the generated content.

Table 4 shows the experiment result of the ablation study. We use the same evaluation metrics as in Table 3. It can be observed that MisMitiFact has the best performance on the overall performance in nearly all settings, the performance gain is achieved by both types of critiques. Removing NNE degrades numerical accuracy and faithfulness in specific settings (e.g., Vicuna on PUBHEALTH), while removing T degrades faithfulness (e.g., COVID-19 dataset). In FActScore, MisMitiFact has comparable performance to the best-performing settings, demonstrating the robust ability to generate counterresponses faithful to the evidence.

6 Conclusion

This study tackled the challenge of generating grounded responses to counter misinformation. We proposed to generate simple critique feedback for LLMs to refine their initial generation and ensure responses are grounded in evidence. Our MisMitFact framework trains lightweight critique models on data sourced from readily available fact-checking sites to pinpoint errors in key elements like numbers, named entities, and topics in LLM generations. Experimental results show that MisMitiFact can generate counter-responses of comparable quality to LLMs' self-feedback while using significantly smaller critique models. Additionally, it achieves ∼5x increase in feedback generation throughput, making it highly suitable for cost-effective, large-scale misinformation mitigation. Our future work will focus on counter-response generation in the presence of noisy evidence.

Acknowledgments

This research is supported in part by the ARC Discovery Projects DP200101441, DP210100743 and ARC Linkage Project LP180100750.