Intensity-Valued Emotions Help Stance Detection of Climate Change Twitter Data

1 Introduction

Changing climate has catastrophic effects on all ecosystems in the world, including humans. As climate change continues to worsen, social media platforms such as Twitter plays a critical role in creating awareness among the public. Despite the scientific evidence of the current climate crisis [P¨ortner et al., 2022], the public remains skeptical, leading to polarized discussions about climate change on Twitter, which often result in misinformation, fake news, and bias that influence public attitudes towards climate change [Jang and Hart, 2015; Zhou and Shen, 2021]. Recent articles 1,2 published on the Verge news (an American technology news website) and the Euro news (European television news network) also claim that there has been a sharp increase in the number of tweets and retweets referring to skeptical content towards climate change and these climate deniers frequently abuse, use hate speech and promote violent behavior towards climate activists and scientists. It is, therefore, needful for government organizations, technology companies, and concerned authorities to recognize and interfere in order to stop such climate change deniers' spread of content that can lead to violent activities, disbelief in government policies, and mitigation of climate change. Since stance detection helps to understand the viewpoint of the author whether it is in favor or against the target topic, we perform stance detection to help identify climate change denier and believer tweets.

Existing literature on climate change has largely focused on examining the polarising effects of climate change conversations [Jang and Hart, 2015; Tyagi et al., 2020b; Falkenberg et al., 2022], while some of the more recent climatespecific studies have conducted stance identification. [Vaid et al., 2022] developed the dataset and proposes a BERT embedding architecture to detect the stance of tweets. Other models use basic model components and lack efficiency and more advanced architecture [Chen et al., 2019; Kabaghe and Qin, 2020]. [Upadhyaya et al., 2022a] used sentiment analysis for the stance task but suffered from the drawback of not properly detecting similar sentiments in tweets from supporters and deniers. Moreover, several previous works performed stance detection on the SemEval 2016 dataset including 364 climate-related tweets [Reveilhac and Schneider, 2023; Fu et al., 2022; Wang and Wang, 2021], but this limited number of tweets did not help the models to focus on the climate change domain, which is currently one of the biggest crises facing humanity. Therefore, these works motivated us to develop an efficient model that can detect the climate change attitude of tweets by incorporating advanced components and other auxiliary tasks to avoid the drawbacks of previous works.

1 https://www.theverge.com/2022/12/5/23494220/elon-musktwitter-climate-misinformation-rise-analysis

2 https://www.euronews.com/green/2022/11/17/how-climatedisinformation-is-spreading-after-elon-musks-twitter-takeover

Emotions and their intensity values have helped various domains [Upadhyaya and Chandra, 2022; Abro et al., 2022]. Moreover, emotional and sentiment aspects in climate change conversations often shape discussions and influence public opinion toward the climate crisis [Salama and Aboukoura, 2018; Upadhyaya et al., 2023b]. Therefore, we focus on emotion recognition and emotion intensity prediction as auxiliary tasks to identify the different attitudes of tweets based on the presence of different emotions. Even in the case of similar emotions, the intensity of those emotions, whether high or low, can further help to identify the attitude. The following are examples of tweets on climate change with their emotion and intensity values: (i.)Deny: "Another fucking hypocrite. There is no climate crisis just the biggest scam in history...#climatehoax (Anger:0.58, Disgust:0.67, Sadness:0.31)"; "oh fuck off with your climate bullshit..(Anger:0.82, Disgust:0.71)". (ii.)Believe: "This is what you get when you #votegreen-hope for a better world...(Anticipation:0.51, Trust:0.73)";"How dare you continue to look away #ClimateEmergency #ClimateAction.." (Anger:0.32, Disgust:0.21, Surprise:0.63)". The deniers' tweets show higher levels of anger and disgust, while expectation and trust are more prevalent in the believers' tweets. Even though similar negative feelings of anger and disgust are present in the believers' tweets, they have a softer(more gentle) tone with lower intensity scores (refer Section 3). This relationship between different emotions or similar emotions with different levels of intensity motivated us to use these tasks to support the detection of attitudes.

The main contributions of our work are as follows:(i.) We create a new climate change dataset consisting of tweets with annotations of stance, emotions, and intensity scores (code and dataset are available here 3 ). (ii.) To the best of our knowledge, this is the first cross-sectional study to use emotions and their intensity scores to identify the attitude of the tweet. (iii.) We propose amulti-task systemSIMS (IntenSItyValued EMotions Help Stance Detection of Climate Change Twitter Data) for stance detection by utilizing emotion recognition and emotion intensity prediction as auxiliary tasks. Our SIMS extracts the words from the tweet text that affect the emotions present in the tweets by using the affect word extractor component. The Emotion Affect Inducer is then responsible for injecting the emotional aspects into the tweet text to capture the overall attitude of the tweet in relation to the associated emotions. The final task-specific and shared representations of the input feature are then fused using the integration module of our proposed approach to determine the correct attitude label of the input tweet. (iv.) Extensive experiments are conducted on our curated dataset, two publicly available climate change datasets (ClimateICWSM2023 and ClimateStance-2022), and benchmark stance detection dataset (SemEval-2016). The experimental results show that our proposed SIMS outperforms other computational methods by benefiting from auxiliary tasks and the proposed model architecture. Please note that we refer to the task of stance detection as SD, emotion recognition as ER, and emotion intensity prediction as EI for the rest of the study.

3 https://github.com/apoorva-upadhyaya/Emotion Intensity Stance

2 RelatedWorks

Climate Change and Stance Detection (SD). As climate change continues to worsen, social media plays a crucial role in spreading awareness [Jang and Hart, 2015]. Recently, [Upadhyaya et al., 2022b] explores the information-seeking behavior of students about climate change on YouTube. However, discussions on Twitter often get polarized by users' beliefs about climate change, dividing them into two communities of deniers and believers of climate change [Jang and Hart, 2015]. Therefore, we focus on the task of automatically identifying the viewpoint of the tweet towards a target to help identify denier statements as climate deniers often spread misinformation and fake news, leading to issues of climate delay and opposing climate action [Zhou and Shen, 2021]. Earlier work on climate change has largely focused on examining the impact of polarised beliefs on Twitter [Tyagi et al., 2020b], while others identify polarised users based on statements [Tyagi et al., 2020a]. However, some of the more recent works have focused on identifying attitudes from statements toward climate change. [Vaid et al., 2022] introduced the ClimateStance-2022 dataset and offered BERT architectures for stance task. [Upadhyaya et al., 2022a] used sentiment analysis as an auxiliary task for detecting stances in their curated data (ClimateICWSM-2023), but suffered from the drawback of sarcasm and the presence of similar words in tweets from deniers and believers. Moreover, [Kabaghe and Qin, 2020] uses naive bayes and does not have an advanced architecture. Hence, these works motivated us to develop an efficient model that uses BERTweet embeddings and attention, and combines the emotional aspects and their intensity values to better distinguish tweets with different attitudes, even if they have similar sentiments or words. There are several studies that use the SemEval-2016 dataset to detect stance [Reveilhac and Schneider, 2023; Fu et al., 2022; Wang and Wang, 2021]. However, the dataset contains only 29 denier and 335 believer tweets, hence, these works do not focus on understanding the effective characteristics of denier or supporter tweets and also ignore the intensity of similar emotions/feelings in the tweet, which could help in identifying the correct attitude of the tweet despite the presence of similar emotions. Therefore, we propose an approach that uses advanced architecture and combines the emotion and their intensity values to perform the SD task.

Emotion Recognition (ER) and Intensity Prediction (EI). Several climate-specific studies have examined the importance of feelings and emotional aspects in the climate change conversations on social media [Brosch, 2021; Upadhyaya et al., 2023a], proving the importance of the sentiments embedded in tweets for determining the tweeter's attitude towards climate change. In addition, the intensity of emotions has been used in different domains [Zirikly et al., 2019; Abro et al., 2022] and helps us understand how similar emotions can be severe or gentle based on their intensity values, which can also be useful to distinguish between different attitudes toward climate change (refer Section 3 of our study). Hence, these studies have inspired us to explore how emotions and their intensity can further support the stance task.

3 Dataset

We initially use climate change denier and believer query hashtags to collect real-time Twitter data, similar to the existing literature [Tyagi et al., 2020a; Upadhyaya et al., 2022a]. We collected 31, 546 denier and 82, 010 believer English language tweets from 25 July 2021 to 05 Dec 2022 using query hashtags and Tweepy API 4 (after deduplicating the tweets based on tweet text). However, it is suggested that the stance of a tweet may change after removing the query hashtag [Sobhani et al., 2016]. Therefore, we randomly select 10, 000 tweets from collected data and perform manual annotation.

3.1 Data Annotation

Stance Detection (SD). We first remove the query hashtags from the randomly selected set of 10, 000 tweets. The three trained annotators perform the task of manual annotation. In line with the existing literature [Sobhani et al., 2016; Vaid et al., 2022], the annotators tagged viewpoints on climate change for 3 categories: (i.)Believe(Favour): contain expressions that agree and suggest that climate change is real and happening; (ii.)Deny(Against): consist of opinions against climate action, climate change, and government policies; (iii.)Ambiguous: tweets do not contain a clear expression/stance toward climate change. To determine the quality of the annotations, we calculated the agreement between the annotators, as evidenced by the Fleiss-Kappa score [Spitzer et al., 1967] of 0.80, indicating that the annotation and the dataset presented are of considerable quality. We found a total of 5362 believe, 1726 deny, and 2912 ambiguous tweets.

Emotion Recognition (ER) & Intensity Prediction (EI). Previous works have used the NRC Emotion Intensity Lexicon (NRC-EIL) [Mohammad, 2018], which indicates the presence of 8 basic emotions (anger, anticipation, disgust, fear, joy, sadness, surprise, and trust) with their realvalued intensity values [Shoeb and de Melo, 2020; Qamar et al., 2021; Qin and Ronchieri, 2022]. We also use the python library of NRC-EIL 5 to compute the labels for ER and EI tasks. The preprocessed tweet text is fed to the weighted emotion scores() function of the NRC-EIL library, which returns a dictionary of emotions and their intensity scores for each tweet ( for example, output dictionary: {'anger': 0.698, 'disgust': 0.54, 'fear': 0.31} ). For the task ER, we create a list of length 8 and mark all the emotions present as 1, while others that are not predicted by the library are marked as 0 ( ER label=[1,0,1,1,0,0,0,0] ). Similarly, for the EI task, we create a list of length 8 with values as intensity scores for the corresponding emotions provided by the library and mark the rest as 0 ( EI label=[0.698,0,0.54,0.31,0,0,0,0] ). To assess the quality of the labels predicted by the NRCEIL library, three trained annotators manually annotated 1000 randomly selected tweets from our dataset for both ER and EI tasks. It should be noted that the annotators indicate the intensity labels as "high", "moderate" and "low" for their ease of labeling. After careful analysis, we convert the values according to the criteria if the intensity > 0.8 is classified as high, < 0.3 as low and the rest as moderate intensity. However, we keep the intensity values as real values and consider EI as a multiple-output regression task for our study, which corresponds to the real environment. We found a Fleiss-Kappa [Spitzer et al., 1967] score of 0.79 and 0.76 between the semisupervised and our manual annotations for the ER and EI tasks respectively, indicating that the predicted labels are of considerable quality. The annotations provided by NRC-EIL are therefore taken into account for ER and EI tasks in order to save time and costs.

4 http://docs.tweepy.org/en/latest/streaming how to.html

5 https://pypi.org/project/emotion-nrc-affect-lex/

Data Pre-processing. We first remove the query hashtags from the tweets, as mentioned in Section 3.1, and then remove URLs, punctuation marks, and stopwords. All the text of the tweets is converted to lowercase. We then use NLTKbased 6 TweetTokenizer to tokenize the tweets, followed by NLTKWordnet Lemmatizer to combine inflected words into root form, and PorterStemmer for stemming.

The percentage of each emotion found in the tweets is shown in Table 1. We report the percentage of tweets of different stances having high/moderate/low-intensity scores of corresponding emotions in Table 2 of Supplementary 3 .

4 ProposedMethodology

Problem Statement. Propose a stance detection approach that combines the tweet text with the emotions-inducing words and utilizes these embedded emotions and their corresponding intensities to further classify the attitude of a climate change tweet into one of the polarized classes (ambiguous/believe/deny). Our SIMS approach consists of the following components: Embedding Component, Feature Encoder, Attention, and Classification Layer (as shown in Figure 1). The input tweet text is initially passed through an affect word extractor that extracts the words influencing the emotions in the tweet. The affect words along with the tweet text are passed through the embedding component containing an emotional affect inducer to generate efficient embeddings to capture the overall representation of the tweet with respect to the emotions present in the tweet. The emotionally affected tweet features are then passed through three separate feature encoders to encode the final input representation specific to each task. The attention component then integrates task-specific and shared attention, followed by the classification layer to obtain the output label for each task. We now describe each component in detail.

6 https://www.nltk.org/index.html

4.1 Embedding Component

The input tweet text is first passed through an affect word extractor (AWE) to formulate a set of words in the tweet that affect the emotional presence of the tweet. The AWE consists of the function affect dict() of the NRC-EIL library 5 which returns the keys of the dictionary as the affect words that influence the presence of emotions in the tweet. The embedding component consists of two modules (Embedding Module and Emotion-Affect Inducer):

Embedding Module. The tweet text and affect words are fed to the embedding module (EM), which consists of the pretrained model BERTweet [Nguyen et al., 2020]. BERTweet has a similar architecture to BERTbase but is explicitly trained on tweets, which allows for greater efficiency in identifying the semantics and syntax of words in the text and set of affect words. The tweet text and affect words containing nt and naw number of words respectively, where the embedding of each word is fetched from BERTweet with dimension de, are then flattened to yield Et ∈ Rnt(de) and Eaw ∈ Rnaw(de) , respectively (refer Figure 1). The embedded Et and Eaw vectors are then passed through a dense layer of dimension df , resulting in Et ∈ Rdf and Eaw ∈ Rdf representations for text and affect words.

Emotion-Affect Inducer. This module is responsible for incorporating the effect of emotion into the tweet in order to capture the overall representation of the tweet in terms of the emotions involved. In conjunction with the existing literature demonstrating the importance of discrete linear convolution for modeling the effect of one function on another [Wikipedia, 2001; Bahri et al., 2013; Wu et al., 2021], we also use the numpy operator convolve 7 to obtain the convolution between the embedded text vector (Et) and the embedded affect words vector (Eaw). The equation 1 represents the convolution operation showing the effect of emotional aspects (affect words vector) onto the text representation, where, n is the dimension of Eaw(Rdf ) and m is the dimension of Et(Rdf ), resulting in the output of Emotion-Affect Inducer (EAIaw ∈ Rdf ) that consists of the tweet text representation with embedded emotional aspects.

Finally, we average the tweet text embedding vector (Et) and the output of the emotion-affect inducer (EAIaw) so as not to miss any stance-specific features of the tweet while retaining the emotional aspects adjoined in the tweet (see equation 2), resulting in the final output of the embedding component, i.e. an emotionally affected representation of the tweet (as in Figure 1), which is then reshaped and passed to the feature encoders (Eeat ∈ Rdf×1 ).

4.2 Feature Encoder

The emotionally affected tweet (Eeat) obtained from the embedding component is then passed to the three discrete BiGRU layers with dimension dg specific to each task so as to incorporate past and future context information and sequentially encode these long-term semantic dependencies into hidden states (H ∈ R2dg×1 ). Hence, the feature encoder results in 3 outputs, shown by BiGRUsd, BiGRUer, and BiGRUei for SD, ER, and EI tasks respectively (Figure 1).

4.3 Attention

The module initially extracts the task-specific attention and then fuses the shared attention vector with each of the taskspecific attention vectors using the integration module. In line with the existing literature [Vaswani et al., 2017], in order to extract the important and relevant parts of the input tweet representation, we first create a set of queries, keys, and values by passing the output of the feature encoder to three separate dense layers of dimension (da), which results in Qtask ∈ Rda×1 , Ktask ∈ Rda×1 , and Vtask ∈ Rda×1 , for each task. The task attention vector (TAtask) specific to each task is then calculated using the equation 3, where TAtask ∈ Rda×1 .

Here, the three TAsd, TAer, and TAei vectors are formulated for SD, ER, and EI tasks respectively (shown in Figure 1). In addition, to take advantage of the shared attention features and to use the features that are common to all tasks, we averaged the task-specific attention vectors to obtain the shared attention vector, where SAsh = Average (TAsd, TAer ,TAei). The SAsh together with the task-specific attention vector is then fed to Integration Module.

7 https://numpy.org/doc/stable/reference/generated/numpy. convolve.html

IntegrationModule. integrates the TAtask and SAsh vectors by using the fusion technique of absolute difference and element-wise product that proves to be effective in various previous works [Mou et al., 2015]. The final output of the attention component (IMop) is fetched using equation 4, is then flattened, and passed to the classification layer to provide outputs for each task.

4.4 Classification Layer

The final representation of the tweet (IMop) of the attention framework is then passed through three output separate channels for stance (Outputsd), emotion (Outputer), and emotion intensity (Outputei) tasks separately. We compute categorical cross-entropy loss (Lsd c ) for SD, binary cross-entropy loss for ER (Ler b ), and mean squared error loss for the EI task (Leimse ). The integrated loss function (L) of our SIMS framework is realized in equation 5:

where x, y, and z represent the constants between 0 and 1 indicating the per-task loss-share to the overall loss.

5 Experiments

5.1 Dataset

Experiments are conducted on the following datasets: (i.) Our curated dataset is described in Section 3 in detail; (ii.)ClimateICWSM-2023 [Upadhyaya et al., 2022a]: consists of climate change tweets of believe (60, 430) and deny (13, 125) stances that are collected using hashtags; (iii.)ClimateStance-2022 [Vaid et al., 2022]: In this benchmark dataset, 3, 777 climate change tweets are included that support, oppose, and have ambiguous views regarding climate change prevention. (iv.)SemEval-2016 [Mohammad et al., 2016]: is a benchmark stance detection dataset used in SemEval-2016 shared task 6.A that covers Atheism, Climate Change is a Real Concern, Feminist Movement, Hillary Clinton, and Abortion as targets with tweets having favor, against, or neutral stances. The distribution of emotion and intensity scores in all 3 publicly available datasets can be found here 3 .

5.2 Implementation Details

We perform stratified 5-fold cross-validation and report accuracy, macro precision, macro recall, and macro F1 scores. Using Sklearn resampling, we oversample the minority classes in training data. We execute experiments on NVIDIA GeForce GTX 1080Ti GPU servers (TDP of 250W) with the carbon efficiency of 0.38 kgCO2eq/kWh. Carbon footprint is calculated using the Machine Learning Impact calculator [Lacoste et al., 2019]. A cumulative of 10 hours of computation was performed on the hardware, including single and multitask models training and evaluation on all datasets used in our work, resulting in total emissions to be ≈ 0.95 kgCO2eq . However, the SIMS model required 50 minutes to be trained on our curated dataset, resulting in ≈ 0.11 kg CO2eq. emission. The best parameters for experiments are: Embeddings dimension for BERT(de): 768, Bi-GRU memory cells (dg): 128, fully connected layer dimension of embedding module (df ) and attention component (da) [with ReLu activation]: 128, output neurons/channels: 3 [softmax activation] (SD), 8 [sigmoid activation] (ER) and 8 [ReLU activation] (EI), loss: categorical cross-entropy (Lsd c ) for SD, binary cross-entropy loss function for ER (Ler b ), andmean squared error (MSE) for EI (Leimse ) tasks; optimizer: Adam(0.001). The best parameter values are selected using TPE in the Hyperopt 8 python library that minimizes loss functions. Furthermore, we finetune the loss weights for all tasks by using Grid Search from Scikit-learn (SD (x)=1, ER (y)=0.5, and EI (z)=0.3).

5.3 Baselines

We compare our SIMS with the below baselines on our curated dataset. Semi-Supervised(Model3) [Reveilhac and Schneider, 2023]: performs stance detection using the semisupervised approach with stance, linguistic, entity, and other features on SemEval dataset. RoBERTa-Base [Vaid et al., 2022]: a stance detection framework for climate change tweets (ClimateStance-2022). MT-LRM-BERT [Fu et al., 2022]: Use SemEVAL-2016 and other benchmark datasets to detect stances with a multi-task approach that considers sentiment and opinion as additional tasks. SP-MT [Upadhyaya et al., 2022a]: a novel multi-task framework that performs stance detection with the help of sentiment analysis on the ClimateICWSM-2023 dataset. ESD [Vychegzhanin and Kotelnikov, 2021]: is an optimal ensemble of classifiers and feature set to detect stance using SemEval and other datasets. S-MDMT [Wang and Wang, 2021]: a multi-task multi-domain framework to perform stance detection using SemEVAL-2016 dataset. HAN [Wang et al., 2020]: a hierarchical attention neural model proposed for stance detection. MNB [Kabaghe and Qin, 2020]: Tweets about climate change are classified by multinomial naive bayes into positive, negative, and neutral beliefs. AT-JSS-LEX [Li and Caragea, 2019]: a multi-task framework for stance detection by using sentiment lexicon loss on SemEval dataset. DNN [Chen et al., 2019]: a neural network that classifies users as climate change deniers/believers on Twitter.

8 http://hyperopt.github.io/hyperopt/

6 Results

We report the results with respect to our main task i.e. stance detection (SD) as the current work aims to improve the performance of the main task using ER and EI as auxiliary tasks.

Significance of SIMS Components. Tables 2 and 3 show the performance improvement of the SD task when the proposed components are used. As can be seen in Table 2, using the BERTweet embedding in the single-task stance detection results in a 2.86% increase in the F1 score, as BERTweet trained on tweets only helps to detect the tweet text features more efficiently than BERTbase embeddings. In the singletask model variant (Table 2), adding affect words and capturing their emotional aspects in the tweet improves performance by 4.91% in the F1 score, confirming the importance of the Emotion-Affect Inducer (EAI) model component. Furthermore, the introduction of task attention (TA) leads to an average F1 score of 83.02 for the single-task model. Similarly, in our multi-task frameworks (see Table 3), the addition of the Emotion-Affect Inducer (EAI) along with the task (TA) and shared attention vectors (SA) improves model performance in all variants (SD + ER, SD + EI, SD + ER+ EI). Moreover, the use of integrating the task and shared attention vectors over the concatenation operation enhances the performance of SD + ER + EI model variant combination with the F1 value from 88.71 to 91.84, resulting in 3.53% (row 2 & 4 of Table 3) increase rather than 1.52% increase in the F1 when using the concatenate operation (row 2 & 3 of Table 3), thus demonstrating the importance of the fusion mechanism of the integration module (IM).

Effectiveness of Auxiliary Tasks. The performance improvement of the single-task variant framework when using emotions and their intensity values as input features (with 85.19 F1 score in Table 2) motivates us to analyze the effect of ER and EI when used as auxiliary tasks. Table 3 shows that the combination of SD + ER performs slightly better than SD + EI with an average F1 score of 90.01 and 89.31 respectively, indicating that ER task has contributed more than EI for capturing stance features. The better and clearer separation of emotions between the believers' and deniers' tweets contributed to a more efficient SD task and justifies the assignment of higher loss weights to the ER task (refer Table 1). However, the combination of the ER and EI tasks further improves the performance of the SD task, resulting in an accuracy of 93.72, as the presence of similar emotions can be distinguished by their intensity values (see Table 2 of Supplementary), proving the importance of ER and EI for SD.

Comparisons With Baselines. (i.)Our curated climate dataset: Our SIMS overtakes the other baselines when these methods are trained and tested on our curated dataset, confirming the effectiveness of our approach (refer Table 4 for the results). SIMS outperforms the semi-supervised approach (Model3) and ROBERTa-Base shows that the multi-task setting with the better embedding technique of BERTweet can improve the stance task. The methods SP-MT, MT-LRM- BERT, and AT-JSS-LEX perform better than the other models because these approaches use sentiment information to detect the attitude of the tweet. However, SIMS outperforms these methods because separating the sentiments into multiple emotions at the more granular level and further distinguishing the presence of similar emotions in different stances according to their intensity values makes our approach more efficient for the SD task. S-MDMT uses target classification as a separate task, while ESD and HAN use different hierarchical and attentional features. However, our SIMS performs better by using the EI and ER as auxiliary tasks and incorporating the emotional aspects into the tweet representation. Our single-task method surpasses the MNB and DNN methods, which shows the importance of an advanced architecture to improve the task SD. During the writing of this paper, we were unaware of any other work which utilized emotion task for identifying the stance of climate change tweets. (ii.)ClimateICWSM-2023: We use the baselines from the work of [Upadhyaya et al., 2022a], who created this dataset. Based on Table 5, our approach SIMS surpasses other methods, suggesting that the fine-grained separation of positive, negative, and neutral sentiments into 8 categories of emotions can better identify the attitude of the tweet, while their emotion intensity scores and inducing the words influencing these emotions further help to distinguish the presence of similar emotions but with different intensity levels and context among the believe, deny and ambiguous stances. (iii.)ClimateStance-2022: We use the baseline methods from the [Vaid et al., 2022] work that created the dataset. SIMS performs significantly better than the other models (refer Table 6), proving the importance of capturing emotional aspects and that the use of BERTweet embeddings together with the fusion of task and shared attention can support the SD.(iii.)SemEval-2016: It can be observed from Table 7, SIMS outperforms other models with an overall F1 score 71.24, especially in climate (C), Hillary (H), and abortion (AB) target domains as these domains have clearer separation of emotions and their intensity levels among different stances while performing similarly with S-MDMT for feminism (F) target with F1 score as 63.27. Hence, it proves that our model generalizes well with different targets and domains and can be suited to other topics as well for the task of SD.

6.1 Error Analysis

We identify some scenarios where our SIMS fails to correctly predict the attitude, apart from the skewness of the dataset, which we have tried to address using the resampling technique (deny: 17.26%, ambiguous: 29.12% and believe: 53.62%). Incomplete context: We note that in some cases where the tweet text is insufficient, the tweet's attitude is labeled as ambiguous. However, the presence of other modalities could help identify the correct attitude, e.g. "The Real Reason We'll Freeze To Death This Winter "https://t.co/xyzabc via @YouTube #climatehoax #climatechange", based on tweet text (without hashtags), groundtruth annotation: ambiguous, SIMS prediction: believe (due to the close proximity between ambiguous and believe stance through similar words such as "real", "freeze", and anticipation emotion), however, the video attached to the tweet provides the correct stance of the tweet, motivating us to focus on the presence of other modalities that could efficiently help with the stance task. Implicit/Hidden Stance: In some of the tweets (example: "Which begs the question; How many @COP26 attendees will burn FF's? When will COP26 set a leading example by going..#climatechangeisreal..", "How many more countries will you be visiting his year so you pollute the planet further?#climatehoax", it is evident that the authors support climate change but are opposed to government action to combat it. Such tweets are often difficult for the model to identify, so further categorization of stance classes and goals, such as "believer of climate change but against climate action" or similar terms, can further improve SD and could also help authorities to understand viewpoints.

7 Conclusion

In this paper, we curate a novel dataset with annotations for stance, emotion, and emotion intensity labels. Our proposed SIMS induces the emotional aspects associated with the attitude of the tweet and fuses the task-specific and joint feature representations obtained from embedding and attention components using the integration module. Our model outperforms other baselines on climate change datasets by avoiding the drawbacks of previous works that lack an efficient architecture and distinguishes different attitudes even if they consist of similar emotions with different intensity levels. The model's performance on the SemEval dataset further suggests that our framework generalizes well across a variety of domains. Future work on the potential pathways that support the stance detection process will be useful. These include focusing on other modalities to address insufficient context, finer categorization, and other tasks like aspect-based sentiment.

Ethical Statement

We address the United Nations Sustainable Development Goal 13: Climate Action by proposing an approach that helps identify climate change deniers on social media platforms. As climate skeptics on social media platforms such as Twitter spread more and more misinformation and disinformation, it is crucial for government agencies and concerned authorities to identify such misleading content and stop its spread before it becomes harmful to society. Our proposed approach will therefore be beneficial to these organizations when implemented in real-time, as our approach can predict the stance of the tweet based on the textual content as soon as the user posts something. Detailed analysis of the real-time suitability of the model, such as latency and prediction time, is part of our future work depending on the deployment environment. In addition, we are conducting our work with public data from social media. however, to ensure individual privacy, we do not share any personal information. As a result, our publicly available dataset consists only of the tweet IDs and the annotations.

Acknowledgments

This research was partially funded by the SoMeCliCS project under the Volkswagen Stiftung and Nieders¨achsisches Ministerium f¨ur Wissenschaft und Kultur and by the Federal Ministry of Education and Research (BMBF), Germany under the project LeibnizKILabor with grant No. 01DD20003.