SEMANTIFY: Unveiling Memes with Robust Interpretability beyond Input Attribution

1 Introduction

In recent times, memes have emerged as a ubiquitous form of online expression, blending humor, satire, and social commentary to encapsulate complex ideas in a single image or short video. While created to disseminate humor, it is often misused to perpetuate societal harm [Kiela et al., 2021]. A significant portion of the meme ecosystem is tainted with content that is offensive, hateful, or even dangerous. Therefore, it is crucial to develop effective tools for the automated detection of offensive memes, to preserve the integrity of online spaces.

However, a simple classification of memes as offensive is often insufficient. Making the system interpretable is paramount as it can elucidate whether the system learns from spurious correlations in the data or whether it can reliably classify a meme as offensive. This clarity aids in enhancing user trust on these systems through transparency. Further, interpretability methods help users to know if the model acquired some kind of inadvertent biases while training.

Existing input-attribution-based explainability methods like LIME [Ribeiro et al., 2016], SHAP [Guo et al., 2019], and GradCAM [Selvaraju et al., 2019] work well in practice but suffer from two issues, viz. i) Semantic Irrelevance: The keywords that are attributed to model predictions are often semantically irrelevant to the model input, making it hard for humans to assess their effect on the model's behavior; and ii) Incohesive: Existing input-attribution methods operate on the input space, generating a set of keywords that often lack a central theme and consequently miss crucial words essential for a more comprehensive explanation of the model and its predictions.

An example of the above drawbacks can be shown in Figure 1, where we compare our method, SEMANTIFY with Integrated Gradient [Sundararajan et al., 2017], which is an input-attribution based explainability method. The tokens retrieved by input attribution are 'Mexican', 'Child', and 'Good', which are Semantically Irrelevant to the hidden meaning of the meme which is associated with 'Racism'. In contrast, our method, SEMANTIFY consults a large set of vocabulary space and retrieves a much more relevant set of keywords (e.g. 'Refugees', 'Slavery' etc.). Also, the set of retrieved keywords from Integrated Gradient lacks a central theme, which makes them Incohesive.

We enumerate the major contributions/attributes of our current work as follows 1 :

1. We propose a theoretically grounded technique that (refer alignment-optimization correlation in Section §3.3) could explain a model's behavior by retrieving 'tokens' from a global vocabulary space. The retrieved tokens are compared with input attribution based baselines and found to carry both a 'faithful' representation of the input meme as well as semantically relevant information.
2. Our method is extensively evaluated for both automatic and human evaluation. A detailed analysis is performed to assess its effectiveness.
3. While we demonstrate the application of our method in the realm of internet memes, it is fundamentally model and task-agnostic. Expanding its application to other tasks is beyond the scope of this paper and is left for future research.

2 RelatedWork

Multimodal Offensiveness Detection. In the realm of Natural Language Processing (NLP), previous research has primarily concentrated on identifying offensive content [Waseem and Hovy, 2016; Sarkar et al., 2021], addressing cyberbullying [Van Hee et al., 2018], hate speech [Caselli et al., 2021], and similar issues within social media posts [Roberts et al., 2012]. Nevertheless, these computational methods have predominantly been evaluated using textual data. Turning to visual offensiveness detection, earlier investigations have centered on identifying sexually explicit images [Ganguly et al., 2017]. The pivotal moment came when Kiela et al. [2021] introduced a set of benchmarks and released the Facebook Hateful meme dataset, which ignited research in this field. This led to a series of research on detecting offensiveness in multimodal media [Yang et al., 2022], particularly in memes [Sharma et al., 2020]. Suryawanshi et al. [2020] used an early fusion method for combining visual and textual features, leading to more accurate detection. Chen and Pan [2022] stacked visual features, object tags, and text features to Vision-Language Pre-Training Model with anchor points to detect offensive memes. While these models are proven to be useful for predictions, their outputs are not interpretable and cannot be reliably used in real-world use cases.

Multimodal Interpretability. Recently, there have been a notable number of multimodal models [Du et al., 2022; Li et al., 2023; Liu et al., 2023b; Zhu et al., 2023] for various tasks. However, there is a dearth of research on generating explanations or justifications around their predictions. Researchers predominantly relied on interpretability techniques.

1 Code and Supplementary Material available at: https://github.com/newcodevelop/semantify

LIME [Ribeiro et al., 2016] explains predictions of any classifier by fitting a sparse linear model locally around the instance being explained. It converts the instance into a binary vector, indicating the presence or absence of interpretable components (like words). SHAP [Lundberg and Lee, 2017a] explains machine learning model's predictions by computing Shapley values, inspired from game theory. These values represent each feature's contribution to a prediction. Gradient heatmap [Guo et al., 2019] explains predictions by computing gradients of the model output concerning the input features.

However, recent years have witnessed a shift in the focus of interpretability research, recognizing the potential for generating natural language explanations for both unimodal and multimodal systems [Kayser et al., 2021]. Instead of traditional end-to-end training, Koh et al. [2020] first predicted concepts and used those to predict the labels such that the model could be interpreted by changing the concepts. There exist some natural-language-based techniques like wt5 [Narang et al., 2020], which is available for text-only systems. Some recent methods like NLX-GPT [Sammani et al., 2022] bridges the gap between text-based and multimodal natural language generation. Cross-modal attention, which attends to the distinguishing features between text and image modalities, is used in the transformer encoder for sarcasm explanation [Desai et al., 2021]. Sharma et al. [2022] generates explanations for visual semantic role labeling in memes. These methods together can generate textual explanations for the behavior of multimodal models. As shown in Figure 1 and mentioned in the Introduction, all of these current methods fall short in adequately explaining model behavior for inputs with implicit meaning. To tackle this, we aim to address the issue by retrieving keywords aligned with the model's inner workings.

3 Methodology

The proposed systems combine a multimodal encoder for an input meme and a language model (LM). The LM acts both as the final classifier and interpretable keyword retriever. It categorizes the input meme as either non-offensive or offensive, and, through sampling, can extract a set of keywords to elucidate its prediction. Thus, our system follows a two-step strategy, i.e. i) Multimodal encoding followed by ii) Classifying via a language model (LM). We elaborate the steps in details as below:

3.1 SystemDesign

Multimodal Encoding. LetM denote the input meme (single instance), consisting of an image V and accompanying text T. We utilize a pre-trained and frozen CLIPmodel [Radford et al., 2021] to obtain textual (ft) and visual (it) representations. These features, with dimensions Rm×1 and Rn×1 respectively, are used to generate a multimodal representation Mt ∈ Ro×1 (here, m = n = 512).

The fusion process employs trainable weight matrices U and V with dimensions Rm×ko . The multimodal representation is calculated as follows: Mt = AveragePool(U T ft ◦ V T it, k), where ◦ denotes element-wise multiplication, k represents the stride for the overlapped window used in the pooling operation, and o denotes the output dimension. This encoding scheme, inspired by a similar approach [Bandyopadhyay et al., 2023], maintains high performance with a low parameter count. This multimodal encoding is passed through two feed-forward neural networks (FFNs) which produce predicted labels l (0 or 1) corresponding to the input meme.

Using LM as the Classifier cum Retriever. We utilize a GPT2 model [Radford et al., 2019] as the classifier as well as for generating the interpretive keywords. The input prompt of GPT2 constitutes of the plain text input of the 'meme text' and the 'meme caption', which is automatically generated via OFA [Wang et al., 2022] module. Along with these plain text input prompts, it also takes an embedding representation m ∈ R1×1024 , which is obtained by projecting Mt via an FFN. Further, to facilitate backpropagation, the predicted label from the first stage, l, is passed through Gumbel softmax [Jang et al., 2017] which produces a soft logit score, which is then passed as an input token. The input prompt is then appended by a fixed string: 'This meme is'. The model is then trained to output 'offensive' or 'normal' (the label associated with the meme in the training data) as the next token of the input prompt via causal language modeling objective.

The system architecture is visualized in Figure 2.

3.2 Retrieving Keywords Beyond Input Space

Obtaining a comprehensive set of human-comprehensible and interpretable keywords that effectively encapsulate the operational characteristics of a classifier when presented with a given input typically poses a challenge. One plausible approach to address this issue involves the utilization of inputattribution-based methodologies, such as Integrated Gradient or Attention Heatmaps. Such techniques serve to highlight specific parts of the input data that are correlated to the classifier output. However, it is noteworthy that these methods only yield set of tokens present in the input space, thereby limiting their applicability due to lack of diversity, as it is illustrated by an example in Figure 1. The proposed method for extracting the set of relevant keywords (interchangeably called tokens) involves four filtering steps involving the vocabulary set of the language model used (i.e. GPT2).

1. Maximally Relevant: First, we filter out the keywords that are not relevant continuation of the input prompt. The GPT2 model is trained via causal language modeling (CLM) objective to predict either 'offensive' or 'normal' as the next token of the input prompt. The CLM objective ensures that Top-K sampling of the next token gives the k most plausible set of tokens that are maximally relevant continuation of the input prompt.
2. Alignment - Optimization Correlation: The set of extracted keywords from the first step (their embedding is denoted as e) should be such that they belong in a special ϵ neighborhood. This is an additional filter that theoretically ensures that the set of keywords does not possess redundant information while also not completely alienated from the optimization objective. The definition and interpretation of this is presented in Section 3.3.
3. Semantically Relevant: In this step, we filter additional keywords that are semantically irrelevant to the input meme. Mathematically, we take the cosine similarity between m and ti, where ti is the text embedding of i-th keyword obtained through the CLIP Text encoder. Finally, we sort them by values of cosine distance and only keep top 20 keywords out of them. CLIP being trained with contrastive learning only preserves tokens that are semantically relevant to the input meme.
4. Prediction Preserving : The fourth step is the most stringent one. First, we use the trained LM in inference mode to generate knowledge text by passing extracted tokens as input prompt. Again, together with the extracted tokens, we pass their generated knowledge text to the LM. If the model predicts the same class as it has predicted before, we call the passed token prediction preserving. If the passed token flips the actual prediction then we can confidently say that the token is not causally related to model prediction and thus it is not faithful. We

filter out only top four keywords after this step by sorting with respect to the log-likelihood score of the predicted tokens in decreasing order.

3.3 Alignment - Optimization Correlation

In the pursuit of optimizing machine learning models, we often encounter the challenge of striking the right balance between the alignment of information vectors and optimization efficiency. To explore this delicate trade-off, we introduce the following theorem.

We first assume that our objective function ˆ y = f(m) is strongly convex. Here, ˆ y shows the model predicted class. Also, we consider two non-zero real column vectors e (token embedding) and m (multimodal embedding of the input meme). ∇mˆ y = ∇mf(m) refers to the gradient of the prediction with respect to m such that m+ = m+ γ∇mf (m).

This theorem carries substantial empirical implications: i) Sampling tokens with their token embedding e such that eT · ∇mf(m) > 0 would imply alignment between e and ∇mf(m), i.e. moving m in the direction of e aids optimization. As demonstrated by the left-hand side (LHS) of the inequality, successive gradient ascents on m progressively reduce the angle between e and ∇mf(m). Intuitively, this entails e aids in the model optimization process, thus its corresponding tokens are also interpretable.

ii) With ∇mf(m) being smooth and differentiable, when eT ·∇mf(m) → 0, we find that eT ·∇mf(m+) > 0. Even as e and ∇mf (m) approach near-orthogonality, indicative of e carrying diverse information rather than perfect alignment with the gradient, the positive value of eT ·∇mf(m+) signifies e as aligned for subsequent gradient-based optimization steps w.r.t m. We term this phenomenon the 'Alignment - Optimization Correlation Criteria'. In practical applications, this serves as a filtering mechanism to retain tokens relevant to regions where eT ·∇mf (m) → 0. This condition is henceforth referred to as ϵ ball or ϵ neighborhood constraint interchangeably. This theoretically grounded motivation significantly enhances our ability to extract diverse yet interpretable tokens, as shown through results and analysis. The proof of this Theorem 1 and the physical implication are described in detail in Technical Supplementary Material Section A and B, respectively. The Algorithm for our filtering process is shown in Algorithm 1.

4 Experiments and Analysis

4.1 Experimental Setup

Our proposed model was constructed using PyTorch, a Python-based deep-learning library. In our experimentation, we imported GPT-2 from the Huggingface transformers package. All experiments were conducted on a single Nvidia A100 80GB GPU. We employed the Adam optimizer [Kingma and Ba, 2017] with a learning rate of 0.005 for optimization. We use the Facebook Hateful Meme dataset [Kiela et al., 2021] for performing the experiments.

To ensure robust evaluation on simulatability, we conduct a 5-fold cross-validation for testing the surrogate models

Algorithm 1 Retrieve explainable keywords with four step filtering

explain out = [] ; /* Final token placeholder */ first stage = [] ; /* Placeholder or TopK & ϵ neighborhood constraint */ rclip ←Meme Image Embedding from CLIP { ti }←Top-k tokens fromVocabulary set V ; /* TopK Filtering */ for ti ∈ {ti} do ei ←GPT2Embedding( ti ) if ∥ei · ∇mˆ y∥ ≤ ϵ then ticlip ← Text Embedding from CLIP(ei) simcosine ← rclip · ticlip first stage . append({ti : simcosine}) ; /* filtered tokens from ϵ neighborhood */ end end {t′i } ← Top 20 tokens decreasingly sorted by simcosine from first stage. ; /* CLIP filtering */ for t′i ∈ {t′i } do e′i ←GPT2Embedding( t′i ) if f(e′i ) = ˆ y then explain out . append(t′i ) ; /* Output preservation filtering */ end end explain out ← Top 4 tokens from explain out sorted by log likelihood. ; /* Final step */

(§Section 4.2) after running experiments for 3, 500 steps on the respective train set. We report averaged scores obtained from 5 experiment runs. Additionally, we maintain a consistent random seed of 42 across all our experiments.

4.2 Results and Analysis

Automatic Evaluation

For Automatic evaluation, we resort to using 'model faithfulness' as a guiding principle to evaluate the effect of the obtained keywords on model behavior. Especially, we measure 'simulatability', which can be defined as how well we can use the extracted keywords to predict (i.e. simulate via another surrogate model) the model's output. In Table 1, we depict the effect of various filtering mechanisms forming an ablation and compare our proposed method with the well-known input attribution-based methods, e.g. Integrated Gradient and KernelSHAP. For comparison, we use i) Leakage adjusted simulatability (LAS) [Hase et al., 2020] score, which measures the effect of predicted keywords/explanation on simulating model prediction opting for explanation leakage. A positive LAS score reflects better 'simulatability' or 'faithfulness' of the extracted keywords corresponding to the model prediction. For evaluating the effect of extracted keywords on simulator confidence, we use ii) Comprehensiveness (↑) and iii) Sufficiency (↓) metrics [DeYoung et al., 2020]. Intuition and technical details of the measurement of these metrics can be found in Supplementary Section I. We also list three accuracy-based measures for the simulator: i) F1 score using both generated explanation and input meme as input to the model (denoted as F1), ii) F1 score using only input (denoted as F1 w/ inp) and iii) F1 score using only explanation (denoted as F1 w/ exp). We also propose two metrics: i) Inter-sample diversity defined as Div (Inter) and ii) Intra-sample diversity defined as Div (Intra) elaborated in the Supplementary Material Section H. These metrics respectively measure the average diversity/exhaustiveness of all the retrieved keywords across the dataset and for a specific example. The trade-off between these two illustrates various properties of the retrieved keywords, as illustrated in the discussion below.

Comparison with Baselines. In the first five rows of Table 1, we describe the effect of extracting keywords from various input attribution-based explanation methods which are compared with random keywords. As expected, the random keywords obtained very low scores for all metrics reflecting the input attribution-based methods which work well in practice. For every setup of our proposed model in the remaining rows, we observe the superiority of our proposed approach by observing better obtaining scores for all the metrics. We also observe that although F1 w/ exp score is better for the baselines compared to the 'Random' explanations, the model performance remains the same when explanation and input both are used as input, as seen through the same F1 score obtained. This intuitively illustrates the fact that the extracted explanations do not provide extra information compared to inputs, such that the before and after F1 scores remain the same. Comparison with LLaVA-13B [Liu et al., 2023a] can be found at Supplementary Material Section D.

1. Does epsilon ball constraint work in the absence of Top-K constraint? Firstly, in rows 6 to 8, we consider dropping off the Top-K sampling restriction (first stage) and observing the effect of disabling and enabling the second stage with different values of ϵ. Without any ϵ constraint, we obtain a negative LAS score along with a low 'comprehensiveness' score. It shows that only selecting the keywords using CLIPbased representation does not retrieve semantically relevant keywords. Next, we enable the second stage with two separate values of ϵ ∈ {0.05, 0.01}. As can be seen through tabulated metrics, enabling the second stage has a positive effect on the quality of retrieved keywords. Also, ϵ = 0.01 works better than ϵ = 0.05 in terms of performance, suggesting that our theoretical justification of retrieving tokens in the neighborhood of eT ·∇mf(m) → 0 indeed works well in practice. However, lowering the ϵ by too much (ϵ ∼ 0) would result in non-retrieval of any keyword.

ii) Why would a larger Top-K be associated with a lower comprehensiveness score? From the Inter sample diversity score, we observe that a higher Top-K value relates to higher Inter sample diversity, which entails that diversity between two explanation sets will be larger. Intuitively, it can be seen that evaluating the simulator model on a more diverse explanation set leads to a lower probability of the predicted class due to lower model confidence. This consequently leads to lower comprehensiveness and higher sufficiency scores. It may be observed that there is a steady increase in inter-sample diversity with increasing Top-K value, which further leads to lower comprehensiveness scores.

• iii) For the same Top-K value what would the effect of enabling ϵ constraint be? Comparing scores from the third and fourth set of rows, we observe that enabling the ϵ constraint seems to be beneficial for the 'simulatability', as can be seen by higher LAS scores for the same Top-K value without the ϵ constraint. This can be attributed to the same inter-sample diversity (indicating variations among samples) but lower intra-sample diversity (indicating lesser variation among retrieved keywords specific to an input meme). Less variation among the retrieved keywords for an input meme would intuitively mean better simulatability. However, this case is not always true, as a very low intra-sample diversity score would entail that the retrieved keywords are very similar and would result in a low LAS score (observe the sixth row). Intuitively, there is an optimal spot where the ratio of inter-sample and intra-sample diversity would indicate optimally selected retrieved keywords.
• iv) What is the similarity of the retrieved explanations using a specific Top-K value w.r.t various ϵ balls? We observe that enabling the Top-K constraint unequivocally retrieves better tokens as illustrated by Table 1. To theoretically justify it, we measure Jaccard similarity between the a) set of tokens retrieved using a specific Top-K value and b) tokens retrieved from the open ϵ neighborhood [-ϵ,+ϵ]. This test is done on a randomly selected set of 100 test memes, keeping equal representation from all over the full test set. From Figure 3, we observe Jaccard similarity value spikes at [-0.01, +0.01] when TopK ∈ {3500, 2500, 1500} and at [+0.02,+0.03] when TopK ∈ {500}. This entails Top-K retrieved tokens mostly lie in the neighborhood where eT · ∇mf(m) → 0, which is theoretically justifiable.

v) Is the Output preservation stage necessary? We state that this stage is of utmost importance. From Figure 4, we observe that for different Top-K values (both with and without ϵ constraint enabled), enabling the output preservation stage always retrieves more interpretable keywords rather than disabling it. The same test subset is used to perform these experiments, as illustrated in the previous point. Also, the performance gap is quite significant, as seen in Figure 4.

• vi) What is the interpretive impact of keywords on model decisions? We input the extracted keywords as knowledge tokens during testing to empirically verify their influence on the model. The analysis reveals that in the majority of cases (86.23%), the predicted probability for the modelassigned class increased, with a mean rise of 0.06 and a standard deviation of 0.07. Moreover, the predicted class remains consistent in 97.41% of cases. Additionally, incorporating keywords does not adversely affect the model; instead, it boosts confidence in predicting the class, emphasizing the interpretive value of keywords in shaping the model's decisions.
• vii) Is semantic relevance (CLIP filtering) stage necessary? For every set-up in Table 1, we manually tested a random set of same 30 memes with and without the CLIP filtering stage enabled. Without CLIP filtering, the quality of

the retrieved keywords are worse such that they do not semantically match with the input meme, which renders them unusable for explanation purposes.

viii) How well does it classify input memes? The macroF1 and Accuracy score on the offensiveness classification task using our proposed system are 73.46% and 75.64% respectively on the test set. This is at par or better than the standard visuo-lingual baselines used in [Kiela et al., 2021] where the FB hateful meme dataset was originally proposed. We also observe that the GPT2 prompt without the projected multimodal embedding Mt obtains only around 57% F1 score.

Human Evaluation

We perform a human evaluation of the generated keywords using two metrics, viz. Relatedness and Exhaustiveness. Relatedness is defined as how much a set of generated keywords is relevant to the content of the input meme, and Exhaustiveness is defined as how much of the aspects of an input meme are correctly represented in a set of retrieved keywords. The scale of these metrics is illustrated in Supplementary Material Section G. Based on these definitions, five people (three authors of this paper and two undergraduate students) are chosen to rate the generated explanations (a randomly chosen set of 200 examples) on a scale of 1-5 in the 5 point Likert scale. The inter-rater agreement (Cohen kappa score [Cohen, 1960]) for all the settings is more than 0.7, indicating fair agreement. For both exhaustiveness and relatedness, our methods achieve the best performance as observed from Table 3, followed by ϵ-ball, CLIP, and lastly Integrated Gradient.

01276

13875

98724

Figure 5: Memes corresponding to the examples in Table 2

4.3 InterpretingModel fromRetrieved Keywords

In our analysis, we employ sampling from GPT-2 for both keyword extraction and prediction to enhance interpretability. Our focus is on understanding the model's decisionmaking process in meme explanation and assessing offensiveness, leading us to prioritize interpretability over utilizing more advanced language models like GPT-3, which may generate unfaithful yet plausible explanation [Filippova, 2020; Maynez et al., 2020].

For the correctly classified memes (with IDs 01276, and 98724), our proposed approach (Top-K=3500 with ϵ = 0.1 and other filters enabled) provides a relevant and exhaustive set of keywords for the input meme which may adequately represent the correct model prediction obtained. These explanations are also intuitive and help us to clarify that the model is not relying on spurious correlations to predict its decision for that particular meme. For other variations of our proposed methods and the baseline method, we observe the quality of the retrieved keywords seems arbitrary to the input meme. Thus, they do not adequately reflect the reason based on which the model might have predicted the correct label.

Even though the CLIP retrieves semantically relevant tokens, they are not exhaustive and often repetitive.

From meme ID 91768, we observe that the model predicts the meme as offensive even though it is a funny meme about Hitler. Due to the presence of Hitler's face, the model classifies it as offensive, which may be correctly illustrated by the retrieved keywords using our method. The baseline performs poorly and the variations of our method yield retrieved keywords that are either repetitive or not very semantically relevant to the input meme. Another example is shown for meme ID 13875, where the the model predicted an offensive meme as normal. The prediction appears to be influenced by the presence of the word 'cat,' which the model uses as a determining factor. This phenomenon can be attributed to the model's lack of exposure to relevant memes during training, resulting in an inability to recognize the underlying racism.

5 Conclusion

Classifying hateful content on social media and generating explanations for moderation is crucial. Existing interpretability methods operate on the input space, making it difficult to explain content with hidden meanings. Our work not only identifies hateful content that requires moderation but also provides explanations for hidden meanings. This improves the efficiency of uncovering hidden meanings in memes and clarifies the model's decision-making process. It also helps identify model bias, as shown in qualitative evaluations. Our method outperforms various baseline models in both automated and manual evaluations and is applicable to social media, with potential real-life impact. Additionally, our taskagnostic method can be applied to various visual-linguistic tasks (e.g., Visual Question Answering, Visual NLI), presenting an important challenge for future studies.

Acknowledgements

Dibyanayan Bandyopadhyay and Baban Gain acknowledge support from the Prime Minister Research Fellowship (PMRF). The authors thank valuable feedback from Samrat Mukherjee and comments from anonymous reviewers.