Visual Attention Prompted Prediction and Learning

1 Introduction

The "black box" nature of deep learning models often obscures the decision-making process in AI, leading to the emergence of Explainable AI (XAI) aimed at demystifying the rationale behind models [Adadi and Berrada, 2018]. Techniques like CAM, Grad-CAM, and integrated gradients, pivotal in XAI, produce saliency maps highlighting the model's focus areas in the input data [Zhou et al., 2015; Selvaraju et al., 2017; Qi et al., 2019; Bai et al., 2023]. While XAI has advanced in explaining model reasoning, the ultimate aim extends beyond this. It is crucial to leverage XAI to enhance the reasoning and predictive capabilities of Deep Neural Networks (DNNs). Key challenges include incorporating human insight into the model's reasoning and ensuring that such insights positively impact future predictions.

Explanation-guided learning in natural language processing (NLP) task has been extensively researched, with studies such as [Hsieh et al., 2023; Raffel et al., 2020; Narang et al., 2020] extracting rationales as additional supervision, alongside label supervision, for training language models. In contrast, the visual domain remains not well explored. In recent years, there has been emerging research in visual explanation-guided learning. Studies like [Chen et al., 2020; Zhang et al., 2023; Shen et al., 2021; Gu et al., 2023] utilizes human-annotated attention maps to guide the reasoning process of DNNs in Computer Vision (CV) tasks by simultaneously minimizing prediction errors and the disparity between the model's reasoning and the human-crafted true reasoning within the training set, as shown in Figure 1(a). However, visual explanation-guided learning is labor-intensive, timeconsuming, and computationally expensive due to the need for large amounts of human-annotated attention maps. In many practical applications, users may have easy and quick (high-level) guidance toward the model for a prediction, e.g., which areas are roughly more important are which are not. For instance, in cancer imaging, clinicians can quickly sketch out areas that are interesting or irrelevant in terms of determining whether the image indicates "cancerous", as shown in Figure 1(b). Hence, there is a need for an efficient way to incorporate these cues to assist the model's decisionmaking process. Despite many existing works about textual prompts [Oymak et al., 2023; Liu et al., 2023; Ye et al., 2022; Ling et al., 2023], how to prompt in visual space is much less underexplored and the focus of this paper.

Despite the interestingness and significance, achieving visual attention-prompted prediction requires working on visual patterns on images that trigger unique unsolved challenges 1) How to incorporate the visual attention prompt into the model's prediction? Traditional classifiers only take images for decision-making, but how to embed the prompt such that it can guide the model reasoning as instructed in the visual attention prompt is crucial and very challenging. 2) What if some samples do not have visual attention prompts? Many samples may not come with prompts, but can their predictions still get some guidance from the samples with prompts? And how? and 3) How to handle the incomplete visual attention prompts? In practice, it is much easier and quicker to provide an imperfect prompt which may not cover the whole image but just its regions where the user can quickly have some intuition. How can we address the incompleteness and sufficiently utilize it?

To tackle the aforementioned challenges, we introduce the Visual Attention-Prompted Prediction and Learning framework. To address the first challenge, we proposed an attention-prompted prediction framework for integrating visual attention prompts into the model's decision-making process. To counter the second challenge, we developed an attention-prompted co-training mechanism that distills knowledge from the prompted model to the non-prompted model, thereby enhancing future prediction performance for samples without provided prompts. Finally, to tackle the third challenge, we proposed a novel architecture to achieve attention prompt refinement by automatically learning the saliency of "undecided" areas for each pixel. In summary, the main contributions of this paper are as follows: 1) A new framework designed to integrate visual attention prompts into the model's decision-making process; 2) A new attentionprompted co-training algorithm developed to improve predictions for samples without attention prompts by distilling knowledge from the attention-prompted model to the nonattention-prompted model; 3) A novel architecture to refine the incomplete visual attention prompts by automatically learning the saliency of "undecided" areas at the pixel level; 4) Comprehensive experiments on four datasets to demonstrate the effectiveness of our framework in enhancing model predictability for image classification tasks.

2 RelatedWork

Privileged information The learning with privileged information paradigm, as proposed by Vapnik [Vapnik et al., 2015], introduces a "teacher" that provides additional information to a "student" model during the learning process. The underlying idea is that the teacher's extra explanations help the student develop a more effective model. Lopez et al. [Lopez-Paz et al., 2015] integrated the concepts of distillation and privileged information into 'generalized distillation,' a framework for learning from multiple machines and data representations. Garcia et al. [Garcia et al., 2018] introduced a method for multimodal video action recognition. Pan et al. [Pan et al., 2024] applied privileged information from Large Language Models (LLMs) to Graph Learning.

Attention-guided Learning The integration of human knowledge into interpretable models has been extensively studied in CV and NLP tasks. Attention maps, such as those generated by Grad-CAM [Montavon et al., 2019; Selvaraju et al., 2017] or intrinsic attention mechanism [Vaswani et al., 2017], have been utilized as supervision signals. These signals are aligned with prediction loss to further enhance model performance [Shen et al., 2021; Camburu et al., 2018; Hajialigol et al., 2023; Bai and Zhao, 2022]. HAICS [Shen et al., 2021] proposed a conceptual framework for image classification that incorporates human annotations in the form of scribble annotations as the attention signal. RES [Gao et al., 2022] developed a novel objective to handle inaccurate, incomplete, or inconsistently distributed issues of explanation. Attention Prompt Prompting, originating from NLP tasks as shown by Devlin et al. [Devlin et al., 2018], has been adapted for CV applications [Dosovitskiy et al., 2020]. Oymak et al. [Oymak et al., 2023] examine prompt-tuning for one-layer attention architectures in contextual mixture models. Jia et al. [Jia et al., 2022] introduced Visual Prompt Tuning (VPT) to add adaptable prompt tokens to the Vision Transformer (ViT) model's patch tokens. Paiss et al. [Paiss et al., 2022] proposed an explainability-based method for improving oneshot classification rates. Finally, Li et al. [Li et al., 2023] developed "saliency prompts" from saliency masks to highlight potential objects in scenes.

3 ProblemFormulation

In the context of visual attention-prompted learning and prediction, we consider the samples from a dataset U to be provided in pair as (I,D, y) ∈ U, where I ∈ RC×H×W represents the original image, with C, H, and W denotes the number of channels, height, and width, respectively, y is the class label, and visual attention prompt for the image corresponding to its class label is denoted by D ∈ RH×W , with dimensions identical to the original image but with one channel. Attention prompt D serves as an instruction or signal indicating which parts of the input image are particularly relevant or should be prioritized. The visual attention-prompted prediction process of model f can be denoted as f : (I,D) ↦→ y.

Despite the merit of the potential of embedding the visual attention prompt into a prediction to guide it better, achieving it requires tackling nontrivial technical challenges in three key aspects. Firstly, how to use the prompt to guide the model's reasoning so that it can attend to the right places? Secondly, image samples I may not always come with prompts D, but can they benefit from the prompts of the others? Moreover, it is usually much more efficient to provide incomplete prompts covering only part of the image, but how to handle such incompleteness?

4 Methodology

To address the challenges outlined above, we propose a new framework for visual attention-prompted prediction and learning. Section 4.1 introduces the overall framework we propose. Section 4.2 details the attention prompts refinement architecture. Finally, in Section 4.3, we propose the attentionprompted co-training mechanism.

4.1 Overall Framework

In our Visual Attention Prompted Prediction Framework, attention prompts are employed to effectively guide the reasoning process of the AI model. This is achieved by utilizing the attention prompt to mask the original image. Consequently, the model's decision-making process is influenced by areas categorized as "indispensable," "precluded," and "undecided," ensuring that these guided insights are integral to the AI's analytical procedures. In the left of Figure 2(a), "indispensable" and "precluded" areas are shown in red and yellow, respectively, with the remaining areas being "undecided" ones. To address the incompleteness of the prompt, we will respect certain parts that are either "indispensable" or "precluded", but impute the undecided part by eliciting the model's explanation of the reasoning process and aligning it with the given attention prompt. As illustrated in Figure 2(b), the given incomplete prompt is initially inputted into the Visual Attention Prompt Refiner. This step serves to activate and constrain the generation of post-hoc explanations. Subsequently, these elicited post-hoc explanations are refined and utilized to mask the input image, thereby facilitating the attention-prompted prediction process. The details are elaborated on in Section 4.2. As illustrated in Figure 2(a), to handle samples without prompts and allow them to benefit from those with prompts, we propose another non-prompted model, fo, that takes only the image. This model is co-trained with the prompted model fm by aligning both the model parameters and activation, as indicated by the dashed rectangles in the figure. S(fo) and S(fm) denote the sets of model architectures before the fully-connected layers, while FCo and FCm represent the fully-connected layers of two models respectively. The details will be elaborated in Section 4.3.

4.2 Visual Attention Prompt Refinement

In this section, we propose an attention-prone refinement architecture. First, we introduce prompt-guided incomplete prompt refinement architecture. Second, we present an adaptively learnable mask aggregation method.

Prompt-Guided Incomplete Prompt Refinement

To refine incomplete prompts containing areas that are "undecided" yet crucial for prediction, we propose a promptguided incomplete prompt refinement architecture designed to enhance the tolerance of our framework to incompleteness and errors within the prompt. This approach aims to learn the saliency of each pixel in the "undecided" areas by aligning the given incomplete prompt with post-hoc explanations. Consider a prompt D ∈ RH×W that includes annotations distinctly highlighting indispensable, precluded, and undecided areas, we aim to leverage the post-hoc explanations of the prediction for the image prompted with this incomplete prompt, to refine the prompt. However, those gradient-based explainers are usually time-consuming due to the involvement of back-propagation during explanation generation. Therefore, post-hoc explanations obtained without backpropagation such as perturbation-based methods that can directly predict the importance of perturbations to compose explanations are preferred here.

To be specific, each element λ ∈ D corresponds to a pixel in the image and takes a value from the set {-1, 0, +1}. Here, λ = 0 indicates that the pixel is precluded, λ = +1 signifies that the pixel is indispensible, and λ = -1 denotes that the pixel is undecided. To generate the post-hoc explanation guided by prompt D, we first randomly perturb D as P(D) to generate N binary masks M = {P (i) (D) = Mi}Ni=1 , where P (i) (·) denotes the i-th preturbation process, by setting each pixel with a value equal to -1 to +1 with probability p and to 0 otherwise. After obtaining the perturbed masks, we first perform an element-wise multiplication of the original image with each randomly perturbed mask to obtain the masked image. Subsequently, we compute the confidence scores for each mask in M as {(I ⊙Mi)}Ni=1 , where I is the image and ⊙ represents the element-wise multiplication operation. We then calculate the confidence score for each masked image relative to its corresponding class label by wi = softmaxk fm(I ⊙Mi), i = 1, . . . , N, where k represents the index of the label y and softmaxk denotes the output of the softmax layer corresponding to the k-th class. Upon calculating the confidence scores for each perturbed mask, we applied weighted averaging and normalized it with the expected pixel value N · p for aggregation. The process resulted in the refined prompt, denoted as A. To summarize, the above computation process can be represented as

where the refined prompt is obtained by learning the saliency of "undecided" areas to align with the post-hoc explanation, which is guided by the incomplete prompt.

Adaptively Learnable Masks Aggregation

Aggregating masks by the confidence score can ensure the relationship between confidence and importance, as a higher confidence score indicates greater importance in the aggregation process. However, the confidence score may not necessarily accurately quantify the importance. To overcome this, we require a function that can adaptively learn to quantify the importance score accurately. In the following, we propose to learn such a weight-learning function by satisfying the following inherent constraints of the confidence score:

• (Monotonicity): The function needs to be monotonically non-decreasing, ensuring masks with higher confidence scores receive greater importance in the aggregation.
• (Endpoint-preserving): The function must ensure that inputs of 0 and 1 yield outputs of 0 and 1, respectively. This respects the input range's limits, preserving the key relationship between confidence scores and weights, and mitigates the effects of extremely large or negative values.

To achieve the formulation and learning of such as weightlearning function, we build a constrained multilayer perceptron (MLP) g(·), to quantify the importance scores. To be more specific, as shown in Figure 3, ensuring the monotonically non-decreasing behavior, g(·) is achieved by the principle that the weight matrix must consist of non-negative entries. This crucial constraint is enforced by applying weights derived from a function with a range of positive numbers [Nguyen et al., 2023]. To satisfy the endpoint-preserving property, two specific measures have been taken: 1) To guarantee that the function produces an output of 0 when the input is 0, the bias term has been eliminated from all layers of g(·). Additionally, an activation function σ(·) has been selected such that σ(0) = 0, and 2) To ensure that the function yields an output of 1 when the input is 1, a constraint term g(1) = 1 is incorporated into the training phase. In details, the mapping between layer k and layer k + 1 of g(·), which is depicted in Figure 3, is as

where ϕ is a function that has a range within the positive numbers and can be exemplified by functions like the exponential function or a translated hyperbolic tangent, and L + 1 denotes the total number of layers in the model. With proposed function g(·), the mask aggregation function, as described in Equation 1, can be redefined as

where the refined prompt is obtained by aggregating masks with the learned weights, as shown in Figure 2(b).

4.3 Attention Prompted Co-training

In this section, we delineate the attention-prompted cotraining mechanism. Initially, the framework for parametersharing and co-activation is introduced. This is succeeded by a detailed exposition of the learning algorithm, which is predicated on an alternating optimization training strategy.

Parameter-Sharing and Co-Activation Framework

In instances where a prompt is unavailable, the proposed function fm, which necessitates a prompt, becomes inapplicable. Under such circumstances, an alternative predictor, designated as fo, is employed. This predictor operates exclusively based on the image data. Rather than learning fm and fo independently, we investigate their interrelation. For each training sample accompanied by a prompt, both fm and fo are executed, with the anticipation that fo will mirror the correct reasoning dictated by fm and guided by the prompt. This approach not only aims to encourage two models to have similar parameters but also to foster similar activation patterns. For instance, if a prompt directs fm to disregard the region of an artifact in the image, which results in no activations in that region, fo should also follow it by disregarding this region. This alignment of activation patterns enables the transfer of knowledge from fm to fo in a joint parameter-sharing and co-activation framework. To concretize this concept, we propose the integration of two regularization terms: model parameter-sharing regularization, and co-activation regularization, which collectively embody this approach as

where Wfm and Wfo represent the convolutional layer parameters of two models and ∥ · ∥2F represents the squared Frobenius norm [Ma et al., 1994]. Furthermore, the crossentropy loss associated with the predictions made by the two models, when compared to the target, can be articulated as

where ˆ yia is the ground truth label for class a of the ith data point in one-hot encoding, pia m = softmaxa (fm (Ii ⊙Ai)) predicted probability for class a of the ith sample for the prompted model, and pia o = softmaxa (fo (Ii)) is for the nonprompted model. In conclusion, considering the constraints imposed by the weight-learning function, the learning objective of our study can be formally expressed as

where λ1 and λ2 are the weighting hyper-parameters for parameter sharing loss and activation sharing loss, respectively. If we treat the constraint as a Lagrange multiplier [Gordon and Tibshirani, 2012] and solve an equivalent problem by substituting the constraint to a regularization term LAgg(θg), our overall objective function can be rewritten as

where LAgg(θg) = ∥g(1) - 1∥ is commonly chosen to be the ℓ2-norm, λ3 is the weighting hyper-parameters for weightlearning function constraint regularization term.

Alternating Training Algorithm

As illustrated in Figure 2, the prompted model fm exhibits an output-as-input structure during the prompt refinement stage. Specifically, fm's output is fed back into its input during the prompted prediction stage, creating circular dependencies in training [Han et al., 2017; Xu et al., 2023]. These circular dependencies may introduce challenges in the convergence of the model, potentially leading to instability or oscillatory behavior during training. To mitigate this, we propose an alternating training strategy to break this cyclic dependency and

Algorithm 1 Alternating Training

Require: I,D, y Ensure: fm, fo, g 1: for t = 1 : T do 2: for q = 1 : F do 3: Compute ∇θfm based on Equation 9 4: Compute ∇θfo based on Equation 10 5: θfm ← θfm - η∇θfm 6: θfo ← θfo - η∇θfo 7: end for 8: for q = 1 : G do 9: Compute ∇θg based on Equation 11 10: θg ← θg - η∇θg 11: end for 12: end for

ensure better convergence. This strategy involves three models: the non-promptedmodel fo, the promptedmodel fm, and the weight-learning function g.

The training algorithm is summarized in Algorithm 1. our algorithm is to fix the parameter for g while updating fm and fo with a learning rate η for F iterations from Lines 2-7. The gradients w.r.t. θfm and θfm are computed as

From Lines 8-11, our algorithm fixes the parameter for fm and fo while updating g with a learning rate η for G iterations. The gradients w.r.t. θg are computed as

We repeat the alternating training process outlined in Lines 2-11 for T iterations, continuing until optimal performance is achieved (e.g., no further increase in prediction accuracy on the validation set).

5 Experimental Evaluation

5.1 Datasets

To assess our framework's effectiveness, we employed four datasets: two from real-world scenarios, sourced from MS COCO [Lin et al., 2014], and two from the medical field, namely LIDC-IDRI (LIDC) [Armato III et al., 2011] and the Pancreas dataset [Roth et al., 2015].

Specifically, LIDC dataset, featuring lung CT scans with annotated lesions, was preprocessed into 2D images (224×224) and augmented with noise to simulate incomplete prompts. Negative samples were created by slicing surrounding areas of nodules. The final dataset included 2625 nodules and 65505 non-nodules images, split into 100/1200/1200 for training, validation, and testing to reflect limited access to human explanations. Pancreas dataset, with normal images from the Cancer Imaging Archive and abnormal images fromMSD, was preprocessed similarly to LIDC. It included 281 CT scans with tumors and 80 without. Tumor lesions were treated as inaccurate explanations, while pancreas segmentations were ground truth. Data was split into 30/30/rest for training, validation, and testing, maintaining class balance. For Gender classification dataset [Gao et al., 2022], involved extracting 1,600 images from MS COCO based on captions mentioning "men" or "women". Images with both genders, multiple people, or unclear figures were excluded. For Scene [Gao et al., 2022] recognition, balanced nature, and urban categories from Places365, again selecting 100 images for training. each image from two datasets comes with explanations that were obtained through a custom user interface (UI). Human annotations were collected for all images, and only 100 were randomly chosen for training to simulate limited access to explanations. For each sample image, the factual region of the human-explanation annotation is treated as the "indispensable" area, the counterfactual region is considered the "precluded" area, and the remaining regions are categorized as "undecided" areas, thereby facilitating the creation of an incomplete visual attention prompt.

5.2 Experimental Setup

Comparison Methods To evaluate the effectiveness of our proposed Visual Attention-Prompted Prediction and Learning framework (hereafter referred to as "VAPL"), comparative studies were conducted with four prominent attentionguided learning methods, namely, GRAIDA [Gao et al., 2022], HAICS [Shen et al., 2021], RES-G, and RES-L [Gao et al., 2022]. These comparison methods were trained following the respective implementation guidelines presented in their papers. Additionally, comparisons were made with a baseline model, comprising a ResNet-18 architecture [He et al., 2016], trained exclusively using prediction loss and employing original images as input.

Implementation Details In this study, the ResNet-18 architecture was uniformly employed as the backbone model across all evaluated methods. The experimental setup was standardized with a batch size of 16, and the number of perturbed masks was set to 5000. Furthermore, a pixel conversion probability of 0.1 was established. The training was conducted over 10 epochs, each comprising 5 iterations for the alternating updating phase, effectively resulting in 50 training epochs for each model. The Adam optimization algorithm [Kingma and Ba, 2014] was utilized with a learning rate of 0.0001. Regarding computational resources, all experiments were executed using an NVIDIA GTX 3090 GPU. To assess the classification performance of the proposed framework, we employed conventional metrics such as accuracy, precision, recall, and the F1 score.

5.3 Results Evaluation

Table 1 shows the quantitative evaluation of two medical image classification tasks: LIDC and Pancreas. Overall, our method outperforms all other explanation-guided learning methods on both LIDC and pancreas datasets. In particular, our model achieves the best accuracy and F1 on the pulmonary nodule classification task and the best accuracy, precision, recall, and F1 on the pancreatic tumor classification task. Our method achieved an accuracy of 92.30% and 69.45% on the two respective datasets, representing an improvement of 8.5% and 4.6% compared to the baseline Resnet-18 approach. In addition, our method attained F-1 scores of 95.30% and 71.13% on the two datasets, marking an enhancement of 5.6% and 12.1% respectively when compared to the baseline Resnet-18 method. Our findings demonstrate that the performance of our prediction methodology is enhanced when incorporating explanations into the prediction phase. While the precision or recall scores reported on the LIDC dataset are marginally lower, our approach achieves the highest F1 scores across both datasets. This outcome signifies that our overall performance surpasses that of all other attention-guided learning methods.

Table 2 presents a comprehensive comparison of various models across two image classification tasks: Gender and Scene. Notably, the VAPL model demonstrates superior performance, achieving the highest scores in almost all metrics for both tasks, with particularly outstanding results in the Scene task where it achieves an impressive 96.39% in Accuracy and 97.43% in Precision. The RES-G model also shows commendable performance, especially in the Scene task, where it records the highest Recall of 95.35% and a competitive F1 score of 95.78%. In contrast, the Baseline model, as expected, trails behind in most metrics, underscoring the effectiveness of advanced models like VAPL and RES-G. This analysis underscores the significance of specialized models in enhancing classification accuracy in attention-guided learning tasks, with VAPL excelling in precision-oriented metrics and RES-G showing strength in recall-focused areas.

We further provide a sensitivity analysis of the hyperparameters λ1, λ2, and λ3, which denote the weights of the parameter sharing regularization loss, activation sharing regularization loss, and weight-learning function regularization loss, respectively. Figure 4 illustrates the accuracy of the proposed model for various weights in the context of pancreatic tumor classification, with the red dashed lines representing the baseline model's performance. Overall, the model exhibits relatively high sensitivity to variations in the weights for the parameter sharing and activation sharing regularization losses, while demonstrating lower sensitivity to the weight-learning function regularization loss. Additionally, a concave curvature is observed for all three hyper-parameters. Optimal overall performance is achieved when λ1, λ2, and λ3 are set to 0.1, 1, and 10, respectively.

5.4 Ablation Study

To evaluate the efficacy of the proposed visual attentionprompted learning and prediction (VAPL) framework, an ablation study was conducted with four distinct variants:

(VAPL-1), which involves the omission of the visual attention prompt refinement architecture, thereby relying exclusively on the incomplete visual attention prompt during the learning and prediction phases. (VAPL-2), which pertains to the exclusion of parameter-sharing regularization between the two models, while retaining co-activation regularization during training. (VAPL-3), which entails the removal of co-activation regularization, and maintaining only parametersharing regularization during the training process. (VAPL-4), characterized by the absence of the proposed weight-learning function and instead employing a weighted averaging method based solely on confidence scores computed by classifiers.

The outcomes of the ablation study conducted on the LIDC and Pancreas datasets are presented in Table 3. While each component individually contributes to the overall performance of the model, it is observed from VAPL-2 and VAPL-3 that co-training with parameter and activation sharing regularization offers a notably larger enhancement in predictability. Furthermore, the results about VAPL-1 highlight the significant contribution of the proposed visual attention prompt refinement method to improved performance. This is particularly evident in the Pancreas dataset, which demonstrates a greater prevalence of prompt incompleteness issues.

6 Conclusion and Discussion

This research paper introduces a Visual Attention Prompted Prediction and Learning Framework, a novel approach that integrates visual attention prompts into the decision-making process of models. The framework effectively tackles challenges like incomplete information from visual prompts and predictions for samples lacking such prompts through an attention-prompted co-training mechanism with parametersharing and co-activation regularization. This helps align activation patterns and facilitates knowledge transfer between prompted and non-prompted models. Additionally, the framework incorporates a prompt-guided refinement method with an adaptively learnable mask aggregation function to manage prompt incompleteness. Its efficacy is validated across four datasets from various domains, showing improved predictive accuracy for both samples with and without attention prompts. Despite the aforementioned advantages, we also acknowledge that prompts from users can introduce bias, though our framework is tolerant of it. In the future, we plan to explore the application of our proposed framework to graph learning and prediction, incorporating prompts with graph attention mechanisms, such as subgraphs.

Acknowledgments

This work is supported by the National Science Foundation (NSF) Grant No. 1755850, No. 1841520, No. 2007716, No. 2007976, No. 1942594, No. 1907805, a Jeffress Memorial Trust Award, Amazon Research Award, Oracle for Research Grant Award, Cisco Faculty Research Award, NVIDIA GPU Grant, Design Knowledge Company (subcontract number: 10827.002.120.04), CIFellowship (2021CIFEmory-05), and the Department of Homeland Security under Grant No. 17STCIN00001.