FCKT: Fine-Grained Cross-Task Knowledge Transfer with Semantic Contrastive Learning for Targeted Sentiment Analysis

1 Introduction

Unlike traditional sentiment analysis tasks [Liu, 2012], aspect-based sentiment analysis (ABSA) [Wang et al., 2016; Tang et al., 2020] requires a deeper contextual understanding and the extraction of more fine-grained information. ABSA has been widely applied in tasks such as customer feedback analysis and academic paper review evaluation [Nath and Dwivedi, 2024]. ABSA typically involves identifying (1) aspects (entities or attributes discussed), (2) opinions (sentiment expressions), and (3) sentiment polarity. However, research indicates that over 30% of sentiment expressions are The lens and autofocus are impressive, except low-light capability.

† Corresponding author.

implicit [Cai et al., 2021; Chen et al., 2024a], meaning the ABSA must infer aspects and sentiments from the context without predefined opinions. In this paper, we address these challenges by developing targeted sentiment analysis (TSA). It involves two sub-tasks, i.e., identifying relevant aspects from reviews and explicitly associating them to their corresponding sentiments [Chen et al., 2022b; Zhou et al., 2024].

When TSA was first proposed, a two-stage pipeline method was developed [Hu et al., 2019; Kalbhor and Goyal, 2023]. However, this approach often lead to error propagation between sub-tasks. Moreover, aspects are intrinsically linked with sentiment, and this staged pipeline approach disrupts this crucial interaction [Xu et al., 2021]. To address these issues, many end-to-end approaches have been proposed [Chen et al., 2022b; 2022a; Li et al., 2023; Zhu et al., 2024], aiming to jointly extract aspect terms and classify their sentiments by modeling the interactions between aspects and sentiments. A widely studied technique in this field is the task-specific feature alignment approach [Chen et al., 2024a]. This method involves two key steps: first, encoding task-specific features for both aspects and sentiments; second, aligning these features in the latent space, even when some aspects differ in sentiments, to enhance cross-task knowledge interaction.

Despite recent advancements, these methods still rely on coarse-grained knowledge transfer, which fails to provide explicit and fine-grained control over aspect-sentiment relationships. This limitation often leads to negative transfer, as coarse-grained approaches assume uniform sentiment polarity across related aspects, ignoring key contextual differences. For instance, as illustrated in Figure 1, coarse-grained methods treat all camera-related aspects ("lens", "autofocus", and "low-light capability") share the same sentiment representation. Consequently, these methods incorrectly classify "lowlight capability" as positive, overlooking the negative sentiment implied by the contrastive cue "except". This misalignment, stemming from the lack of explicit control over knowledge transfer, is particularly problematic when aspectsentiment relationships are subtle or context-dependent.

Along this research line, explicit fine-grained knowledge transfer has emerged as a promising solution to enhance the interaction between aspects and sentiments. It disentangles aspect-sentiment relationships and enables dynamic alignment tailored to each aspect. By isolating sentiment representations for each aspect, it accurately associates "lens" and "autofocus" with positive sentiment and "low-light capability" with negative sentiment, leveraging contextual cues like the contrast implied by "except". This ensures more precise and context-aware sentiment classification. However, despite its potential, explicit fine-grained cross-task knowledge transfer faces the more severe challenge of error accumulation that must be addressed: (1) In fine-grained knowledge transfer, the performance of tasks such as aspect extraction and sentiment classification is highly interdependent, as information flows directly between tasks. While leveraging aspect extraction to assist sentiment classification fosters cross-task collaboration, it also introduces the risk of error propagation. Errors or ambiguities in aspect extraction, such as incorrectly identified or missing aspects, are directly transferred to the sentiment classification phase, compounding inaccuracies in the final predictions. To address this, a more effective approach is required to constrain aspect extraction, ensuring more accurate identification and minimizing the influence of ambiguity. (2) Moreover, even with perfect aspect extraction, fine-grained knowledge transfer may still struggle due to insufficient supervisory signals for sentiment classification. As the sentiment classifier relies heavily on aspect extraction, its training can become overly dependent on these features, potentially overlooking other crucial contextual cues that are not explicitly tied to aspect information. This lack of comprehensive supervision can hinder the classifier's ability to generalize effectively. This underscores the need for balanced, holistic supervision to optimize fine-grained knowledge transfer.

To address these challenges, we propose FCKT, a finegrained cross-task knowledge transfer framework tailored for the TSA task. Specifically, we design a token-level semantic contrastive learning mechanism. In this approach, the start and end tokens of the same aspect are treated as positive pairs, while tokens from unrelated aspects serve as negative pairs. This refines the model's understanding of aspects and enhances its ability to capture subtle contextual dependencies. To mitigate the lack of supervisory signals in the sentiment classifier, we introduce an alternating learning strategy. This approach trains a fixed proportion of samples using real labels, while the remaining samples are updated based on predictions from the previous model. This enables FCKT

to leverage both real labeled information and predicted transfer knowledge, synergistically improving the effectiveness of TSA. Furthermore, while Large Language Models (LLMs) like GPT-3.5 and GPT-4 excel in many general-purpose scenarios, they often struggle to adapt to task-specific challenges in fine-grained TSA tasks. Our proposed framework surpasses these models in both few-shot learning and chainof-thought reasoning scenarios by leveraging task-specific knowledge transfer mechanisms and tailored training strategies. These results highlight the capability of FCKT to address the unique complexities of TSA tasks, providing valuable insights for advancing task-specific methods in the LLM era. Our key contributions are summarized as follows:

• We propose a token-level semantic contrastive learning mechanism that enhances the model's understanding of aspects by treating start and end tokens as positive pairs and unrelated tokens as negative pairs, enabling it to effectively capture subtle contextual dependencies.
• To address insufficient supervisory signals in sentiment classification, we introduce an alternating learning strategy integrates real labeled data with predicted transfer knowledge, significantly improving task-specific performance.
• We conduct extensive experiments on three real-world datasets, demonstrating the superiority of FCKT over existing approaches. We also demonstrate the effectiveness of our approach across various LLM models, showcasing its superior performance in the era of LLMs.

2 RelatedWork

Targeted Sentiment Analysis. Targeted sentiment analysis (TSA) involves extracting aspect terms and their associated sentiment polarities, a key focus in recent research. Early work approached aspect extraction and sentiment analysis as separate tasks, employing methods such as Long Short-Term Memory (LSTM) networks [Luo et al., 2019] and Bidirectional Encoder Representations from Transformers (BERT) [Yang et al., 2020]. Recent studies have shifted toward end-to-end solutions using multi-task learning frameworks [Luo et al., 2019; Lin and Yang, 2020; Chen et al., 2022b], highlighting the interdependence between aspects and sentiments. Techniques such as shared-private feature interaction [Lin and Yang, 2020] and task-regularization interaction [Chen et al., 2022b] have been introduced to improve performance by leveraging cross-task knowledge transfer. However, current approaches mainly rely on coarse-grained alignment approches , which lacks fine-grained control over aspect-sentiment relationships. This often leads to negative transfer, as coarse-grained approaches assume uniform sentiment polarity within related aspects, overlooking contextual cues that differentiate sentiments. More recently, large language models (LLMs) like GPT have demonstrated strong generalization capabilities, outperforming fine-tuned BERT in some low-resource ABSA scenarios [Wang et al., 2023; Zhou et al., 2024]. Nevertheless, they continue to underperform compared to state-of-the-art task-specific models, underscoring the need for further adaptation to address the unique challenges of resource-constrained TSA tasks.

Contrastive Learning. Contrastive learning has recently achieved significant success in various domains [Jaiswal et al., 2020; Zhang et al., 2022; Luo et al., 2024; Zhong et al., 2024b; 2024a; 2025; Yuan et al., 2025a; 2025b; 2023]. In ABSA, several studies have explored its integration into model training to improve performance. For example, Liang [Liang et al., 2021] leveraged contrastive learning to distinguish between aspect-invariant and aspect-dependent features, leading to better sentiment classification. Similarly, Xiong [Xiong et al., 2022] proposed a triplet contrastive learning network that combines syntactic and semantic information via an aspect-oriented sub-tree and sentence-level contrastive learning. Chen [Chen et al., 2024a] further utilized contrastive learning to model adaptive task-relatedness between aspect and sentiment co-extraction. Different from these works, we leverage contrastive learning specifically for aspect extraction to enhance knowledge transfer in TSA.

3 Methodology

In this section, we will introduce proposed FCKT framework in detail, which is depicted in Figure 2. The comprehensive task consists of two base components: first, extracting opinion aspects (AE), and second, predicting the sentiment polarity (i.e., positive, neutral, or negative) of each aspect (SP).

3.1 ProblemDescription

Formally, given a sentence x = {x0, x1, . . . , xn-1}, where xi represents the ith token in x and n denotes the sentence length, the main objective of FCKT is to extract all possible aspects and predict their corresponding sentiments in x.

To enhance the representation of contextual semantics, we leverage the widely recognized BERT model [Devlin et al., 2019], an effective multi-layer bidirectional Transformer designed to construct rich contextual embeddings by simultaneously capturing both left and right word dependencies. BERT comprises a series of T Transformer layers. The working principle of BERT is summarized as follows:

where Wi is the one-hot representation of the sub-word in the input sentence, E denotes the sub-word embedding matrix, P represents the positional embedding, T is the Transformer block, and Ht is the output of the tth Transformer layer. For the given sentence x = {x0, x1, . . . , xn-1}, we assume the output of the final Transformer layer is H = HT = {h0, h1, . . . , hn-1} ∈ Rn×d , where d is the embedding size.

Remark. In our modeling process, sentences containing multiple aspect terms are split into separate sentences during training, with each focusing on a single aspect, as illustrated in Figure 3. This preprocessing step allows the model to effectively learn aspect-specific representations by isolating each aspect, avoiding interference from others in the same sentence. It is important to note that this splitting strategy is only applied during training to facilitate the learning process. During testing, the original sentence structure is preserved, ensuring consistency in evaluation and practical application. Our approach, FCKT, leverages this strategy without altering the original optimization objectives or knowledge transfer mechanisms. Further details are provided in the Appendix A, where we demonstrate that this strategy maintains optimization consistency while enabling fine-grained modeling of aspect-specific interactions.

3.2 Semantic Contrastive Learning for AE

In this article, aspects are detected by predicting their start and end boundaries through a linear transformation operation [Lin and Yang, 2020]. Specifically, the start and end boundary distributions are predicted as follows:

where Fφ1 and Fφ2 are MLP layers used for extracting aspects. The length of ˆ ps matches that of the sentence, and each element in ˆ ps represents the probability of a word being the start of an aspect. It is important to note that this step only predicts the boundary distributions of aspects. To determine the final aspects, a heuristic extraction algorithm [Hu et al., 2019] is subsequently employed. During optimization, the learning objective for boundary distributions is formulated as:

where pTi,s ∈ Rn and pTi,e ∈ Rn are the boundary ground truths (i.e., 0-1 vectors), ˆ pi,s and ˆ pi,e are the predicted boundary distributions, n is the length of sentence.

Compared to conventional coarse-grained transfer methods [Chen et al., 2024a; Sun et al., 2024], fine-grained knowledge transfer imposes stricter requirements on the precision of aspect extraction. Any incorrectly identified or missing aspects are directly transferred to the sentiment classification phase, accumulating prediction inaccuracies. To mitigate this issue, we introduce a token-level semantic contrastive learning framework to improve aspect representation quality.

Intuitively, tokens within an aspect inherently exhibit strong semantic compatibility, as they frequently co-occur in similar contexts. For instance, as illustrated in Figure 2, the end boundary distribution suggests that both the 2nd and 3rd words have comparable probabilities of being the end of an aspect. However, from a semantic perspective, the word "is" is unrelated to the preceding words and is therefore less likely to be part of the aspect boundary. Semantic information can serve as a complementary signal to boundary distributions by verifying whether different tokens form a "semantically reasonable" aspect. To leverage this property, we treat the start and end tokens of the same aspect as positive pairs, encouraging their representations to be closely aligned. For negative pairs, we construct two types: (1) the start token of the aspect paired with end tokens from other, unrelated aspects, and (2) the end token of the aspect paired with start tokens from unrelated aspects. The InfoNCE loss [Chen et al., 2024b; 2024c] is then employed to optimize these token-level embeddings. Formally, the contrastive loss is defined as:

where (hs, he) are positive pair embeddings, while (hs, hi) and (he, hj) are negative pairs. τ is the temperature parameter, s(·, ·) denotes the cosine similarity function, and E represents the set of all positive pairs in each sentence.

3.3 Fine-Grained Knowledge Transfer for SP

In sentiment prediction task, the sentiment polarity for a given aspect is determined based on the words within its boundaries. Let the start and end boundaries be denoted as s and e, respectively. The sentiment is then predicted as:

where ˆ y(ψ) represents the predicted class probability distribution, Cθ is a nonlinear classifier that maps input features to the range [0, 1], effectively modeling the likelihood of each sentiment class. Hs→e denotes the aspect representation, aggregated from the start boundary s to the end boundary e.

As previously mentioned in Section 1 , these two sub-tasks exhibit a logical sequential dependence, implying that aspects can provide valuable signals for sentiment prediction. However, sentiment prediction in previous coarse-grained transfer methods relies on ground truth boundaries, which essentially disconnects the sequential connections between these two sub-tasks. Therefore, we employ aspect features to supervise the sentiment prediction process. In this way, aspect extraction can provide sequentially useful supervised signals for sentiment prediction, while sentiment information can influence aspect detection through backpropagation.

To achieve this, we refrain from using the ground truth boundaries as input. Instead, we derive an expectation of word embedding based on the boundary distributions. Consequently, Eq. (5) can be redefined as:

where E denotes the expectation operator over the indices i and j, which are sampled from the predicted start boundary distribution ˆ ps and the end boundary distribution ˆ pe, respectively. Furthermore, since ˆ ps and ˆ pe are both discrete distributions, we have the following derivation:

where we assume n is the sentence's length. In this way, these two sub-tasks enable genuine end-to-end learning.

In practice, to compute Eq. (7) , one needs to traverse the whole sentence, and the computational complexity is O(n2 ), which is very difficult to calculate. Fortunately, this complexity can be reduced based on two facts: (i) The end point is not smaller than the start point. (ii) The length of an aspect term is usually not large. We then revise ˆ y(ψ) to ˆ y(ℓ), as follows:

where h is the maximum length of aspect. Thus, the computation complexity is reduced to O(nh) (i.e.,h ≪ n ).

Actually, in the testing phase, the sentiment is inferred based on the predicted aspects, which is more aligned with our fine-grained transfer modeling. By inputting the distributional boundaries, the sentiment information can be back propagated to supervise the aspect detection process, and enhance its accuracy. However, if we only base on the distributional input, we may still face challenges due to insufficient supervisory signals for sentiment classifier. In such a scenario, providing some ground truth data can reduce the error propagation path, and thus alleviate this problem. Specifically, during training, we use an alternate training strategy: a fixed ratio (ξ) of samples are trained based on Eq. (5), while the remaining samples (1 - ξ) are optimized using Eq. (8). The model parameters are optimized through the following cross-entropy loss formulation [Chen et al., 2024a]:

where N represents the number of samples, K indicates the number of sentiment types, ˆ y(ψ) and ˆ y(ℓ) are predicted sentiment distributions, and y is the ground truth. This strategy effectively combines real labels with transferred knowledge, leveraging their synergistic integration to enhance FCKT.

Algorithm 1: Training Framework

Input: Training dataset D, learning rate α, sampling ratio ξ, model parameters Θ = {φ1, φ2, θ}

while the convergence criterion is not met do

for each batch B in Dataloader(D) do

Compute ˆ ps and ˆ pe using Eq. (2); Evaluate aspect extraction loss Lae via Eq. (3); Evaluate contrastive loss Lcl via Eq. (4); Sample a random value p ∼ Uniform(0, 1); if p > ξ then

Construct predicted E(i∼ˆ ps,j∼ ˆ pe)Hi→j ; Compute the predicted ˆ y(ℓ) via Eq. (8);

else

Select real aspect embeddingHs→e; Compute the real ˆ y(ψ) via Eq. (5);

Evaluate sentiment loss Lsp via Eq. (9); Aggregate total loss L = Lae + Lsp + λLcl; Update parameter Θ = Θ - α∇ΘL.

3.4 Model Optimization

Overall Training. To enhance the performance of FCKT framework, we jointly optimize the aspect extraction loss Lae, sentiment prediction loss Lsp, and contrastive loss Lcl. The combined objective function is defined as:

where λ is a trade-off parameter for contrastive loss. The entire optimization process is illustrated in Algorithm 1, with all parameters trained in an end-to-end manner.

Complexity Analysis. We further analyze the time and space complexities of FCKT. The time complexity is composed of three main components: (1) the aspect extraction module, with a complexity of O(n + n2 ) per training epoch; (2) the sentiment prediction module, with a complexity of O(h); and (3) the expectation input module, with a complexity of O(nh). Consequently, the overall time complexity of FCKT is O(n2 + nh). In terms of space complexity, the model incurs costs primarily from word embeddings and the contrastive learning process, resulting in a total space complexity of O(nd + n2 ), where d represents the embedding size.

4 Experiments

In this section, we conduct experiments to answer the following five research questions: RQ1: How does the proposed FCKT perform overall compared to baseline approaches? RQ2: How effectively does FCKT perform across different sub-tasks? RQ3: How do the various components of FCKT contribute to the final results? RQ4: How do different parameter configurations in the proposed method impact its performance? RQ5: Does fine-grained knowledge transfer help address the aforementioned challenges effectively?

4.1 Experimental Setup

Datasets. To fairly assess our proposed FCKT, we conducted our experiments using three public datasets, Laptop, Restaurant, and Tweets. The data statistics are presented in Table 1. More dataset details are shown in Appendix B.

Baselines. We compare three categories of models: Pipelinebased models, End-to-End models, LLM-based models. More baseline details are listed in Appendix C.

Evaluation. We adopt three widely used metrics, precision, recall, and F1 score , to evaluate the FCKT. Please refer to Appendix D for more metric details.

Implementation Details. They are provided in Appendix E, with the full Appendix available in our arXiv version.

4.2 Main Experimental Results (RQ1)

We conduct experiments on three public datasets and the comparison results between FCKT and the baselines are presented in Table 2. This table shows the following:

Among the baselines, we find that our proposed FCKT consistently achieves the best results in all cases on F1 score. More precisely, when compared to the state-of-the-art approach AIFI, FCKT demonstrates notable enhancements in F1 score performance across three distinct datasets, with an average improvement of approximately 1.38% 1 . These findings strongly suggest that the meticulously crafted FCKT exhibits a potential for superior performance. This can be attributed to two primary factors: first, the inclusion of a fine-grained knowledge transfer framework that effectively captures mutual information between sub-tasks, and second, the introduction of a token-level contrastive learning mechanism that facilitates the extraction of more fitting aspects. However, the recall of FCKT is lower than several baseline models. One possible reason is that our heuristic extraction method filters out semantically irrational aspects based on the cumulative start/end scores, which may inadvertently exclude some false negatives, leading to a lower recall score.

FCKT demonstrates notable advantages in both performance and computational efficiency compared to advanced LLMs. As illustrated in Tables 2, even the state-of-the-art LLM, GPT-4, with its vast number of parameters, fails to deliver satisfactory results for TSA, despite leveraging FewShot learning and Chain-of-Thought (CoT) [Wei et al., 2022] enhancement. Moreover, the use of LLMs incurs substantial computational overhead, further underscoring the efficiency of our approach. This suggests that improving performance in these low-resource domains with LLMs remains a challenging task. Despite advancements in LLMs, some traditional methods for TSA are still valuable, offering insights that complement the limitations of current techniques.

4.3 Analysis on Both Sub-Tasks (RQ2)

To verify model's performance in individual tasks, we conduct a comparative analysis tailored for both sub-tasks in Table 3. In AE, our proposed FCKT is able to achieve the best overall performances in terms of F1 score. Compared with the other two datasets, the improvement of model on Tweets is more obvious (+2%). Since the context length of Tweets is usually shorter, our proposed fine-grained transfer methods are more adequate between two sub-tasks, leading to more significant improvements compared to other two datasets.

1 Noted TSA remains a challenging task with minimal progress in recent research, making a 1.38% improvement significant.

For SP, FCKT consistently outperforms other approaches across the three datasets. However, its improvement on the Restaurant dataset is less pronounced compared to the other datasets. We hypothesize that this discrepancy may stem from the relatively high sample diversity in the Restaurant dataset, where the limited interaction information might be insufficient to substantially enhance sentiment prediction.

4.4 Ablation Study (RQ3)

To examine the contributions of various components, we delve further into FCKT and carry out ablation studies. The results are shown in Table 4. It is evident that the removal of specific modules leads to a decrease in model performance, highlighting the indispensable nature of each module. This underscores the critical role that various model components play in achieving optimal performance. If we delve deeper into the comparisons in TSA task, we can find the "AKT" module holds greater significance, indicated by its lower F1

score compared to the other module. This is not surprising, as cross-task knowledge transfer is the core focus of our work, emphasizing the importance of effectively leveraging mutual information between tasks to enhance performance in TSA.

4.5 Parameter Analysis (RQ4)

Effect of parameter ξ. In FCKT, a critical parameter is the ratio ξ, which controls the connection between different tasks. This parameter determines how much of the predicted aspect knowledge is transferred to the second task. We tune ξ within the range of 0 to 1, and the results, based on the TSA task, are presented in Figure 4. Across these datasets, we observe that the F1 score initially increases rapidly, then stabilizes, and finally declines as ξ increases from 0.8 or 0.9 to 1. The reason lies in that when ξ is small, the model lacks sufficient supervision from the ground truth, leading to suboptimal performance. And when ξ is very large, the model fails to leverage the valuable interactions between the two sub-tasks, also resulting in unsatisfactory performance. We observe that the best performances are usually achieved when ξ is moderate on two datasets, which implies that neither the ground truth boundary nor distributional boundary is dominantly superior, and a mixture of them can be more favorable.

Effect of parameter h. In this study, we investigate the influence of the parameter h as it regulates the length of the aspect in Eq. (8). Theoretically, when h takes on a large value, the model ends up with an enormous number of computing memory, making it challenging to achieve convergence. Conversely, when h is excessively small, accurately extracting aspects becomes non-trivial. Considering that aspect term lengths are typically not extensive, we conduct experiments by varying h within the range of {1, 2, 3, 4}. In our experimental results, presented in Figure 5, we showcase F1 scores for both the Restaurant and Laptop domains. As h increases, we observe a gradual rise in performance up to a peak, followed by a decline. This suggests that maintaining a moderate h value (such as 2 or 3) can not only reduce the parameter count but also ensure accurate aspect extraction.

Effect of parameter λ. We also analyze the effect of the contrastive weight λ on F1-score across two datasets: Restaurant and Laptop. The results indicate that the optimal performance is achieved when λ = 0.1. As λ increases beyond this point, the F1-score decreases steadily for both datasets, with a sharper decline in the Restaurant domain. This suggests that a smaller λ effectively balances the contrastive and taskspecific objectives, while higher values overly emphasize the contrastive loss, negatively impacting overall performance.

4.6 Case Study and Error Analysis (RQ5)

Case Study. To offer a deeper understanding of FCKT's exceptional performance, we present several case studies in a qualitative manner in Figure 6. In 1st case, the aspect detection and sentiment prediction in AIFI may be compromised due to weak sequential associations. For instance, it successfully identifies the aspect "chefs" but falters in accurately predicting the sentiment associated with "chefs" in the 1st case. Conversely, FCKT allows for the backpropagation of sentiment information into the boundary detection process, thereby bolstering correlations and leading to accurate predictions. By modeling semantic compatibility for aspect extraction via token-level contrastive learning, FCKT can extract more precise aspects. For example, in the 2nd case, word "separate" should not be included in the aspect of "ram memory", a distinction that FCKT successfully identifies.

Error Analysis. While the FCKT model has shown favorable performance, it still encounters challenges when handling complex sentences. For example, in the 3th sentence, due to the ambiguously expressed sentiment associated with "restaurant", the model struggles to make precise predictions.

5 Conclusion

In this paper, we addressed the problem of targeted sentiment analysis (TSA) by proposing a fine-grained cross-task knowledge transfer (FCKT) framework. By explicitly integrating aspect-level information into sentiment prediction, FCKT enables fine-grained knowledge transfer, effectively mitigating negative transfer and improving task performance. Extensive experiments conducted on three real-world benchmark datasets, including comparisons with diverse baselines and state-of-the-art LLMs validate the effectiveness of FCKT.

Acknowledgments

We sincerely thank all the anonymous reviewers for their valuable comments to improve this paper. This work was supported by the National Key Research and Development Program of China under Grant Nos. 2024YFF0729003, the National Natural Science Foundation of China under Grant Nos. 62176014, 62276015, 62206266, the Fundamental Research Funds for the Central Universities.