Mitigating Entity Type Confusion in Cross-Domain NER via Multidimensional Quantification and Reasoning Enhancement

1 Introduction

Named Entity Recognition (NER) is a core task in information extraction, which aims to identify specific entities in texts that belong to predefined types, such as person, location, and organization [Li et al., 2022; Hu et al., 2024; Esmaail et al., 2024]. NER plays a critical role in various tasks, including information retrieval [Long et al., 2024; Cong et al., 2023; Nguyen et al., 2024], knowledge graphs [Chen et al., 2024; Jin et al., 2022; Gao et al., 2025], and recommendation [Li et al., 2024; Li et al., 2025]. With large-scale annotated data, neural network-based methods have performed remarkably in traditional NER [Zhu and Li, 2022; Shen et al., 2023; Liu et al., 2022]. However, in cross-domain NER, several challenges persist, especially in low-resource scenarios where model performance experiences a significant decline.

∗Corresponding authors.

Figure 1: song and album share similar contextual features and entity forms, which can lead to confusion.

Recent studies [Xu and Cai, 2023; Zhang et al., 2024a; Xu et al., 2024] decompose cross-domain NER into subtasks to capture transferable patterns across domains. These methods focus on identifying shared feature patterns between domains through specialized components to strengthen sourcedomain knowledge transfer. Other approaches [Chen et al., 2023; Zhang et al., 2024b; Nandi and Agrawal, 2024] employ generative pre-trained language models, transforming NER into a generation task through the use of task-specific input prompts. These methods can effectively utilize largescale knowledge from pretraining and exhibit high flexibility. Although both paradigms have demonstrated effectiveness in improving cross-domain NER performance, they remain susceptible to entity type confusion, leading to significant performance degradation in entity type classification.

major challenges. (1) Granularity differences between coarse-grained entity types in the source domain and finegrained entity types in the target domain. Entity types in the source domain are coarse-grained, such as PER and LOC, whereas entity types in the target domain are more specific and domain-specialized, such as scientist and country. Compared to coarse-grained types, distinguishing fine-grained types often requires more detailed and nuanced features. Relying solely on the coarse-grained features learned from the source domain makes it difficult for the model to accurately classify fine-grained types in the target domain, such as song and album (as shown in Figure 1). (2) The lack of contextual information in the text further limits the model's ability to classify fine-grained entity types accurately. In the target domain, some samples lack the necessary contextual cues, making it difficult for the model to effectively learn and capture the required fine-grained features for type classification. As shown in Figure 1, we selected a sample from CrossNER [Liu et al., 2021], where the song and the album are two entity types that are difficult to distinguish. Entities of both types share similar forms and contextual environments. Due to insufficient information in the text, the model misclassifies these entities as the song. The lack of adequate contextual information prevents the model from effectively learning the feature differences between similar fine-grained entity types, which reduces the precision of type classification.

To address the challenges in cross-domain NER, we propose the Multidimensional Confusion Quantification Model (MCQM) and the Progressive Bidirectional Reasoning Chain (PBRC). (1) We propose the MCQM that quantifies the model's confusion extent from three dimensions: sourcetarget hierarchy analysis, semantic similarity analysis, and explicit data evaluation. MCQM enables in-depth analysis and quantification of the model's confusion extent between fine-grained entity types in the target domain, providing a foundation for addressing the entity type confusion. (2) Furthermore, we innovatively leverage the source-target hierarchy, along with the confusion analysis from the MCQM, to propose the PBRC, which employs a two-stage reasoning strategy. Specifically, PBRC first instructs the LLM to perform initial reasoning based on contextual information using coarse-grained entity types from the source domain. Then, it instructs the LLM to perform bidirectional reasoning using fine-grained entity types from the target domain and confusion analysis from the MCQM. Finally, the LLM generates two-stage reasoning information for knowledge augmentation. The two-stage reasoning strengthens the fine-grained features and alleviates the granularity gap between the source and target domains, while external knowledge provided by the LLM solves the issue of insufficient contextual information. Our method significantly mitigates entity type confusion. Experimental results demonstrate that our method effectively improves the model's generalization ability, with the average F1 score increasing by more than 10.00%. In summary, our contributions are as follows:

• We systematically analyze the entity type confusion in cross-domain NER. Then, we propose the MCQM to quantify the model's confusion extent effectively.
• We propose the PBRC based on the source-target hierarchy and confusion analysis from the MCQM, which can leverage the LLM to mitigate entity type confusion in cross-domain NER significantly.
• Extensive experiments show that our method significantly improves the model's generalization ability, achieving new state-of-the-art performance across all domains of the CrossNER dataset.

2 RelatedWork

Named Entity Recognition (NER). Traditional methods rely on handcrafted features or rules combined with machine learning models like CRF [Lafferty et al., 2001] for entity recognition, which are labor-intensive and lack adaptability to complex contexts. With the development of deep learning, neural network-based methods, such as LSTM [Huang et al., 2015] and transformer [Vaswani et al., 2017] models, have become mainstream. Models [Raffel et al., 2020; Radford et al., 2019; Devlin et al., 2019] based on these architectures capture sequential relationships and contextual dependencies, offering improved performance and flexibility. However, existing methods still face challenges in crossdomain NER due to the variations across different domains.

Cross-Domain NER. Current studies can be broadly categorized into two paradigms: decomposition-based and generation-based. Decomposition-based methods [Xu and Cai, 2023; Xu et al., 2024; Zhang et al., 2024a; Zhang et al., 2022] reformulate cross-domain NER into multiple subtasks. These methods improve knowledge transfer by learning shared cross-domain patterns through specialized subtask modules. Generation-based methods [Bao and Yang, 2024; Chen et al., 2023; Zhang et al., 2024b; Nandi and Agrawal, 2024] transform the NER into a generation task. These methods employ prompts containing task descriptions and auxiliary information, achieving robust performance through instruction fine-tuning. Although both paradigms effectively improve cross-domain performance, particularly in entity span detection, they still suffer from significant accuracy degradation in entity type classification.

LLMs for NER. With the emergence of LLMs, their reasoning capabilities have been leveraged to improve NER performance. Recent studies [Ashok and Lipton, 2023; Wang et al., 2025; Zhu et al., 2024; Ji, 2023] employ prompt learning for NER, where task-specific templates are augmented with demonstration examples and knowledge. LLMs are also used for cross-domain NER. Nandi and Agrawal [Nandi and Agrawal, 2024] perform instruction fine-tuning of LLMs by retrieving and leveraging similar examples. Zhang et al. [Zhang et al., 2024b] utilize the LLM to generate taskoriented knowledge, used to conduct additional task-oriented pre-training of the backbone model for domain adaptation. These methods demonstrate strong few-shot performance.

Compared to fine-tuning LLMs [Nandi and Agrawal, 2024] and additional pre-training corpora [Zhang et al., 2024b], our method reduces training time for domain adaptation.

3 Methodology

3.1 Task Description

Given a sentence X = {x1, x2, . . . , xN}, where wi represents the i-th token in the sentence and N is the sentence length, the goal of NER is to identify all entities E within the sentence and classify each entity into a specific type. The set of entity types is denoted as L. Each entity ei ∈ E can be represented as ei = (y, xl:r), where y ∈ L indicates the entity type, and l and r represent the start and end boundary indexes of the entity within the sentence, respectively. In the cross-domain NER, two distinct datasets are considered: the source domain dataset Dsrc and the target domain dataset Dtgt. The set of entity types in the source domain is denoted as Ls = {s1, s2, . . . , sn}, and the set of entity types in the target domain is denoted as Lt = {t1, t2, . . . , tm}. The goal is to leverage the knowledge learned from Dsrc to improve recognition performance on Dtgt. Specifically, we focus on low-resource scenarios, where the training data in the target domain is significantly smaller than that in the source domain, i.e., |Dtgt| ≪ |Dsrc|.

3.2 Multidimensional Confusion Quantification Model

Entities of the same type share certain common features, referred to as entity type features, and entity types with similar features are more prone to confusion. We quantify entity type confusion from three dimensions in Figure 2.

Source-Target Hierarchy Analysis

Entity types in the source domain are conceptually coarsegrained, whereas those in the target domain are fine-grained. In the feature space, the fine-grained entity types in the target domain can be viewed as extensions of the entity types in the source domain. For example, PER in the source domain can be expanded into fine-grained entity types in the target domain, such as politician and scientist. The expanded entity types retain the coarse-grained features in the source domain. Therefore, fine-grained entity types derived from the same entity type in the source domain exhibit similar features, meaning closer in the feature space and more prone to confusion among these entity types.

To quantify the model's confusion arising from the sourcetarget hierarchy between the source and target domains, inspired by [Bao and Yang, 2024], we propose a reliable quantification method. For each entity in the target domain, its feature vector He can be considered as a combination of coarsegrained features Hcoarse and fine-grained features Hfine.

We only focus on extracting Hcoarse. We utilize the model Msrc, trained on Dsrc, to extract Hcoarse of entities in the target domain and perform predictions based on the source domain type set Ls. We calculate the proportion of entities corresponding to each fine-grained type ti in the target domain Dtgt that are classified as the coarse-grained type sj in the source domain Dsrc:

where P(ltgt = ti) is the proportion of entities in Dtgt that belong to type ti, P(ltgt = ti ∧ lpred = sj) is the proportion of entities in Dtgt that belong to type ti and are classified as sj, and the proportion of entities of type ti classified as sj is denoted as RH(ti → sj ).

We use the source domain entity type lsrc with the highest proportion as the prefix for the target domain entity type ltgt, denoted as lsrc → ltgt. The target domain entity type ltgt is considered a fine-grained extension of the source domain entity type lsrc:

where prefix(ti) is the prefix of ti, and Ls is the set of entity types in the source domain.

RH(ti → sj) measures how much of the coarse-grained features of sj are contained in ti. The more entities corresponding to ti are classified as sj, the more coarse-grained features of sj are contained in ti. Therefore, for two finegrained entity types, ta and tb, if they share the same prefix, it implies that ta and tb exhibit a significant overlap in their coarse-grained features, which originate from the same source domain entity type. This overlap increases the likelihood of confusion between ta and tb. We quantify the confusion arising from the source-target hierarchy using RH(ti → sj), based on the prefixes of fine-grained entity types:

where u(ti ,tj ) hie is the extent of hierarchical confusion from the fine-grained entity type ti to tj, and δhie is the scaling factor. This method quantifies the model's confusion arising from the source-target hierarchy, serving as an important metric in the MCQM for quantifying entity type confusion.

Semantic Similarity Analysis

The semantics of entity types reflect the common features of their entities. When the model maps the embedding vectors of the semantics of two entity types to closer positions in the semantic space, it considers the entity features of the two types as more similar, which increases the probability of confusing the two types in classification.

Since entity type labels are short words, they cannot sufficiently reflect the features of the entity types. To enrich the semantics of each target domain entity type ti ∈ Lt, we utilize the WordNet [Fellbaum, 2010] to obtain a brief description Cs of ti. Then, Cs is further refined into a final description Ce of ti by leveraging the LLM. Then, Ce is fed into the encoder of the model, from which the hidden layer feature sequence H = [h1, h2, . . . , hZ] ∈ RZ×d can be extracted:

where hi denotes the feature vector from the final hidden layer for the i-th token, Z is the number of tokens, and d represents the dimension of the final hidden layer of the encoder. This study [Reimers and Gurevych, 2019] demonstrates that Mean Pooling outperforms both [CLS] token and Max Pooling strategies in semantic textual similarity tasks. Therefore, we use Mean Pooling to calculate the global feature vector H, which better integrates the semantic information of all tokens and provides a more comprehensive representation of the entity type features:

where mi is the mask value of the i-th token, used to ignore the influence of padding.

In high-dimensional space, the features of entity types are amplified, and the mapping rules for similar features are also similar. Therefore, we calculate the semantic similarity between entity types using H. We quantify the confusion arising from the semantic similarity by computing the cosine similarity between the global feature vectors H of entity types:

where u(ti,tj) sim is the extent of semantic confusion from the fine-grained entity type ti to tj. This method effectively quantifies the model's confusion arising from the semantic similarity between entity types, serving as an important metric in the MCQM for quantifying entity type confusion.

Fusion of Implicit Confusion Factors

The quantification of confusion arising from the source-target hierarchy and the semantic similarity of entity types is considered implicit confusion analysis. We use two fixed mixing ratios, α and β, to balance the weights of these two factors:

Then, we use the softmax function to model the implicit confusion and define the implicit confusion distribution among fine-grained entity types in the target domain:

where w0 ∈ R+ is a temperature coefficient.

Explicit Data Evaluation

We denote the model obtained by fine-tuning Msrc on the target domain dataset as Mtgt. We use the dev set of the target domain to perform a statistical analysis of the misclassification proportions for all entity types. Evaluating Mtgt reflects the model's ability to distinguish between different fine-grained entity types and explicitly reveals the model's confusion extent between entity types:

where P(ltrue = ti) is the proportion of entities that truly belong to type ti, P(ltrue = ti ∧ lpred = tj) is the proportion of entities that truly belong to type ti but are misclassified as tj , and the proportion of entities of type ti misclassified as tj is denoted as R(ti,tj) M .

The value of R(ti,tj) M explicitly reflects the model's learning effectiveness in distinguishing the fine-grained features of ti and tj. A larger value indicates that ti and tj are more prone to confusion. We use R(ti,tj ) M as an important metric for confusion analysis in the MCQM. Then, we use the softmax function to model explicit confusion and define the explicit confusion distribution among fine-grained entity types in the target domain:

where w0 ∈ R+ is a temperature coefficient.

Confusion Analysis Results

We integrate the implicit and explicit evaluations to obtain the final confusion analysis results for fine-grained entity types in the target domain:

where vconf(ti → tj) is the model's confusion extent from ti to tj. For each type ti, we select the entity type with the highest confusion extent as its most easily confused type:

where T (ti) conf is the most easily confused entity type for the entity type ti. MCQM provides a comprehensive analysis of the model's confusion extent between fine-grained entity types, playing a critical role in the bidirectional reasoning of PBRC.

3.3 Progressive Bidirectional Reasoning Chain Contents of PBRC

We propose PBRC based on the source-target hierarchy and the confusion analysis from MCQM. As shown in Figure 2, PBRC prompts the LLM to output two-stage reasoning information. Figure 3 shows the contents of PBRC.

Step1: Identifying Named Entities. This step instructs the LLM to identify all potential named entities in the text. With pre-trained multi-domain knowledge and contextual understanding, the LLM is capable of recognizing potential entities in texts across various domains.

Step2 and Step3: Coarse-grained Classification and Initial Reasoning. We provide all source domain coarsegrained entity types and explanations. For each entity in Step1, we instruct the LLM to conduct coarse-grained classification and provide explanations, leveraging both its prior knowledge and an initial analysis of the context. This step unifies entity types across different domains, serving as an initial classification and reasoning process for entities.

Step4 and Step5: Fine-grained Classification and Bidirectional Reasoning. We provide all fine-grained entity types of the target domain along with their corresponding source domain prefixes and explanations. For each entity, we instruct the LLM to conduct fine-grained classification based on the initial classification and reasoning information from Step2 and Step3, and then conduct bidirectional reasoning through a deep analysis of contextual information and the LLM's rich external knowledge. As shown in Figure 3, we present the confusion analysis result of the MCQM, where A → B indicates that A's most easily confused type is B. Bidirectional reasoning consists of two parts: (1) forward reasoning, which infers that the entity type is A; (2) backward reasoning, which infers that the entity type is not B.

Two-stage Reasoning Information. The LLM only outputs the initial reasoning from Step3 and the bidirectional reasoning from Step5. The two-stage reasoning information is fed into the backbone for knowledge augmentation: the initial reasoning guides the backbone to leverage the knowledge from the source domain, strengthening knowledge transfer; the bidirectional reasoning guides the backbone to differentiate easily confused entity types, mitigating the entity type confusion. Besides, external knowledge mitigates the insufficiency of contextual information.

Knowledge Augmentation

The two-stage reasoning information generated by the LLM is utilized to augment the knowledge of Msrc. Through supervised fine-tuning, the backbone learns to leverage this reasoning information, yielding the final backbone model M†tgt .

• Input: Find all entities of types {politician, person, country ...} in {text}.

Reasoning information: {...}.

Output format: {"type 1": ["entity 1", "entity 2"], "type 2": ["entity 3"], ...}.

• Gold Sequence: {"country": ["Afghanistan"], "politician": ["Barack Obama"], ...}.

4 Experiments

4.1 Experimental Setup

As shown in Figure 1, we evaluate on two source domains (CoNLL2003 [Sang and De Meulder, 2003] and Twitter [Lu et al., 2018]) and five target domains (CrossNER [Liu et al., 2021]: politics, science, music, literature, and AI). We employ Flan-T5 [Chung et al., 2024] as the backbone model and GPT-4o-mini as the LLM. To evaluate the performance of our proposed method, we compare it with the following baselines: (1) MTD [Zhang et al., 2022]: A modular learningbased method that decomposes NER into span detection and type classification. (2) CP-NER [Chen et al., 2023]: Transforms NER as text-to-text task and introduces collaborative domain-prefix tuning based T5 as well. (3) DH-GAT [Xu and Cai, 2023]: Applies Graph Attention Networks to encode syntactic and semantic information while embedding words into hyperbolic space. (4) PromptNER [Ashok and Lipton, 2023]: Uses GPT4 for NER through prompt templates. (5) Dual-CL [Xu et al., 2024]: Uses dual contrastive learning to refine ambiguous representations and learn generalizable features. (6) DT-MPrompt [Zhang et al., 2024a]: Splits the cross-domain NER task into subtasks and uses separate functional modules for learning and knowledge transfer. (7) IF-WRANER [Nandi and Agrawal, 2024]: Finetunes 7B LLaMA with instruction fine-tuning and employs word embeddings to retrieve examples for in-context learning. (8) TOPT (SOTA) [Zhang et al., 2024b]: Utilizes LLaMA to generate task-oriented knowledge for flan-T5 and adopts task-oriented pre-training for domain adaptation.

4.2 Main Results

The main results are shown in Table 2.

CoNLL2003 as the Source Domain. It is observed that the F1 score improves as the parameter scale of the model increases. On average, our method achieves consistent F1 score improvements of +1.61% (Flan-T5-base) and +3.01% (FlanT5-large). Our method surpasses prior SOTA results in science, politics, literature, and AI domains (effective for both Flan-T5-base and Flan-T5-large), achieving improvements exceeding 2.10% in these domains. These results demonstrate the strong effectiveness of two-stage information generated by the LLM in cross-domain NER. We observe that IF-WARNER-7B (85.43%), which employs a fine-tuned 7B LLaMA, and PromptNER (84.26%), which utilizes GPT-4, both achieve significantly higher F1 scores in the music domain compared to other baselines using smaller models. This demonstrates the advantage of LLMs in the music domain, ensuring the quality of the information generated by our twostage process using the LLM. We observe that Flan-T5-large achieves only a marginal improvement of 0.19% over prior SOTA in the music domain, while Flan-T5-base fails to surpass the prior SOTA. Through careful analysis, we have identified the main reasons. First, the high overlap between the domain knowledge of the LLM and Flan-T5 in the music domain diminished their complementary effects, resulting in limited benefits from two-stage reasoning. Then, though the two-stage information significantly improves F1 scores for certain entity types, their limited entity count results in a negligible impact on the overall F1 metric.

Twitter as the Source Domain. Our method consistently achieves significant improvements across all domains. Since TOPT is not tested with Twitter as the source domain, we cannot compare with it directly. Compared to CoNLL2003, the performance of both Flan-T5-base and Flan-T5-large shows a slight decline. We attribute this to the smaller dataset size of Twitter, which limits the model's ability to learn coarse-grained features of entities. This highlights the importance of learning the coarse-grained features of entities and further demonstrates that our method enables effective knowledge transfer from the source to the target domain.

4.3 Entity Type Confusion Analysis

To verify that the model's performance improvement is attributed to enhanced entity type classification capabilities, we evaluate its performance in both entity span detection and overall performance using precision, recall, and F1 score. The results are shown in Table 3, where the values in parentheses indicate the performance on entity span detection. Before incorporating MCQM and PBRC, we observe a substantial performance gap between the entity span detection and overall performance, which indicates that the model's ability of entity type classification is the primary bottleneck, suffering from significant entity type confusion. After incorporating MCQM and PBRC, we observe that our method improves the F1 score for entity span detection by 2%-4% across the five domains, with slight improvements also observed in precision and recall. More notably, the overall performance increases by over 10% on average. These results demonstrate that our method significantly enhances the model's ability in entity type classification and effectively mitigates the model's entity type confusion.

4.4 Ablation Study

Ablation Study on MCQM

Table 4 presents the effects of MCQM components: semantic similarity (SS), source-target hierarchy (STH), and data evaluation (DE), on model performance (F1 score). By comparing the results of completely removing all components (w/o) with those of introducing only SS, STH, or DE, we observe that, across all target domains, incorporating SS, STH, or DE into confusion analysis and subsequently introducing backward reasoning based on the corresponding analysis improves model performance. This indicates that semantic similarity analysis, source-target hierarchy analysis, and explicit data evaluation can all effectively quantify the entity type confusion extent. Moreover, simultaneously introducing SS and STH (+SS&STH) or incorporating all components (+ALL) into confusion analysis further improves model performance, suggesting that leveraging multiple analytical dimensions can reduce the errors associated with a single dimension and enhance the quantification accuracy of MCQM.

Ablation Study on PBRC

We conduct ablation studies to evaluate the effectiveness of each reasoning component in PBRC: forward (FR), backward (BR), and initial reasoning (IR). As shown in Table 5, adding forward reasoning (FR) alone yields an average performance gain of 6.55%. This demonstrates that the reasoning information provided by the LLM can effectively support the backbone model in performing NER. Further incorporating initial reasoning (IR) brings an additional average improvement of 1.35%, demonstrating that the two-stage reasoning from source to target domain can enhance the backbone's crossdomain transferability. Moreover, when backward reasoning (BR) is incorporated, the average performance rises by 3.54% compared to using FR alone. This indicates that the MCQM effectively analyzes and quantifies the model's confusion among entity types, while backward reasoning can alleviate entity type confusion.

4.5 Runtime Analysis

We report the average training time of different methods across the five target domains, the average F1 score on the dev set, and the inference time per text (in seconds), as shown in Table 6. First, although our method requires the LLM to generate reasoning information for the training set in advance, the overall training time is still lower than that of using Flan-T5 alone. We attribute this to the fact that, once reasoning information is introduced, Flan-T5 converges faster and reaches the best F1 score of the dev set with fewer epochs. Besides, for the inference time of each text, our method first leverages the LLM to generate reasoning information before feeding it into Flan-T5 for NER. Since Flan-T5 has a very short inference time, the final inference time of our method is comparable to that of the LLM. Our method maintains training time comparable to that of using Flan-T5 alone, attains inference efficiency that is nearly equivalent to the LLM, and achieves performance that substantially exceeds both Flan-T5 and the LLM when employed alone.

5 Conclusion

In this paper, we propose the MCQM, which effectively quantifies the model's confusion extent among fine-grained entity types from three dimensions: source-target hierarchy analysis, semantic similarity analysis, and explicit data evaluation. Furthermore, we propose the PBRC based on the sourcetarget hierarchy and the MCQM, which can significantly mitigate entity type confusion and improve the model's generalization ability. Our method achieves SOTA results on all domains of the CrossNER dataset.

Acknowledgments

This work is partially supported by the National Natural Science Foundation of China (No. 62272045).