Integration of Old and New Knowledge for Generalized Intent Discovery: A Consistency-driven Prototype-Prompting Framework

1 Introduction

Intent detection is a core task in both Natural Language Understanding (NLU) and Task-Oriented Dialogue (ToD) systems . Its primary goal is to identify the intent or objective of a user from their natural language input. Intent detection is a critical step for dialogue systems to understand user needs and take appropriate actions. In recent years, with the rise of deep learning, data-driven fully supervised methods have achieved significant success. However, these methods largely rely on a substantial amount of in-domain annotated data and can only handle a limited set of in-domain intents [Yang et al., 2024], which poses numerous challenges in practical applications. Generally, users will always inquire about some out of scope intents after the deployment of a dialogue system, which traditional dialogue systems may fail to correctly identify and respond to [Siddique et al., 2021].

* Corresponding authors.

Therefore, to enhance the generalization, scalability, and adaptability of dialogue systems, it is necessary to study Generalized Intent Discovery (GID) [Mou et al., 2022a; Mou et al., 2023]. As the Fig.1 shows, GID uses labeled in-domain (IND) data and unlabeled out-of-domain (OOD) data for training, which aims to extend the traditional closeddomain intent set by leveraging such OOD data generated by dialogue systems. This reduces reliance on supervised data and gradually expands the system's intent detection scope, enabling it to recognize both known IND intents and unknown OOD intents, thereby improving the adaptability and intelligence of dialogue systems.

The GID task is an emerging research direction in taskoriented dialogue systems, whose core challenge lying in accurately classifying OOD intents in the absence of corresponding supervised data. Current mainstream methods are based on the EM algorithm [Dempster et al., 1977], which uses clustering methods to assign pseudo labels to unsupervised OOD data (E-Step) and then trains a joint classifier with supervised IND data (M-Step), repeating this process until the model converges. However, these methods focus on learning new knowledge in target domains while neglecting the transfer of knowledge from external sources [Wang et al., 2022; Wang et al., 2025; Wang et al., 2023]. This is manifested in two aspects. First, previous methods did not fully utilize prior knowledge when generating pseudo labels, resulting in low quality pseudo labels, introducing noise into the joint training phase and limiting the performance of the model. Second, the fine-tuning paradigm is used during model training, which cannot fully eliminate the gap between pretraining tasks and downstream tasks due to insufficient supervised data in low resource scenarios, thus inadequately transferring knowledge from the pre-trained model.

To address these challenges, we propose a Consistencydriven Prototype-Prompting framework (CPP) for GID, from the perspective of integrating old and new knowledge. Our work makes three main contributions:

• A prototype-prompt framework for transferring old knowledge from external sources, including prototypeoptimized metric learning and prior-enhanced prompt learning. We exploit large language models (LLMs) to introduce different external knowledge in prototypes and verbalizers, and explore different update methods to improve the utilization of external knowledge.
• Hierarchical consistency constraints for acquiring new knowledge from target domains, including consistency regularization and symmetric cross-prediction loss. We follow and extend the consistency principle to construct different constraints, which can also synergistically conduct metric learning and prompt learning.
• Extensive experiments on up to 9 settings with different domain setups on two datasets, whose results show that our method significantly outperforms all baselines and achieves state-of-the-art results, proving the effectiveness and generalization of our method in generalized intent discovery.

2 RelatedWork

OOD Intent Detection & Discovery. There are two tasks related to GID that are also designed to address OOD intent. OOD intent detection aims to identify whether a user query belongs to the predefined IND intent set and to reject OOD intents [Lang et al., 2023; Zheng et al., 2020]. However, the task cannot further utilize the potential information in these OOD queries. OOD intent discovery aims to cluster unlabeled OOD data and discover new intent categories [Vedula et al., 2019], iteratively optimizing intent representations and cluster assignments [Lee and others, 2013]. However, it cannot effectively integrate with existing IND intent classifiers. The former is a fully supervised task, whose main methods includes distance-based detection [Xu et al., 2020], generative model-based detection [Xu et al., 2021], and contrastive learning-based detection [Zhou et al., 2022a]. The latter is an unsupervised task, whose main methods includes K-means clustering [Lloyd, 1982], deep alignment-based clustering [Mou et al., 2022b], and contrastive learning-based clustering [Zhou et al., 2022b].

OOD Intent Detection & Discovery. GID is a semisupervised task, where the current challenge lies in the high coupling between OOD pseudo label generation and representation learning [Sohn et al., 2020; Mou et al., 2023]. Specifically, the quality of pseudo labels affects the performance of subsequent joint representation learning, while the effectiveness of representation learning influences the quality of pseudo label generation in turn.

Existing GID methods can be divided into two categories: pipeline or end-to-end. Pipeline methods are unidirectional, leading to error propagation and thus limiting model performance. Current mainstream methods are based on end-toend approaches, such as E2E [Mou et al., 2022a] and DPL [Mou et al., 2023]. E2E introduces a framework that combines pseudo label generation and representation learning for joint optimization. Additionally, it employs swap prediction for OOD data to prevent the model from collapsing into degenerate solutions during clustering. Based on the end-to-end framework, DPL further introduces contrastive learning to enhance feature representation learning for OOD data in the target domain. In summary, existing methods focus on optimizing OOD data clustering algorithms while neglecting domain adaption [Farahani et al., 2021] from external sources.

3 Consistency-driven Prototype-Prompting Framework

3.1 ProblemFormulation

Input Data. Labeled INDdata: DIND = {(xIND i , yIND i )}ni=1 , where yIND i ∈ YIND, YIND is known and |YIND| = N; unlabeled OOD data : DOOD = {(xOOD i )}mi=1 , where yOOD i is unknown, while YOOD is known and |YOOD| = M. YIND ∩ YOOD = ∅.

Objective. Use DIND and DOOD to train a joint classifier to classify input queries into the total label set Y = YIND∪YOOD.

The model training in this domain is generally divided into two stages: the pretrain stage, where the model performs fully supervised learning on the labeled IND data, and the discover stage, where the model undergoes joint training on the labeled IND data and unlabeled OOD data, and then both IND and OOD data are used for evaluation.

3.2 Overview

The overall framework of our method is illustrated in the Figure 2. For the discover stage, firstly, we design a prototypeprompt framework for transferring old knowledge from external sources, setting up two classifiers: a prototype-optimized similarity comparison module based on metric learning, and a prior-enhanced verbalizer layer based on prompt learning. Secondly, we construct hierarchical consistency constraints for acquiring new knowledge from target domains, and first set up consistency regularization. And then, we extend the idea of consistency principle and construct symmetric swapping prediction loss.

3.3 Prototype-Prompting Framework for Old Knowledge Transfer

Prototype-Optimized Metric Learning

Prototype-based metric learning [Kaya and Bilge, 2019; Wei et al., 2022] aims to classify or cluster data by learning a prototype representation for each category and measuring the distance between samples and prototypes, which is particularly effective in low-resource scenarios.

We propose external knowledge-enhanced prototype metric learning based on LLMs, whose core idea is to leverage the semantic understanding capabilities of LLMs to generate high-quality initial prototypes for each category. We experiment with category names, descriptions, representative samples, and keywords.

For prototype computation, specifically, we use the category name as the basic prototype text, denoted as C = {c1, c2, . . . , cC}. We predefine a template and use a LLM (here, DeepSeek-V3) to generate corresponding metainformation denoted as mi like explanatory descriptions, representative samples, or keywords [Li et al., 2023; Wei et al., 2024]:

where t(·) denote template function, including LLM role definition, dataset background and task description, input and output format description. And then we extract the corresponding embedding using our encoder denoted as E(·) to obtain the prototype for that category. Finally, we concatenate the prototype vectors of all categories to form the overall prototype vector:

For distance measurement, we compare the cosine similarity between the sample representation and the prototype representation to obtain its logits. For each training step, we repeat Formula (2) to update the prototype representation simply and effectively.

Prior-Enhanced Prompting Framework

Prompt learning aims to reduce the gap between pre-training tasks and downstream tasks, especially in low data scenarios, typically by restructuring downstream tasks into selfsupervised tasks of pretrained models [Brown et al., 2020; Liu et al., 2023; Ding et al., 2022]. Based on prompt learning, we predefine a template to reformulate the task into the form of Masked Language Model (MLM) [Devlin et al., 2019] head, guide pretrained models to generate the expected output and design a verbalizer for downstream adaption [Zhuang et al., 2025; Li et al., 2025], thus fully transferring the knowledge of the pretrained models. The framework consists of two main components: template and verbalizer, where we focus on the latter.

For an input text x, we embed it into a template T(x):

where '[MASK]' is the position where the model predicts the label word. We use handcrafted templates without any trainable parameters.

The verbalizer acts as a function that maps the model's predicted label words to actual labels. It plays a crucial role in transferring prior knowledge in two ways: label words and MLM head reuse.

The construction of label words is similar to the initial text of prototypes, therefore we reuse the label-related metainformation generated from the LLM in Formula (1) as label words, Let Wi denote the set of label words associated with the category label yi, and Wi = mi. Assuming each category yi has k label words, Wi can be also expressed as Wi = {wi1, wi2, . . . , wik}.

We construct a soft verbalizer to reuse the MLM head more efficiently. Concretely, we introduce a downstream linear layer Wverbalizer ∈ Rd×C , where d is the embedding dimension and C is the number of categories. The weights of this layer are initialized by a subset of the MLM head's weights. The steps are as follows:

For each category label yi, we extract the corresponding weights from the MLM head's weight matrix WMLM based on its label words Wi:

whereWverbalizer[:, i] is the weight column for label yi in the downstream linear layer, and wij represents the j-th label word for yi. And then we compute the logits for each label:

where h[MASK] is the hidden representation of the '[MASK]' token. Finally, the logits is normalized to obtain the probability distribution:

3.4 Hierarchical Consistency Constraints for New Knowledge Acquisition

In unsupervised learning scenarios, consistency regularization [Sohn et al., 2020], is a crucial technique that leverages the inherent properties of data to train models without relying on labeled data. The core idea is that for reasonable perturbations of the input data, the model's predictions should remain consistent. This allows the model to learn effective feature representations from the intrinsic structure of the data.

Consistency Regularization Loss

We propose a consistency regularization (CR) [Fan et al., 2023] loss based on the principle of consistency, including data consistency and prediction consistency.

As for the data consistency, we generate two different representation views of the same input with the dropout mechanism [Little, 1995], denoted as H1 and H2. And then we use symmetric KL divergence [Kullback and Leibler, 1951] to align the distributions of these two views:

where H1 and H2 are obtained by applying mean pooling to the final hidden states of the encoder.

As for the prediction consistency, the two classifiers, denoted as C1 and C2, process the same input and obtain two probability distributions for each view. And then we again use symmetric KL divergence to align these distributions:

where P1 and P2 are the probability distributions from the two classifiers. The final consistency regularization loss is the sum of these two losses:

Symmetric Cross-Prediction Loss

Since GID is a semi-supervised task, where unsupervised data lacks labels during training. To address this, we propose a symmetric cross-prediction loss, based on the two classifiers we constructed. Specifically, for each view:

1). Take argmax of the logits to generate pseudo labels for OOD data:

2). Combine these pseudo labels of OOD data with the true labels of IND data to form hybrid labels.

where the Hybrid1 and Hybrid2 are the hybrid labels.

3). Compute the cross-entropy (CE) loss [Shannon, 1948; Goodfellow et al., 2016] with hybrid labels and the other classifier's logits, and then symmetrically calculate and accumulate it:

Multi-View Contrastive Learning

For unsupervised data, we also incorporate a classic contrastive learning approach [Chuang et al., 2020]. Specifically, based on the two views mentioned above, we apply multiview contrastive learning with the NT-Xent loss [Chen et al., 2020]. The goal is to maximize the similarity of positive pairs (views of the same data) and minimize the similarity of negative pairs (views of different data):

where zi and ˆ zi are the feature representations of the two views of the same data (positive pair), which are obtained by projecting the averaged hidden states of last layer, sim(·) represents a similarity function (here cosine similarity), B is the batch size and τ is a temperature parameter.

Our final loss is obtained by adding up the three losses:

4 Experimental Results

4.1 Datasets and Settings

We conducted experiments on two classic intent detection datasets: Banking [Casanueva et al., 2020] and CLINC [Larson et al., 2019]. Banking is specifically designed for banking scenarios, containing 77 different banking-related intents with approximately 13,000 samples. CLINC covers 10 domains and includes 150 different intent categories, with approximately 22,500 samples.

We adopted the experimental setups from previous work [Mou et al., 2022a; Mou et al., 2023] and further extended data settings . We followed the setups of GID-SD, GID-CD, and GID-MD, which means single-domain, cross-domain and multi-domain respectively. Specifically, GID-SD refers to extracting a certain proportion of data as OOD unsupervised data on Banking, with intent categories as the smallest partition unit, GID-MD refers to performing the same operation on CLINC, while GID-CD refers to obtain such data on CLINC with domain as the smallest partition unit. The remaining data is used as IND supervised data for all the three setups. The model is required to use both IND data with label and OOD data without labels for joint training, and then is expected to predict both intents of IND and OOD. For each setup, previous work used OOD ratios of 20%, 40%, and 60%. We follow the 60% OOD ratio and expand it to 80% and 90%, make it more challenging and discriminative.

4.2 Baselines

We compared our method with the following baselines:

Kmeans [Lloyd, 1982]: A pipeline approach that first clusters OOD data with k-means to generate pseudo labels and then jointly trains with IND data.

DeepAligned [Zhang et al., 2021]: A pipeline approach that improves upon Kmeans by using the Hungarian algorithm to align cluster centers after clustering, addressing label inconsistency.

DeepAligned-Mix [Mou et al., 2022a]: A variant of DeepAligned that differs mainly in the inference phase, where predictions are made using the classification head of the new classifier.

E2E [Mou et al., 2022a]: An end-to-end approach for pseudo label generation and intent classification, using Sinkhorn-Knopp (SK) [Sinkhorn and Knopp, 1967] algorithm to optimize pseudo label assignment and employing multi-view swap prediction [Chen et al., 2020].

DPL [Mou et al., 2023]: An end-to-end approach that builds on E2E, using prototype contrastive learning to learn and generate pseudo labels for OOD data. It achieved stateof-the-art results on the previous 40% OOD ratio settings.

For fair comparison, all baselines use BERT (bert-baseuncased) [Devlin et al., 2019] as the encoders, and are evaluated in terms of accuracy and weighted F1-score follow previous settings.

4.3 Implementation Details

All text is tokenizedwithwordpieces, themaximumsequence length is set to 128, and dynamic padding is used; the batch size is set to 64, the maximum training epoch is set to 25, and the early stop is set to 10; the bert-base-uncased pretrained model is loaded as the backbone, and the dropout is set to 0.1; the optimizer is set to AdamW [Loshchilov and Hutter, 2017], the weight decay is 0.01, the initial learning rate is set to 5e-5, and a linear warm-up strategy [Goyal et al., 2017] with 500 warm-up steps is adopted; we utilized prompt-based fine-tuning and set up an additional projection layer with 256 dimensional for NT-Xent loss. To ensure comparability, we directly use the code released in previous studies, especially E2E and DPL. All of the experiments were conducted on 8 NVIDIA RTX 4090D GPUs.

4.4 Main Results

We conducted extensive experiments for the three setups (GID-SD, GID-CD, and GID-MD) with three data settings (60%, 80%, and 90% OOD ratios) . We compared the performance with baselines, and the results are shown in Tables 1, 2, and 3. Following convention, we use 5 evaluation metrics for each experiment, i.e., accuracy of IND data, accuracy and weighted-F1 score of OOD data, and overall accuracy and weighted-F1 score. Note that for the experimental results not provided by the baselines themselves, we reproduced them via their source codes.

The experimental results show that our method (CPP) significantly outperforms the baselines across all settings, achieving state-of-the-art performance. Overall, compared to the suboptimal model and in terms of total accuracy, our method leads by 8.60%, 7.17%, and 9.90%, respectively for the 60%, 80%, and 90% OOD ratios in the SD setup. In the CD setup, these numbers are 3.73%, 3.87%, and 13.11%, and in the MD setup, they are 2.45%, 1.91%, and 6.98%. These results strongly demonstrate the effectiveness and generalization of our method. Further observation reveals that our method achieves the best results on almost all metrics (40/45). The performance of different methods on IND data is roughly comparable, while our improvement on OOD data is more critical. Baseline methods tend to focus on clustering learning for unsupervised data while neglecting domain adaptation. Our improvement is attributed to the old-new knowledge fusion mechanism, which effectively transfers external knowledge and source domain knowledge to target domains while learning new knowledge from unsupervised data through hierarchical consistency constraints.

We conducted experiments on three setups: SD, CD, and MD. The performance of all methods gradually improves across these setups. This is because, in the SD setup, the model needs to recognize multiple fine-grained intents within a single domain, increasing the classification challenge. Compared to the MD setup, the CD setup makes it difficult for the model to transfer shared knowledge from source domains to the target domains due to the cross-domain setting between IND and OOD data. The improvement of our method over the suboptimal methods in these three settings generally follows this trend. For example, for the 60% OOD ratio, our method improves over E2E by 8.60%, 3.73%, and 2.45%, respectively. Additionally, we evaluated three OOD ratios: 60%, 80%, and 90%. The performance of all methods gradually declines as the OOD ratio increases, which aligns with our intuition. The most significant improvement of our method over the suboptimal model occurs at the 90% OOD ratio. The larger the OOD ratio, the more urgent the need for relevant knowledge in GID methods. For example, in the GID-CD-90% setup, our method improves over DPL by 13.11%, demonstrating that even in extremely resource-scarce scenarios, our method can effectively transfer old knowledge from external sources and learn new knowledge from OOD data.

5 Quantitative Analysis

5.1 Comparison with LLMs

In the field of Natural Language Processing (NLP), LLMs have demonstrated powerful performance across a variety of tasks. However, these models typically require substantial computational resources and data, which limits their applicability in resource-constrained scenarios. Our approach integrates both large and small language models (SLMs), leveraging the strengths of LLMs to assist SLMs in decisionmaking, thereby combining their complementary advantages to achieve low resource consumption and high performance.

For further analysis, we select the recently popular DeepSeek-V3 and the established GPT-4o as baselines, and conduct zero-shot and few-shot experiments. Note that 1shot refers that for each IND category, one random sample with the corresponding label is used , while for OOD categories, |OOD ∗ 1| samples are randomly chosen from the OOD training data in total, due to the absence of their labels.

As shown in Table 4, our method significantly outperforms these LLMs in terms of overall metrics. The accuracy of our method is higher than that of DeepSeek 1-shot by 4.02% and higher than GPT-4o 1-Shot by 4.90%. In a closer observation, this improvement is primarily attributed to the strong performance on IND data, where our method surpasses DeepSeek 1-shot by 6.49% and GPT-4o 1-Shot by 9.68%. In contrast, the gap narrows significantly on OOD data, where our method maintains a slight lead in accuracy but falls slightly behind in F1 scores. This suggests that our method still have potential for handling of OOD data. Overall, our method demonstrates superior performance on specific tasks compared to LLMs, particularly in resource-constrained scenarios, making it a competitive choice.

5.2 Utilization of Meta-information

In few-shot learning and zero-shot learning scenarios, the performance of SLMs is often limited by their constrained parameter scale and training data. To enhance the performance of small models in complex tasks, an effective approach is to leverage meta-information related to labels generated by LLMs to augment the decision-making capabilities of small models. This method not only utilizes the powerful semantic understanding abilities of large models but also compensates for the shortcomings of small models in data-scarce situations through knowledge transfer. We conduct this method by injecting label-related descriptions, such as paraphrases, keywords, and examples, into the prototype and verbalizer of our SLM. As shown in Table 5, the experimental results demonstrate that this method significantly improves the performance of small models, highlighting its important application value in resource-constrained scenarios.

Specifically, the performance across all configurations on IND data remains relatively stable, indicating that the enhancement effect of meta-information on IND data is limited, with the primary improvement observed on OOD data. When only label names or paraphrases are used, the performance on OOD data is relatively low, suggesting that they provide limited semantic information. Keywords and examples are more effective. In the GID-SD-60% setup, as examples increases, the performance on OOD data improves significantly, reaching its peak when using 4 examples. This suggests that a greater number of examples can provide richer contextual information. However, when examples increases to 5, the performance slightly declines. It is worth noting that we conducted this experiment across all other setups as well, and the conclusions vary slightly across different setups. In 6 out of 9 setups, using 5 examples yields the best performance.

5.3 Ablation Study

Parameter Settings

Prompt learning and parameter-efficient fine-tuning have emerged as highly regarded research directions in recent years. The effectiveness of prompt learning heavily depends on the parameter settings. Therefore, we first conducted parameter update experiments, as shown in Table 6. We systematically explore the impact of different parameter configurations on model performance, through ablation experiments on the parameter settings of our prompt framework, the encoder, and the verbalizer. PLM Freeze v1 refers to freezing all parameters in the pretrained language model (PLM), while PLM Freeze v2 refers to unfreezing its last layer on the basis of v1.

Our method, which combines hard templates, promptbased fine-tuning, and soft verbalizers, achieves the best performance. Upon closer examination, although completely freezing the PLM parameters leads to a significant performance drop, the model still achieves an accuracy of 47.44%, demonstrating that our framework is highly effective in transferring knowledge from external sources into target domains. Both manually designed and rule-based PTR [Han et al., 2021] verbalizers perform poorly, indicating their lack of flexibility. In contrast, the soft verbalizer dynamically generates label word mappings, better capturing task semantics and significantly improving the model's performance on both IND and OOD data.

Loss Function Settings

To better acquire new knowledge from target domains, we designs three types of loss functions: consistency regularization loss (including data consistency loss and prediction consistency loss), symmetric cross-prediction loss, and contrastive learning loss. Each loss function plays a distinct role in model training. Through ablation experiments, we can clearly evaluate the specific contribution of each loss function to model performance, thereby validating their rationality and necessity. By analyzing the impact of each loss function on model performance, we can further optimize the weights of the loss functions or design more efficient training strategies.

As shown in Table 7, the ablation experiment results demonstrate that each of these loss functions contributes to model performance to varying degrees. Specifically, removing the contrastive learning loss results in the most significant performance drop on OOD data, indicating its irreplaceable role in enhancing the feature representation of unlabeled data. Following that, the symmetric cross-prediction loss also shows its importance in improving the model's classification capability on unlabeled data. Additionally, the data consistency loss plays a certain role in enhancing the quality of representation learning, while the prediction consistency constraint helps reduce the negative impact of pseudo-label noise on the model.

6 Conclusion

In this study, we propose our method from the perspective of integrating old and new knowledge, which includes a prototype-prompt framework for transferring old knowledge from external sources and hierarchical consistency constraints for acquiring new knowledge in target domain. For the former, we introduce LLMs to generate label-related meta-information and apply it into metric learning and prompt learning to reinforce SLMs decisions, thereby achieving complementary advantages of large and small PLMs. For the latter, we synergistically conduct prompt learning and metric learning to further improve performance. We conducted extensive experiments and analysis on various settings to validate the effectiveness and generalization of our method. In the future, we will attempt to load more powerful LLMs to explore parameter-efficient fine-tuning methods in this scenario, further improving its performance under extremely low resource conditions.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (No. 62302333 and No. U23B2053) and the Open Research Fund from Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) (No.GMLKF-24-16).