Instance Relation Learning Network with Label Knowledge Propagation for Few-shot Multi-label Intent Detection

1 Introduction

Multi-label Intent Detection (MID) [Zhang et al., 2021; Pham et al., 2023; Tu et al., 2023] is a crucial component in dialogue systems [Zhang et al., 2020; Han et al., 2021], which aims to identify multiple user intents in a given utterance. For example, in the sentence "What time is my meeting and what day?", the MID task could detect two user intents, "request time" and "request day". With conversational artificial intelligence, MID has attracted widespread attention in academia and industry. Previous methods heavily rely on large amounts of labeled data, but their performances drop significantly in low-resource dialogue domains. Few-shot MID is indispensable because it can detect multiple intents of utterances when only a few labeled data are provided.

∗Corresponding author

Existing methods [Hou et al., 2021] mainly focus on a twostage pipeline: (1) utterance representation learning and (2) multi-label results identification. Specifically, they first learn different representations for an utterance with multiple intents and then use these representations to obtain multi-label results through a threshold-based strategy. However, these methods heavily rely on the representation classification of utterances, leading to error propagation easily. For example, in Figure 1, they are difficult to obtain well-separated representations for intents "request time" and "request date" because the utterance representations of these two intents could be closer to each other due to originating from the same utterance and expressing similar label semantics, as well as "request traffic" and "request route". Affected by multi-label noise, they may fail to obtain reliable representations, leading to low performance. Although DCKPN [Zhang et al., 2023] learns utterance representations with class information to alleviate this issue, it still underperforms in this case due to error propagation.

To solve the above issues, we are the first to propose a multi-label joint learning method for few-shot MID, which designs an instance relation learning network with label knowledge propagation to directly gain multi-label results in an end-to-end manner, addressing error propagation. Concretely, the proposed method explicitly models the relationship between intra- and inter-class instances and propagates label knowledge between instances to capture strong interaction. With label knowledge propagation, the proposed method could use the relation strength between an instance pair to indicate whether these two instances belong to the same label well for multi-label inference, eliminating the negative effect caused by error propagation. Besides, a dual relation-enhanced loss is designed to enforce support-level and query-level relation strength to further improve performance. The support-level loss enhances the relation strength between instances with the same label connected to the support instance and weakens the relation strength between those with different labels to promote label knowledge propagation. The query-level loss encourages the maximization of relation strength between a query instance and its multiple relevant support instances while minimizing relation strength between the query and its irrelevant support instances to promote multi-label prediction. Extensive experiments show that our proposed method performs well, especially an average improvement of 11.19%Macro-F1 score in the 1-shot setting. The contributions are summarized as follows:

• We propose a multi-label joint learning method for fewshot MID, which constructs an instance relation learning network with label knowledge propagation to guide query inference explicitly. The method eliminates error propagation and works well in low-resource domains.
• We model the relationship between intra- and interinstances to directly indicate whether two utterances belong to the same intent for multi-label prediction instead of depending on representation classification.
• We design a dual relation-enhanced loss to optimize the support- and query-level interaction relations between instances and strengthen label knowledge propagation to further improve performance.
• Extensive experiments show that our proposed method outperforms strong baselines and obtains significant performance on the few-shot MID task.

2 RelatedWork

2.1 Few-shot Learning

Few-shot learning has achieved great progress in Computer Vision (CV) [Ouyang et al., 2022; Chen et al., 2025] and Natural Language Processing (NLP) [Li et al., 2023; Chan et al., 2023; Zhou et al., 2025]. Meta-learning is the mainstream method [Zhang et al., 2022b; Liang et al., 2023; Zhou et al., 2024b; Zhou et al., 2024a] for few-shot learning, and it includes model-based [Tsendsuren and Hong, 2017], optimization-based [Lee et al., 2019], and metricbased [Wang et al., 2021; Lv et al., 2021; Zhao et al., 2022a; Chen et al., 2023] methods. Among these methods, metricbased methods are the most popular and potential research works. However, few-shot learning methods mostly focus on single-label prediction [Zhang et al., 2022a; Yehudai et al., 2023; Du et al., 2023], where they assign an intent to each utterance. Obviously, these methods become problematic in real-world scenarios because an utterance may have multiple intents. Few-shot MID could be regarded as a multi-label classification task. Previous few-shot multi-label classifications focus on CV domain [Yan et al., 2022], audio domain [Cheng et al., 2019], and sentiment analysis [Hu et al., 2021; Zhao et al., 2023a]. Few-shot MID is still in its infancy, and a few methods are available.

2.2 Few-shot Multi-label Intent Detection

Existing methods mostly focus on a two-stage pipeline. Hou et al. [2021] learn instance representations for multiple labels in the first stage. The second stage computes the relevance scores between the learned representations and different labels. Then, it uses a meta-calibrated dynamic threshold to perform multi-label classification based on relevance score ranking. However, they ignore the interaction relations between instances and do not handle the identical representation of instances with multiple labels. Graph Neural Networks (GNNs) [Zhao et al., 2023b; Sun et al., 2023; Chen et al., 2022] have shown great progress in modeling the relationship between instances using graph structure. Zhang et al. [2023] propose a dual-class knowledge propagation network, which uses an instance-level GNN and a class-level GNN to learn instance representations. Then, they use a label count estimation to predict the label number of each instance. However, these methods rely on the representation classification in a pipeline, inevitably causing error propagation.

Different from the aforementioned methods, our proposed method is the first to propose multi-label joint learning for few-shot MID in an end-to-end manner. We design an instance relation learning network with label knowledge propagation to explicitly explore the interaction relations between instances, which eliminates the negative effect caused by error propagation and works well in low-resource domains.

3 Methodology

3.1 Task Formulation

We formulate the few-shot MID task in meta-learning, which is commonly employed in current studies [Yu et al., 2022; Fang et al., 2023; Ma et al., 2023] to generalize prior knowledge to a low-resource target domain from resource-rich source domains. Our proposed method trains the model in a source domain Dsource by building a collection of metatasks and tests it in a target domain Dtarget with other metatasks. Each meta-task consists of a support set and a query set. Specifically, the support set S = {(xi, yi)}N×K i=1 includes N classes (N-way), and each class has K instances (K-shot). As a multi-label task, few-shot MID could assign more than one label to each instance. Briefly, for an instance xi, its corresponding label is yi = {y1 i , y2 i , ..., yN i } ∈ {1, 0}N , where yck i = 1 represents xi belongs to class ck. Besides, Q = {(xj)}Tj=1 is called the query set, and T is the number of query instances. The training objective is to minimize the query loss of Q based on S in Dsource. In the testing phase, we construct S and Q in Dtarget. Therefore, the proposed method predicts the query labels of Q based on S to verify the effectiveness of the model in the target domain Dtarget.

3.2 Overall Framework

This section describes the proposed method for few-shot MID, as illustrated in Figure 2. We use the support set and the query set to construct a fully connected instance relation graph. Each node is an utterance, and each edge represents the relation strength between two connected nodes. In the graph network, our proposed method explicitly models the relationship between intra- and inter-class instances, which propagates label knowledge from support instances to query instances. Besides, a dual relation-enhanced loss is introduced to promote inference by enhancing support-level relation strength and query-level relation strength.

3.3 Instance Feature Extraction

Given an utterance x with n words, the utterance is defined as x = {w1,w2, ...,wn}. We use a pre-trained text encoder like BERT [Devlin et al., 2019] to encode each word and generate hidden states Hx = [h1, h2, ..., hn] ∈ Rn×d . d is the dimension of hidden states. Besides, the corresponding class description c with m words is encoded into hidden states Hc = [h1, h2, ..., hm] ∈ Rm×d .

Support Instance Feature

As an utterance has multiple intents (a.k.a. labels), it could be classified into different classes. Support utterances for each class could express better class-related intent semantics. To extract class-related semantic features, we concatenate a support utterance and its class text Hs = [Hx : Hc ] ∈ R(n+m)×d and utilize a self-attention mechanism [Yan et al., 2020; Liu et al., 2022; Du et al., 2022] to learn attention weights A for each word, which generates the final representation of the support utterance. Specifically,

where us ∈ Rd is the final feature representation of the support utterance. W1, W2, W3, b1, b2 and b3 are trainable parameters.

Query Instance Feature

For each query utterance, a mean pooling layer [Chen and Sun, 2023] is utilized to calculate the average of hidden states across all words, serving as the feature representation of the query utterance.

where uq ∈ Rd is the query feature.

3.4 Instance Relation Graph Construction

Graph structure is beneficial for exploring the interaction relations between instances. Therefore, the proposed method utilizes the graph network to model the relationship between instances. As illustrated in Figure 2, the instance relation graph is denoted as G = (V, E), where V is the node set and E is the edge set. The nodes in V correspond to instance features, and the edge eij ∈ E encodes the relationship between the ith instance and the j th instance.

Node Initialization

The nodes in the graph G are defined as V = {vi}Mi=1 , where M = |S| + |Q|, and vi is the feature representation of the ith instance in the graph network. Nodes are initialized as v(0) i = femb(xi), where femb(∗) is the feature extraction function.

Edge Initialization

The edges in the graph G are defined as E = {eij}Mi,j and represent the relationship between nodes. Besides, eij expresses the relation strength between two connected nodes and could be regarded as a probability that vi and vj are from the same class, reflecting intra-class similarity and inter-class dissimilarity. To better model the relationship between instances, the instance features are projected as key and query [VS et al., 2023]. For the ith instance, its key (ki), query (qi) and node pairwise logits (eij) are as follows:

where W (0) k and W (0) q are trainable parameters. The edge features are written as follows:

where e(0) ij represents the edge feature between nodes vi and vj in the 0th layer.

3.5 Label Knowledge Propagation

The proposed method incorporates class information into representations of support instances in Equation 1 and Equation 2. With message passing, the proposed method propagates label knowledge between support and query instances to enrich node and edge features.

Node Update

The proposed method propagates label knowledge from neighbors to enrich node features.

where v(l) i is the ith node feature in the lth layer. MLP(∗) is a multi-layer perceptron. W4 and b4 are trainable parameters.

Edge Update

With the graph network, the proposed method propagates label knowledge to adjust edge features from the latest node features. The edge features are written as follows:

where W (l) k andW (l) q are trainable parameters.

3.6 Dual Relation-enhanced Loss

The dual relation-enhanced loss includes a support-level loss and a query-level loss to promote label knowledge propagation and multi-label inference.

Support-level Loss

The support-level loss enhances the relation strength between instances with the same label connected to the support instance and weakens the relation strength between those with different labels. The ground-truth label relation of an instance pair is given as follows.

where yi and yj are the labels of vi and vj , respectively. yij = 1 denotes that vi and vj belong to the same class.

Any ith support instance is considered an anchor, where vi ∈ S. The remaining instances are constructed a set as J = {S,Q}\{vi}. A positive set is Ωspos = {e(l) ij |yij = 1, i ̸= j}, and a negative set is Ωsneg = {e(l) ij |yij = 0, i ̸= j}. The support-level loss could be formulated as follows.

From the above formula, we can observe that the relation strength scores of connected instances derived from the same class are greater than value 0, while those with different classes are less than value 0. In this way, the support-level loss encourages the target node to learn information from similar neighbors and pushes it away from dissimilar neighbors to guide label knowledge propagation effectively.

Query-level Loss

The edge feature eij is regarded as a probability that two connected nodes vi and vj are from the same class. After L layers of network propagation, the latest edge features from support nodes to query nodes are used for query inference. Specifically, each query node could be classified by final edge voting with support labels. The prediction probability that a query node vi belongs to class ck could be formulated as pck i .

where ys j is the label of the support node vj. eji is the edge between the support node vj and the query node vi. δ(ys j = ck) is the Kronecker delta function that is equal to one when ys j = ck and zero otherwise.

Considering that a query may contain multiple user intents, the positive set for the ith query node is defined as Ωqpos = {pck i |yq i = ck}, which indicates the prediction probability set between a query node and its corresponding classes. In contrast, a negative set is defined as Ωqneg = {pck i |yq i ̸= ck}, which indicates the prediction probability set between a query node and its irrelevant classes. The training objective is used to minimize the following loss function.

The aforementioned loss function aims to ensure that the scores in Ωqpos are greater than value 0 and the scores in Ωqneg are less than value 0. Therefore, the label ˆ yi of the query instance is formulated as follows:

where ˆ yi ck = 1 shows that vi is predicted to class ck, whereas ˆ yi ck = 0 shows that vi does not belong to ck.

3.7 Training Objective

So far, we have introduced an instance relation learning network to effectively tackle the few-shot MID task. The overall training objective is written as follows:

where α and β are hyper-parameters.

4 Experiments

4.1 Experimental Setup

Datasets. We conduct experiments on the benchmark dataset TourSG [Williams et al., 2012]. TourSG comprises six domains: Itinerary (It), Accommodation (Ac), Attraction (At), Food (Fo), Transportation (Tr), and Shopping (Sh). The detailed statistics are presented in Table 1.

Evaluation Metric. Following previous works [Hou et al., 2021; Zhang et al., 2023], we use AUC and Macro-F1 scores to evaluate the performance of our proposed method.

Implementation Details. The proposed method is implemented with PyTorch (version 1.10.0) on a single GPU (RTX 3090 Ti) with CUDA version 11.3. We use the pre-trained language model BERT-base [Devlin et al., 2019] as our encoder for H (see Feature Extraction). The AdamW optimizer trains the model with a learning rate of 5e-5. Meanwhile, we use the GradualWarmupScheduler to optimize the learning rate and set the warmup proportion to 0.05. The hyperparameters α and β are fixed as 0.1 and 1. We set L to 2 as the number of cycles for the updates in the overall architecture. We test and validate the model on two domains, respectively, and train it on the remaining domains. 100 meta-tasks are randomly sampled for training, validation, and testing in every epoch. We evaluate Macro-F1 scores in each meta-task and obtain the average F1 scores for all meta-tasks. Finally, the results of the test domain are reported when the validation domain obtains the best results.

4.2 Baselines

We compare the proposed method with a series of strong baselines. AWATT [Hu et al., 2021] designs a support-set attention mechanism and a query-set attention mechanism for learning representations of multi-label instances. Then, they use reinforcement learning to obtain a fixed threshold to identify multiple categories from these representations. HATT [Gao et al., 2019] and LDF [Zhao et al., 2022b] use an empirical threshold to achieve multi-label results. Besides, HATT utilizes instance-level and feature-level attention mechanisms to learn robust instance representations. LDF uses contrastive learning to learn different representations of multi-label instances. CTLR [Hou et al., 2021] combines an instance and its label text to learn representations for multiple labels. Then, it leverages a meta-calibrated method to learn a dynamic threshold. In addition, LPN [Liu et al., 2022] and DCKPN [Zhang et al., 2023] use label count estimation. They use an adaptive neural network to predict the label number of each instance. Besides, we also compare our method with Large Language Models (LLMs), such as Llama3-70b [Touvron et al., 2023] and GhatGLM4 [GLM et al., 2024].

4.3 Main Results

We conduct extensive experiments and report results in Tables 2, 3, 4 and 5. The best results are highlighted in bold, with the following observations.

(1) Our proposed method achieves an average of 5.24% AUC and 7.62% Macro-F1 improvement in the 5-way scenario. In Table 2 and Table 3, the proposed method improves upon the most competitive baseline DCKPN by 6.49%-9.88% AUC and 10.07%-17.21% Macro-F1 scores in the 5-way 1shot setting, with an average improvement of 7.60% AUC and 11.89% Macro-F1 scores. In terms of the 5-way 3-shot setting, the proposed method improves upon the most competitive baseline DCKPN by 1.57%-5.79% AUC and 0.77%- 8.47% Macro-F1 scores, with an average improvement of 2.89% AUC and 3.36% Macro-F1 scores.

(2) Furthermore, in Table 4 and Table 5, our proposed method achieves an average of 9.60% AUC and 5.73% Macro-F1 scores improvement in the 10-way scenario. In terms of the 10-way 1-shot setting, the proposed method improves upon the most competitive baseline DCKPN by 10.34%-12.69% AUC and 4.97%-15.68% Macro-F1 scores, with an average improvement of 11.50% AUC and 10.49% Macro-F1 scores. In terms of the 10-way 3-shot setting, the proposed method improves upon the most competitive baseline DCKPN by 6.46%-8.87% AUC, with an average improvement of 7.71% AUC. Compared to other methods, our 1-shot setting outperforms the 3-shot setting. Our proposed method focuses on edges in the graph network, using these edges to directly indicate whether two instances belong to the same label. Therefore, each query node is classified based on final edge voting with support labels. In 1-shot scenarios, fewer edges make optimization easier within the dual relation enhancement loss compared to in 3-shot scenarios, which is why 1-shot scenarios outperform 3-shot scenarios. This advantage demonstrates that our proposed method can achieve better results with fewer samples. However, DCKPN or LPN focuses on representation classification and ignores explicit interaction between instances, leading to low performance in 1-shot scenarios due to error propagation. Therefore, our proposed method guides the multi-label inference effectively.

(3) Compared to LLMs, our proposed method consistently outperforms GhatGLM4 and Llama3. For few-shot MID, the task requires LLMs to extract all relevant intents from a given utterance. However, LLMs cannot understand dialogue diversity in different domains. Even with a few demonstration examples, LLMs may generate outputs that are not fully aligned with the expected labels. In the example "What time is my meeting and what day?", LLMs might only generate outputs like "request time", whereas the ground-truth labels are "request time" and "request day". These incomplete outputs can degrade the model's performance, making it difficult for LLMs to handle multi-label tasks effectively.

4.4 Impact of Shot Number

To investigate the impact of the shot number for each class, we evaluate our proposed method with one to eleven shots in the food domain. Figure 3 shows the performances of methods in 5-way scenarios. The performances of DCKPN, LDF, AWATT, and HATT improve at first then fluctuate. The reason is that they focus on representation classification and rely on the number of labeled data to learn great instance representations. However, these methods underperform in the 1shot setting. Our proposed method focuses on edges in the graph structure and utilizes the relation strength between instances to guide multi-label inference. As discussed in the main results, optimizing the training objectives in fewer instances would be easy, which achieves great performance. Our proposed method obtains the best results on the 1-shot setting, demonstrating the effectiveness of the model.

4.5 Ablation Study

In Table 6, we conduct an ablation study. (1) "w/o label knowledge". We remove class descriptions in the support set. The negative results suggest that class descriptions promote label knowledge propagation in the network. (2) "w/o node pairwise logits". We replace node pairwise logits with cosine similarity as edge features in the network. The low performance indicates that node pairwise logits play a positive effect by utilizing key and query of instances. (3) "w/o dual loss". After we remove the dual relation-enhanced loss, the performance drops a lot. The negative results suggest that enhancing support- and query-level relation strength is a key point for multi-label prediction. (4) "w/o support-level loss". Though we only remove the support-level loss, the performance is reduced. In conclusion, the complete model outperforms all ablation studies and achieves the best performance.

5 Conclusion

We propose a multi-label joint learning method for few-shot MID, addressing the negative effect caused by error propagation. The proposed method explicitly models the relationship between instances and utilizes the relation strength between an instance pair to directly indicate whether these two instances belong to the same label, guiding multi-label inference effectively in an end-to-end manner. Besides, we design a dual relation-enhanced loss, which enhances interaction relations between support- and query-level instances and strengthens label knowledge propagation to further improve performance. Experiments show that we significantly outperform strong baselines, esp. the 1-shot setting.

6 Limitation

We incorporate class descriptions into the graph network to facilitate label knowledge propagation. Given that class descriptions are often brief and may provide limited context, their contribution to the overall model may be constrained. To improve performance, we plan to integrate more comprehensive external knowledge sources (e.g., domain-specific ontologies) into the graph construction, ultimately leading to better classification and prediction outcomes. Besides, our proposed method focuses on the few-shot MID task in dialogue systems. In the following work, we will explore other applications, such as health care.

Acknowledgements

This research is supported by the National Science and Technology Major Project (No. 2021ZD0111202).