Attention Based Document-level Relation Extraction with None Class Ranking Loss

1 Introduction

Relation extraction (RE) is an important subtask in information extraction (IE) and plays a key role in constructing large-scale knowledge graphs for many applications such as recommendation systems. Considerable effort has been made to extract all entity relation triples within a sentence, yielding some success in what is known as sentence-level RE [Zhang et al., 2018; Soares et al., 2019]. In contrast, there is much less work in document-level RE [Zeng et al., 2020;

James William Murphy (April 17, 1858-July 11, 1927) was a U.S. Representative. (...2 sentences ...) He was elected mayor of Platteville for a two-year term in 1904, and was then elected to the United States House of Representatives as a Democrat in 1906, defeating Joseph W.Babcock for the seat from Wisconsin's 3rd congressional district.

Subject: Wisconsin

Object: U.S.

Relation: country;

located in the administrative territorial entity

Subject: the United States House of Representatives

Object: U.S

Relation: country;

applies to jurisdiction

Figure 1: An example of document-level RE.

Zhou et al., 2021; Jiang et al., 2022; Zhou and Lee, 2022]. Document-level RE is more challenging than sentence-level RE since documents often contain a lot of implicit relations that can only be identified with the help of relation reasoning [Xu et al., 2023; Zhang et al., 2023]. It is necessary to consider the information across sentences and establish the relation between entities, rather than just extract the relation from a single sentence. In addition, there may be cross-sentence, cross-paragraph or even cross-chapter relations in the document, which requires cross-text relation reasoning. For example, as shown in Figure 1, the subjects "Wisconsin", "the United States House of Representatives" and the object "US" are located in different sentences, which makes it hard to discover the relations between them. To be specific, documentlevel RE is facing two main challenges as follows:

Challenge 1: Long-tail Problem. In document-level RE, there are numerous low-frequency or sparse relation instances, known as the long-tail problem. In the context of long-tail data, the model tends to prioritize high-frequency categories while struggling to recognize the tail ones, exacerbating the challenge in document-level RE. This imbalance makes it more difficult to train the model, as the model can not fully learn the context of the relations and capture their distinctive features.

Challenge 2: Multi-label Problem. In document-level RE, a single document can involve multiple relation types simultaneously, leading to multiple labels. For instance, a news report may encompass various aspects such as relations between individuals, location information, and event timing, which correspond to the labels. Consequently, the task of document-level RE becomes a multi-label classification problem where the model needs to predict all existing relation types within the document concurrently.

To address these challenges, most existing approaches apply Transformer [Xie et al., 2022; Zhou et al., 2021] and graph-based approaches [Nan et al., 2020; Peng et al., 2022; Xu et al., 2021b] to solve the long-tail problem and multilabel classification problem of relation types. However, these approaches ignore that relation, as an important component in the triple, exhibits informative correlations together with entities and contexts. For example, the two relations "last name" and "inclusion" often co-occur between the same pair of entities, but these two relations occur at different rates cue with the specific contexts. Thus, the ability to model and fully exploit the rich interactions across relations, entities, and contexts is critical to the task. Although the researchers [Xu et al., 2022] have proposed EmRel to fusion representations of relations, entities and contexts, they still ignore the correlations between relation types.

Moreover, the majority of existing methods for documentlevel RE rely on binary cross entropy (BCE) loss [Bishop, 1995], which solely considers the case of predefined relations independently while disregarding the case of "no relation". However, this loss overlooks the crucial insight that both the cases of none class and predefined classes are pivotal for multi-label prediction due to their high correlation (increased cases of the none class translate to decreased cases of all predefined classes). Without consideration of the case of "no relation", whether an entity pair has certain relations is determined via global thresholding (i.e., the labels, corresponding to the certain relations, have higher confidence scores than a defined threshold will be selected as final predictions). However, this approach fails to take the context of each entity pair into account and produces too many or too few labels, thereby yielding poor predictions.

To deal with the above issues, we propose a novel approach with none class ranking loss, which leverages embedded representations of relations. Specifically, we treat relation representations as embedding vectors and model the correlations among relation types and the indivisibility among relations, entities, and contexts via an attention-based fusion module. Then, we employ a novel multi-label loss named none class ranking loss to improve the prediction performance.

To demonstrate the validity and generality of our method, we conducted comprehensive experiments on public datasets, and the results showed that our method consistently outperformed all competing baselines. The main contributions of this paper are as follows:

• We model the correlations among relation types and indivisibility of relations, entities, and contexts through graph attention and multi-head attention networks.
• We use a novel multi-label loss named none class rank-
• ing loss to improve the prediction performance through a novel multi-label loss named none class ranking loss.
• Intensive experiments are conducted on the two commonly used datasets, which proves that capturing rich semantic information contained in relations, entities and contexts and taking the case of "no relation" into consideration help improve the accuracy of joint extraction of entity and relation. The sourcecode is available at https://github.com/sugarman490/Document-RE.

2 RelatedWork

Extracting multi-triples from document-level text has recently aroused increasing interest [Yao et al., 2019]. Existing methods take the entity perspective that proposes various techniques to model entity interactions. LSR [Nan et al., 2020] and GAIN [Zeng et al., 2020] construct an entity graph, and perform graph-level reasoning to refine the entity node representations. SSAN [Xu et al., 2021a] introduces entity structure as useful prior, and models such information within the transformer attention layer. DocuNet [Zhang et al., 2021] utilizes a segmentation network to model the interdependency among entity pairs. DRE [Xu et al., 2023] proposes a new evidence extraction model which explicitly extracts the precise evidence for each entity pair. [Zhang et al., 2023] emphasizes the bidirectional semantic interaction between the document encoder and the graph encoder. In conclusion, intertriple correlations are only captured at the entity level by these methods. However, the correlations and indivisibility among relations, entities and contexts are neglected.

Relation Embedding. There is one specific previous work [Chen and Badlani, 2020] that also considers modeling relation representations. CRE uses the sentence representation as relation embeddings, and scores them with the entity embeddings trained along with the knowledge base. EmRel [Xu et al., 2022] jointly represents relations with entities. However, they still ignore the correlations among relation types which are important for improving the accuracy of extracting triples. This raises significant advantages with our method in design that our method models both the correlations among relation types and the inseparability between relations and entities or contexts. SRLR [Huang et al., 2023] separates the entity pair to the mention-level that can capture more fine-grained relation representation and elicit logical reasoning based on evidence sentences to enhance the performance.

Multi-label Learning. Multi-label classification is a wellstudied problem with rich literature. Here, we focus on loss functions for multi-label learning. BCE is the most popular multi-label loss, which reduces the multi-label problem into a number of independent binary (one-vs-all) classification tasks, one for each label. Another common multilabel loss function is pairwise ranking loss, which transforms multi-label learning into a ranking problem via pairwise (onevs-one) comparison. Many attempts have been made to improve these two types of multi-label loss functions [Yeh et al., 2017; Li et al., 2017; Durand et al., 2019; Xu et al., 2019; Wu et al., 2019; Ridnik et al., 2021]. However, as far as we know, none of the existing loss functions as well as theoretical studies consider none class instances, which commonly appear in real-world multi-label applications.

Contrary to previous approaches, we consider the correlations between relation types and the indivisibility among relations, entities, and contexts by utilizing graph attention networks and multi-head attention networks. Furthermore, we employ a novel loss to improve the poor prediction caused by ignoring the case of "no relation". With these changes, our method achieves higher accuracy than the previous ones.

3 Methodology

In this section, we provide a detailed description of our method. As illustrated in Figure 2, our method comprises three components: the fusion module for representation, the alignment module, and the none class ranking loss. Firstly, within the fusion module for representation, interactive modeling is conducted between relations using graph attention networks (GAT) [Veliˇckovi´c et al., 2018] to capture the correlations among relation types (Section 3.1), which is ignored by previous methods. Subsequently, we briefly describe a representation alignment module that aligns the triple structures (Section 3.2). Finally, we utilize the none class ranking loss to improve the poor prediction performance existing in current methods (Section 3.3). Many ablation experiments demonstrate the effectiveness of our designed modules.

3.1 FusionModule for Representation

The pre-trained language model BERT [Kenton and Toutanova, 2019] is utilized to construct the entity representation and extract the context representation from the final layer at the corresponding position:

where we denote it as H ∈ R | {𝑤𝑖 } |×𝑑ℎ . We then construct each entity representation e𝑖 ∈ R 𝑑𝑒 by applying the pooling operation to its corresponding reference location, and further map it to the corresponding subject and object representations e 𝑠 𝑖 , e 𝑜 𝑖 . Thus, we represent all extracted entity representations as E 𝑠 , E 𝑜 ∈ R |𝐸|×𝑑𝑒 .

Relation Correlation. Using several relation labels in Figure 1 as an example, we can see correlations among relation types in Figure 3. The relation correlation interdependence is initially modeled using conditional probability, where 𝑃 (𝐿𝑏 | 𝐿𝑎) denotes the likelihood of relation type 𝐿𝑏 occurring given the occurrence of relation type 𝐿𝑎. To account for the symmetry in conditional probability 𝑃 (𝐿𝑏 | 𝐿𝑎) ≠ 𝑃 (𝐿𝑎 | 𝐿𝑏), a relation-dependent digraph is constructed based on prior relation knowledge from the training set used for modeling, resulting in an asymmetric adjacency matrix.

To construct the correlation matrix, we calculate the cooccurrence of the relations and obtain the co-occurrence matrix 𝐶 𝑟×𝑟 , where 𝑟 is the number of predefined relation types. To obtain the conditional probability among relations, we divide each element of the co-occurrence matrix 𝐶 𝑟×𝑟 by the total number of co-occurrences of the relations:

where 𝑃𝑖 𝑗 = 𝑃 � 𝐿 𝑗 | 𝐿𝑖 � represents the probability of relation type 𝐿 𝑗 when relation type 𝐿𝑖 occurs.

However, the previous method of constructing a correlation matrix may present two drawbacks. Firstly, certain relations seldom occur in conjunction with others, leading to an inflated probability value of co-occurrence that is deemed unreasonable. Secondly, there may exist a statistical bias between the training dataset and test dataset. Utilizing precise numbers tends to overfit the training data, potentially compromising generalization. To mitigate these issues, we introduce a threshold value 𝜏 to filter out these infrequent co-occurrence relations and address the aforementioned concerns. Then we binarize the conditional probability matrix 𝑃 by

where 𝐵 is the binarized correlation matrix. 𝛿 is the conditional probability threshold. Besides, We add the self-loop by setting 𝐵𝑖𝑖 = 1, 𝑖 = 1, ..., 𝑟.

One challenge associated with utilizing a binary correlation matrix B in graph neural networks is the issue of oversmoothing [Zhou et al., 2020], wherein node attribute vectors tend to converge towards similar values. This occurs due to the absence of inherent weight differentiation between relational features and those of neighboring nodes. To address this problem, we employ the following reweighting scheme:

where 𝑅 is the re-weighted relation correlation matrix and 𝑝 is a hyperparameter. In this way, the fixed weights for the relation feature and its neighbors will be applied during training, which is beneficial for alleviating this issue.

Relation features are embeddings obtained similarly to word embeddings. We employ GAT networks with K-head attention mechanisms to aggregate these relation features for two primary reasons. Firstly, GAT is well-suited for directed graphs, making it an appropriate choice. Secondly, GAT offers enhanced expressive capabilities as the weights of each node can vary. The resulting feature after the GAT transformation is denoted as 𝑅 ∈ R 𝑟×𝑑𝐵 , where 𝑟 represents the number of relation features and 𝑑𝐵 denotes the dimensionality of the transformed feature.

Representation Fusion. To effectively capture the indivisibility between entities and relations within a shared knowledge representation space, we fuse them to facilitate mutual learning. To model the interactions between these components, we employ the attention network [Bahdanau et al., 2015], which has demonstrated great success in capturing intricate relations within contexts [Yu et al., 2018] and modalities [Lu et al., 2016]. In particular, we adopt the widely-used multi-head Attention (MHA) network [Vaswani et al., 2017] in our work. In the context of our task, where the target represents X𝑄 and the source represents X𝑆, each head of the MHA network operates as follows:

Figure 3: An example of the correlation calculation among the relations in Figure 1.

where �X𝑄 represents the updated representation of X𝑄 with respect to X𝑆, all heads function concurrently and their outputs are concatenated together. To take advantage of the full interaction among all components, we first build an entity/context aware representation of relations 𝑅:

which are then aggregated together using layer normalization:

We generate relation-aware entity representations based on the relation features acquired from the Relation Correlation

Module as follows:

The abbreviations "s", "o" and "c" stand for subject, object and context respectively. Each attention module is enveloped with residual connections, feedforward layers, layer normalization, and instantiated with distinct 𝑊 𝑄 , 𝑊 𝐾 , and 𝑊 𝑉 parameters to capture diverse attention patterns. The fusion module produces comprehensive representations of relations, subjects and objects denoted as ˆR,�E 𝑠 ,�E 𝑜 .

3.2 Alignment Module

We extracts triples by aligning their ternary components 𝑅, �E 𝑠 , and �E 𝑜 . To fully leverage their expressive capabilities, we propose factory-based alignments utilizing Tucker factorization [Tucker and others, 1964]. In this approach, we introduce a core tensor Z ∈ R 𝑑𝑒∗𝑑𝐵∗𝑑𝑒 . The validity score for each < 𝑠𝑖 , 𝑟𝑘, 𝑜 𝑗 > is calculated as follows:

where ˆe 𝑠 𝑖 = �E 𝑠 𝑖,: , ˆr𝑘 = R𝑘,:,ˆe 𝑜 𝑖 = �E 𝑜 𝑗,: , and ×𝑛 indicates tensor product along the 𝑛-th mode, 𝜎 denotes sigmoid function.

3.3 None Class Ranking Loss

The task of Document-level RE can be viewed as a multilabel classification problem, where each entity pair represents an instance and the association between them serves as a label sample. Let X denote the instance space and Y = {0, 1} 𝐾 represent the label space, with 𝐾 being the number of predefined classes. An instance 𝑥 ∈ X is associated with a subset of labels, represented by the binary vector 𝑦 ∈ Y = (𝑦1, . . . , 𝑦𝐾), where 𝑦𝑖 = 1 if the 𝑖-th label is positive for 𝑥; otherwise 𝑦𝑖 = 0. To indicate that there are none class instances, all labels can be set to negative (𝑦𝑖 = 0, 𝑖 = 1, . . . , 𝐾). For ease of notation and clarity in presenting the loss function, we introduce a special class called none and assign it the 0-th label 𝑦0 without loss of generality. Consequently, the absence of any class instance can simply be indicated by setting 𝑦0 = 1.

Performance Measure. In contrast to document-level RE, traditional multi-label datasets typically assume that each instance contains at least one positive label, ensuring that no instances lack a class for training. Consequently, existing multilabel performance measures primarily focus on the prediction of predefined labels, disregarding the prediction impact of the none class. However, the probability of the predefined class is closely associated with the probability of the none class. Neglecting this label correlation in performance measures can lead to the oversight of poor predictions that should be penalized during training. This oversight increases the risk of misclassification, particularly in the case of document-level RE, where a significant portion of entity pairs falls within the none class.

To address the need for improved evaluation of documentlevel RE performance, we transform the multi-label prediction of each instance into a label ranking, where positive predefined labels should be ranked higher than the none class label, while negative ones should be ranked below. Formally, we introduce a multi-label performance measure. The formulation is as follows:

where 𝑓𝑖 is the confidence score of class 𝑖, and ⟦⟧ is a mapping that returns 1 if its content is true and 0 otherwise. The multi-label performance measure treats the none class label as an adaptive threshold that distinguishes positive from negative labels and counts the number of reverse-arranged label pairs between the none class and each predefined class, which takes into account errors when sorting both the none class and the predefined labels.

Loss Function. For practical multi-label learning, it is essential to minimize surrogate loss, which should be simple and amenable to optimization. Therefore, we employ none class ranking loss:

where 𝑚 + 𝑖 = 𝑓𝑖 - 𝑓0 and 𝑚 - 𝑖 = 𝑓0 - 𝑓𝑖 represent positive and negative label margins, respectively, and 𝜎(𝑥) is the sigmod function that converts label margins to the probability output in [0,1]. Intuitively, minimizing encourages positive (negative) labels to score as high (low) as possible over none class labels. When testing, we can return a label with a score higher than 𝑓0 as a prediction label, or NA if no such label exists.

Margin Regularization. In document-level RE datasets, the number of positive samples per relation is much larger than the number of negative samples. This positive and negative sample imbalance allows the loss to push a large number of negative samples away from the threshold 𝑓0 to achieve low training loss. Therefore, training with unbalanced data tends to make negative predictions for predefined labels with rare positive samples. To solve this problem, we propose to control the average margin and design the following regularization term:

The average positive and negative margins, denoted as 𝑚 + 0 = 𝑓0 - 1 𝐾 � 𝐾 𝑖=1 𝑓𝑖 and 𝑚 - 0 = 1 𝐾 � 𝐾 𝑖=1 𝑓𝑖 - 𝑓0 respectively, are utilized in the margin regularization 𝐿NA0 (𝒚, 𝒇 ) for instances with 𝑦0 = 0 to penalize the average score overall pre-defined labels if it is smaller than 𝑓0. This approach avoids oversuppressing scores of negative samples. For none class instances with 𝑦0 = 1, a threshold value of 𝑓0 larger than the largest score among all pre-defined labels is required. However, directly maximizing the margin 𝑓0 -max𝑖≠0 𝑓𝑖 can lead to unstable results due to changes in the pre-defined label with max𝑖≠0 𝑓𝑖 during training. Instead, 𝐿NA0 (𝒚, 𝒇 ) maximizes the upper-bound of 𝑓0 - 1 𝐾 � 𝐾 𝑖=1 𝑓𝑖 which is given by the average margin value of 𝑓0 -max𝑖≠0 𝑓𝑖, encouraging that value to be at top position in label ranking.

Margin Shifting. Another practical problem in documentlevel RE is the problem of labeling errors. Because documents can describe a variety of things, it is difficult for human annotators to accurately and broadly annotate all the relations of each entity pair. As a result, document-level RE datasets often contain considerable noise in markup. Among mislabeled data, false-negative samples are particularly common in practice.

To obtain stronger robustness to mislabeled samples, we suggest shifting the transformed negative margin 𝜎(𝑚 - 𝑖 ) in formula (10) as follows:

where 0 < 𝛾 < 1 is a hyper-parameter. This margin shifting approach increases the probability of all negative samples by a factor 𝛾, thereby attenuating the impact of very hard negative samples with small 𝜎(𝑚 - 𝑖 ) = P(𝑦𝑖 = 0|𝒙) during training, which are suspected to be mislabeled. Additionally, very easy samples with 𝜎(𝑚 - 𝑖 ) > 1 - 𝛾 get a zero loss and are ignored, enabling the loss to focus more on harder samples.

Despite the existence of similar shifting methods [Lin et al., 2017; Ridnik et al., 2021; Wu et al., 2020], they fail to consider none class labels and shift the score 𝑓𝑖 instead of the margin 𝑚 - 𝑖 . Since the none class score 𝑓0 aligns the margins 𝑚 - 𝑖 of pre-defined labels in the same range, margin shifting is more suitable than score shifting in adjusting the prediction probability for different training samples. Incorporating the margin regularization and shifting, our final loss function is defined as follows:

4 Experiments

4.1 Datasets and Settings

Datasets. The experiments are conducted on two widely used datasets, shown as follows:

• DocRED [Yao et al., 2019] is a common document-level RE dataset used to evaluate and promote research on document-level RE tasks. The goal of this dataset is to extract relation triples from a given document. The documentation for the DocRED dataset is sourced from Wikipedia and news articles, covering multiple fields and topics. Compared with traditional sentence-level RE tasks, DocRED datasets examine more complex RE scenarios that require cross-sentence, cross-paragraph and even cross-discourse relation inference and reasoning.
• Re-DocRED [Tan et al., 2022b] is another revised version of DocRED. Since DocRED contains a considerable number of false-negative samples, we still conduct experiments on this revised version.

Settings. We use BERT-Base-Cased [Kenton and Toutanova, 2019] as the text encoder. The dimension of embedded relation representation is set as 768. The learning rate for BERT parameters is 5𝑒 -5 and 𝑒 -4 for other layers. In the relation correlation module, we set the threshold 𝜏 to be 10 for filtering noisy co-occurred relations, and 𝛿 is set to be 0.05 in equation (3). We set p to be 0.3 in equation (4). We employ 2-layer GAT networks with k = 2 attention heads computing 500 hidden features per head. We utilize the exponential linear unit (ELU) as the activation function between GAT layers. The number of attention heads in the fusion module is simply set as 4.

4.2 Baselines and EvaluationMetrics

Baselines. We conduct a comparison between our method and the Transformer-based and Graph-based methods.

• Transformer-based methods directly employ PLMs to enhance the acquisition of entity representations for document-level RE. Examples of such methods include BERT-TS [Wang et al., ], HINBERT [Tang et al., 2020], CorefBERT [Ye et al., 2020], ATLOP-BERT [Zhou et al., 2021] and SRLR-BERT [Huang et al., 2023].
• Graph-based methods leverage GNNs to improve the reasoning capability of document-level RE models. Examples of such methods consist of EoG [Christopoulou et al., 2019], DHG [Zhang et al., 2020], GEDA [Li et al., 2020], LSR [Nan et al., 2020], HeterGSAN [Xu et al., 2021c], SIRE [Zeng et al., 2021], and SSAN [Xu et al., 2021a]. Furthermore, for comparison purposes, we also include several recent competitive methods, namely DocuNet [Zhang et al., 2021], KD-DocRE [Tan et al., 2022a], and KMGRE [Jiang et al., 2022].

Evaluation Metrics. Consistent with prior researches conducted [Yao et al., 2019; Zhou et al., 2021; Tan et al., 2022b], we employ micro F1 and micro Ign F1 as our evaluation metrics. Ign F1 excludes the relational facts that are shared between the training and the development/test sets.

4.3 Main Results

In this part, we carry out a lot of comparative experiments with existing methods in terms of accuracy on F1 and Ign F1 evaluation metrics.

The results, presented in Table 1, indicate that our method universally outperforms its baselines on the two datasets. On DocRED, our method improves the baseline by +1.46 Dev F1, +1.32 Test F1, and also outperforms several previous studies including BERT-TS [Wang et al., ], CorefBERT [Ye et al., 2020], LSR [Nan et al., 2020], and SSAN [Xu et al., 2021a]. On Re-DocRED, our method obtains improvements of 1.61 Dev F1 and 1.79 Test F1 points. These results demonstrate that our method can better capture the indivisibility among relations, entities, and contexts, and well handle the presence of the none class between entity pairs, thus improving the inference ability of the model.

Furthermore, our method surpasses recent state-of-theart models such as DocuNet-BERT and KD-DocRE-BERT which incorporate dependencies between entity pairs to enhance their reasoning capabilities. This further attests to the exceptional reasoning prowess exhibited by our method.

4.4 Ablation Studies

Effect of Modules. We validate the design of the modules in our method. Table 2 shows that the fusion module for representation, the alignment module, and the none class ranking loss all contribute to the improvement. Furthermore, it can be observed that our method exhibits higher performance consistency across multiple iterations, which is influenced by the effectiveness of our alignment function. Notably, upon its removal, the standard deviation increases from ±0.25 to ±0.52.

Matrix Construction Threshold. We investigate the effect of key components in the relation correlation module. As shown in Table 3, we analyze the effect of probability filtering threshold 𝛿 in equation (3). It can be observed that the highest F1 score when 𝛿 = 0.05 for all experiments. Besides, the results indicate that 𝛿 = 0.03 will lead to more performance degradation compared with 𝛿 = 0.07. The above phenomenon is due to the smaller threshold value resulting in more noise edges.

Number of GAT Layers. We report the results of different GAT layers with two heads in Table 4. Results demonstrate that 1-layer and 2-layer GAT networks achieve relatively similar results, while 3-layer GAT networks lead to greater performance degradation. The probable reason for the performance degradation is the over-smoothing issue that the node feature vectors are inclined to converge to comparable values.

Different Losses. To study how none class instances affect the performance of the none class ranking loss, we conduct experiments on our method with different losses. We still use the BERT-base for text encoding. Table 5 shows the accuracy comparison results. As can be seen, ATL and the none class ranking loss are better than BCE. On the other hand, due to the tuned threshold, the none class ranking loss obtains better accuracy than ATL when facing none class instances. Different from sentence-level datasets, document-level RE datasets contain many none class instances, which can be utilized by the none class ranking loss for better classification performance.

5 Conclusion and FutureWork

In this paper, we propose a document-level RE model consisting of a fusion module for representation, an alignment module and a none class ranking loss. In particular, the fusion module for representation includes two parts: 1) the relation correlation part captures the correlations among relations; 2) the representation fusion part explicitly merges the representation of relations, entities, and contexts. Finally, the none class ranking loss maximizes label margins between none class and each predefined class, captures classless label dependencies, and enables adaptive thresholds for label predictions. Experimental results on two commonly used datasets show that our model outperforms all existing competing baselines. In the future, we consider applying the modules to other document-level tasks to verify their generality.

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grant No. 42050102 and Dou Wanchun Expert Workstation of Yunnan Province (No. 202105AF150013). Dr Xuyun Zhang is the recipient of an ARC DECRA (project No. DE210101458) funded by the Australian Government.