None Class Ranking Loss for Document-Level Relation Extraction

1 Introduction

Identifying relationships among entities from unstructured text, namely relation extraction (RE), is a fundamental task in information extraction. Previous works have studied sentence-level RE that aims at extracting relations between two entities in a single sentence [Miwa and Bansal, 2016; Zhang et al., 2018]. More recently, document-level RE, which aims to identify the relations of various entity pairs expressed across multiple sentences, has received increasing research attention [Yao et al., 2019; Yu et al., 2020].

An extended version of this paper with appendices is available at http://arxiv.org/abs/2205.00476.

Since one entity pair can express multiple relations in a document, document-level RE can be formulated as a multilabel classification problem. Different from conventional multi-label settings where each instance at least has one positive pre-defined label, most instances (entity pairs) in document-level RE express none of the pre-defined relations, where all the pre-defined labels are known to be negative. For good document-level RE performance, it is crucial to distinguish such none class instances (entity pairs) from those of pre-defined classes (relations).

However, most existing methods adopt binary cross entropy (BCE) loss for document-level RE. They only estimate the probability of pre-defined relations independently without considering the probability of "no relation". This ignores the fact that the probabilities of both the none class and predefined classes are important for multi-label prediction, as they are highly correlated (the increased probability of the none class implies the decreased probabilities of all the predefined classes). Without knowing the none class probability, whether an entity pair has certain relations needs to be determined via global thresholding (i.e., selecting labels whose confidence scores are higher than a single threshold as final predictions). This fails to take the context of each entity pair into account and may produce either too many or too few labels, leading to sub-optimal predictions.

In this work, we propose a new multi-label performance measure and its surrogate loss, called None Class Ranking Loss (NCRL), to address the aforementioned disadvantages of BCE in document-level relation extraction. NCRL aims to maximize the margin of label confidence scores between the none class and each pre-defined class, such that positive labels are ranked above the none class label and negative labels are ranked below. In this way, the none class score becomes an instance-dependent threshold that treats labels being ranked above the none class as positive, and the label margin reflects the uncertainty of such prediction. By formulating the probability of each class as label margins, NCRL not only takes the context of entity pairs in multi-label prediction but also (as will be shown later) enables captured label correlations without directly comparing the scores of positive and negative labels. Moreover, we theoretically justify the effectiveness of NCRL by proving that NCRL satisfies Bayes consistency w.r.t. the targeted multi-label performance measure, which is a desirable property for any surrogate loss in multi-label learning.

In a document, entity pairs generally have a few positive relations and many negative ones. This positive-negative imbalance makes multi-label losses focus more on negative samples and down-weight the contributions from the rare positive samples [Ben-Baruch et al., 2021]. Another practical problem of document-level RE is that entity pairs are prone to being mislabeled as "no relation", which results in deteriorated classification performance. To alleviate the positive-negative imbalance problem, we propose a margin regularization technique, which penalizes negative samples whose confidence scores are far below that of the none class. For robustness against mislabeled samples, we propose a margin shifting approach to attenuate the impact of negative samples with a very large score, which are suspected as mislabeled. This helps to reduce the label noise in document-level RE datasets.

Our NCRL loss is general enough and can be applied to other multi-label tasks when none class instances are available for training. Experimental results on two documentlevel RE datasets, DocRED [Yao et al., 2019] and DialogRE [Yu et al., 2020], and one fine-grained emotion classification dataset, GoEmotions [Demszky et al., 2020], show that NCRL achieves significantly better results than existing multi-label losses. The main contributions of this paper are summarized as follows:

• We propose a new multi-label performance measure along with its surrogate loss, NCRL, that prefers large label margins between the none class and each predefined class. Our loss captures the label correlations and enables context-dependent thresholds for multilabel prediction.
• We prove that NCRL satisfies multi-label consistency, which guarantees that minimizing NCRL can achieve the expected objective.
• We propose a margin regularization and a margin shifting technique to alleviate positive-negative sample imbalance and gain further robustness against mislabeled samples, respectively.

2 RelatedWork

Multi-label learning. Multi-label classification is a wellstudied problem with rich literature. Here, we focus on loss functions for multi-label learning. BCE is the most popular multi-label loss, which reduces the multi-label problem into a number of independent binary (one-vs-all) classification tasks, one for each label. Another common multilabel loss function is pairwise ranking loss, which transforms multi-label learning into a ranking problem via pairwise (one-vs-one) comparison. Many attempts have been made to improve these two types of multi-label loss functions [F¨urnkranz et al., 2008; Yeh et al., 2017; Li et al., 2017; Durand et al., 2019; Xu et al., 2019; Wu et al., 2019; Ben-Baruch et al., 2021]. However, as far as we know, none of the existing losses as well as theoretical studies consider none class instances, which commonly appear in real-world multi-label applications.

Document-level relation extraction. Existing works on document-level RE mainly focus on learning better entity pair representations to improve classification performance. Some representative approaches include incorporating longdistance entity interactions into recurrent neural networks or transformer-based models [Christopoulou et al., 2019; Nan et al., 2020], aggregating local and global entity information with graph neural networks [Wang et al., 2020; Zeng et al., 2020; Xu et al., 2021b; Dai et al., 2021], and integrating latent word semantics with pre-trained language models [Zhou et al., 2021; Xu et al., 2021a].

All the aforementioned document-level RE methods, except [Zhou et al., 2021], use BCE as the objective function for training and do not consider the none class. ATL [Zhou et al., 2021] is probably the most related method to our NCRL loss function. Both ATL and NCRL treat the none class label as an adaptive threshold to separate positive relations from negative ones, leading to context-dependent multi-label prediction. However, the proposed NCRL can learn a more discriminative classifier with theoretical guarantees and consistently outperforms ATL in our experiments. For clarity, we defer the detailed comparisons between NCRL and ATL in Sec. 3.5.

3 None Class Ranking Loss

In this section, we first introduce None Class Ranking Error (NCRE), a new multi-label performance measure that takes into account the none class in multi-label learning. Then, we propose None Class Ranking Loss (NCRL), which is a surrogate loss of NCRE with Bayes consistency. Finally, we propose a margin regularization and a margin shifting strategy to address the positive-negative imbalance and mislabeling problems that commonly appear in real-world RE datasets.

3.1 Problem Statement

Document-level RE is essentially a multi-label classification problem, where each entity pair is an instance and the associated relations are label samples. Let X be an instance space and Y = {0, 1}K be a label space, where K is the number of pre-defined classes. An instance x ∈ X is associated with a subset of labels, identified by a binary vector y ∈ Y = (y1, . . . , yK), where yi = 1 if the i-th label is positive for x, and yi = 0 otherwise. In particular, none class instances can be represented by setting all the labels to be negative (yi = 0, i = 1, . . . ,K). To simplify the notation and facilitate deriving the proposed NCRL, we introduce the none class and assign it with the 0-th label y0 without loss of generality. Then none class instances can be simply labeled with y0 = 1.

3.2 Performance Measure

Different from document-level RE, conventional multi-label datasets often assume that each instance at least has one positive label, which means that none class instances are not available for training. For this reason, existing multi-label performance measures only focus on the predictions of pre-defined labels individually, and do not consider how well the none class is predicted. Nevertheless, the probabilities of the predefined classes are highly correlated with the probability of the none class. Failing to consider such label correlations in the performance measure could overlook bad predictions that should have been penalized during training, and thus increases the risk of misclassification, especially for documentlevel RE, where most entity pairs are from the none class.

To better measure the performance of document-level RE, we transform the multi-label prediction of each instance into a label ranking, where positive pre-defined labels should be ranked higher than the none class label, while negative ones should be ranked below. Formally, we propose NCRE, a new multi-label performance measure as follows:

where fi is the confidence score of the i-th class, and �π� is the mapping that returns 1 if predicate π is true and 0 otherwise. NCRE treats the none class label as an adaptive threshold that separates positive labels from negative ones, and counts the number of reversely ordered label pairs between the none class and each pre-defined class, which considers errors in ranking the none class and pre-defined labels simultaneously.

3.3 NCRL Surrogate Loss

Although our goal is to find an optimal classifier w.r.t. NCRE, it is difficult to minimize NCRE directly. For practical multilabel learning, we need to minimize a surrogate loss of NCRE instead, which should be simple and easy for optimization. To this end, we propose None Class Ranking Loss (NCRL) as follows:

wherem+i = fi-f0 andm-i = f0-fi represent positive and negative label margins, respectively, and σ(x) is the sigmoid function that transforms label margins into probability outputs within [0, 1]. Intuitively, minimizing NCRL encourages positive (negative) labels to have higher (lower) scores than that of the none class as much as possible. At the test time, we can return labels with higher scores than f0 as predicted labels or return NA if such labels do not exist.

Captured correlations. BCE assumes pre-defined classes are independent and thus ignores label correlations and dependencies. Pairwise ranking loss is commonly used to capture label correlations in multi-label learning, which aims to maximize the margin between each pair of positive and negative labels. Compared with BCE and pairwise ranking loss, NCRL relaxes the independent assumption of BCE and captures label correlations by connecting the probability of each pre-defined class P(yi = 1|x) to the label margins m+i = fi - f0. In this way, maximizing P(yi = 1|x) not

Figure 1: An illustration of NCRL. For yi = 1, NCRL maximizes the positive margin fi - f0 ∝ P(yi = 1|x) (see Theorem 1). For yj = 0, NCRL maximizes the negative margin f0 - fj ∝ p(yj = 0|x). In this way, the margin between positive and negative labels fi - fj = (fi - f0) + (f0 - fj ) is also maximized.

only leads to increased fi but also decreased f0, which in turn affects the margins (probabilities) of all the other classes.

Moreover, thanks to the introduced none class, the margin between any pair of pre-defined labels can be induced from the sum of m+i andm-j , i.e., fi -fj = (fi -f0)+(f0-fj). Therefore, NCRL captures pairwise label correlations and indirectly maximizes the margin between every pair of positive and negative labels without expensive pairwise comparisons. Figure 1 illustrates how NCRL learns a proper label ranking by separating positive and negative labels via f0.

Consistency analysis. Now we theoretically justify the effectiveness of NCRL by proving that NCRL (2) is Bayes consistent w.r.t. NCRE (1). Bayes consistency is an important property for a surrogate loss. It guarantees that the classifier obtained by minimizing the surrogate loss converges to the one with minimal multi-label risk, and thus is likely to achieve good classification performance w.r.t. the corresponding multi-label performance measure.

Given an instance x, let ∆i = P(yi = 1|x) be the marginal probability when the i-th label is positive. Then the conditional risk w.r.t. NCRE (1) can be written as follows:

which corresponds to the expected penalty for an instance x when f(x) is used as the score function. From (3), the Bayes optimal score function f ∗ ℓNA that minimizes the multilabel risk (3) is given by:

Theorem 1. NCRL (2) is Bayes consistent w.r.t. NCRE (1).

Proof sketch. We prove the Bayes consistency of NCRL by showing that given an instance, the optimal label margin f ∗ i - f ∗ 0 obtained by minimizing NCRL coincides with the marginal label probability ∆i = P(yi = 1|x), such that f ∗ i - f ∗ 0 > 0 ⇒ ∆i > 1 2 , which is an optimal solution with the minimal NCRE risk according to (4). The complete proof is provided in Appendix.

3.4 Margin Regularization and Shifting

Margin regularization. In real-world document-level RE datasets, it is common that only a small number of entity pairs express pre-defined relations, implying that the number of positive samples per relation is much larger than that of negative ones. Such positive-negative sample imbalance makes NCRL push a large number of negative samples away from the threshold f0 to achieve a low training loss. As a result, training with imbalanced data tends to make negative predictions for pre-defined labels with rare positive samples.

To address this problem, we propose to control the average margin and design the following regularization term:

wherem+0 = f0- 1K ∑Ki=1 fi andm-0 = 1K ∑Ki=1 fi-f0 are the average positive and negative margins, respectively. For instances with y0 = 0, the margin regularization LNA0 (y, f) penalizes the average score over all pre-defined labels if it is smaller than f0, and thus avoids over-suppressing scores of negative samples. On the other hand, for none class instances with y0 = 1, the threshold f0 should be larger than the largest score among all the pre-defined labels. However, directly maximizing the margin f0 -maxi̸=0 fi can lead to unstable results, as the pre-defined label with maxi̸=0 fi would keep changing during training. Instead, LNA0 (y, f) maximizes the average margin f0 - 1K ∑Ki=1 fi, which is an upper-bound of f0 -maxi̸=0 fi, to encourage f0 being at the top of the label ranking.

Margin shifting. Another practical issue in document-level RE is the mislabeling problem. As a document can describe various contents, it is very difficult for a human annotator to annotate all relations accurately and extensively for each entity pair. As a result, real-world document-level RE datasets often contain considerable noise in labeling. Among mislabeled data, false negative samples are particularly common in practice.

To gain more robustness against mislabeled samples, we propose to shift the transformed negative margin σ(m-i ) in NCRL (2) as follows:

where 0 < γ < 1 is a hyper-parameter. Such margin shifting approach increases the probability of all negative samples by a factor γ such that the impact of very hard negative samples with small σ(m-i ) = P(yi = 0|x), which are suspected as mislabeled, are attenuated during training. In addition, very easy samples with σ(m-i ) > 1 - γ get a zero loss and are ignored, which makes NCRL focus more on harder samples.

Although similar shifting approaches have been proposed [Lin et al., 2017; Ben-Baruch et al., 2021; Wu et al., 2020], they do not consider the none class label and shift the score fi rather than the margin m-i . Since the none class score f0 aligns the margins m-i of pre-defined labels in the same range, margin shifting is more suitable than score shifting in adjusting the prediction probability for different training samples. Armed with the margin regularization and shifting, our final loss is given as follows:

3.5 Comparison with ATL

There are a few related works that consider the none class in document-level RE. To the best of our knowledge, adaptive thresholding loss (ATL) [Zhou et al., 2021] is the most related method to our NCRL. ATL also treats the none label class as a learnable threshold and encourages positive labels to be ranked above the none class label and negative labels to be ranked below for document-level RE. Specifically, ATL is given as follows:

Although ATL and NCRL share the same spirit of learning the none class label, they essentially learn distinct label rankings. ATL is constructed in two terms. The first term prefers the none class score f0 being ranked at the bottom among f0 and the scores of positive pre-defined labels, and the second term prefers f0 being ranked at the top among f0 and the scores of negative pre-defined labels. ATL only focuses on positioning f0 between positive and negative sets of labels, whereas fails to maximize the margin between each pair of positive and negative labels, which is desirable for obtaining discriminative classifiers. Due to the softmax function in the first term of ATL, increasing the score of one positive labels fi will penalize the scores of all the other positive labels, and could leads to large loss values, which is not reasonable in the multi-label setting. On the contrary, our NCRL separates positive labels from negative ones as much as possible by maximizing the label margins, and enjoys the theoretical guarantee of Bayes consistency. In addition, NCRL is equipped with the margin regularization and shifting to alleviate the positivenegative imbalance and mislabeling problems, which are not applicable for ATL.

4 Experiments

In this section, we evaluate NCRL on two document-level RE datasets. To demonstrate the effectiveness of NCRL for other multi-label tasks, we also test NCRL for fine-grained emotion classification.

4.1 Experimental Setups

Datasets. DocRED [Yao et al., 2019] is a large-scale document-level RE dataset, which is constructed from Wikipedia articles. DocRED consists of 5053 documents with 96 pre-defined relations, where 3053 documents are used for training, 1000 for development, and 1000 for testing. DocRED also provides a large distantly supervised dataset, which includes 101,873 documents with noisy labels. DialogRE [Yu et al., 2020] is a dialogue-based RE dataset, which is constructed from the transcripts of the American TV situation comedy, Friends. DialogRE consists of 1788 dialogues with 36 pre-defined relations, where 60% of dialogues are used for training, 20% for development, and 20% for testing. GoEmotions [Demszky et al., 2020] is an emotion classification dataset of 58009 Reddit comments labeled for 27 emotion categories or Neutral, where 80% data are used for training, 10% for development, and 10% for testing.

Implementation details. For each dataset, we adopt an encoding model upon pre-trained language models for representation learning. The classification results are then obtained by training the encoding model with different loss functions. We use Huggingface's Transformers [Wolf et al., 2020] to implement all the models. We use AdamW [Loshchilov and Hutter, 2019] as the optimizer with learning rates ∈ {1e-5, 2e-5, . . . 5e-5}, and apply a linear warmup [Goyal et al., 2017] at the first 10% steps followed by a linear decay to 0. The number of training epochs is selected from {5, 8, 10, 20, 30}. At the test time, the best checkpoint on the development set is used for the final prediction. For BCE, we follow the same settings with ATL [Zhou et al., 2021] to use a global threshold for multi-label prediction and select the threshold from {0.1, 0.2, ..., 0.9}. For NCRL, the hyper-parameter γ in margin shifting (6) are selected from {0, 0.01, 0.05}. All hyper-parameters are tuned on the development set, where those with the highest F1 scores are selected for testing. All the experiments are conducted with 1 GeForce RTX 3090 GPU. We provide detailed configurations of the encoding models and the hyper-parameter settings on each dataset in Appendix. Our code is available at https://github.com/yangzhou12/NCRL.

Competing methods. We compare our NCRL with existing multi-label loss functions including BCE and ATL [Zhou et al., 2021]. We also test pairwise ranking loss and the logsum-exp pairwise loss [Li et al., 2017], but do not report their results for simplicity, as they are consistently worse than BCE in our experiments. For each method, we report the mean and standard deviation of performance measures based on 5 runs with different random seeds. The results with citations are taken from the corresponding papers.

4.2 Results and Discussions

Results on DocRED. On DocRED, we follow the same setting of ATL and use its encoding model called ATLOP [Zhou et al., 2021] for fair comparisons, which is built on RoBERTa-large [Liu et al., 2019]. To demonstrate our NCRL can still improve the performance on stronger encoding models, we also test a more recent pre-trained language model, DeBERTa-large [He et al., 2021]. In addition, we consider an improved RE model, which is first trained on the distantly supervised DocRED dataset and then finetuned on the humanannotated dataset. For comprehensiveness, we also compare recent document-level RE methods including HIN [Tang et al., 2020], Coref [Ye et al., 2020], CFER [Dai et al., 2021], and SSAN [Xu et al., 2021a], which use BCE for training. Following [Yao et al., 2019], we use F1 and Ign F1 in evaluation, where the Ign F1 denotes the F1 score excluding the relational facts that already appear in the training and dev/test sets.

Table 1 shows the document-level RE results on DocRED, where the test scores are obtained by submitting the predictions of the best checkpoint on the development set to the official Codalab evaluation system. Overall, BCE with a manually tuned threshold already achieves comparable or better results than most competing methods such as CFER and SSAN. By considering the none class in multi-label learning, ATL and our NCRL outperform BCE in most cases, and avoid the overhead of threshold tuning. This indicates that exploiting the correlations of the none class label can improve the performance of document-level RE. Moreover, our NCRL consistently outperforms ATL in all cases. This demonstrates the advantage of NCRL in learning a label ranking with maximized label margins. When the distantly supervised data are used for training, all the methods obtain much better results. In particular, NCRL achieves a state-of-the-art Ign F1 score of 65.81% on the leaderboard. This indicates that NCRL is robust against noisy labels and can better transfer the information from the distantly supervised dataset for documentlevel RE. On the other hand, ATL seems to be sensitive to label noise and gets much worse results than NCRL.

Results on DialogRE. On DialogRE, we adopt the official baseline, BERTsBase [Yu et al., 2020], as the encoding model, where the dialogue inputs are augmented with entity type and coreference information. We follow [Yu et al.,

https://competitions.codalab.org/competitions/20717

2020] to use F1 and F1c in evaluation, where F1c is the F1 score computed by taking the first few turns instead of the entire dialogue as input. Table 2 shows the dialogue-based RE results on DialogRE. As can be seen, our implementation of BERTsBase with BCE obtains significantly better results than the reported ones and is slightly worse than the stateof-the-art method, GDPNet, which combines the graph neural networks and BERTsBase for better representation learning and uses BCE for training. ATL and our NCRL outperform both BCE and GDPNet by a large margin, showing the benefit of learning the none class for document-level RE. Moreover, NCRL achieves the best and most stable results in most cases. This is because NCRL is more robust against imbalanced and mislabeled samples by applying the margin regularization and shifting approaches. It is worth noting that our margin regularization and shifting techniques are inapplicable for BCE and ATL, as they do not formulate the probability of each class as label margins.

Results onGoEmotions. For the GoEmotions dataset, sentences labeled with Neutral are considered as none class instances. Originally, there are a small number ambiguous instances labeled with both Neutral and certain emotions. We remove these instances from the train/dev/test set for simplicity. We use BERTBase as the encoding model as in [Demszky et al., 2020], and finetune it with the plain NCRL (2) without any regularization, which is enough to achieve good performance in our experiments. We use mean Average Precision (mAP) for evaluation, which summarizes a precision-recall curve at each threshold and measures the model ability in label ranking for each instance. Table 3 shows the classification results on GoEmotions. Again, NCRL obtains the best results, indicating that NCRL works well on multi-label tasks when none class instances are available for training. Specifically, our NCRL improves BCE and ATL by 1.49% and 1.25% in mAP, respectively. This demonstrates that NCRL produces more accurate label rankings by capturing the correlations between the none class and pre-defined classes via the label margins.

Ablation studies. Finally, we perform ablation studies of NCRL on DialogRE. Table 4 provides the results of NCRL when each component of NCRL is excluded at a time, where M-Reg and M-shift denote the margin regularization and the margin shifting, respectively. In addition, we also compare BCE with probability shifting (P-shift) that is used in [Ben- Baruch et al., 2021; Wu et al., 2020] for multi-label learning. We can observe that all components of NCRL contribute to multi-label learning. Both the margin regularization and shifting techniques are important for NCRL to achieve good performance. Even without these two approaches, the plain NCRL still outperforms BCE and its variant, which implies that learning the none class from none class instances is indeed helpful for document-level RE.

5 Conclusion

We have proposed NCRL, a new multi-label loss that considers the probability of "no relation" in document-level RE. NCRL maximizes the label margin between the none class and each pre-defined class, capturing the label correlations of the none class and enabling adaptive thresholding for label prediction. Through theoretical analysis, we justify the effectiveness of NCRL by proving that NCRL is Bayes consistent w.r.t. its targeted performance measure NCRE. In addition, a margin regularization and a margin shifting technique are proposed to gain robustness against imbalanced and mislabeled samples. Experiments on three benchmark datasets demonstrate that NCRL significantly outperforms existing multi-label loss functions for document-level RE and emotion classification.

Acknowledgments

This research is supported in part by the National Research Foundation, Singapore under its AI Singapore Program (AISG Award No: AISG2-RP-2020-016).