Document-level Relation Extraction via Subgraph Reasoning

1 Introduction

Relation extraction (RE), which aims to identify semantic relations between entities from plain text, is one of the most fundamental tasks in information extraction and natural language processing. Previous research efforts on RE mainly concentrate on sentence-level, attempting to predict the relations between two entities within a sentence [Zeng et al., 2014; Zhou et al., 2016; Zhang et al., 2018; Wei et al., 2020]. However, a large number of relations are expressed by more than one sentence and can only be extracted by reading and reasoning over multiple sentences. For this reason, many researchers start to explore document-level RE [Quirk and Poon, 2017; Gupta et al., 2019; Christopoulou et al., 2019].

∗Equal contribution.

†Corresponding author.

same sentence and thus have long-distance dependencies is not trivial. Therefore, the identification of these two relational facts requires reasoning over more than one sentence. For example, the relation between Espoo Cathedral Parish and Finland is indirectly established by a reference The Cathedral at the beginning of sentence 2, which requires performing coreference reasoning over sentence 1 and sentence 2. While the relation between Espoo Cathedral and Espoon keskus is indirectly established by a bridge entity Espoo Cathedral, requiring performing logical reasoning over sentence 1 and sentence 6.

These examples indicate that a model for document-level RE needs to be able to effectively capture complex interactions between long-distance entities across multiple sentences in the document. Most recent studies construct a documentlevel graph based on syntactic dependency, heuristics, and other factors, then use graph neural networks to perform multi-hop reasoning to obtain meaningful entity representations [Zhang et al., 2018; Nan et al., 2020; Zeng et al., 2020]. However, these methods only take into account the overall graph structure, so they may ignore local contextual information surrounding the target entity pair. Moreover, as a result of the over-smoothing problem, these methods may fail to model the interactions between long-distance entities. In addition to graph-based models, some path-based models attempt to extract paths between the target entity pair that retain sufficient information to predict the relations [Xu et al., 2021; Huang et al., 2021]. These methods explicitly consider the above-mentioned reasoning skills and can alleviate the problem of modeling long-distance entity interactions, but they treat each path separately and not all entity pairs can be connected by a path.

In this paper, we propose a novel SubGraph Reasoning (SGR) framework for document-level RE. SGR combines the benefits of both graph-based and path-based models, integrating various paths into a much simpler subgraph structure to perform various reasonings at once. More specifically, we start by building a heterogeneous graph with three different types of nodes: entity nodes, mention nodes, and sentence nodes. Then, considering the reasoning skills required for document-level RE, we heuristically design a simple but extremely effective strategy to generate reasoning paths. These paths not only intuitively model all potential reasoning skills, but they also roughly cover the annotated supporting sentences in practice. Furthermore, the path generation strategy ensures that all entity pairs could be connected by a path. After the reasoning paths are generated, inspired by the success of incorporating subgraph structure in knowledge graph completion [Teru et al., 2020], question answering [Yasunaga et al., 2021], link prediction [Cai and Ji, 2020], etc., we extract a subgraph around the target entity pair based on the previously generated paths and finally apply an R-GCN [Schlichtkrull et al., 2018] on the subgraph. In this way, the model could focus on the most crucial entities, mentions, and sentences and perform joint reasoning over various reasoning paths.

We evaluate SGR on DocRED, experimental results show that SGR achieves better performance than existing models, and further analyses demonstrate that our method is both effective and explainable.

2 Methodology

In this section, we first briefly introduce the document-level RE task formulation, and then describe the proposed model in detail. Figure 2 gives an illustration of our approach.

2.1 Task Formulation

Given a document d containing sentences {si}ns i=1 and entities {ei}ne i=1 , the task of document-level RE is to predict the relations R′ ⊂ R between each entity pair (eh, et), where R is a pre-defined relation set, eh and et refer to head and tail entities, respectively. An entity ei may appear multiple times in the document via its corresponding mentions {mij }nei j=1 . A relation r ∈ R exists between entities eh and et if it is expressed by any pair of their mentions.

2.2 Document Encoding

For a document d = {wi}|d| i=1 , we encode it into a sequence of vectors {gi}|d| i=1 :

where xi is the concatnation of wi's word embedding and entity type embedding.

where Ew(·) and Et(·) denote the word embedding layer and entity type embedding layer respectively. ti is the corresponding entity type id of wi.

2.3 Subgraph Reasoning

Document Graph Construction

Since an entity may spread across multiple sentences in the document via its corresponding mentions, we build a heterogeneous graph G comprised of entity nodes, mention nodes, and sentence nodes. The node embeddings are initialized as follows:

Then, to explicitly model the interactions among entities, mentions, and sentences, we introduce three types of edges.

• Mention-Entity Edge: We connect a mention node to an entity node if the mention refers to the entity in order to model the co-reference of mentions.
• Mention-Sentence Edge: The co-occurrence of mentions in a sentence may indicate a relation. Therefore, we add an edge between a mention node and a sentence node if the mention resides in the sentence.
• Sentence-Sentence Edge: Unlike previous methods that connect all sentence nodes with each other [Christopoulou et al., 2019; Wang et al., 2020], we only add edges between two sentence nodes that corresponding sentences are adjacent in the document to keep the sequential information.

Reasoning Path Generation

Our approach aims to predict the relations by focusing on relevant entities, mentions, and sentences rather than the entire document. To this end, we assume that the paths between two target entities, containing a subset of all nodes in the graph, provide sufficient information to determine the relations. Therefore, inspired by [Xu et al., 2021], we introduce the following reasoning paths, which connect two target entities and explicitly model the four types of reasoning skills we discussed earlier. The meta-path schemes of these paths are illustrated in Figure 3.

• Intra-sentence Reasoning Path: The relation between two entities co-occurred in the same sentence (e.g., 1st relational fact in Figure 1) can be formulated as a path in the form of E-M-S-M-E. "E", "M", and "S" are entity node, mention node, and sentence node, respectively, "-" denotes edge. The intra-sentence reasoning path models two types of reasoning: pattern recognition and common-sense reasoning, as they generally perform reasoning inside the sentence.
• Inter-sentence Reasoning Path: For two entities that have no mentions in the same sentence, we use the intersentence reasoning path to model the relations between them. It can be thought of as adding additional bridges to the intra-sentence reasoning path, as shown in Figure 3. According to the different kinds of bridges, we introduce logical reasoning path and co-reference reasoning path. Logical Reasoning Path: The relation indirectly established by a bridge entity (e.g., 2nd relational fact in Figure 1) can be formulated as a path in the form of E-M-SM-E-M-S-M-E, which models logical reasoning by performing reasoning over sentences containing the head, bridge, and tail entities, respectively.

Co-reference Reasoning Path: The relation between two entities in adjacent sentences is mostly established by a reference (e.g., 3rd relational fact in Figure 1). It can be formulated as a path in the form of E-M-S-S-M-E, which models co-reference reasoning by performing reasoning over two adjacent sentences containing the head and tail entities respectively.

Naturally, these two types of reasoning paths are just special cases of the inter-sentence reasoning path, which contain only a single bridge. In fact, with repetition, there could be several different kinds of bridges in an inter-sentence reasoning path.

As a result of reasoning path generation, all entity pairs could be connected by at least one path and these paths are able to capture both the semantic and structural correlations between the entity pair. The problem is that even if we generate paths under the meta-path scheme, there would still be a large number of paths for an entity pair, but only few of them are necessary.

Hence, we simply constrain the scale of paths by limiting the number of bridges, thus generating fewer but better paths to remove the noise from irrelevant paths. Specifically, given an entity pair, we start with searching for intra-sentence reasoning paths (0 bridges) between them, if none are found, turn to search for inter-sentence reasoning paths without repetitions (1 bridge), gradually relaxing the limit on the number of bridges by allowing more and more repetitions until at least one path is found.

To verify the effectiveness of our heuristic path generation strategy, we compare the generated paths with the annotated supporting sentences on the development set of DocRED. The results show that the supporting sentences for 75.2% of relation instances are covered by the sentences contained in the generated paths and 53.5% are exactly the same. Moreover, we find that the paths between each entity pair contain an average of 1.8 sentences, implying that up to 80% of the sentences in the document could be filtered out. Consequently, the reasoning paths generated by our strategy are both sufficient and non-redundant for relational reasoning.

Subgraph Extraction

In document-level RE, there could be multiple relations between an entity pair, corresponding to multiple reasoning paths. In addition, determining a single implicit relation may also require multiple reasoning paths.

Previous works [Huang et al., 2021; Xu et al., 2021] tend to use each path to predict the relations independently and then aggregate the results. In contrast, we extract an enclosing subgraph G ′ around the entity pair in the graph G to integrate different paths. Specifically, the subgraph G ′ is an induced subgraph of G, formed from the nodes that appear at least once on the paths and all of the edges in G among those nodes.

As an example shown in Figure 2, for two target entities that do not appear in the same sentence but can be associated by a bridge entity or a bridge sentence, we generate two inter-sentence reasoning paths without repetitions (a logical reasoning path and a co-reference reasoning path) between them, and then use these paths to extract a subgraph.

Subgraph Encoding

In this stage, we first introduce a super node z, and connect z to the target entity nodes in the subgraph. The super node embedding hz is initialized as the max-pooling of the target entity node embeddings.

Then we employ an L-layer stacked R-GCN [Schlichtkrull et al., 2018] on the subgraph, which applies message passing for different types of edges separately in each layer. Given node u at layer l, the aggregation of u's immediate neighbors is defined as:

where σ(·) is an activation function, N tu is the neighbors of node u under edge type t ∈ T , cu,t = |N tu | is a normalization constant, W (l) t ,W (l) 0 ∈ Rdh×dh are trainable parameters.

In particular, we set the L to 2 to enable the super node to aggregate information from the target entity nodes and their subordinate mention nodes, as well as the sentence nodes to aggregate information from all entity and mention nodes in the subgraph.

2.4 Classification

We formulate the document-level RE task as a multi-label classification task. To calculate the probabilities of relations between an entity pair (eh, et), we use information from both the global document encoder and the local subgraph. Specifically, we concatenate the following representations: (1) the linear combination of the initial target entity node embeddings hc = Wc([heh ; het ]), which provides global entity-aware information of the target entity pair; (2) the learned super node embedding h(L) z , which provides local entity-aware information of the target entity pair; (3) the max-pooling of all learned sentence node embeddings h(L) G′ = Max ({h(L) si ), which provides local contex-

tual information of the target entity pair; (4) the entity distance embedding Ed(dht), where dht is the relative distance between the first mentions of the target entity pair in the document, Ed(·) is the relative distance embedding layer. The concatenated representation is then passed through an MLP:

We use binary cross entropy loss to train our model:

where S denotes the whole corpus, I(·) refers to the indication function.

3 Experiments

3.1 Dataset

We evaluate our model on DocRED, a large-scale humanannotated dataset for document-level RE constructed from Wikipedia and Wikidata. DocRED contains 3,053 documents for training, 1,000 for development, and 1,000 for testing, involving 96 relation types, 132,275 entities, and 56,354 relational facts. Moreover, more than 40.7% of relational facts can only be extracted from multiple sentences, and 61.1% of relation instances require multiple reasoning skills. It also provides supporting sentences for each relation instance as part of the annotation.

3.2 Experimental Settings

Following the settings of [Yao et al., 2019], We use GloVe (100d) [Pennington et al., 2014] and BiLSTM (256d) [Hochreiter and Schmidhuber, 1997] as word embedding and encoder, respectively. With setting the batch size to 4, we train our model using AdamW [Loshchilov and Hutter, 2019] optimizer, a linear learning rate scheduler with 6% warmup, and a maximum learning rate of 0.01. All hyperparameters are tuned based on the development set.

3.3 Baselines and EvaluationMetric

We compare our model with existing sequence-based models, including CNN/LSTM/BiLSTM based models evaluated in [Yao et al., 2019] and HIN [Tang et al., 2020]; graphbased models, including GAT [Velickovic et al., 2018], GCNN [Sahu et al., 2019], EoG [Christopoulou et al., 2019], AGGCN [Guo et al., 2019], LSR [Nan et al., 2020], GAIN [Zeng et al., 2020], and SIRE [Zeng et al., 2021]; and path-based models, including DRN [Xu et al., 2021] and Paths+BiLSTM [Huang et al., 2021].

Following [Yao et al., 2019], we use F1 and Ign F1 as the evaluation metrics. Ign F1 denotes F1 scores excluding relational facts that exist in both training and development/testing sets. Moreover, F1 scores for intra- and inter-sentence entity pairs are also evaluated on the development set. The results on the testing set is evaluated through CodaLab 1 .

3.4 Main Results

As shown in Table 1, our proposed model SGR achieves 57.49% F1 and 55.82% Ign F1 on the development set, 57.15% F1 and 55.12% Ign F1 on the testing set, significantly outperforming all strong baselines. Specifically, SGR obtains 3.85 F1 and 3.97 Ign F1 points higher than the sequencebased model HIN on the testing set. Compared with the recent path-based models DRN and Paths+LSTM, SGR also achieves a better performance. Moreover, SGR outperforms the state-of-the-art graph-based model SIRE in the same experimental setting by 1.19% and 1.08% in terms of F1 and Ign F1 on the testing set. These results indicate that our proposed model combines the benefits of both path-based models and graph-based models by leveraging both well-designed reasoning paths and subgraph structure to learn more expressive features.

1 https://competitions.codalab.org/competitions/20717

Meanwhile, we can see that SGR outperforms all baseline models on both intra- and inter-sentence settings, which demonstrates that subgraph reasoning improves performance on inter-sentence relation instances, while intra-sentence relation instances benefit from it as well.

3.5 Performance Analysis

To further assess the effectiveness of SGR, we divide the development set into several subsets based on different analytical strategies, and evaluate models trained with or without subgraph reasoning on each subset.

Entity Distance. In this part, we explore our model's capacity for capturing the relations between entity pair in longdistance. For this purpose, we examine the model performance in terms of entity distance, which is defined as the relative distance between the first mentions of the target entity pair in the document, and report the F1 score in Figure 4a. The first thing to note is that as the entity distance increases, the F1 score gradually decreases, which is consistent with human intuition that capturing long-distance relations is still a challenging problem for document-level RE. Also, the model with subgraph reasoning consistently outperforms the model without subgraph reasoning, especially when the entity distance ≥ 64. This is because explicitly incorporating the graph structure breaks the limitations of sequence modeling. In addition, the finer subgraph structure further makes model focus more on the target entity pair and related entities, thus reducing the influence of unrelated entities in long-distance.

Average Number of Entity Mentions. In order to investigate our model's capacity for aggregating information from multiple mentions, we examine the model performance in terms of average mention number of the target entity pair. From Figure 4b we can first observe that models do not perform well when the average mention number is small, which reveals that the information carried by a single mention is quite limited, making relations harder to be predicted. When the average mention number is large, however, not all mentions are necessary to predict the relations, and aggregating information indiscriminately may introduce noise from irrelevant mentions. Nevertheless, the subgraph structure makes model focus more on mentions that are relevant to the target entity pair, so the model with subgraph reasoning consistently maintains a relatively high performance with the increase of average mention number, especially when the average mention number ≥ 4.

3.6 Case Study

In Figure 5, we present two case studies to better illustrate the reasoning process and the effectiveness of our proposed model. In the first case, we can first infer that the Skai TV in sentence 1 is part of the Skai Group in sentence 2 by coreference reasoning. Then, using Skai TV as a bridge entity, we can further infer that Greece is the country of Skai Group

[1] Skai TV is a Greek free-to-air television network based in Piraeus. [2] It is part of the Skai Group, one of the largest media groups in the country. ... [5] Skai TV is also a member of Digea, a consortium of private television networks introducing digital terrestrial transmission in Greece.

Relation: P17

Supporting Sentences: [1], [2], [5]

Prediction: P17

Sentences contained in Subgraph: [1], [2], [5]

[1] The Beibu Gulf Economic Rim or Beibu Economic Belt also known as Gulf of Tonkin Economic Belt defines the economic region or rim surrounding around China's southwestern coastal region and cities around the Gulf of Tonkin. [2] The region is part of Chinese government's "Go West" strategy, to boost its less developed western regions. [3] The implementation of the campaign, has resulted in many construction projects in cities on the Beibu Gulf Rim, especially in Guangxi. [4] The Beibu Gulf economic rim has emerged as a new highlight of China - ASEAN cooperation, especially between Vietnam, who is also cooperating in this economic zone. [5] It covers Guangdong, Hainan and Guangxi, and northern and central Vietnam.

Relations: P17, P131

Supporting Sentences: [2], [3], [4], [5]

Predictions: P17, P131

Sentences contained in Subgraph: [2], [3], [4], [5]

Subgraph:

by logical reasoning. The preceding reasoning process can be expressed as an inter-sentence reasoning path with both a bridge sentence and a bridge entity. In the other more complicated case that two target entities, China and Guangxi, are mentioned in each of the five sentences, we can generate four reasoning paths between them and use these paths to extract a subgraph to make predictions. It should be noticed that, in these two cases, not only does our proposed model correctly predict the relations, but the sentences contained in the subgraph extracted by our proposed model also accurately cover the supporting sentences.

4 RelatedWork

Various approaches have been shown effective in documentlevel RE, which can be roughly divided into sequence-based models, graph-based models, and path-based models.

Sequence-based Models. These models mostly rely on LSTM- or Transformer-based architectures, which can model complex interactions among entities by implicitly capturing long-distance dependencies. [Yao et al., 2019] use CNN/LSTM/BiLSTM to encode the document into a sequence of hidden state vectors, and then compute entity representations to predict the relations for each entity pair. The hierarchical inference network (HIN) is proposed by [Tang et al., 2020], which uses multi-granularity inference information including entity-level, sentence-level, and documentlevel to better model complex interactions between entities. [Ye et al., 2020] propose a language representation model to better capture and represent the co-reference information. [Zhou et al., 2021] use attentions learned from transformer to locate relevant context and adopt an adaptive threshold for each entity pair.

Graph-based Models. These models generally construct a graph with words, mentions, entities, or sentences as nodes, and predict the relations by reasoning on the graph. [Christopoulou et al., 2019] propose an edge-oriented graph neural model that focuses on encoding information into edge representations rather than node representations. [Nan et al.,

2020] propose the latent structure refinement (LSR) model, which dynamically induces the latent document-level graph to empower the relational reasoning across sentences. [Zeng et al., 2020] propose the graph aggregation and inference network (GAIN), which constructs two graphs of different granularity: heterogeneous mention-level graph and entity-level graph to respectively model the interactions among different mentions and conduct a path reasoning mechanism to predict the relations. [Zeng et al., 2021] propose the separate intra- and inter-sentential reasoning (SIRE) model to represent intra- and inter-sentential relations in different ways, as well as a new and straightforward form of logical reasoning module to cover all cases of logical reasoning chains.

Path-based Models. These models typically extract reasoning paths between the entity pair, and perform relational reasoning with them instead of reading the entire document. [Xu et al., 2021] model the paths of reasoning skills explicitly, propose the discriminative reasoning network (DRN) to encode the reasoning paths, and then estimate the relation probability distribution of different reasoning paths. [Huang et al., 2021] select supporting sentences from the document using several well-designed handcrafted rules and combining them with a BiLSTM to predict the relations.

5 Conclusion

In this paper, we propose a novel subgraph reasoning (SGR) framework for document-level RE that combines the advantages of both graph-based models and path-based models. In particular, we extract a subgraph around the target entity pair based on corresponding reasoning paths and perform relational reasoning by applying an R-GCN on the subgraph, which considers all potential reasoning skills. Besides, the reasoning paths generated by our designed heuristic strategy explicitly model the document-level RE required reasoning skills and roughly cover the supporting sentences for each relation instance. Experimental results show that SGR outperforms existing models on both intra- and inter-sentence settings. Further analyses demonstrate that our method is both effective and explainable.

Acknowledgments

This research was supported by the National Natural Science Foundation of China under grant 61932002.