Legal Judgment Prediction via Multi-Perspective Bi-Feedback Network

1 Introduction

The task of legal judgment prediction (LJP) is to determine the judgment results based on the fact descriptions of the cases. Such techniques are crucial for legal assistant systems, which are helpful for ordinary people who are unfamiliar with legal terminology and complex procedures in legal consulting. Moreover, it can also serve as the handy reference for professionals (e.g., lawyers and judges) and improve their work efficiency.

So far, LJP has been regarded as a classic text classification problem and studied for many years [Segal, 1984; Lauderdale and Clark, 2012; Ye et al., 2018; Luo et al., 2017].

∗Corresponding Author

However, there still exist two major challenges that make existing methods not performed well as detailed below:

The first challenge is to exploit the topological dependencies among multi-task results efficiently. Generally, LJP consists of multiple subtasks, such as the charge prediction and the term of the penalty prediction [Xiao et al., 2018]. Moreover, [Zhong et al., 2018] consider that the law article prediction is also one of the fundamental subtasks in countries with the civil law system (e.g., China and Germany), and these multiple subtasks have a strict order in the real world. Specifically, appropriate law articles are decided by human judges at first, and then charges are determined according to the law articles. Finally, the term of the penalty is further confirmed based on laws and charges. [Zhong et al., 2018] explore the topological dependencies of LJP multiple subtasks according to the above order, and propose a topological multi-task learning framework. However, their work ignores the interaction of task results. Most of the multi-task learning methods ignore the dependencies among multi-task results too, since they generally focus on sharing representations (or some encoding layers) among relevant tasks [Liu et al., 2015] or sharing hard/soft parameters [Zhang and Yang, 2017]. In some scenarios where multitasking exists topology dependencies as LJP, the prediction results of each task can also affect and verify its related tasks. For example, if the applicable laws are predicted as property infringement, then the content related to property infringement in the fact description should be emphasized when predicting charges. Meanwhile, the prediction of the charges can also verify the rationality of predicted applicable laws. Therefore, it is a major and meaningful challenge to exploit the result of each task to infer and validate other tasks based on the topological dependencies.

The second challenge is to distinguish the cases of similar descriptions with different penalties accurately. In LJP, some cases have similar fact descriptions with different judgments. [Hu et al., 2018] manually introduce several discriminative attributes of charges as the internal mapping between fact descriptions and charges, which provide effective signals for distinguishing confusing charges (also suitable for laws). However, the term of penalty is difficult to distinguish with limited attributes. For example, the cases "Zhang smuggled gold 10kg and drugs 10g", and "Zhang smuggled drugs 10kg and gold 10g" have the same charge, but completely different penalties in the real world. Most existing work cannot distinguish them, which is mainly because the word collocation information in the fact description is ignored. Moreover, the semantics associated with numbers are not well interpreted. Existing natural language techniques often ignore the semantics of numbers because numbers are not crucial in general problems. However, when predicting the term of the penalty, the semantics of number in the text is quite crucial. Therefore, how to utilize word collocations effectively and extract the semantics of numbers to distinguish cases with similar descriptions but different penalties is also a major challenge.

To solve the above challenges, in this paper, we propose a Multi-Perspective Bi-Feedback Network (MPBFN) with Word Collocation Attention (WCA) mechanism. To incorporate the predicted results of subtasks into the network, we design the MPBFN with multi-perspective Forward Prediction (FP) and Backward Verification (BV), i.e., bi-feedback, which improves the overall performance of LJP. Specifically, we first map the intermediate prediction result of each task into the latent state space. Then we merge the latent state with the semantic vector of fact and obtain the task resultbased semantic vector, which is utilized to make the FP for its follow-up tasks. Meanwhile, we also make a BV to check the rationality of its pre-order tasks based on the latent state. Finally, for each task, we combine all prediction and verification results, which we call multi-perspective (MP), to obtain the final prediction results. To better distinguish cases with similar descriptions but different penalties, we extract the collocation information and the semantics of numbers with attention mechanism. Specifically, we first consider the number as a digit string and embed it by digit to obtain the number semantic vector. Then, we extract the collocations in the fact description and obtain the weights of collocations by result-based attention mechanism. Finally, we combine weighted collocation features with MPBFN to optimize the performance of the penalty prediction.

The main contributions of this paper are presented as follows:

1. We are the first to focus on and utilize the dependencies among prediction results of multiple subtasks in LJP and propose the MPBFN.
2. We extract the semantics of numbers and utilize the word collocation information in the fact description, and propose a WCA mechanism to distinguish cases with similar descriptions but different penalties.
3. We evaluate the MPBFN-WCA with two real-world datasets, and our model significantly outperforms all baselines on all subtasks.

2 RelatedWork

In this section, we discuss the related work in two aspects.

Legal Judgment Prediction has been studied for decades and most existing work formalizes this task under the text classification framework [Kort, 1957; Segal, 1984; Lauderdale and Clark, 2012]. Since machine learning has been proven successful in many areas, researchers begin to formalize LJP with machine learning methods. [Sulea et al., 2017] develop an ensemble system, which averages the output of multiple SVM classifiers to improve the performance of LJP related text classification. Recently, neural network-based methods have been applied to LJP and have achieved better results than traditional machine learning methods. [Luo et al., 2017] propose an attention-based neural network method to jointly model the charge prediction task and the relevant article extraction task in a unified framework, which solves multi-label tasks in LJP. Besides, [Ye et al., 2018] explore charge labels to solve the non-distinctions of fact descriptions and adopt a Seq2Seq model to generate court views and predicted charges. [Hu et al., 2018] propose an attributeattentive charge prediction model to infer the attributes and charges simultaneously, which is the first to focus on few-shot and confusing problems. Moreover, legal judgment usually consists of complicated subclauses, and there exists a strict order among the subtasks of legal judgment. [Zhong et al., 2018] first explore and formalize the multiple subtasks of legal judgment, and propose a topological multi-task learning framework. However, their work ignores the interaction of task results, which is quite meaningful in LJP.

Multi-Task Learning has numerous successful usages in NLP tasks. Sharing representations (or some encoding layers) among relevant tasks [Liu et al., 2015] or sharing hard/soft parameters [Zhang and Yang, 2017; Liu et al., 2018] are two general ways. For instance, [Luong et al., 2015] share encoders or decoders to improve one (many) to many neural machine translation. [Guo et al., 2018] improve abstractive summarization via multi-task learning with the auxiliary tasks of question generation and entailment generation. [Lin et al., 2018] propose a multi-lingual multi-task architecture to develop supervised models with a minimal amount of labeled data for sequence labeling. Different from existing methods, in this paper, we utilize the dependencies between the prediction results of multiple subtasks and proposed a MultiPerspective based Bi-Feedback Network.

3 Method

In this section, we describe the Multi-Perspective based BiFeedback Network (MPBFN) and a Word Collocation Attention (WCA) mechanism. Specifically, the problem formulation and overview are first provided in Section 3.1 and Section 3.2, respectively. Then, we provide a neural encoder for fact descriptions in Section 3.3. Next, we introduce the MPBFN and a WCA mechanism in Section 3.4 and 3.5, respectively. The training process of the model is finally introduced in Section 3.6.

3.1 ProblemFormulation

For each case, suppose the fact description is a word sequence A = {a1, ..., al}, where l is the number of words. Moreover, we extract n word collocations from the fact description by Stanford CoreNLP toolkit [Manning et al., 2014], and obtain the word collocation sequence COL = {col1, ..., coln}. Given A and COL as input, our task is to predict the judgment results of applicable law articles, charges and term of penalty, which is a multi-task classification problem.

3.2 Overview

Our MPBFN-WCA consists of three parts, i.e., the encoder, MPBFN, andWCAmechanism. In this paper, we employ the CNN encoder [Kim, 2014] to generate the semantic vector of the fact description.

After getting the encoder, we design the MPBFN as the decoder. Before proposing MPBFN, we first introduce the topology dependencies among subtasks in LJP. As mentioned in Section 1, there exists a strict order among subtasks of LJP [Zhong et al., 2018], which is shown in Figure 1. To facilitate the following calculations, we define the law article prediction, charge prediction, and penalty term prediction as task 1, task 2, and task 3, respectively.

Based on the topology, we design the MPBFN to utilize the dependencies among prediction results of subtasks in LJP. In MPBFN, the result of each task can infer its follow-up tasks, and can also verify its pre-order tasks. For instance, according to the law article prediction (task 1), we emphasize the content related to applicable laws in the fact description and make a more attentive FP for the charge (task 2), denoted as FP (1, 2). Conversely, based on the charge prediction, we make a BV(2, 1) to check whether the law articles are applicable. That is the philosophy of bi-feedback. In Figure 1, the solid red lines express the FP, and the dotted blue lines express the BV. Besides, a task may have more than one follow-up task and pre-order task. For each task, different pre-order/follow-up tasks may have different prediction/verification results because different subtasks pay attention to different parts of the fact description. Combining the results based on different related tasks can obtain more accurate results. That is what we call the MP.

Finally, to distinguish cases with similar descriptions but different penalties, we propose the WCA mechanism. We extract the semantic features of numbers and collocations in the fact description and combine them with MPBFN to predict task 3 more accurately.

The general framework of MPBFN-WCA is shown in Figure 2, where factori expresses the original fact semantic vector, resi is comprehensive FP result of task i, and ˆ yi denotes the final prediction result of task i.

3.3 Neural Encoder for Fact Descriptions

In this section, we employ the CNN encoder [Kim, 2014] to generate the semantic vector of the fact description.

Given a word sequence A = {a1, ..., al}, we first convert each word ai into its word embedding xi ∈ Rdw , and get the word embedding sequence X = {x1, ..., xl}, where dw is the dimension of the word embedding.

For X, we make a convolution operation with the convolution martix Wc ∈ Rdc×(h×dw) by

where xi:i+h-1 is the concatenation of word embeddings in the i-th window, bc ∈ Rdc is the the bias vector, dc is the number of filters and h is the length of a sliding window. We apply the convolution over each window i and get C = {c1, c2, ..., cl-h+1}.

Finally, we apply the max-pooling over C by

and obtain the fact semantic vector

3.4 Multi-Perspective Based Bi-Feedback Network

In this section, we first introduce the FP and BV, respectively. Then, we calculate the final result of each task by combining results of multi-perspective FP and BV.

Forward Prediction

In LJP, each category of a subtask result has a specific meaning. For example, in the law article prediction task, each category of the result represents a specific law article. Therefore, for task i, we employ a latent state matrix Si ∈ Rti×ds (an embedding layer) to express the latent state of the result, where ti is the number of result categories of task i, and ds is the dimension of the latent state.

As shown in Figure 2, we build the network according to the topology. Assume that we have predicted the result distribution of task i, resi ∈ Rti based on its pre-order tasks (see Section "Multi-Perspective Integration" for details). We combine the result distribution resi with the latent state matrix Si , and obtain the result-based latent state vector lsvi ∈ Rds by

where j expresses the j-th category of the result of task i.

Then, we map the latent state vector lsvi to the corresponding semantic space semi ∈ Rdc by

where Wsi ∈ Rdc×ds is the transformation matrix from latent state space to semantic space of task i and elu is an activation function proposed by [Clevert et al., 2015].

We merge the result-based semantic vector semi with the semantic vector of the fact description factori, and obtain the task i based fact semantic vector by

where ⊗ is an element-wise product function. Finally, we predict each follow-up task j of task i by

where Wfi,j ∈ Rtj×dc is the fully connected matrix and bfi,j ∈ Rtj is the bias vector.

Backward Verification

Conversely, the result of each task can verify pre-order tasks. For task i, we first calculate the result-based latent state vector lsvi by Eq. (4). Then we map lsvi to the gate vector space of its pre-order task j, and obtain gatei,j ∈ Rtj by

where Wgi,j ∈ Rtj×ds is the fully connected matrix and bgi,j ∈ Rtj is the bias vector.

We employ the sigmoid function to determine whether each category of the result of task j is available based on the result of task i. That is why we call gatei,j the gate vector.

Multi-Perspective Integration

After we obtain FP and BV results, in this section, we merge these multi-perspective results and calculate the final result of each task.

First, we summarize multi-perspective FP results for each task in topological order, i.e., before solving task i, each task j that j < i in topological order has been solved. For task i, we calculate the comprehensive FP result resi ∈ Rti by

where pi is the number of pre-order tasks of task i and norm(x) is the normalization function. We take the intersection of each FP result and obtain a more accurate result.

Similarly, the comprehensive BV result of task i veri ∈ Rti can be calculated by

where ui is the number of follow-up tasks of task i and e is the vector with 1 in each dimension.

Finally, we merge the results of comprehensive FP and BV of task i by

which is the final prediction result of task i.

3.5 Word Collocation AttentionMechanism

In this section, we propose a number embedding method and a word collocation attention mechanism to increase the accuracy of the penalty prediction in cases where the fact description is similar.

Number Embedding

In the penalty prediction, numerals with their units have a significant impact. In this section, we propose a digital bitwisebased embedding method to obtain the number embedding.

In particular, we first convert a number i into a fixed lnbit vector digi = {digi,1, digi,2, ..., digi,ln}. The unit digit corresponds to dig1, the tens digit corresponds to dig2, and so forth. We pad zeros when number i is less than ln-bit and discard high digits when exceeds.

Then, we map each digit into the semantic space by embedding layer, and obtain

where numi,j ∈ Rdn is the embedding vector of digi,j and we assure that dn × ln = dc.

Finally, we merge the number semantic vector numi with its corresponding unit's word embedding vector xk, and obtain:

where k is the position of the unit in the word sequence A (labeled by Stanford CoreNLP toolkit [Manning et al., 2014]), Wn ∈ Rdc×2dc is the fully connected matrix and bn ∈ Rdc is the bias vector.

Since the semantics of numerals is mainly used for the penalty prediction, we do not utilize the number embedding directly in the fact description encoder, but in the collocation embedding that will describe in the next section.

Collocation Attention

With the semantics of numbers, we further focus on the impact of word collocation on the penalty prediction in this section.

In the word collocation sequence COL = {col1, ..., coln}, each collocation coli = (wi,1, wi,2) contains two words. We convert each word to its corresponding semantic vector and obtain the collocation embedding ecoli = (ewi,1, ewi,2) and the sequence ECOL = {ecol1, ..., ecoln}, where ewi,j is the word embedding vector of word wi,j. Particularly, if a word in the collocation is a number with its unit, we replace the original word embedding vector with the vector that calculated by Eq. (13).

Then, we merge the word vector ewi,1 with ewi,2 and obtain the collocation semantic vector eci ∈ Rdc by LSTM [Hochreiter and Schmidhuber, 1997]

where hj is the hidden state vector of LSTM, and obtain the collocation semantic vector sequence EC = {ec1, ..., ecn}.

Different word combinations have different influences on different task results. Therefore, for each task j, we design an attention mechanism to obtain a result-based collocation semantic vector ectj ∈ Rdc by

where αj,i is the attention values of collocation i based on task j that can be calculated by

The Waj ∈ Rdc×dc is the a weight matrix and semj is calculated by Eq. (5).

Finally, for task j, we merge its collocation-based semantic vector ectj with the fact semantic vector factj, and get the mixed semantic vector fcj by

where Wm ∈ Rdc×2dc is the fully connected matrix and bm ∈ Rdc is the bias vector.

In this paper, we employ the mixed semantic vector to predict task 3, the term of penalty. Specifically, we replace the fact1 and fact2 with fc1 and fc2 when calculating pred1,3 and pred2,3 by Eq.(7).

3.6 Training

In this paper, we assume that the prediction result of each task is single-label. We use cross-entropy loss function for each subtask and take the sum as an overall loss by:

We train our model in an end-to-end fashion, and use Adam [Kinga and Adam, 2015] for optimization. Besides, we utilize the dropout after the fact semantic vector layer factori to prevent overfitting.

4 Experiments

In this section, we demonstrate the effectiveness of our model. We first introduce datasets and data processing. Then we provide the necessary parameters of our model. Finally, we compare the performance of our model with baselines and analyze the effect of each module in our model.

4.1 Dataset Construction and Experimental Setup

In this section, we use two public datasets from Chinese AI and Law challenge (CAIL2018) [Xiao et al., 2018], i.e., CAIL-small (the exercise stage data) and CAIL-big (the first stage data).

The cases in the datasets contain fact descriptions, applicable law articles, charges and the terms of penalty. Also, there exist cases with multiple applicable law articles and multiple charges. As our model aims to explore the effectiveness of considering result-based topological dependencies between various subtasks, we filter out these multi-label samples.

Moreover, there are also some infrequent charges and law articles, for example, insulting the national flag and national emblem. We filter out these infrequent charges and law articles and only keep those with frequencies greater than 100. For the term of penalty, we divide the terms into nonoverlapping intervals as [Zhong et al., 2018]. The detailed description of the datasets is shown in Table 1. Particularly, there exists a validation set with 12,787 cases in CAIL-small.

Since the cases are written in Chinese, we use THULAC [Sun et al., 2016] for word segmentation. For word embedding, we set the frequency threshold as 25, and initialize the word embedding vector with a uniform distribution directly. We extract the word collocations in the fact description by Stanford CoreNLP toolkit [Manning et al., 2014]. Specifically, we employ the CoreNLP to extract the basic dependencies in each sentence of the case as collocations and set the maximum number of collocations in each case as 128. Particularly, we treat the number and its unit as one word. For CNN-based model in baselines, we set the maximum document length as 512 words; for LSTM-based model in baselines, we set the maximum sentence length to 64 words and maximum document length to 64 sentences.

Meanwhile, we use Keras (https://keras.io/) framework to build neural networks. We set the word embedding size dw as 200, and the dimension of the latent state ds as 256. For CNN encoder, we set the number of filters dc as 256, and the length of sliding window h as 2,3,4,5, respectively (each kind of sliding window contains 64 filters) as [Kim, 2014]. Besides, we set the dimension of number embedding dn as 32, and the number of digit ln as 8, which meet dn · ln = dc.

In the training part, we set the learning rate of Adam optimizer as 0.001, and the dropout probability as 0.5. The batch size of all models is 128. We train every model for 16 epochs and evaluate the final model on the testing set.

4.2 Results

In this section, we compare our methods with existing methods and analyze the effort of each module in our methods.

To evaluate the performance of the proposed methods, we compare our methods with the following methods:

• FLA: a neural network method that capturing the interaction between Fact descriptions and applicable Laws with Attention mechanism. [Luo et al., 2017] .
• HARNN: A Hierarchical Attention based RNN for document classification [Yang et al., 2016].
• CNN: A CNN-based textual classification model [Kim, 2014].
• TOPJUDGE: A topological multi-task learning framework for LJP, which is the state-of-the-art method [Zhong et al., 2018].

For each method in the baselines, we train each method by the multi-task way and select a set of the best experimental parameters according to the range of the parameters given in their experiments. We adopt accuracy (Acc.), macroprecision (MP), macro-recall (MR) and macro-F1 (F1) which are widely used in the classification task to evaluate the performance of the baselines and our methods. Here, the macro-precision/recall/F1 are calculated by averaging the precision/recall/F1 of each category [Zhong et al., 2018]. All the models are repeated for 5 times, and we report the average values as the final results for clear illustration.

Experimental results on the test sets of CAIL-small and CAIL-big are shown in Table 2 and Table 3, respectively. The results show that MPBFN-WCA achieves the best performance on all metrics. Compared with TOPJUDGE, which performs best among baselines, MPBFN-WCA has enhanced the performance by about 2.75%, 1.54%, and 13.3% on F1score for law article prediction, charge prediction and term of penalty prediction in CAIL-small test set and about 2.24%, 1.64% and 8.92% in CAIL-big test set. In other methods of baselines, the performance of CNN and HARNN are not as good as TOPJUDGE because they do not take the topological properties of multi-task into consideration. FLA is the worst performer because their work does not focus on the singlelabel problem. The performance of all methods in CAIL-big dataset is better than in CAIL-small, which is because the training data of CAIL-big is more adequate.

Besides, we also compare the performance of the different modules in our methods. In Table 2 and Table 3, MPFP expresses the network of Multi-Perspective based Forward Prediction (no backward verification), MPBFN expresses the Multi-Perspective based Bi-Feedback Network, and MPBFNWCA denotes MPBFN with WCA mechanism. Our MPBFN has a significant improvement in the term of penalty prediction. Even without WCA and BV, the penalty prediction result of MPFP is still better than baselines. We believe that this is caused by the multi-perspective mechanism since the penalty prediction has more than one pre-order task (it based on the result of law prediction and charge prediction). Compare with MPBFN, the WCA mechanism has enhanced the performance of term of penalty prediction by 4.81% on F1score in CAIL-small, and 3.11% in CAIL-big. Even only employ MPBFN, the performance is still better than TOPJUDGE in two datasets, which indicates that the result-based multi-perspective bi-feedback network has a stable improvement in the performance of LJP.

5 Conclusion

In this paper, we focus on result-based multi-task topological dependencies of LJP. Specifically, based on the topology structure between multi-tasks, we design a network structure with multi-perspective forward prediction and backward verification to improve the performance of multitasking. Moreover, we extract the collocations in the fact description and combine the attention mechanism to reduce the misjudgment of penalty prediction. The experimental results show that our model achieves significant improvements over baselines on all judgment prediction tasks.

Acknowledgments

This work is supported by Chinese National Research Fund (NSFC) Key Project No. 61532013 and No. 61872239. FDCT/0007/2018/A1, DCT-MoST Jointproject No. (025/2015/AMJ), University of Macau Grant Nos: MYRG2018-00237-RTO, CPG2018-00032-FST and SRG2018-00111-FST of SAR Macau, China.