Incorporating Legal Logic into Deep Learning: An Intelligent Approach to Probation Prediction

1 Introduction

In recent years, the application of artificial intelligence (AI) in the Intelligent Judicial Assistant System (IJAS) has increased steadily [Zhong et al., 2020; Medvedeva and Mcbride, 2023; Dong and Niu, 2021; Niklaus et al., 2021], primarily with the aim of improving judicial efficiency. Research has mainly focused on improving the performance of AI from two perspectives. The first focuses on continuously refining the extraction and learning of key information from judgment documents [Miao et al., 2024; Bhattacharya et al., 2022; Xiao et al., 2021]. These methods, however, mainly concentrate on superficial information and fail to deeply analyze the legal interpretation of texts. The second adopts data-driven techniques to build effective neural network models for legal judgment prediction (LJP) [Liu et al., 2023; Zhang et al., 2023; Zhang et al., 2024a]. However, these approaches often overlook the inherent legal logic.

Probation is a crucial component of sentencing [Matczak, 2021; Berryessa, 2022; Ruhland and Scheibler, 2022]. On the one hand, it offers criminals an opportunity for rehabilitation through non-custodial measures, enabling them to reform and reducing the likelihood of re-offending [Applegate et al., 2009; Phelps et al., 2022; Norman and Ricciardelli, 2022]. On the other hand, probation helps alleviate the burden on the prison system and contributes to social harmony by facilitating the reintegration of criminal [McNeill and Dawson, 2014; Harding et al., 2022; Okonofua et al., 2021]. Despite its importance, current IJAS lack dedicated methods for probation prediction, and research on the underlying factors influencing probation decisions remains limited.

Article 72 of the Criminal Law of the People's Republic of China 1 (Art. 72) stipulates that criminals sentenced to detention or fixed-term imprisonment of three years or less may be granted probation if they meet the following conditions at the same time:

• Condition (a). Mild circumstances of the crime.
• Condition (b). Evidence of remorse.
• Condition (c). No danger of re-offending.
• Condition (d). No significant negative impact on the community.

In addition, criminals sentenced to detention or fixed-term imprisonment of 3 years or less must be granted probation if they are under 18 years old, pregnant, or over 75 years old. In the Chinese judicial system, both the prerequisite and substantive conditions for probation form a crucial theoretical foundation for determining probation eligibility, which guide judges to assess a criminal's probation eligibility. Figure 1 illustrates a case example of intentional injury, highlighting key information relevant to probation eligibility of criminal. In this case, the defendant Chen was sentenced to a threeyear prison term, meeting the prerequisites for probation. In addition, his evidence of remorse and a low risk of recidivism further support his probation eligibility.

Due to limited AI adoption in probation research, we aim to enhance deep learning models with legal logic for more accurate predictions. However, this poses several challenges:

Diverse Legal Information in LJP Tasks. Different prediction tasks focus on distinct types of legal information. For example, in charge prediction tasks [Zhao et al., 2022; Liu et al., 2021; Zhang et al., 2024b], it is essential to incorporate not only the fact description but also the specific elements that constitute each charge, enabling a deeper understanding of its legal implications. In case matching tasks [Bi et al., 2022; Ge et al., 2021], the relevant legal articles and the semantic relationships among entities must be integrated for similarity matching. Similarly, in probation prediction tasks, the PLEs play a particularly crucial role. Therefore, combining fact description with PLEs is critical for improving the performance of probation prediction models.

Legal Logic in Multi-Task Learning. Multi-task learning approaches [Yao et al., 2020] have been employed to predict multiple labels simultaneously. However, these methods typically share the same input across all tasks, neglecting the logical dependencies that exist between tasks. However, in probation prediction task, different information must be provided for each subtask, based on the specific legal logic underlying probation decisions. This task-specific approach aligns with legal logic and has the potential to significantly enhance both the accuracy and interpretability of the predictions.

To address the first challenge, we construct a dataset tailored for probation tasks, focusing on the crime of intentional injury as a case study. This crime exemplifies the conflict between its inherently high subjective malice and the "minor circumstances" required for probation eligibility. To navigate this conflict in probation cases involving intentional injury, we begin by analyzing the interpretation of "minor circumstances" within the substantive conditions of probation. Currently, twomainstreamviewpoints exist. First, the crimemust involve relatively minor circumstances, indicating a low degree of personal danger posed by the criminal, thereby fulfilling the retribution purpose of probation, referred to as the "retributive factor". Second, the minor circumstances of the crime also suggest a low risk of re-offending, which satisfy the preventive purpose of probation, referred to as the "preventive factor". In this study, we extract the fact description to represent the retributive factors. Additionally, guided by legal expertise, we obtain PLEs reflecting preventive factors and other substantive conditions.

1 http://xingfa.org/

To address the second challenge, we first propose the TwoStage Probation Prediction Model Based on PLEs (TS-LE), which focuses on preventive factors using a cascading prediction framework. Building upon TS-LE, we introduce the Two-Stage Dual-Theory Probation Prediction Model (TSDT), which integrates both retributive and preventive factors, aligning with the Dual-Track Theory of Punishment to achieve a more balanced decision-making approach. Finally, extending TS-DT, the Multi-Task Dual-Theory Probation Prediction Model (MT-DT) adopts a multi-task framework, treating the prediction of prerequisites as an auxiliary task. This enhancement allows for the simultaneous optimization of prerequisites and substantive conditions, further improving prediction accuracy and interpretability. These models incorporate legal interpretation encoding of PLEs to better simulate the judicial decision-making process. By combining fact description with PLEs, these models provide a comprehensive and systematic assessment of probation eligibility. Validation on our constructed dataset demonstrates their effectiveness, with each successive model offering improved performance and deeper insights into the legal logic underlying probation decisions.

To sum up, this paper has three main contributions, summarized as follows:

• Dataset Construction. We construct a probation prediction dataset, which includes fact description and PLEs with legal interpretations.
• Legal Logic Analysis and Model Development. We conduct an in-depth analysis of the legal logic behind probation eligibility and proposed three progressively advanced models, TS-LE, TS-DT and MT-DT, to simulate the decision-making process for probation.
• Model Validation and Insights. Experimental results on the constructed dataset show that the MT-DT model effectively integrates both the retributive and preventive factors, while also addressing the "error amplification" problem in cascading tasks.

2 RelatedWork

2.1 Exploration of Factors in Probation

Several studies have explored the factors influencing probation eligibility. Cordier et al. [Cordier et al., 2021] analyzed the link between legal measures and recidivism risk, offering insights into probation outcomes. Similarly, Sims et al.

[Sims and Jones, 1997] studied 2,850 felony probationers in North Carolina and identified key factors such as demographics, criminal history, and probation conditions. Furthermore, Louden et al. [Louden and Skeem, 2012] used simulationbased experiments to highlight the complex role of mental health issues in probation management. Their findings emphasized the need for customized support and diverse intervention strategies. Williams et al. [Williams et al., 2014] developed a tool to assess recidivism risk among juvenile offenders. All of these approaches primarily rely on mathematical and statistical methods.

2.2 Application of Multi-TaskModels in LJP

Multi-task learning models have been applied to various LJP tasks. For instance, Lyu et al. [Lyu et al., 2022] proposed a method to enhance legal judgment prediction by reinforcing the extraction of criminal elements, integrating these elements into content inputs for multi-task predictors, with the goal of improving model accuracy. Yao et al. [Yao et al., 2020] introduced a gated hierarchical multi-task learning network, which models different legal tasks at multiple granularities, thereby improving the predictive performance of judicial decisions. Additionally, Yue et al. [Yue et al., 2021] proposed the NeurJudge, an environment-aware approach that divides fact description into different scenarios using intermediate subtask results, which are then leveraged to predict other subtasks. These studies provide important insights for our research on intelligent probation prediction models.

To the best of our knowledge, this is the first work to integrate legal logic with AI techniques for probation prediction. Our approach focuses on developing deep learning models that incorporate legal logic, with the aim of achieving efficient and reliable probation prediction.

3 Method

3.1 Overview

Figure 2 showcases the overall pipeline of our model-data Processing, text Encoder, and multi-task classifier-into a cohesive system.

3.2 Preliminaries

Probation Prediction. According to Art. 72, probation applies only to criminals sentenced to three years or less in prison or detention. Therefore, the probation prediction consists of two subtasks: Task 1: Predict whether the criminal's sentence meets the prerequisite of three years or less, or detention. Task 2: Predict the probation eligibility of criminal.

Retributive (Preventive) Factors. By analyzing the substantive conditions for probation, Condition (a) incorporates retributive or preventive factors. Retributive factors refer to elements that embody the principle of retribution within the probation system, primarily focusing on the criminal's personal danger or culpability. In this study, we represent retributive factors using fact description. Preventive factors, on the other hand, reflect the preventive purpose of probation and are included in the PLEs.

The Dual (Single)-Track Theory of Punishment. When Condition (a) accounts for the retribution factor, it is known as the Dual-Track Theory of Punishment. In contrast, when only the prevention factor is considered, it is known as the Single-Track Theory of Punishment. This distinction arises because the retribution factor has been incorporated into the sentencing process as a prerequisite for probation. Reincorporating this factor would constitute double retribution.

Probation Legal Elements. Guided by legal expertise, we analyze the substantive conditions for probation eligibility as outlined in Art. 72, as follows:

• Mild Circumstances of the Crime. For retribution, the fact description includes numerous retributive factors, such as the criminal's modus operandi and the consequences caused. For prevention, the underlying causes are considered, along with other subjective and objective circumstances, such as whether the criminal was forced to commit the crime.
• Evidence of Remorse. According to Article 225 of the Interpretation on the Application of the Criminal Procedure Law of the People's Republic of China, remorse requires a comprehensive assessment of the criminal's post-crime behavior, including actions such as compensation, expressions of remorse, and whether forgiveness has been obtained.
• No Danger of Re-offending. This condition should be analyzed from two perspectives. One is the dangerousness of the criminal's behavior, which focuses on the motivations behind the crime. Elements such as excessive self-defense (i.e., the criminal was coerced into committing the offense) are indicative of a lower risk of recidivism. The other perspective is the dangerousness of the individual, which considers the criminal history, as well as personal factors such as age, occupation, education, family environment, and social circumstances.
• No Significant Negative Impact on the Community. This condition consists of a variety of factors, including the criminal's motives, methods, and the outcomes of the crime, to determine the extent of social harm caused by the offense.

Based on the analysis of the substantive conditions, we identify 33 distinct probation legal elements. The distribution of their frequencies is presented in Figure 3.

3.3 ProblemFormulation

We assume that the fact description is denoted as F and the text sequence of PLEs as Q. Based on F and Q, the probation prediction task aims to learn a function ϕ, where ˆ y = ϕ(F,Q), to predict the probation eligibility of the criminal. This task consists of two subtasks: T = {Taux, Tmain}. The corresponding prediction results are ˆ yaux ∈ {0, 1} and ˆ ymain ∈ {0, 1}.

3.4 Legal-Enhanced Sequence Generator

As illustrated in Figure 2 (b), to provide legal implication to PLMs, we obtain the interpretation of each element from the encyclopedia entries and generate a corresponding text sequence. More specifically, we first apply regularization to match the elements of each case, as follows:

where Vi = {vi1 , vi2 , . . . , vik } is a vector that represents the presence of elements in the i-th judgement document, and vik ∈ {0, 1} indicates whether the k-th element is present (vik = 1) or absent (vik = 0). Here, H is the extraction function, R is the extraction rule, E represents the set of PLEs, and N is the number of judgement document.

Then, we extract the legal interpretation for each element from the encyclopedic entries, forming A as:

where Ak mains the legal meaning of the k-th element.

Finally, we search the knowledge base for the legal interpretation corresponding to the elements in the case and generate a text sequence of elements unique to the case Qi as:

where G represents the generating function.

3.5 Training and Prediction

In the multi-task training of MT-DT, the fact description is encoded as input for the auxiliary task, represented as:

where F denotes a sequence representing the fact description, F = {f1, f2, . . . , fm}, and m is the length of sequence.

According to the Dual-Track Theory of Punishment, we concatenate the fact description and the text sequences of PLEs, encoding them jointly as input for the main task. Specifically, this process can be represented as follows:

where Q denotes a word sequence representing the elements, Q = {q1, q2, . . . , qn}, and n is the length of sequence.

(b) Results on TS-LE, TS-DT, and MT-DT

Table 1: Performance comparison of three legal logic enhanced probation prediction models.

To obtain prediction results for the auxiliary and main task, we use two MLP classifiers:

In order to train the multi-task probation predictor, we compute the cross-entropy loss function for both the auxiliary and main task. The loss for the auxiliary task is formally computed as:

where yauxc represents the ground truth for the auxiliary task prediction associated with the c-th judgment document. Similarly, the loss for the main task is formally computed as:

where ymainc represents the ground truth for the main task prediction associated with the c-th judgment document.

During multi-task training, we sum the losses of the different subtasks as the total loss to train the multi-task model:

where λ represents the weight of the auxiliary task loss.

4 Analysis

In this section, we aim to answer the following research questions:

• (RQ1) How can the underlying legal logic of the methods be interpreted? (Section 4.1)
• (RQ2) Is the text sequence of PLEs valid? (Section 4.2)
• (RQ3) What is the impact of the hyperparameter λ in the multi-task model? (Section 4.3)

4.1 Legal Logic Analysis

As shown in Table 1 (a), to model the legal logic in the probation decision process, we establish three models with different frameworks and legal logic. In Table 1 (b), the results show that the multi-task model framework based on the DualTrack Theory of Punishment performs the best.

TS-LE. According to the two subtasks of the probation prediction task, in the TS-LE model, we use the fact description as input for Task 1. If the prediction result meets the prerequisites for probation eligibility, we use the text sequence of PLEs as input for Task 2 to predict the probation eligibility of criminals.

As shown in Table 1 (b), the TS-LE method achieves an accuracy of 95.17% in Task 1. The legal logic behind this lies in Article 234 of the Criminal Law of the People's Republic of China, which specifies that in cases of intentional injury, sentencing is determined by the means employed by the criminal and the resulting consequences. Therefore, determining whether a criminal meets the prerequisites for probation relies on the fact description, which includes the criminal's actions and the outcomes.

Next, we select cases from the dataset where the prediction result of Task 1 meets the prerequisites, and we input the text sequence of PLEs into the model to predict the probation eligibility of criminals. The results show that the accuracy is only 69.65%, which is less than the accuracy of the MT-DT method (88.15%). This reflects a legal logic where the independent training approach employed by the TS-LE method causes the two models trained in Task 1 and Task 2 to focus respectively on retributive and preventive factors. This approach fails to capture the dual retribution principle in Dual-Track Theory of Punishment, which requires revisiting the retributive factors in Task 2.

TS-DT. Based on the analysis in TS-LE, we propose a revised two-stage probation prediction method that aligns more closely with the Dual-Track Theory of Punishment in legal logic. Specifically, in Task 2, we incorporate both the fact description and the text sequence of PLEs as joint inputs to the model for prediction. The experimental results in Table 1 (b) demonstrate that this model outperforms the TS-LE method in probation prediction.

Furthermore, as shown in Figure 4, due to class imbalance in dataset, the cascade method used for training in Task 2 may lead to suboptimal model performance. The imbalance significantly hampers the model's ability to effectively predict probation eligibility. Moreover, since the prediction accuracy of Task 1 does not reach 100%, any errors made in this task are likely to be amplified in Task 2. In practice, for cases where the true sentence is three years or less, but the Task 1 prediction incorrectly exceeds three years, these cases will not proceed to Task 2. Consequently, PLEs will not be considered and the probation eligibility cannot be evaluated. This results in disproportionately harsh sentencing outcomes for such criminals.

MT-DT. To address the error amplification issue discussed in TS-DT, we adopt a joint training approach by modifying Task 1 to an implicit intermediate task. Thus, we propose a multi-task probation prediction model based on the DualTrack Theory of Punishment. This model treats Task 1 as an auxiliary task, jointly training Tasks 1 and 2.

In practice, we redefine the loss function of the model to ensure that, during training, the model simultaneously focuses on both the retributive and preventive factors of the criminal. According to the experimental results in Table 1 (b), the MT-DT method improves accuracy by 15.27% compared to the TS-DT.

Additionally, we visualize the attention mechanism of the model, as shown in Figure 5. The visualization demonstrates that the model effectively captures both retributive and pre-

(a) Different combinations of input information.

(b) Results with different input information

Figure 6: Performance of the MT-DT model across different λ. The parameter is varied from 0 to 1 to assess its influence on the model's overall performance. The stars indicate maximum accuracy

ventive factors in MT-DT. This indicates that our joint training framework effectively enhances the focus of the model on retributive factors, leading to better alignment with the legal logic of the Dual-Track Theory of Punishment.

4.2 Ablation Studies

As shown in Table 2 (a), we perform an ablation study on the MT-DT model. To examine the contribution of PLEs to performance, we consider the following three model variants:

• A. Fact description only.
• B. Fact description and the vector of PLEs.
• C. Fact description and the text sequence of PLEs.

As shown in Table 2 (b), replacing the vector with the text sequence of PLEs as input improves the accuracy by 4.73%. Furthermore, the accuracy of both configurations involving PLEs (vector and text) exceeds that of using fact description alone. This suggests that the legal elements provide additional semantic information, thereby enhancing the model's ability to make accurate predictions.

4.3 Influences of the Parameter λ

We further investigate the effect of the auxiliary task loss ratio in the overall loss on the performance of the MT-DT model. The results are shown in Figure 6. From the results, we observe the following:

• When the auxiliary task loss is excluded from the loss function, the approach is comparable to directly using the pre-trained model for probation prediction. As a result, the model demonstrates suboptimal performance.
• When the auxiliary task loss is incorporated, the performance of the MT-DT model stabilizes in the range of 86% to 88%, significantly surpassing the baseline. This demonstrates the robustness and effectiveness of the proposed method.

5 Experiments

5.1 Dataset

We obtain 29,105 judgment documents from publicly available legal documents on China judgements 2 . Of these, 8,351 cases involve probation, representing 28.69% of the total. The dataset consists of first-instance intentional injury cases with a single defendant and a single charge, spanning from 2011 to 2021. The dataset was randomly split in an 8:1:1 ratio into training sets (23284 cases), validation sets (2911 cases), and test sets (2910 cases).

5.2 Experimental Settings

We set the encoder feature dimension at 768, with the maximum document length limited to 512 words. To prevent overfitting, dropout regularization was applied to the feature vectors. The batch size was set to 16. The models were finetuned using a batch contrastive loss function. For optimization, we employ the Adam optimizer [Kingma and Ba, 2014] with a learning rate of 10-5 and a dropout rate of 0.3. The model was trained for 10 epochs. The hyperparameters that performed the best, determined in the validation set, were evaluated in the test set, and the process was repeated six times to compute the average prediction results. The hyperparameters were set as follows: λ = 0.1.

For evaluation, we use the Accuracy (Acc), Macro Precision (MP), Macro Recall (MR), and Macro F1 (F1) metrics.

2 https://wenshu.court.gov.cn/

5.3 Baseline

In this section, we conduct a comprehensive comparison between our proposed MT-DT model and a range of conventional text classification models, as well as pre-trained language models widely used in natural language processing tasks. The detailed performance metrics for all models are summarized in Table 3. The empirical results clearly demonstrate that MT-DT consistently outperforms all baseline models across various evaluation criteria, establishing its superior predictive capability in the context of the given tasks. Among the baseline models, LawFormer achieves the highest prediction accuracy, which is designed to effectively capture intricate patterns and perform sophisticated information extraction from large-scale, domain-specific legal corpora. However, despite its impressive accuracy, the complexity of the model leads to longer prediction times, which presents a challenge for real-time applications.

6 Conclusion

In this study, we propose a dataset for probation prediction, grounded in legal knowledge, and design an intelligent prediction model based on legal logic. By analyzing the substantive conditions for probation, we extract legal elements from legal documents and generate a text sequence, enriching the model with semantic information beyond the fact descriptions. We also introduce a novel loss function that enables the multi-task model to simultaneously focus on both facts and legal elements, thereby improving accuracy. Extensive experiments demonstrate that our method outperforms others and aligns with the legal logic of probation. Additionally, we assess the sensitivity of the model to parameters.

In future research, we will conduct a deeper investigation into the key factors influencing probation sentence lengths, as well as systematically analyze the underlying legal principles governing such judicial determinations. Furthermore, we plan to extend our dataset to encompass cases involving a broader range of charges and incorporate more sophisticated legal reasoning frameworks to enhance the model's robustness and generalizability.

Ethical Statement

The dataset used in this study has been fully anonymized to protect the privacy of individuals. All data processing and analysis were conducted in compliance with relevant ethical guidelines and legal requirements.

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant T2293773 and Grant 12471488. Qinghua Wang and Xu Zhang contributed equally to this work. Please ask Dr. Cunquan Qu (cqqu@sdu.edu.cn) for correspondence.