Research Experience
Legal Judgement Prediction
Legal Judgment Prediction (LJP) is a subfield of Natural Language Processing (NLP) focused on predicting legal judgments in court cases. LJP utilizes various NLP techniques to make predictions based on metadata and case facts, such as violations, charges, and sentences. With the imminent introduction of lay judges in Taiwan, there is a growing need to improve public understanding of the judicial process. LJP aims to provide recommendations for verdicts, sentencing, and relevant legal provisions, making it a valuable tool in enhancing legal transparency and accessibility.
Problem to be Solved
LJP encounters unique hurdles due to the distinctive nature of legal judgments, which can vary significantly from country to country. The fundamental questions arise: Can a model developed for one country be effectively transplanted to fit another country's legal system? Is it possible to adapt and curate a dataset to tailor it for a specific LJP model designed for another country? These questions underscore the need for innovative solutions to bridge the gap between legal systems and languages, making LJP more accessible and accurate for the Taiwanese context.
Research a Possible Solution
TopJudge is an innovative Chinese framework designed from thunlp, focusing on predicting legal outcomes based on case facts. Unlike many existing LJP approaches that often focus on isolated subtasks, TopJudge takes a holistic approach. It recognizes that legal judgments involve multiple intricate components, such as the interpretation of applicable law articles, determining charges, fines, and the length of penalties. Moreover, these components are interrelated, forming complex dependencies. To address this challenge, TopJudge formalizes these dependencies as a Directed Acyclic Graph (DAG) and introduces a topological multi-task learning framework. This approach allows TopJudge to effectively incorporate multiple subtasks and dependencies, resulting in significant improvements in judgment prediction accuracy across a range of real-world datasets.
Drawing inspiration from the TopJudge framework, which successfully addressed similar challenges, I proposed solutions that take into account the inherent characteristics of Chinese legal texts
In this project, I undertook the task into two sections: Re-implementing the LJP system from the open-source TopJudge framework, and porting to Taiwanese Dataset.
Reproduce
While TopJudge provided open-source code, it lacked critical training parameters and required several preliminary steps and the necessary model components for successful replication. The absence of comprehensive datasets posed a challenge, as only CAIL 2018 data was mentioned in the reference but required preprocessing to be used effectively. The preprocessing phase involved complex tasks like vectorizing case facts using THULAC Chinese word segmentation, word2vec conversion, and creating the necessary input data for training the Transformer model. During training, modifications were made to the original code to accommodate the available metadata. These challenges underscored the intricacies of adapting TopJudge for use in different legal contexts.
Porting to Taiwanese Court Dataß
In the next phase, the adapted TopJudge framework was applied to Taiwanese judicial judgment data. However, a key challenge arose due to the absence of mature datasets tailored to the Taiwanese legal context. To overcome this, data was collected from the publicly available judgments of the Taiwanese Judicial Yuan, accessible through the https://opendata.judicial.gov.tw/api/ platform. The data spanned from April 1996 to February 2022 with a specific focus on the years from 2012 to 2022 to ensure relevance and mitigate inconsistencies resulting from legal reforms over time. Notably, the structure of Taiwanese judgments significantly differs from those in China, typically featuring standardized sections such as plaintiff, defendant, verdict, facts (not always present), rationale, relevant legal articles, and attachments. Retrieving crucial information like case facts proved challenging, as they were not always explicitly stated, and often referred to attached documents. Yet, some judgments included attachments below the main text, although no batch download API was provided by the Ministry of Justice. These complexities were addressed during preprocessing, where specific keywords were used to filter and identify prosecution-provided case facts.
This process also extracted sentencing lengths from the verdict section, captured charges from JTITLE metadata, and identified relevant legal articles by searching for keywords such as "論罪科刑" and "刑法第," iteratively ascertaining multiple applicable legal articles when necessary. Despite data reduction due to these challenges, a substantial dataset comprising approximately 320,000 entries was maintained. This dataset was then employed for training, resulting in extended training times compared to the CAIL2018 dataset due to its volume and complexity.
The project's experimental results provide valuable insights despite facing significant challenges. The precision and F1 scores for different judgment prediction tasks — Crit, Time, and Law — indicate room for improvement. Several factors contributed to these less-than-ideal outcomes. First, data imbalance presented challenges, especially for less common cases like Embezzlement of Public Interest (公益侵占) and Bigamy (重婚) where data availability was limited. Second, inconsistencies in data labeling emerged from Taiwan's judicial judgments, as JTITLE may not have consistently represented detailed information about charges, often providing only a high-level description of the violated legal articles. Lastly, the extraction of prosecution charges proved difficult, as criminal cases in Taiwan encompass various laws beyond the ROC Criminal Law (中華民國刑法), such as money laundering regulations and domestic violence prevention laws, creating complexity in identifying relevant legal articles.
Nonetheless, this project's contribution is noteworthy as it explored the potential for TopJudge and similar Logical Judgment Prediction (LJP) models to operate successfully in diverse legal systems and languages. Even though it encountered various challenges during the model's transplantation, it demonstrated the feasibility of adapting such models with appropriate adjustments, minimizing the need for extensive modifications. This experience represents a significant learning opportunity for the researcher, as it involved comprehending and attempting to adapt a model from scratch. Despite not achieving ideal results when applied to Taiwan's judicial context, prospects appear promising. By addressing the existing limitations and ensuring data accuracy and richness, the project aims not only to facilitate the adaptation of foreign frameworks, but also to establish a specialized dataset for predicting legal outcomes within Taiwan's legal system.