Are Large Language Models Ready for Legal Translation?

Legal translation has long challenged machine translation (MT). Specialized legal translators have traditionally bridged the gap between what MT produces and what courts and institutions need.

However, with the rise of large language models (LLMs), researchers are questioning whether these advanced AI tools can meet the complex demands of legal translation.

In their recent study, Applying Large Language Models in Legal Translation, Martina Bajčić, Associate Professor at the University of Rijeka, and Dejana Golenko, Assistant Professor at the same university, acknowledge the significant impact of LLMs on translation but stress that “the potential of LLMs in relation to specialized translation such as legal translation needs to be comprehensively examined.”

Through an analysis of papers published from 2021 to 2024, Bajčić and Golenko found that research on LLMs in legal translation remains scarce. “To date, there has been scarce research on its application in the field of legal translation,” they observed, highlighting the disparity between the widespread enthusiasm for generative AI and the lack of studies focusing on its application in specialized domains.

High Stakes, High Complexity

Legal translation errors can have far-reaching consequences, particularly in the context of EU law. “Recognizing and applying the most appropriate legal term is indispensable for finding, navigating, and understanding legal regulations,” Bajčić and Golenko explained.

In EU accession contexts, the stakes are even higher. The terminology used in translating the acquis becomes part of national legal languages, making accuracy crucial. Errors at this stage can lead to “faulty implementation of directives,” undermining legal systems and exposing states to liability.

Despite this, political leaders have touted the use of generative AI to streamline legal translation claiming it could eliminate “an army of translators and a battalion of lawyers, costing millions of euros.” 

However, Bajčić and Golenko caution that relying on unvalidated tools risks compromising translation quality and legal accuracy.

LLMs vs. NMT

According to Bajčić and Golenko, LLMs offer certain advantages over traditional neural machine translation (NMT) systems. Unlike NMT models, which rely on extensive domain-specific corpora, LLMs can quickly adapt to new tasks using few-shot or one-shot learning methods.

However, the authors emphasized that available data on LLMs in legal translation is limited, often based on small-scale studies of contracts or patent claims. While LLMs show promise in producing accurate translations with high terminology consistency and learning and adapting to new terminology, their performance on larger legal texts, such as legislative acts, remains unclear. These texts often pose interpretation challenges that require a nuanced understanding of legal context.

“Translating larger samples of legal texts […] needs to be further investigated in order to be able to draw conclusions about their “terminology competence” and parity with NMTs,” Bajčić and Golenko noted.

To harness the potential of LLMs in legal translation, Bajčić and Golenko advocate for expanding high-quality, domain-specific corpora to improve training. They also call for empirical studies to assess how LLMs handle complex legal texts and concepts. 

Given the sensitive nature of legal translation tasks, privacy and data security must also be addressed. In conclusion, the researchers stress the need for a holistic approach, integrating insights from linguistics, law, and AI development.