Today my timeline is filled with stories of the Greek Minister of Justice wanting to use AI-powered machine translation tools for interpreting in court settings and translating legal documents in various judicial proceedings. Settings where attention to detail should certainly matter.
The law is all about wielding words accurately (or deliberately wielding them in deliberately ambiguous ways). But let’s focus on the scenario where accuracy is what you’re after because you want / need to understand your rights and obligations in a legal setting; you want/need to make an important legal decision, etc.
In a post I did yesterday I stressed that the constancy of legal words is important; such constancy is important for a host of important reasons lawyers will immediately recognise, just some of which are legal certainty and the rule of law.
So I thought I’d run a little test connected to something I was working on.
I asked ChatGPT to translate a legal provision from Greek to English:
This is the output:
Sounds quite plausible and convincing.
But let’s not forget that MT/AI systems used in translation are known for creating the “illusion of fluidity”[1].
They’re also known for a whole series of other problems (omitting bits of the text, changing negative obligations into positive ones, inconsistent use of terms, made-up words, etc.) but they are not relevant to us today.
What’s relevant today is this “illusion of fluidity”. On first reading the translation seems to be ok. It seems right.
On closer reading, especially if you compare it with the original, you start to spot “issues”.
Are you even able to compare it to the original to be able to identify any issues?
In this particular case, the article comes from a convention that already exists in English so we can easily determine what is right and what’s not. The relevant article reads as follows:
Admittedly, the original and “translation” are very similar.
There are differences you can easily spot:
- “Entitled to benefit” vs. “Entitled to avail himself of”:
- “Willful misconduct on its part” vs. “His wilful misconduct”:
- “According to the law of the court having jurisdiction over the matter” vs. “In accordance with the law of the court or tribunal seized of the case”:
- “Considered equivalent to willful misconduct” vs. “Considered as equivalent to wilful misconduct”:
- “Omission” vs. “Default”
Some are probably not that important (you get the general idea whether worded in one way or the other). Are you only after a general idea though? Or as a lawyer/client do you want to precisely understand what the text is saying?
Others change the legal meaning utterly.
It does get the specialist term “wilful misconduct” right. Other online tools get it wrong (deliberate poor management / deliberate mismanagement / wilful mismanagement).
Although I didn’t ask for it, ChatGPT added “its” “view” about the output generated:
Was it asked to provide simpler, more straightforward phrasing? We see a clear translational strategy here: opting for plainer language to make the text easier to read. Probably not a bad thing. Not the appropriate strategy though in this context.
It makes the bold assertion that both the original English text and its “output” are legally correct.
So I asked the obvious question:
ChatGPT generated this response:
If lawyers use precise wording “to avoid ambiguity” why does this system generate a “rough” translation?
So while I was focused on the details, and getting the translation “right”, our online tool was not.
ChatGPT provided an unofficial rendering of the provision. While the meaning may be “roughly the same”, it does not carry the same legal weight or and certainly doesn’t contain the precise wording as the official version.
Why does this matter?
Words matter in law … even translated words.
Established wording needs to be maintained.
Established wording is what lawyers will recognise and are used to working with.
Any deviation from established wording creates headaches.
Sticking to established wording saves users of the translation time, and avoids a lot of head-scratching and bewilderment of the type “so legally speaking what does that actually mean”?
Legal language is a “controlled” language. You cannot just use any old words you want.
Randomly generated “rough” translations introduce inefficiencies into lawyerly processes.
Randomly generated “rough” translations create false impressions of legal rules / legal obligations, especially if you aren’t an expert in that area of law; say, for example, you’re the client rather the lawyer.
Basically with tools like this you get “a translation”.
But …
Is it “the translation” you need?
Is it a translation you can use?
Is it a translation you can rely on in your lawyerly dealings?
Is it a translation you can trust?
It is a translation you can base decisions on?
Work with expert legal translators if you need your legal words to count.
Very interesting article and great illustrative example. I don’t know that your prompt looked like, but what if you refined it to specify using, let’s say, most recent case law or EU multi-lingual documents only, or using the legal language of English law. Would it make a difference in terms of using “established wording”?
It generates “roughly the same” output but there are clear differences.
This is today’s output with a prompt based on your suggestions:
“The carrier shall not be entitled to benefit from the provisions of this chapter which exclude or limit its liability, or which shift the burden of proof, if the damage was caused by its wilful misconduct or by such an omission on its part which, under the law of the court having jurisdiction over the case, is deemed equivalent to wilful misconduct on its part.”
Whereas yesterday the output was:
“The carrier shall not be entitled to benefit from the provisions of this chapter which exclude or limit its liability, or that shift the burden of proof, if the damage was caused by wilful misconduct on its part, or by an omission on its part that, according to the law of the court having jurisdiction over the matter, is considered equivalent to wilful misconduct.”