The “uses and abuses” of Generative AI in litigation
The use of Generative AI, including Large Language Models (LLMs), has become a hot-button issue in the legal profession in recent years.
As covered in a previous article on the rising use of AI by the legal profession, Generative AI is seeing increasing and rapid adoption across the profession, including in litigation, but it is not without its potential pitfalls.
What is Generative AI?
In short:
- Generative AI is a catch-all term for a type of AI which, by analysing patterns in large sets of data (training), can then generate new content.
- LLMs (such as OpenAI’s Chat GPT or Microsoft’s Copilot) are a type of Generative AI trained by analysing patterns in vast quantities of human-authored text, which can produce seemingly original new text by placing one statistically-related word after another.
How can Generative AI be used in litigation?
The use of AI in litigation is not a new concept. Technology Assisted Review (TAR) or “predictive coding” has been increasingly common in electronic disclosure since 2016, analysing the findings of human reviewers to predict which documents are more likely to be relevant.
However, the ability of Generative AI to process large quantities of information and generate entirely new content leads to several potential new use-cases.
Discussion has so far tended to focus on undertaking traditional time- and cost-intensive tasks such as reviewing large volumes of documents, routine drafting, and legal research – with several products being described explicitly as “AI paralegals”.
Access to justice is another common theme, with the potential costs savings presented by AI – and the availability of open source models to litigants in person – touted as potential benefits for improving access to the courts.
The key issue – “convincing but false”
The issue at the heart of almost all (mis)use of Generative AI is that it does not – and cannot – understand the data which it “learns” or verify the accuracy of its outputs. However, it is excellent (by design) at making those outputs a convincing imitation of a human response.
This can lead users, in particular those without specialist knowledge, to place undue trust in the accuracy of those outputs.
What does this look like in practice?
To illustrate some of the potential pitfalls, we put a number of prompts (reflecting commonly discussed use-cases in litigation) to a widely used open-source LLM:
Hallucination
“Hallucination” is the phenomenon where Generative AI may generate false information in response to prompts, which is then presented as fact.
We asked: "Write me an article about the Supreme Court’s decision in URS v BDW".
The LLM produced a thorough case summary including a bullet-point summary of the Supreme Court’s decision, which (at the time of writing) has yet to be handed down!
We also asked: "Write me an article about a recent UK supreme court judgment".
The LLM produced a concise and coherent article which began:
“April 7, 2025 – The UK Supreme Court delivered a significant judgment in the case of Lancashire County Council v Secretary of State for the Environment, Food and Rural Affairs…”
That “recent” judgment does exist – however, it was handed down in December 2019. The AI summary also had significant factual details of the case (such as who had made the application being appealed) wrong and cited four points of law which the Supreme Court had “clarified”. Two did appear in the judgment, obscuring the fact that the other two did not – and were also wrong.
Lack of understanding of procedural rules
Generative AI is incapable of understanding procedural rules (such as the Civil Procedure Rules or the Pre-Action Protocols). It also cannot differentiate data it has “learned” which is outdated or wrong.
We asked: "Write me a template letter for a claim against a UK solicitors’ firm".
The LLM produced a clean and clear template letter – which made no reference to the Pre-Action Protocol for Professional Negligence and was non-compliant with 7 of the 9 Protocol requirements for a Letter of Claim, as well as the Protocol deadlines.
We then asked: "Write me a template letter of claim against a UK solicitors’ firm which complies with the Pre-Action Protocol for Professional Negligence".
With this prompt tailoring, the LLM produced a more appropriate draft, which stated it was a letter under the Protocol – but still excluded 6 of the 9 Protocol requirements.
A litigant in person or non-specialist relying on that template would face immediate pushback from a specialist familiar with the Protocol, and potential sanctions on costs for non-compliance if they followed through with the steps set out in the template.
Incorrect or incomplete legal analysis
Generative AI lacks the ability to ask clarifications that might be obvious to a lawyer, or to think laterally in order to solve problems.
We asked: "My home was badly constructed in 2009. Has limitation run out?".
The LLM correctly identified the limitation periods for breach of contract, negligence, and the extended limitation period for latent damage cases.
However, it then stated that a latent defect claim may still be within the 15-year long stop period (which would have expired in 2024) – unlike a human listener, the LLM has no ability to ask the follow-up question – “has this claim already been issued?”
It also entirely omitted the retroactive 30-year limitation period for Defective Premises Act claims, which could be a potential alternate route to bring such a claim.
Inability to explain outputs
Most current Generative AI are proprietary software with no explanation of their internal decision-making processes, and little ability to explain their own output.
In the now-notorious US case of Mata v Avianca, Chat GPT was asked directly whether the cases it had cited were fake (they were) and responded that the cases “are real and can be found on legal research databases such as Westlaw and LexisNexis” (which they were not, and could not).
We asked: "What is the role for generative AI in the future of litigation?".
We then asked: "Summarise the potential uses of generative AI in litigation".
And then also asked: "List the potential uses of large language models in litigation".
In all three cases, the LLM produced very similar outputs listing the same seven “key roles it could play” in litigation, which included...
Client interaction
Generative AI can improve client interaction by providing instant responses to common legal queries, scheduling appointments, and even offering preliminary legal advice based on the input provided.
One version contained no sources for this at all. Another cited a major UK law firm’s article, which did not contain any such statements. The third cited a paper in the Hungarian Journal of Legal Studies, which in fact stated that the legal chatbot studied in that paper could give incorrect contact details and information and “was not ideal” for booking appointments.
So finally, we asked: "Can you explain your source for item 5 in the list above?"
The LLM output: “Certainly! The use of large language (LLMs) in client interaction within litigation is highlighted in various sources…” – before repeating the text above word-for word and giving a third source which again did not support the output.
Even where outputs are being checked manually, this can cause issues in verifying how the AI has reached a given output – which may be a particular issue for documents which are required to be verified by a statement of truth.
Data bias and data confidentiality
LLMs are only as good as the data they “learn”. Many open-source LLMs are trained on large volumes of data available on the internet, potentially introducing the same biases and misinformation which are present in that training data.
Both the Law Society and Bar Council have also raised concerns over the potential for breaches of confidentiality, legal professional privilege, and data protection laws. Because the prompts entered into open-source LLMs are often themselves used as part of their “training” data, any input containing confidential details of a case could in theory lead to outputs for other users which reproduce that data.
Why this matters, and some useful tips
Generative AI undoubtedly has significant advantages to offer when used properly, and Master of the Rolls Sir Geoffrey Vos recently commented that “silly examples of bad practice” should not derail its adoption.
However, litigators in particular will need to be familiar with its limitations, not only because of the potential for poor outcomes, reputational damage, and exposure for firms if used improperly, but also because these tools are equally available to opposing parties.
Litigants in person are one area of concern – HMCTS notes that AI “may be the only source of advice or assistance some litigants receive”, and they are particularly unlikely to spot obvious errors or understand the limits of AI.
Bad actors are another – the Bar Council’s commentary on this subject notes several examples already of LLMs being used to manufacture fake misconduct allegations.
HM Courts and Tribunals, the Law Society, and the Bar Council, and the Chartered Institute of Arbitrators have all issued guidance for their members attempting to grapple with the use of Generative AI, and there are a number of sensible precautions that can be taken to reduce the risk of poor outcomes, including:
- Ensuring all work produced by Generative AI is checked by someone with the legal and factual knowledge required to spot errors, and without the use of other Generative AI tools.
- Avoiding the use of Generative AI for tasks which cannot be easily verified (eg legal research into unfamiliar topics), or which may require legal analysis or lateral thinking.
- Keeping a clear record of how Generative AI has been used in a case, for what purpose, and both the prompts and outputs used – and being prepared to explain this to the Court if questioned.
- Avoiding entering any privileged or confidential information into open-source platforms.
- Remaining vigilant for tell-tale signs of Generative AI use when dealing with other parties, and in particular litigants in person.
If you're interested in reading more about how AI is impacting on professional life, please sign up to follow our FutureProof series.
Our content explained
Every piece of content we create is correct on the date it’s published but please don’t rely on it as legal advice. If you’d like to speak to us about your own legal requirements, please contact one of our expert lawyers.