A lawyer cited the cases. The judge checked. None of them existed

A lawyer hands the court a list of cases that support their argument. Solid citations, real-sounding judges, confident quotes. The judge goes to read them. And not one of them exists. Not “got the year wrong” or “misremembered the facts”. Entirely invented, by a computer, and handed up as real law.

If that sounds like a nightmare, it’s a real one, and it’s happened in courtrooms around the world, including here in Australia. It’s the clearest warning yet about using AI carelessly in legal work, and the lesson isn’t “never touch AI”. It’s much more specific than that.

The case that put everyone on notice

The famous one is Mata v Avianca, in New York in 2023. A lawyer used ChatGPT to find cases supporting his argument, and it obligingly produced more than half a dozen. They looked perfect: judge names, docket numbers, internal citations, lengthy quotes from the “judgments”. The problem was that ChatGPT had invented all of them from scratch. The cases, the judges, the rulings, none of it was real.

The court was not amused. The lawyer and his firm were fined $5,000 and ordered to write to every real judge whose name the AI had falsely attached to a fake opinion. A career’s worth of credibility, spent in a single filing.

It’s already happened in Australia

This isn’t a faraway curiosity. In 2024, a Victorian solicitor was referred to the Victorian Legal Services Board after handing a court a list of cases, prepared with AI legal research software, that turned out not to exist. The judge’s associates went looking for the authorities and couldn’t find a single one. The same mistake, in an Australian court, with a regulator now involved.

Why AI does this

Here’s the thing every lawyer using these tools needs to understand. AI language models are built to produce text that sounds right, not text that is true. They’re extraordinarily good at the shape of a thing. Ask one for case law and it will generate something that has the exact look of a real citation, because it has seen thousands of them, whether or not the case behind it ever existed. It isn’t lying, because it doesn’t know. It’s filling in the most plausible-looking blank. The polite industry word for this is a hallucination, and it’s not a bug that’s being fixed. It’s how the technology works.

That’s what makes it dangerous for legal work specifically. The output is most convincing exactly where you most need it to be reliable.

The lesson isn’t “don’t use AI”

AI is genuinely useful in a practice. It’s great at first drafts, summarising long documents, and the supporting grind around the lawyering (we wrote about where it actually helps a firm). The line is simple: it is not a source of legal authority, and it can never be treated as one.

So the habit that protects you:

Verify every citation, quote and proposition in a real source before it goes near a court or a client. The same standard you’d apply to a list handed to you by a first-year solicitor.
Use AI for the supporting work, not the truth. Let it draft and summarise. Don’t let it be the thing you rely on for what the law says.
Put the rule in writing for your team. Most of these disasters come from a well-meaning person who didn’t know the tool could simply make things up. A two-minute briefing prevents it.
Remember the mistake is yours, not the machine’s. Every court that has dealt with this has been clear: the lawyer who files it owns it. “The AI did it” is not a defence, it’s an admission.

The takeaway

AI won’t end your career. Trusting it blindly, on the one thing it’s worst at, might. The fix isn’t to avoid the technology, it’s a verification habit you apply every single time. Check the cases before you cite them. It’s the oldest rule in the profession, and it turns out to be the one that saves you from the newest mistake.