Tips for spotting and correcting AI hallucinations

AI has become a powerful research partner for the legal profession, speeding up the process of legal research, drafting documents, summarising documents and analysing complex information. But like every tool, it has its quirks. One of the most important to understand and manage is hallucination.

While AI can be a valuable springboard for legal research, it can’t replace a lawyer. Only a lawyer can interpret the law and apply it to the complex and nuanced facts of their client’s matter. AI provides a helpful starting point, but it’s a lawyer’s legal training and judgement that ensures it is used in a safe and effective manner.

At LawY, we see AI as a trusted research partner. Our goal is to make it simple to build on AI-generated legal research by providing linked resources that let you easily verify any answer. This keeps your work grounded in real and reliable legal resources.

Our goal is to help both users and new Verifiers recognise where hallucinations might appear, whether within the AI’s response or the resources it references.

Because large language models have their natural limitations, LawY’s unique human-in-the-loop verification feature allows users to send answers to be reviewed by qualified lawyers who enhances and builds on the AI answer and in doing so checks and removes any hallucinations. Their subject matter expertise and attention to detail are what transform each response into something accurate and trustworthy to use as the first step to your legal research.

1. What are AI hallucinations and why they matter in legal work

An AI hallucination happens when a model generates content that sounds convincing but isn’t true. In a legal setting, this often means:

• Fake case names
• Invented citations (for example, “HCA 123” that doesn’t exist)
• Incorrect or fabricated facts
• Misleading summaries of real cases

These errors can be subtle, sometimes just a few misplaced details such as the numbers in a case citation or the year of the judgement, but they risk misleading users and creating false confidence in inaccurate legal information. Large language models are trained to produce the most likely response, not necessarily the most accurate one. That’s why human verification remains critical.

2. How to spot and correct hallucinations during verification

When reviewing an AI-answer, a Verifier’s goal is to separate truth from noise. Hallucinations can appear in three key places and each needs its own check.

Name: Does the case name actually exist and if so, does it match the citation? Search trusted databases such as Austlii or LawCite to confirm.

Citation: Is the citation valid? Check both the medium neutral or law report citation.  If it doesn’t match, that’s your first red flag.

Facts: Even if the case exists, the AI might distort its summary. Start a new conversation in LawY and use LawY’s case summary feature to quickly generate a case summary by using the URL of the case.  This allows you to cross check the facts and conclusions drawn about the case in the original AI answer you are verifying.

A note for Verifiers

If you find a hallucination, don’t ignore it. Delete and/or Replace it.

Your final verified answer should include primary resources directly relevant to the question and matter details. Include cases that are binding or persuasive in the relevant jurisdiction, directly on point, and current. Explain how each case applies to the specific issue. If you cannot locate cases supporting the AI's proposition, delete the hallucinated content and include a sentence noting that "There are currently no reported cases in [jurisdiction] that directly address these facts."

Tip: A good Verifier doesn't just fix the error. Verifiers should add value by enbancing it with reliable primary resources, supportive secondary resources and practical tips that would help the user.

3. How LawY helps you work smarter, not harder

‍Using LawY's verification features effectively

LawY's verification features are designed to make spotting and correcting hallucinations faster and more accurate. Follow this process:

A. Evaluate every resource

Click on each resource cited in the AI answer and assess its reliability:

• Is it from a legitimate, authoritative source?
• Is it directly relevant to the question?
• Is it current and still good law?
  ‍

B. Verify cases with primary sources

‍When cases are cited or linked in an answer:

1. Locate the original judgment using LawCite or AustLII
2. Copy the direct source link to the judgment
3. Paste it into a new LawY conversation to generate an instant case summary

This approach is both safe and efficient. LawY's URL feature extracts information directly from the original judgment, your single source of truth, ensuring the summary reflects what the case actually says, not what the AI thinks it says.

C. Dig deeper with follow-up questions

Once you have the case summary, ask LawY targeted questions about the judgment to:

D. Confirm conclusions are sound

‍Verify whether the original AI answer drew appropriate conclusions from the case.

Example: The AI claims Smith v Jones establishes that silence can constitute acceptance in contract law. Ask LawY: "Does Smith v Jones actually support the proposition that silence can constitute acceptance, or did the court distinguish this principle?"

E. Verify quoted material

‍Explore specific statements or passages that the original AI answer claimed to quote.

Example: The AI quotes Justice Brown as saying "economic duress requires proof of illegitimate pressure." Ask LawY: "What did Justice Brown actually say about the requirements for economic duress in paragraph 45?"

F. Assess true relevance

‍Determine whether the case is genuinely relevant to your issue, or if it only tangentially addressed it as a minor point.

Example: The AI cites Property Holdings Pty Ltd v Developer Co for principles about exercising options to renew. Ask LawY: "What was the central issue in Property Holdings? Was the exercise of the option to renew a primary issue or merely a peripheral matter?" You may discover the case was actually about relief from forfeiture, with the option issue only briefly mentioned in passing.

The result

‍This verification workflow transforms lengthy judgments into actionable insights while maintaining accuracy and control over your research.

Once you have reasonable confidence that a case is genuinely relevant to your research question or the issue you're verifying, read the judgment, either in full or the key sections, so you can use it appropriately in your work. This strategic approach ensures you don't waste time reading irrelevant judgments that the AI has incorrectly included, and helps you quickly identify and discard hallucinated cases before they derail your research.

This feature doesn’t just save time it enhances the reliability of every verified answer you produce.

4. Building skill sets to handle hallucinations

Dealing with hallucinations is about developing good habits and using the right features. Here’s what to keep in mind:

• Always verify case names, citations and facts - Don’t assume a professional-looking citation means it’s real. Ensure the case name, citation and facts all align.
• Go back to first principles - Treat every piece of AI-generated text as a starting point for your own legal research.
• Utilise Traditional Legal Research Skills - You already know how to navigate legal research databases like AustLII and LawCite. Use these trusted resources alongside LawY to cross-check and confirm your findings.
• Practice efficiency - Effective verification isn't about being fastest or verifying the most answers. It's about efficiently delivering safe, reliable, and thoroughly verified results. LawY aims to provide quality verification to its users.

At LawY, we’re helping legal professionals work smarter, not harder, giving you tools that strengthen your judgement rather than replace it.