How to Verify AI Answers and Spot Hallucinations

Start by identifying the claims someone might act on. A delivery date, price, citation or decision should have support in the original source. If you cannot find it, mark the claim as unconfirmed or remove it before sharing the answer.

Čítať po slovensky

What is an AI hallucination?

An AI hallucination happens when generative AI presents incorrect or invented information as fact. It might cite a source that does not exist or treat a proposal as an approved decision. Keep contradicted claims separate from those you have yet to verify. A working link can still point to a document that does not support the claim.

A worked example: from meeting notes to a customer email

Consider these fictional meeting notes:

The customer needs a pilot for 12 staff members and suggested 15 May as a start date. The supplier will check availability by Friday. No price was discussed.

A flawed draft reads: “We’ll start the pilot for 12 staff on 15 May for €1,200.” The headcount is supported. The date was only suggested, and the price is absent from the notes.

A corrected version would read:

We’re planning for a pilot with 12 staff members. We’ll check whether we can start on 15 May and let you know by Friday.

Before sending a message like this, check that Friday is still ahead and that the sender can make that commitment.

Four checks before sharing an AI answer

  1. Separate the claims. Check each material detail individually, especially numbers, dates, decisions and commitments.
  2. Open the original source. Check the exact passage and the document version. If you are checking a citation, verify that the source supports the claim in its context.
  3. Assign a status. Mark each claim as supported, contradicted or not stated in the source. Recalculate important figures from the underlying data.
  4. Revise and get sign-off. Keep confirmed details, label proposals as such and ask the person responsible to approve any commitments.

A verification prompt you can copy

Compare the draft with the source below. List each claim about names, numbers, dates, decisions and commitments. Mark each claim as supported, contradicted or not stated in the source. For supported or contradicted claims, quote the exact passage. If you cannot find supporting evidence, leave the citation blank. Suggest revisions for the claims that need attention.

Source: [insert approved source material]
Draft: [insert text to review]

Use the result as a checklist for a human reviewer. For internal documents, use a tool your organisation has approved.

Build verification into your team’s workflow

For recurring work, keep a small set of approved source documents and expected results. Include a case with a missing detail and one where two sources disagree. After changing your instructions, repeat the checks to see whether the model still flags uncertainty and preserves the distinction between a proposal and a commitment.

Separate drafting and verification with prompt chaining. For help writing the initial instructions, read How to Write a Good AI Prompt.

If you want to set up a review process for customer communications or another recurring task, tell us about your workflow.

Further reading on confabulation: NIST Generative AI Profile.