Hallucination
Also called: AI hallucination · confabulation · fabrication
When an AI states something false with the same confidence and fluency as when it states something true.
Why it matters
The danger is not that AI is often wrong; it is that it is wrong in the same tone as it is right. A made-up citation reads exactly like a real one. A wrong number sits in a perfectly formatted table. Unless someone checks, the errors travel: into proposals, into contracts, into decisions, wearing the same confident voice as everything else.
Where it comes from
A language model answers by producing the most plausible next words, given everything in front of it. Most of the time, plausible and true coincide: that is why the technology is useful at all. But when the model has no good material to draw on (a niche topic, a detail it never saw, a question about your company’s internals), it does not stop. It produces the most plausible text anyway.
There is no internal alarm that separates the two cases. Writing a real reference and inventing one are, to the model, the same act. This is the single most important thing to understand about hallucination: fluency is not evidence. The confidence in the prose tells you nothing about the truth of the content.
What it looks like in practice
Invented citations and case law. Statistics that are plausible, well formatted, and wrong. A confident summary of a document that says something subtly different. Product features your product does not have. Policies your company never wrote. The pattern is always the same: specific, polished, false.
Why it survives review
The wildly wrong answers get caught; nobody ships “the contract was signed in 1875”. The answers that survive are the slightly wrong ones, and they are exactly the ones fluent enough to pass a tired skim. Review fatigue makes this worse: nine verified answers lower the guard for the tenth.
The errors also compound. An unverified output pasted into the next prompt becomes ground truth for everything downstream. By the third document, nobody remembers that the number was never checked.
What actually reduces it
Grounding, first: let the model read your actual documents instead of reconstructing from memory, which is the problem RAG exists to solve. Then traceability: ask for sources, and treat an answer without a checkable source as an opinion. Then process: verification against systems of record for numbers, human sign-off where the output touches money, law, or customers.
What does not work: asking the model to promise it is sure. It will promise.
The question to ask
Not “how do we stop hallucinations”, because you will not, entirely. The question is: when one gets through, how quickly would we notice, and what would it have touched by then? If the honest answer is “we might not notice”, the workflow is not ready for the stakes it is carrying.
Common questions
Why does it make things up instead of saying it doesn't know?
Because it is not looking anything up. A language model produces the most plausible continuation of the text in front of it, and most of the time plausible and true point the same way. When they do not, nothing inside the model raises an alarm: generating a real citation and inventing one are the same operation. It is not lying; lying requires knowing the truth and choosing otherwise. It is completing a pattern.
Can we get rid of hallucinations completely?
No, and be wary of anyone who says otherwise. You can reduce them substantially: ground the model in your own documents, keep tasks narrow, ask for sources, and route high-stakes output through review. The goal is not a system that is never wrong. It is a workflow where being wrong is cheap to catch.
Does hallucination mean the model is broken or badly made?
No. It is inherent to how generative models work, and every vendor's model does it. The useful question is not 'which model never hallucinates', because none qualifies. It is 'what happens in our process when it does': who checks, against what source, and before which decisions.
How do we catch hallucinations before they cause damage?
Make claims traceable. Ask for sources and actually click them. Check numbers against the system of record rather than against the model's own summary. Keep a human sign-off wherever the output touches money, law, or customers. And set one cultural rule: an AI claim nobody can trace is a draft, not a fact.
Working through this with your team?
Olaia Labs runs AI training and automation programmes for SMEs. Start with a diagnostic: we look at where your team already uses AI, where it breaks, and what is worth automating.
Request a diagnostic