Most people try to catch an AI's mistakes by asking "is this true?"
Almost nothing can automate that question. The one that actually works is much narrower: "is this in the document the model was handed?"
That is groundedness: every claim in an answer traces back to a specific line in the source the model was given. Usually that source is a page a search step pulled up right before answering (the look-it-up-first setup people call RAG) or a row from a database. A hallucination is the opposite, a confident sentence with nothing behind it.
Grade it like an essay: highlight every sentence and ask which page of the textbook says this. Well written and unsupported still scores zero.
Here is why the narrow question is the useful one. Checking truth means checking against everything anyone knows. Checking support means comparing two pieces of text that are both sitting right in front of you. That is small enough to hand to a machine and run on every answer you ship.
Small enough, literally. Vectara's HHEM-2.1-Open does only this one job, scoring how well a passage supports a sentence. It is about 110 million parameters (tiny, by the standards of the models it grades) and checks a 2,000-token answer in roughly 1.5 seconds on an ordinary CPU, no graphics card. On Vectara's published benchmarks it beats far larger general-purpose models asked to make the same call.
Without a score like that, your testing measures fluency. Ask a bot to summarise a contract, get back a clause the contract never contained, and nothing errors, nothing logs, nothing goes red. It just reads well.
Quick check before you scroll: Your RAG chatbot retrieves a company's refund policy doc, and the answer says "refunds take 30 days" but the doc actually says 14 days. Is this a retrieval problem or a groundedness/hallucination problem?
Full breakdown + the answer: frankduah.me/learnings/2026-09-06-measuring-hallucination-and-groundedness
New here? I post a bite-size AI / ML concept like this every day, follow me for the daily drop, and it compounds fast. Why I do it: https://lnkd.in/gK8knHDH
#MeasuringHallucination #AI #LLM #AIAgents #MachineLearning
The answer
It's a groundedness problem: the right document was retrieved, but the generated answer isn't faithful to it. A faithfulness metric like RAGAS would catch this by checking the claim "30 days" against the retrieved context and finding no support for it.