Try this right now: ask ChatGPT "Why did Einstein win his Nobel Prize for the theory of relativity?" (He didn't win it for the theory of relativity, but for the photoelectric effect. But ask to see what it replies)
Or ask "Summarize the 2019 paper 'Quantum Tax Heuristics in Mid-Size Retail' by Dr. Elena Marchetti", which I completely made up right now. Watch whether it corrects you or just starts explaining. Results depend on the model and whether search is on, but when it goes wrong, you will see the problem: the wrong answer sounds exactly as sure as a right one.
OpenAI published a paper called "Why Language Models Hallucinate" and the reason is funny. Imagine a MCQ exam where you don't know the answer to one question, and marking nothing will guarantee that you don't get any marks. You would try to guess every single time just for the hope even if one is correct. That is how most AI tests are scored, so models learn to guess with confidence instead of admitting they are unsure. The confidence you see is partly a side effect of how they were graded.
So you cannot judge an answer by how it sounds. The fix is one sentence. Paste this at the end of any question that matters: "Rate each claim as sure, likely or guess, and tell me what would change your answer." It will not make the model perfect, but it makes the guessing visible, and that is most of the battle.
One thing deserves extra care: anything that changes every day, like live data. No model knows today's price from memory, so an answer without a live lookup is a guess in a nice voice.
Whenever doing something important where data is really necessary, grab some open source tools from GitHub and use them with your AI. Or connect to some MCP servers for specific tasks. This gives the AI enough data to work on without hallucinating.
Like:
- LetsFG for booking international family trip booking in 10 minutes for the cheapest price
- Consensus for responses from the most accurate research papers out there
- Legal-Data-Hunter for checking any correct information about laws of any countries.
What tricks do you use to catch it when it is wrong?
PS. Links to the tools:
LetsFG- https://letsfg.co/mcp
Consensus- https://mcp.consensus.app/mcp
Legal-Data-Hunter- https://legaldatahunter.com/mcp