Why your AI chatbot keeps giving wrong answers

Your AI chatbot is probably not broken. It is doing exactly what it was built to do, and it is doing it on data that was never as clean as anyone assumed. That is the part most teams miss when the tickets start piling up, and it is the reason the tickets keep coming after every fix.

What hallucination actually is

"Hallucination" is a bad word for this. It implies the model is inventing facts from nothing, and in a company data context that is rarely what happens.

A language model does not check facts. It predicts. Given the inputs it has, it generates the most likely output and delivers it with the same confidence every time. If one table labels a category "Electronics / Consumer" and another labels it "Consumer Electronics," the model picks one and answers. It does not flag the discrepancy or hedge. It has no way of knowing there is a discrepancy to flag.

So the reasoning is usually fine. The problem is what the reasoning is working from.

Where the bad data lives

Almost no chatbot is reading raw files. It reads the transformed layer, the cleaned and modeled tables your data team built on top of the raw sources. That layer is where the trouble is, because that layer is where every undocumented decision from the last five years went to live.

I have inherited a few of these. At one agency, we had a client whose channel mapping table was last touched during a rebrand. Two years later, half the campaign names in the raw feed no longer matched anything in the mapping, so they fell into a bucket called "Other." The dashboard had quietly been reporting "Other" as the third-largest channel for months. Nobody caught it because nobody looked at "Other." The AI, when we pointed one at that warehouse, reported it too, in a nice paragraph, with total confidence.

That is the typical shape of the problem. Transformation logic two engineers interpreted differently and neither documented. A revenue field recalculated every quarter with a definition that drifted between implementation and today. A join that silently drops rows. None of it looks like an AI problem, so nobody debugs it as one.

When a VP asks for last quarter's top campaigns and gets a number that is off by 30%, the instinct is to blame the prompt. Rewrite the system message, swap the model, add more context. None of that repairs a mapping table.

Why teams debug the wrong thing

Prompt engineering is real, and a well-structured prompt produces better output. But it has a ceiling. If the underlying field is wrong, the model will answer accurately about wrong data every single time, and a better prompt gets you a more articulate version of the same wrong answer.

This is uncomfortable to say to a team that just spent a quarter on the AI layer: the model is not confused. It is correct about something incorrect. That is a different problem from a bad prompt, and it needs a different fix.

The teams I have seen get reliable results are not the ones with the cleverest prompts. They treated the data layer as the product before the AI project existed. Documented definitions, consistent naming, tests on the models that feed anything customer-facing. It is not the work that gets demoed at a board meeting, but it is the work that determines whether the demo holds up in week three.

What to check before you file another ticket

Here is a check you can run in an afternoon.

Pull the ten fields the chatbot touches most. Look at the query logs or the semantic layer config. It will be a shorter list than you expect.

Trace each one to the model that produces it. In dbt, that means finding the model and the upstream sources. If you cannot find it in under ten minutes, that is finding number one.

Ask someone to define the field without opening the code. If the definition they give does not match what the SQL does, you have found where the AI is getting its wrong answer. Write the real definition down in the model's YAML before you touch anything else.

Check when the mapping tables were last updated. Compare that date to the last rebrand, platform migration, or reorg. If the mapping predates the change, assume it is wrong until proven otherwise.

Look at whatever bucket is called "Other," "Unknown," or "Uncategorized." Its size tells you how much of your data the model is confidently reporting from a category that means nothing.

If any of those come back bad, that is where the problem lives. Switching models will not help. A retrieval layer on top of dirty data just gives you faster access to wrong answers.

Fix the data first. The chatbot was fine.