Summary: In day-to-day business, the weaknesses of AI systems show up particularly clearly. In one test, ChatGPT could not even reproduce simple basic facts about companies correctly, and many details were made up. With business figures, this weighs more heavily, because the AI lacks two things at once: it does not know the numbers, and it does not know how a company calculates, say, its revenue. Providing the data is the easy part of the solution. AI only delivers reliable numbers once it draws on verified metrics instead of calculating them itself.
96% made-up details: what a study shows about ChatGPT and mid-sized companies
Suppose you ask ChatGPT who the managing director of one of your suppliers is. The answer comes instantly, is fluently worded and even names the year the company was founded. Yet none of it is true.
The agency maxonline examined how often this happens in its KI-Halluzinations-Studie 2026 (AI Hallucination Study 2026, in German). Between March and April 2026, it tested 150 mid-sized companies from 11 industries with annual revenue of €5 million to €50 million, each with 3 standardized prompt types in ChatGPT. The focus was on Austria, supplemented by companies from Germany and Switzerland. The results are clear:
- 96% of the managing directors’ names that ChatGPT actually gave were made up.
- 78% of the founding years it gave were wrong.
- 68% of the employee numbers it gave were made up.
- For only 4 of the 150 companies, just under 3%, were all the details requested correct.
This was publicly available information that ChatGPT could, in principle, have known. For a company’s internal figures, such as revenue in the last quarter, a language model without a connection to the company’s own systems has no basis at all. It gives an answer anyway.
What is an AI hallucination?
An AI hallucination occurs when a language model outputs information that is wrong but sounds convincing. The model does not notice that it is making something up. It generates the answer that, based on its training data, is most likely to fit, and it does not distinguish between an established fact and a plausible-sounding guess.
In advertising copy, an invented phrase may hardly matter. With a business figure that a decision rests on, it is a different story.
Cause 1: The AI lacks the knowledge
Language models such as ChatGPT, Gemini or Claude get their knowledge from training data. What rarely or never appears in it, they do not know. They answer anyway, because they are trained to be helpful. When a piece of information is missing, they fill in whatever fits best statistically. That is exactly what the study shows: there is a lot online about large corporations, but little about a machine builder with 200 employees.
Applied to business data, this cause can be partly fixed: uploading a spreadsheet, attaching a document or connecting the AI to a data source through a connector gives it the missing knowledge. With their built-in data analysis features, ChatGPT and similar tools also actually calculate instead of estimating numbers. They usually add up an uploaded Excel spreadsheet correctly.
That, however, only solves half the problem. The AI now has access to the numbers, but does it also pick the right ones? And with business figures, how large may the gap between right and almost right be? As a rule: almost right is worthless.
Cause 2: The AI lacks the company’s business logic
Even with access to the right data, AI can return a wrong number. The reason lies in the terms. What a company means by revenue, margin or new customer is not in any table. It lives in definitions that the controlling and BI teams have established over the years.
Take the question “What was revenue in October?” In most companies, there are several correct answers to it, depending on what is meant:
| How the question is read | Result (example) |
|---|---|
| Gross revenue by order date | €1,240,000 |
| Net revenue by invoice date | €1,030,000 |
| Net revenue by payment received, excluding returns and cancellations | €945,000 |
A language model settles on one of these readings, usually the one that seems most plausible to it. It does not say which one it chose. The result is a number that looks precise and sounds plausible but does not match the figure in the monthly report.
On top of that, language models answer identical questions differently. If someone asks the same question twice, the AI may choose a different reading the second time and return a different number. For a metric that ends up in a report or a board paper, that is not acceptable.
More data does not help here. The cause of the error only disappears once the definition itself is fixed and the AI has to stick to it.
Why wrong numbers are more dangerous than wrong sentences
A clumsy sentence stands out the moment you read it. A wrong number, by contrast, looks exactly like a right one. Numbers that appear exact are rarely double-checked. They end up unquestioned in reports, forecasts and decision papers, and are passed on from there.
In everyday work, this can look like the following:
- A sales manager asks the AI for the revenue of his region and gets the gross figure. In the forecast meeting, his numbers then do not match those from controlling, which works with net revenue.
- A division head uploads a file to calculate the margin per product group. The AI calculates without the discounts that are booked separately in the ERP.
- Senior management receives two papers on the same topic with two different numbers and first has to work out which one is right.
This happens more often than many BI departments suspect. According to a survey by the German Economic Institute (Institut der deutschen Wirtschaft) (in German) from September 2026, 29.1% of employees who use AI at work use tools their employer does not provide. Nobody controls which data goes into them and which numbers come out.
Our article “Entering Company Data into ChatGPT?” describes the data protection risks that come on top of this.
What helps against AI hallucinations with numbers, and what does not
There are several ways to avoid wrong numbers from AI. How well they work depends on which of the two causes they address.
| Measure | Helps against missing knowledge | Helps against missing business logic |
|---|---|---|
| More precise prompts, such as “Only give numbers you can back up” | hardly | hardly |
| Uploading files or connecting data sources | yes | no |
| Letting the AI calculate with code | yes, the calculation is correct | no, the definition remains open |
| Including the definitions in the prompt every time | partly | partly, depends on each individual person |
| Verified metrics in a semantic layer | yes | yes |
| Transparency in the answer (metric, time reference, filters) | makes errors visible | makes errors visible |
A semantic layer is a layer between the data sources and the application. Every important metric is defined there once, as the binding standard: which source applies, how it is calculated, and which time reference and filters apply. The AI then only has to match the question to the right metric. It no longer has to calculate freely.
How to recognize a reliable answer
Five questions help with every number an AI delivers:
- Which metric was used, and is it defined exactly like that in your company?
- Which time reference applies: order date, invoice date or payment date?
- Which filters were applied, for example for cancellations, returns or sales channels?
- Was the metric verified beforehand, or did the AI calculate it freely?
- What happens if there is no definition for the question yet?
If a tool cannot answer these questions, you should not blindly trust its numbers. You can check whether your BI is ready for the use of AI with our checklist “Is your BI ready for AI?”.
How Lumdona solves this
Lumdona separates the two tasks that a language model otherwise mixes up. The AI understands the question in natural language. The number Lumdona returns comes from a metric that was defined and approved by the business beforehand. Key metrics can additionally be safeguarded with test cases and a comparison against the source system. With one click, you see which metric, which time reference and which filters were used. A number like this carries the “verified metric” seal.
If there is no verified metric for a question yet, Lumdona says so. Lumdona offers to calculate the result itself and only does so after your explicit consent. The solution shows how it calculated the result, and such a result does not carry the “verified metric” seal. The data comes directly from your systems: Lumdona connects to more than 550 data sources, from the data warehouse to the ERP and the CRM.
How metrics are defined, verified and approved in Lumdona
Is your BI ready for AI?
With our checklist, you see in a few minutes where your data and metrics stand today and which starting path suits you. You get the result right away, without signing up.
Frequently asked questions
Where does the figure of 96% made-up details come from?
From the KI-Halluzinations-Studie 2026 by the agency maxonline Marketing hfw GmbH, published in April 2026. For the study, 150 mid-sized companies from 11 industries were tested in ChatGPT between March and April 2026, each with three standardized prompt types. Of the managing directors’ names that ChatGPT actually gave, 96% were made up.
What else did the maxonline study measure?
Besides the made-up names of managing directors, the study found made-up details in 78% of the founding years given, in 68% of the employee numbers and in 58% of the awards. ChatGPT did not know more than every second company at all, 56% to be exact. Only 4 of the 150 companies, just under 3%, were presented completely correctly.
Is this an isolated case or a fundamental problem with language models?
It is a fundamental problem. Language models generate text based on probabilities, not on verified facts. For well-known large companies with plenty of public material, the answers are often correct. For mid-sized companies with little presence online, the model fills the gaps with plausible-sounding details.
Can AI hallucinations be prevented entirely?
Not with language models in general. With numbers, however, the risk can be greatly reduced if the AI does not decide for itself what a metric means and instead draws on verified definitions. It is just as important that every answer shows what it is based on.
Does ChatGPT also hallucinate when I upload an Excel file?
The calculation itself is usually correct, because ChatGPT writes and runs a small program in the background instead of estimating the number. What remains open is which columns, time periods and filters it uses. If these specifications are missing, the model chooses a plausible reading that does not necessarily match the definitions in your company.
What is a semantic layer?
A semantic layer is a layer between data sources and applications in which metrics are defined consistently. It specifies which source a metric comes from, how it is calculated, and which time reference and filters apply. BI tools and AI draw on these definitions and so arrive at the same result.
How does AI with verified metrics differ from ChatGPT and similar tools?
ChatGPT and similar tools generate answers from probabilities and, with numbers, decide for themselves how they understand a question. AI with verified metrics uses the language model only to understand the question. The result comes from a metric whose source, calculation and filters were defined and verified beforehand.
Further reading
- Entering company data into ChatGPT? Why that’s risky
- Can I enter customer data into ChatGPT?
- How data silos hold back the use of AI
- Verified, not guessed: how Lumdona works with metrics
Sources
- maxonline Marketing hfw GmbH: KI-Halluzinations-Studie 2026 (AI Hallucination Study 2026, in German), April 2026
- Barbara Engels, Andrea Hammermann, Institut der deutschen Wirtschaft: Schatten-KI: Beschäftigte sind schneller als ihre Betriebe (Shadow AI: employees are faster than their companies, in German), IW-Kurzbericht No. 63 (IW short report), September 2026