LLMs don't understand what they're saying
4 MINUTES READ
After examining last week how hallucinations can arise in Large Language Models (LLMs) like ChatGPT, today we look at what can be done about them. First, some bad news: given how the models are currently built, it is impossible to completely prevent hallucinations when generating responses. Because LLMs do not understand the content of what they reproduce, they cannot verify it substantively. As a result, false or half-true answers will always appear. However, there are techniques that can check for hallucinations and warn against them. We list a few of these strategies today.
ChatGPT, Claude & Co
Don't ask chatbots questions whose answers could influence critical decisions. What constitutes a critical decision is something each person must decide for themselves. If, for example, I ask for a recipe for cheesecake and ChatGPT forgets to list the sugar, at worst I simply end up with a bad cake. If I ask the chatbot about my child's illness symptoms and it gives incorrect recommendations, the consequences could be far worse. Of course, a Google link or your own neighbour can also give a wrong answer. The rule always applies: those who question answers, verify them, and apply common sense can avoid such pitfalls. However, it must be said again: LLMs like ChatGPT do not respond with intent or intelligence. They string words together based on probability. So one cannot even hope that the answer was well-meant. Of course, this advice also applies to other chatbots, including self-built ones.
AI Agents
To minimise false statements from AI agents, they should be designed to serve only one purpose. These AI agents are not built like ChatGPT to find answers to everything, but rather behave like a single cog that has only one function. For example, one agent might read texts and sort them by topic. Another agent might digitise text via image recognition. Yet another could be responsible for directing incoming customers in customer service to the right departments.
Guidelines
It sounds simple, but good instructions for using supportive AI agents can prevent many errors and hallucinations. This naturally also applies to agents working in the backend to merge and analyse data or advance image recognition. When these receive unsuitable input, errors quickly occur. However, this particularly applies to AI agents used by employees and customers. Expectations of these agents are often high — almost too high. But this is not the developers' fault; rather, it stems from the current landscape of available chatbots like Claude or ChatGPT, as well as reporting on their results. Many see chatbots as miracles that can do everything. No matter what they input or ask, they get back an answer that at least looks correct (and fortunately is often right). On the surface, the AI agents deployed in companies or for specific purposes such as database queries are exactly the same: free text input and free responses. What happens behind the scenes is opaque, and therefore expectations are often the same, even though specialised AI agents operate in a more limited environment. Simple instructions can help here to narrow expectations, formulate requests better, and ideally avoid hallucinations.
Limitations
It can be helpful to limit inputs to streamline results. Rather than allowing all data and requests, it is often useful to filter inputs in advance — not only to help users phrase their questions more precisely, but also to feed the LLMs with data that can generate the best possible answers. Of course, a question built from several dropdown menus does not feel as free as an empty text field where anything can be entered... but that is precisely the point. Expectations are automatically guided in the right direction and answers are more targeted.
Built-in Approaches
Up to this point, all the ways to avoid hallucinations have been on the user side. Whether it's how questions are asked, data is limited, or agents are designed, the work lies with the users. But why not tackle hallucinations at the root? With the ever-growing flood of AI agents being deployed in many business areas, this is precisely a central research focus for many developers. The way LLMs are fundamentally set up, they work quite well — so tinkering at the base is risky. But what if AI could finally understand what it's saying? If not just probabilities string words together, but meaning could be understood and verified?
The paper "Addressing hallucinations in generative AI agents using observability and dual memory knowledge graphs" (https://www.sciencedirect.com/science/article/pii/S0950705126002121) addresses such an approach. The basic ability to generate content is not changed, but the way to verify it is added as an additional step. Put simply, this is what happens: imagine you are asked to write a grammatically correct sentence with any content. The resulting sentence reads: "The cat flies through the clouds." Grammatically, there is nothing wrong here, but in terms of content, something is off. From one's own experience, one knows that cats generally cannot fly. A person with this knowledge would therefore say: sounds right, but is wrong. For the AI to be able to say this as well, the researchers give it additional knowledge databases where information is also stored according to other criteria such as semantic and thematic proximity. In particular, historically correct content is also stored as such so that it can be used as comparison material. However, the paper also finds that even with these steps, AI is still no better than a human in terms of information accuracy when it comes to broad knowledge questions. This is mainly because even with additional "memories," AI still does not understand what it generates — it can simply compare better. Even if such tools become better and more accurate in the future, it is always important in the end that a human makes the final decision.
With increasingly powerful LLMs being deployed ever more broadly, it is now more important than ever to prevent hallucinations through various measures. A system that incorporates these problems into the design from the start is safer for the moment and open to adjustments that will be found in the future.
News & Insights
