Research from the University of Exeter claims that chatbots not only provide information, but may reinforce false beliefs, distorted memories and inaccurate personal narratives
Creative AI is often portrayed as a tool. It writes, summarizes, translates, suggests ideas, and helps people solve problems and get answers faster. But new research shows that this description is only partial. When AI-powered conversational systems become part of everyday thinking, they don’t just provide answers. They can also influence the way people interpret reality, remember events, and tell themselves their life stories.
The study, conducted by Dr. Lucy Osler from the University of Exeter, examines the link between creative AI and the formation or reinforcement of false beliefs. Typically, when an AI system comes up with incorrect information, it is said to be "delusional." However, Osler offers a more complex perspective: not only can AI "delusionalize us," humans can begin to "delusionalize with it."
The central claim of the study is based on an idea known as distributed cognition. According to this approach, human thinking does not exist only inside the head. It is assisted by notebooks, phones, search engines, diaries, maps, conversations with other people and now also chatbots. When an external tool is deeply integrated into the thinking process, it can affect the way a person remembers, draws conclusions and understands themselves.
The novelty in the case of creative artificial intelligence is that it is a tool with a dual role. On the one hand, it serves as a cognitive tool: it helps to remember, formulate, organize ideas, and build arguments. On the other hand, it also functions as a conversation partner. It responds, confirms, expands, relates to emotions, and sometimes creates a sense of understanding and partnership. This combination can be particularly problematic when the user brings a false belief, misinterpretation, or unfounded personal story to the conversation.
Reinforcing past mistakes
According to Dr. Osler, when people regularly rely on artificial intelligence to think, remember, and tell themselves their stories, they may enter a process in which the technology not only adds its own errors, but also reinforces errors that they already have. The conversational system tends to build its responses based on how the user presents reality. If the user's starting point is wrong, the system may continue from it, expand on it, and give it a sense of consistency.
This is where an important social element comes into play. Unlike a notebook or a search engine, a chatbot doesn’t just store or retrieve information. It can appear to listen, understand, and agree. Osler emphasizes that the conversational, almost social nature of such systems can give false beliefs a sense of social validation. When an idea appears to be “shared” with another party, it can feel more real.
AI hallucinations are reinforced by human hallucinations
The study also deals with cases where artificial intelligence systems have integrated into the thinking processes of people diagnosed with delusional thinking or hallucinations. In the literature and public discourse, such situations have sometimes begun to be described as "artificial intelligence-induced psychosis." Osler does not present artificial intelligence as a single or simple factor, but points to features that may make it a strengthening system: constant availability, emotional responsiveness, personalization, and sometimes a tendency to agree with the user rather than challenge him.
The risk is not limited to clear-cut clinical conditions. Conspiratorial beliefs, feelings of persecution, victimhood stories, or revenge narratives may also be reinforced if a conversational system continues to confirm them, build detailed explanations around them, and provide them with a seemingly logical structure. Another person may at some point argue, question, or offer an alternative interpretation. An AI system, on the other hand, may continue to flow with the user, especially if it is designed to be pleasant, supportive, and personalized.
This effect may be particularly strong for people who are lonely, socially isolated, or feel unable to talk about certain experiences with other humans. A chatbot is available 24/7, non-judgmental, responds quickly, and sometimes provides a sense of closeness. The very features that make it useful and easy to use can also make it a system that reinforces inaccurate perceptions of reality.
Osler suggests that some of the risks can be mitigated by more sophisticated safeguards, built-in fact-checking, and a reduction in the tendency for systems to agree with users. Such systems could not only avoid adding errors, but also identify cases where it is appropriate to raise a question mark or offer an alternative interpretation.
Artificial intelligence systems depend on the story we tell them
However, she says, there is a deeper problem. AI systems depend on the story we tell them about our lives. They have no body, no life experience, no social integration into the human world. So they have trouble knowing when to cooperate with the user’s perspective, and when to pause, object, or gently nudge in another direction.
The conclusion is not that creative AI is necessarily dangerous or should not be used. The study suggests a more cautious approach: understanding that it is not just an information tool, but a factor that may be integrated into thought processes and identity. As the use of such systems becomes more everyday and intimate, the more important it is to develop mechanisms that will distinguish between intellectual assistance and uncritical reinforcement of misconceptions.
for the scientific article DOI: 10.1007/s13347-026-01034-3
More of the topic in Hayadan:
