Hallucinations in AI occur when an AI model generates information that is inaccurate or misleading but presents it as if it were true.

For businesses relying on AI customer service, false or misleading information can erode customer trust and lead to operational inefficiencies. Given the stakes, preventing hallucinations is a top priority for me as the VP of Machine Learning and AI at an AI-powered customer service automation company.

There’s no quick fix or magic solution, but we try to prevent hallucinations as much as we possibly can. In our efforts to do this, we’ve come up with some best practices for preventing hallucinations in AI agents. Let’s dive in.

Understanding hallucinations in AI

AI hallucinations aren’t just a minor glitch — they’re a fundamental challenge in AI systems. These hallucinations can range from simple factual inaccuracies to complex, misleading narratives.

For instance, an AI agent might incorrectly inform a customer about company policies, suggest non-existent troubleshooting steps, or provide false order statuses. It could misattribute quotes, timelines, stats, or events incorrectly.

Generative AI tools hallucinate anywhere between 2.5 to 22.4% of the time, according to Vectara , and I’m sure you’re wondering why.

Why do AI agents hallucinate?

Hallucinations in AI can happen for a variety of reasons, including training data and training methods. It’s not just about what data is fed into an LLM, it’s also about how the model is trained. For instance, the model could be trained or fine-tuned to always output an answer — it could even be given the option of saying “I don’t know.”

Here’s a brief breakdown of why AI hallucinates. For a more detailed look, including the greater impact this has on customer service, head here

• Training data quality: Generative AI models are trained on vast datasets that may contain both accurate and inaccurate information. Inaccurate or insufficient data equals inaccurate results.
• Training processes: Incomplete or training that isn’t continuously monitored and tweaked can increase the onset of hallucinations. Again, it could be trained or fine-tuned to always output an answer, even if the answer is “I don’t know.”
• Pattern recognition limitations: AI models generate responses based on patterns they recognize in the training data. If a query doesn't have a clear pattern, the model could create a plausible-sounding but incorrect response.
• Model architecture: AI models still have limitations in understanding complex contexts, reasoning, and generating coherent outputs (another reason why you should be monitoring and optimizing regularly).
• Prompt input: The way input prompts are formulated can impact the model's responses. Conflicting, ambiguous, or leading information can increase the likelihood of hallucination. For example, a newly revealed Apple prompt reads , “Do not hallucinate. Do not make up factual information.” This is too ambiguous. Instead, a more constructive method would be this example from OpenAI : “Information you are providing in your response must be grounded in trusted knowledge.”

Best practices to prevent AI hallucinations

There’s still no way to eliminate hallucinations completely, but following best practices can help you manage them more effectively. Preventing AI hallucinations requires a multi-faceted approach.

​​1. Optimize training data and knowledge bases

• Create a knowledge base: Build a knowledge base that answers FAQs and provides all the relevant information on areas where your customers might need help. Organize this information in a way that’s easy for the AI agent to sort through and pull from whenever needed.
• Train or fine-tune on high-quality data: Ensure AI models are trained on diverse, balanced, and well-structured data. This helps minimize output bias and improves the model's understanding of its tasks, leading to more accurate outputs. Focus on quality over quantity. Sure, quantity is important, but training AI on a smaller, high-quality dataset yields far better results.
• Clean your data: Remove duplicates, outdated information, and irrelevant information. Detox your data before you feed it to the AI tool.
• Test LLMs: Selecting the right LLM makes a world of difference because LLMs differ on various fronts, including speed, accuracy, and scalability.

2. Robust verification systems and knowledge base optimization

Incorporate verification and filtering checks to ensure that the answers generated by the AI agent are grounded in the company's knowledge base and are accurate. This can involve using internal models that compare replies generated by the AI agent to the underlying knowledge documents.

Ensuring consistency and accuracy on a regular basis minimizes the potential for contradictory or outdated information.

3. Adversarial training

Train your AI models on a mixture of normal and adversarial examples to improve their robustness against adversarial attacks. This helps the model better handle unexpected or confusing inputs.

4. Grounding

Grounding is the process of connecting AI outputs to verifiable sources of information, making it more likely that the model’s responses are anchored in fact. Some effective grounding techniques include:

• Retrieval Augmented Generation: Insteading relying solely on its pre-trained knowledge, RAG allows an AI model to pull information from external, verified sources in real-time. When an AI agent uses RAG, it’s able to cross-verify information across multiple sources and use various verification mechanisms to ensure the accuracy of its responses.
• Prompt engineering: You can optimize your writing for AI by using clear and specific language. Avoid ambiguous or leading prompts that could confuse the AI model, and use full sentences and simple language to ensure that the AI agent can accurately interpret the information. You can also provide context and examples to enhance the quality of the AI’s responses and use constraints to narrow down the desired output.

5. Train the team

Educate your customer service team about hallucinations in AI, and give them a better understanding of what causes them.

• Create a clear and concise document that covers common issues and steps for corrections
• Show them past instances when the AI hallucinated
• Consider interactive simulations to help the team practice
• If possible, bring in an expert to discuss new methods to tackle the topic and teach the team new techniques

6. Continuous monitoring and feedback

Implement continuous monitoring and feedback loops to measure, identify, and correct hallucinations. This involves regularly reviewing conversation logs and reasoning logs to determine whether the information provided by the AI agent is accurate and relevant.

In this process, ensure the training data it’s thoroughly vetted, cleaned, and free from biases and inconsistencies.

You should also continuously fine-tune and update the model, with high-quality, domain-specific data to improve its understanding of the customer service context and keep the model(s) current.

Detect and correct hallucinations in AI

Taking proactive steps to prevent hallucinations can greatly impact the quality of responses. Here are a few ways to do it:

1. Measurement and analysis tools

Use measurement and analysis tools to assess where the AI agent went wrong and flag opportunities for improvement. This can involve manual reviews of conversation logs and reasoning logs or automated tools.

• Manual reviews: Test the AI agent with complex questions and scenarios. When you find an error or room for improvement, provide feedback. This feedback loop nudges AI to generate better, more accurate responses. This can be time-consuming, so consider prioritizing based on complexity and previous error rates.
• Use automated tools: Use a combination of automation tools to keep your AI agent on the straight and narrow. You could use fact-checking APIs, other LLMs, data validation tools, and content moderation tools to prevent AI hallucinations.

2. Provide feedback and coaching

Treat your AI agent like an employee by providing continuous feedback and coaching. When a hallucination occurs, analyze whether the AI agent is missing or misinterpreting information and provide guidance to correct its reasoning.

Preventing hallucinations in AI is ongoing — but we’re making progress

I’ll leave you with some ethical considerations for using an AI agent, in light of the state of hallucinations in AI:

1. Let customers know they’re interacting with AI: It’s best to be transparent with customers about using AI, so there’s less shock and confusion if a hallucination occurs.
2. Own your mistakes: When AI hallucinates, own your mistakes and fix it quickly. Allow customers to share feedback and better train your AI based on it.
3. Be vigilant about data and security: When personal information is on the line, you need to be on top of data security and ensuring the AI is properly trained and managed. Not only is this an ethical problem, you could end up violating data protection laws.

AI hallucinations pose a significant challenge, but they can be mitigated with the right strategies. While we can’t eliminate them completely, we have made great strides in preventing, detecting, and correcting them.