AI chatbots waffle on GOV.UK queries, then get facts wrong when told to zip it

The Register

February 19, 2026

AI-Generated Deep Dive Summary

Artificial intelligence chatbots designed to assist with government inquiries often struggle with accuracy and clarity, according to a study by the Open Data Institute (ODI). The research evaluated 11 large language models (LLMs), including ChatGPT and Claude, on over 22,000 questions based on GOV.UK content. Results showed that while the chatbots frequently provided correct answers, they often waffled or buried key facts under verbose explanations, leading to potential misinformation. This tendency to "over-explain" reduced accuracy, particularly when instructed to be more concise. The study revealed several issues with AI-driven government services. For instance, some models gave incorrect advice on sensitive topics like eligibility for benefits and legal requirements. For example, one chatbot mistakenly claimed that a court order was needed to add an ex-partner's name to a child's birth certificate, while the actual process involves re-registration. Another model inaccurately stated that a £500 maternity grant was unavailable in Scotland, despite its availability there. These errors highlight the risks of relying on AI for critical government information. Smaller, cost-effective LLMs were found to perform similarly to larger, more expensive models like OpenAI's ChatGPT. This suggests that organizations could adopt flexible AI solutions without being locked into specific suppliers. However, the study emphasizes the need for transparency and caution when deploying AI in public services. Users must be informed about potential uncertainties, and responses should be tightly aligned with authoritative sources like GOV.UK. The findings raise important questions about the reliability of AI chatbots in government settings. While these tools can streamline information delivery, their current inconsistency poses a significant risk to public trust. As governments move forward with integrating AI into services—such as the UK's planned chatbot for job

Verticals

tech

Originally published on The Register on 2/19/2026