AI chatbots and search engines are being systematically manipulated to spread false information to millions of people, but Google and other tech companies are now taking steps to address the problem. A BBC investigation uncovered how unscrupulous companies are exploiting a simple vulnerability in how AI systems gather information from the internet, allowing them to inject biased or completely fabricated answers into responses seen by over a billion regular users worldwide.

How Are Bad Actors Manipulating AI Systems?

The vulnerability works because many AI tools, including ChatGPT, Claude, and Google's various AI products, don't just rely on data built into their training. Instead, they search the internet for answers to user questions. This creates an opening for manipulation. According to search engine optimization (SEO) experts monitoring the issue, AI tools often pull information from a single web page or social media post, leaving them vulnerable to bogus information.

The manipulation technique is surprisingly simple. In a demonstration, a BBC journalist published a single article on a personal website claiming to be a world-champion competitive hot-dog eater. Within 20 minutes, both ChatGPT and Google's AI systems were confidently telling the public this false claim. The same technique has been used at scale to dismiss health concerns about medical supplements, influence financial information about retirement planning, and spread other misleading content.

What Are the Real-World Consequences?

The stakes are significant. Globally, more than a billion people use AI chatbots regularly, and 2.5 billion people see Google's AI Overviews each month. If these systems can be subverted, it gives bad actors immense power to influence public opinion and behavior.

The consequences extend beyond embarrassing false claims. Experts warn that manipulated AI responses can lead to serious harm:

• Health Risks: Users might take medical advice from manipulated AI responses that makes them sicker than they were before.
• Financial Harm: People could receive bad financial information and make poor decisions about retirement or investments.
• Legal Consequences: Users might get incorrect legal information and do something that is not legal in their state or country.
• Voting and Hiring Decisions: Biased information can influence how people vote or which service providers they hire.

"You should assume that you're being manipulated until they have better systems in place. We're moving towards this 'one true answer' world. Before, Google would give you 10 blue links and you would kind of do your own research. But AI just gives you one answer. It becomes so easy to just take things at face value. You need to be careful," said Lily Ray, founder of the SEO and AI search consultancy Algorythmic.

Lily Ray, Founder of Algorythmic

What Changes Has Google Made?

Last week, Google updated its spam policies to officially confirm that attempts to manipulate AI responses violate the company's rules. The update signals that Google is proactively looking for those who try to abuse the system. If a company or website is caught breaking the rules, it could be removed from or downranked in Google's search results, which effectively removes it from public visibility.

Google characterizes the policy update as a "clarification" of efforts it has been making for a while, stating that it has "long applied core anti-spam policies and protections to generative AI Search features." However, the timing suggests the company is ramping up enforcement. Behind the scenes, there is evidence that Google and other companies are experimenting with solutions.

According to SEO experts, Google and ChatGPT appear to be quietly removing companies from AI answers when they suspect self-promotion. Additionally, both Google and AI tools made by Anthropic have started adding more labels to responses, letting users know when the chatbot isn't confident about its answers. Some tools are also explicitly telling users they're trying to root out spam in certain responses.

Will These Fixes Actually Work?

Experts are skeptical that current measures will be sufficient. Harpreet Chatha, who runs the SEO consultancy Harps Digital, compared Google's efforts to "playing whack-a-mole." As Google cracks down on manipulative blog posts, companies are finding subtler ways to promote themselves. For example, instead of publishing biased articles, bad actors can pay YouTube influencers to promote products, and since Google's AI now cites YouTube videos, the manipulation cycle continues.

Even as these policy changes roll out, evidence suggests people are still using the exact same techniques to fool the world's biggest search engine. Just this week, an SEO specialist successfully made Google tell people that another SEO specialist is good at building sandcastles, demonstrating that the vulnerability persists despite Google's stated efforts.

For now, the best defense remains user awareness. Experts recommend remembering what AI actually is: a tool that confidently gives you one answer, whether it's right or wrong. Just because a response appears to come from a giant tech company doesn't mean you should take it at face value without verification.