Self-driving cars being tested by Waymo in London may fail to detect pedestrians depending on what they're wearing, according to AI safety experts raising concerns just months before the company plans to launch commercial passenger services. The detection gaps stem from limitations in the training data used to teach these vehicles' artificial intelligence systems to recognize people on the road. Research from King's College London found that autonomous vehicles are approximately 20% more likely to detect adults than children, and over 7.5% more likely to detect white people than ethnic minorities.

Why Can't Self-Driving Cars See Everyone Equally?

The core issue lies in how AI systems learn to identify pedestrians. These vehicles rely on machine learning models trained on image datasets, but those datasets often don't represent the full diversity of real-world pedestrians. Professor Siddartha Khastgir, Head of Safe Autonomy at Warwick Manufacturers Group at the University of Warwick, explained the problem during a recent London Assembly meeting focused on autonomous vehicle safety.

"We have experimental evidence from our collaborators in Canada who have shown to us that, depending on the clothing of the pedestrian, the sensors may or may not detect the pedestrian. So, for example, winter clothing could be part of this," said Professor Siddartha Khastgir.

Professor Siddartha Khastgir, Head of Safe Autonomy at Warwick Manufacturers Group, University of Warwick

The detection challenges extend beyond winter coats. Cyclists in London, who make up a significant portion of road users, can appear similar to pedestrians at first glance, but differences in complexion, hair color, and clothing affect how the AI identifies them. This creates a potential safety blind spot in a city with heavy cycling traffic.

What Factors Affect Detection Accuracy?

The training data gaps that cause these detection problems include several specific scenarios that autonomous vehicles struggle with:

• Seasonal Clothing: Winter coats, heavy jackets, and layered clothing can obscure the body shape that AI systems use to identify pedestrians.
• Reflective Gear: Whether a cyclist or pedestrian wears reflective clothing significantly impacts detection rates in different lighting conditions.
• Summer Attire: Shorts and light clothing on hot days present different visual patterns than the training data may have captured.
• Demographic Representation: The open-source image galleries used to train these systems lack diversity, leading to bias against children and people of color.

Professor Khastgir emphasized that addressing these gaps requires deliberate action. "All those things need to be part of the training data set so this bias is not part of the detection process," he noted. He added that his team has worked with regulators and autonomous vehicle developers to create comprehensive classification systems that account for these variations.

Professor Khastgir

How to Improve Autonomous Vehicle Safety Before Deployment

Experts and regulators are pushing for specific measures to address detection bias before self-driving services launch commercially:

• Expand Training Datasets: Include pedestrians and cyclists wearing diverse clothing types, reflective gear, and seasonal attire to ensure AI systems recognize people across all conditions.
• Test for Demographic Bias: Conduct rigorous testing to measure detection rates across different age groups, ethnicities, and body types before vehicles operate on public roads.
• Implement Continuous Monitoring: Track real-world detection performance after deployment to identify and correct blind spots that testing may have missed.
• Establish Clear Safety Standards: Work with government agencies to define minimum detection accuracy thresholds that autonomous vehicles must meet before operating commercially.

Waymo is currently conducting human-supervised testing in London and is aiming to launch a fully driverless, commercial passenger service as early as September 2026. During this testing phase, a human driver sits behind the wheel ready to take control during journeys while the technology is demonstrated to be safe.

What Are Regulators Saying About Waymo's London Launch?

The detection concerns emerged during scrutiny of Waymo's safety protocols at a London Assembly meeting. Labour Assembly Member Elly Baker raised additional concerns about passenger safety features, particularly regarding protection from assault or harassment. When Waymo's Ben Loewenstein, Head of Policy and Government Affairs for UK and Europe, described how the vehicle detects cabin disturbances through seatbelt sensors and movement detection, Baker pushed back on the adequacy of these measures.

"I don't really think that is an appropriate example when I'm talking about somebody being assaulted in the back of a cab to be perfectly honest. Because what you're saying does not replace what a human in the front would be able to pick up on in terms of uncomfortable interactions in the back that didn't involve any major movement or seat belts being undone," stated Elly Baker.

Elly Baker, Labour Transport Spokesperson on the London Assembly

Baker expressed frustration that despite the imminent rollout, the potential impact of autonomous vehicles remains unclear. "It is really frustrating that, despite how close we are to roll out, the potential impact of autonomous passenger vehicles is still totally unclear," she said, adding that the hearing "raises serious doubts about whether this technology is actually ready for London".

Baker

Waymo responded to safety concerns by emphasizing its commitment to passenger protection. A Waymo spokesperson stated that "safeguarding is critically important, and strong protections and support for passengers are at the core of our operations." The company noted it offers 24/7 human Rider Support, direct access to emergency services, and processes to preserve information and support police investigations.

Waymo

The detection bias issue highlights a broader challenge facing the autonomous vehicle industry as it moves from testing to commercial deployment. While Waymo and other companies have made significant progress in self-driving technology, the gap between controlled testing environments and real-world diversity remains a critical safety consideration that regulators, operators, and safety experts are working to address before passengers board these vehicles on London's streets.