How Zoox Is Teaching Autonomous Vehicles to 'Talk' to Pedestrians With Sound
Zoox and Virginia Tech researchers have discovered that custom audio tones can effectively communicate with pedestrians, matching the warning power of traditional car horns while sounding less aggressive. In a novel study published at the Enhanced Safety of Vehicles Conference, the team tested how sound could replace the eye contact and hand signals that normally pass between drivers and pedestrians at crossings. The findings suggest that as autonomous vehicles become more common, sound design may be a crucial tool for keeping vulnerable road users safe.
Why Can't Pedestrians Just Look at an Empty Driver's Seat?
For decades, pedestrians have relied on subtle human cues to judge whether it's safe to cross. A driver's eye contact, a wave, or even a slight nod can signal intent. But autonomous vehicles like Zoox's robotaxi have no one behind the wheel, leaving pedestrians without those familiar signals. Researchers at Virginia Tech Transportation Institute (VTTI) recognized this safety gap and partnered with Zoox, the Amazon-owned autonomous ride-hailing company, to explore whether sound could fill that void.
The challenge was significant: how do you create sounds that feel intuitive to pedestrians without prior training, communicate clear intent, and avoid the jarring aggression of a traditional horn? The research team designed seven unique audio tones to test this question in a controlled environment.
How Did Researchers Test Pedestrian Reactions to Autonomous Vehicle Sounds?
The VTTI team conducted a carefully designed experiment on Virginia Smart Roads, a closed-course test track. They recruited 40 participants to act as pedestrians in realistic crossing scenarios. To make the test feel authentic, the researchers used a "seat suit," a VTTI invention that camouflages a human driver so participants genuinely believed they were interacting with a fully autonomous vehicle.
The study involved two types of scenarios:
- Jaywalking scenarios: Participants attempted to cross mid-block in front of the vehicle, simulating the most dangerous crossing situations where pedestrian fatalities often occur
- Crosswalk scenarios: Participants crossed at marked intersections where the vehicle communicated that it would yield to them
Zoox's sound design team created two categories of tones: alert sounds meant to discourage crossing and waiting sounds meant to signal that it was safe to proceed. Across 200 jaywalking scenarios and 160 crosswalk scenarios, researchers measured how quickly pedestrians responded to each sound and compared the results to their baseline reaction time to a standard car horn.
What Did the Study Actually Find?
The results were most promising in jaywalking situations, the very scenarios where pedestrian safety is most at risk. All alert tones successfully discouraged people from crossing, and importantly, they worked as effectively as a traditional horn but were perceived as less obnoxious and aggressive. Pedestrians made the decision not to cross faster with the custom tones than they did with a horn baseline.
"We know that a lot of fatalities happen mid-block, not necessarily at intersections, but at mid-block. In the jaywalking scenario, they responded very similarly to a horn," said Charlie Klauer, research scientist at Virginia Tech Transportation Institute.
Charlie Klauer, Research Scientist at Virginia Tech Transportation Institute
The crosswalk scenarios showed more mixed results. The waiting tones successfully encouraged safe passage across a crosswalk less than 50 percent of the time, suggesting that sound alone may not be sufficient to communicate that a vehicle is yielding. Researchers hypothesized that repeated exposure to the sounds or additional visual cues from the vehicle's motion might be necessary for pedestrians to fully understand the message.
One notable finding: the study included participants with vision impairments, ensuring that the audio communication system could work for all pedestrians, not just those with perfect sight. This inclusive approach reflects a broader commitment to making autonomous vehicles safe for everyone on the road.
Steps to Designing Safer Autonomous Vehicle Communication
- Test with diverse participants: Include people with varying vision abilities to ensure audio communication works for all pedestrians, not just those with normal sight
- Compare against familiar baselines: Measure new sounds against the standard car horn to ensure they're at least as effective at capturing attention and influencing behavior
- Evaluate emotional perception: Survey participants on qualities like urgency, friendliness, and aggression to ensure sounds communicate intent without causing unnecessary stress
- Focus on high-risk scenarios: Prioritize testing in jaywalking and mid-block crossing situations where pedestrian fatalities are most common
What Makes This Research Different From Other Autonomous Vehicle Safety Studies?
According to Klauer, this is a one-of-a-kind study because it directly addresses the human decision-making process that pedestrians use when crossing streets. Rather than simply measuring whether a vehicle can detect and avoid pedestrians, the research explores what information pedestrians actually need to make safe decisions.
"This is a one-of-a-kind study. We really need to understand how pedestrians are making those decisions and what information they need to make safe decisions," said Charlie Klauer.
Charlie Klauer, Research Scientist at Virginia Tech Transportation Institute
Jeremy Yang, sound design lead at Zoox, emphasized that the company's approach to building autonomous vehicles from the ground up gives it unique opportunities to experiment with communication features like custom audio. Because Zoox designed its robotaxi without a traditional driver's seat, the company could integrate external speakers and audio systems specifically intended to communicate with pedestrians.
"Sound is one of our most innate senses. We wanted to examine the effectiveness of non-traditional automotive sounds in eliciting safer responses from pedestrians," said Jeremy Yang, sound design lead at Zoox.
Jeremy Yang, Sound Design Lead at Zoox
The research also highlights a broader principle: autonomous vehicle safety extends far beyond the passengers inside. Pedestrians, cyclists, and other vulnerable road users must be considered from the earliest stages of vehicle design. As more robotaxis hit city streets, the ability to communicate clearly and intuitively with pedestrians will become increasingly important.
The study's findings suggest that Zoox and other autonomous vehicle companies have a promising tool at their disposal. By carefully designing audio signals that feel natural and intuitive, they can help bridge the communication gap created by the absence of a human driver, making streets safer for everyone.
" }