The next frontier in robotics training isn't about smarter algorithms,it's about capturing the hidden signals in human brains that reveal intent, error detection, and surprise. Companies like Encord and Zander Labs are experimenting with brain wave headsets worn by human trainers to generate higher-quality training data for robots, tackling what may be the biggest constraint holding back physical AI: the sheer scarcity of real-world training data.

Why Is Training Data Such a Bottleneck for Robots?

The challenge facing robotics companies today mirrors the problem that plagued large language models (LLMs), or AI systems trained on vast amounts of text data, before they became powerful enough to be useful. But there's a critical difference: while companies building chatbots could scrape billions of words from the internet for nearly free, robots need physical-world data that must be painstakingly collected and annotated by humans.

Vineeth Velmurugan, head of robot learning at Encord and a veteran of OpenAI's robot lab, explained the scale of the problem: "The data simply does not exist." Velmurugan estimates that breaking through current limitations will require a training dataset roughly five times the size of YouTube's entire video corpus, a staggering amount that underscores why data generation itself has become a business rather than just a research problem.

"The data simply does not exist," said Vineeth Velmurugan, head of robot learning at Encord.

Vineeth Velmurugan, Head of Robot Learning at Encord

Self-driving car companies have solved this by collecting their own data from real-world driving, but that approach doesn't scale easily to warehouse robots or humanoids learning household tasks. Training from video alone can work, but it lacks the fidelity of actual physical interaction. This gap is where brain waves enter the picture.

How Are Brain Waves Improving Robot Training Data?

At Encord's facility in San Leandro, California, human trainers wearing brain wave headsets built by German neuroscience startup Zander Labs perform tasks like carefully removing blocks from a Jenga tower or plugging ethernet cables into servers. The headset includes a camera tracking what the trainer sees, but the innovation is the addition of sensors measuring brain activity to detect mental states like error recognition, intent, and surprise.

The theory is elegant: when a robot trainer's brain shows heightened activity during a particular moment, it signals that something important is happening,a mistake being caught, a difficult decision being made, or an unexpected challenge. By tagging training data with these brain wave markers, robotics companies can identify which moments in a video contain the most valuable learning signals.

Lucas Gehrke, a neuroscientist at Zander Labs supervising the work, explained that brain activity patterns offer clues for model builders about when they need to deploy their highest-effort AI models. Encord is currently running this as a trial, with plans to build an initial brain wave-tagged dataset, test it with customer robotics models, and evaluate whether it actually improves performance before deciding whether to scale the approach.

What Other Data Modalities Are Robotics Companies Exploring?

Brain waves are just one of several new approaches Encord is developing to solve the data scarcity problem. The company is experimenting with multiple techniques to capture richer information about human movement and decision-making:

• Egocentric Video: Workers wear cameras that record their perspective while performing tasks, often supplemented with additional camera angles and other sensor data to create a more complete picture of what's happening.
• Muscle Signal Sensors: Electrodes strapped to the forearm detect electrical signals in muscles, allowing researchers to build a 3D model of hand position and movement that video alone cannot capture, since cameras typically don't record the entire hand.
• Remote-Operated Robots: Paired robotic arms, one controlled directly by a human operator and one that mimics its movements, generate data about complex manipulation tasks like pouring coffee or stacking poker chips.

Velmurugan estimates that densely annotated training data, where each video is tagged with precise descriptions like "right hand tightens bolt," is worth roughly 100 times as much as raw, untagged video for training specific tasks. The catch is that producing this annotated data costs about 20 times more than simply collecting raw footage, creating a real economic constraint on how quickly robotics companies can scale their models.

How Does This Compare to How AI Language Models Were Built?

The fundamental difference between training robots and training chatbots reveals why physical AI faces unique challenges. Companies building large language models scraped text from the internet, Stack Overflow, and countless other sources at virtually no cost. That free data fueled the explosive growth of models like GPT and Claude.

Physical AI cannot follow the same playbook. Robot training data must be manufactured, not just collected. A robot learning to manipulate objects needs video of humans performing those exact tasks, annotated with precise descriptions of what's happening at each moment. This manufactured data is expensive, and that economic reality fundamentally changes how quickly physical AI can advance compared to language models.

At Encord's San Leandro facility, the reality of this data manufacturing is visible in storage racks holding fake flowers in vases, plastic vegetables, kitty litter trays, books, and bundles of wires. These props are the raw materials for training robots to handle household and workplace tasks. Pilots like Sofia Infante and Andrew Ceja spend their days maneuvering robotic arms to perform precise manipulations, generating the training data that will eventually teach robots to do the same work autonomously.

What's the Business Model Behind Robot Training Data?

Encord's position as a data infrastructure company gives it a unique vantage point across the robotics industry. By working with multiple leading robotics firms simultaneously, the company can observe which data techniques are gaining traction industry-wide before any single customer discovers them. This visibility into cross-industry trends becomes part of Encord's value proposition.

The company's workforce of a dozen or so pilots at its San Leandro facility represents a new category of AI workers: people whose job is to generate training data for physical AI systems. Both Infante and Ceja previously worked at Scale, another AI data annotation firm, before joining Encord. Ceja's background is particularly telling; he previously worked at a waste management company where his interest in technology led him to maintain a robotic trash sorter. Now he's helping train the next generation of robots by carefully disassembling Jenga towers while wearing a brain wave headset.

This emerging workforce and the infrastructure around it represent a bet that the next constraint on robotics won't be model architecture or computing power, but rather access to high-quality, diverse, real-world training data. As Velmurugan noted, progress is being made across the industry, with startups and frontier labs alike figuring out what works and what doesn't to improve physical AI models. The brain wave experiment at Encord is part of that broader effort to unlock the potential of humanoid and warehouse robots by solving the data bottleneck that currently limits their capabilities.