Brain wave monitoring is emerging as a new tool for training robots, with companies like Encord and Zander Labs measuring neural activity to create richer datasets that could accelerate progress in warehouse and humanoid robotics. Rather than relying solely on video footage, researchers are now capturing what human trainers are thinking and feeling as they perform physical tasks, adding a new dimension to the data that teaches AI systems how to manipulate objects and navigate real-world environments.

Why Is Physical Training Data Such a Bottleneck for Robotics?

The robotics industry faces a fundamental problem: there simply isn't enough real-world training data to teach AI models how to perform physical tasks reliably. Unlike large language models (LLMs), which are AI systems trained on vast amounts of text from the internet, robots need to learn from expensive, labor-intensive recordings of humans performing actual work. The gap is enormous. According to industry estimates, building a sufficiently large dataset for physical AI would require something like five times the volume of all video content on YouTube, a scale that explains why data generation itself has become a business rather than just a research problem.

Companies building robot brains currently rely on two main approaches to gather this data:

• Egocentric Video: Workers wear cameras that record their perspective as they perform tasks, often supplemented with additional camera angles and other sensors to capture more detail.
• Remote Operation: Humans control robotic arms from a distance using leader-follower rigs, paired robotic arms where one is controlled directly by a human operator and the other mimics its movements to create training examples.

Encord, a company that builds data tooling for training AI models, operates a facility in San Leandro, California where pilots perform tasks like stacking poker chips, pouring coffee, and plugging ethernet cables into servers. These recordings become the foundation for teaching robots how to handle real-world objects with the precision required for tasks like data center maintenance or household automation.

How Do Brain Waves Improve Robot Training?

Encord's partnership with Zander Labs, a German neuroscience startup, introduces a novel element to this process: measuring brain activity while humans perform physical tasks. The brain wave headset worn by trainers captures neural signals that reveal mental states like error detection, intent, and surprise. The theory is that these signals provide valuable clues about when a task is difficult, when mistakes are being made, and where a robot's AI model needs to deploy its highest-effort processing.

"The amount of brain activity used at any point during a given task offers clues for model builders trying to figure out when they need to deploy their highest-effort models," explained Lukas Gehrke, a neuroscientist at Zander Labs supervising the work.

Lukas Gehrke, Neuroscientist at Zander Labs

This approach is still experimental. Encord says the goal is to build an initial brain wave-tagged dataset, run it through customer robotics models, and evaluate whether it actually improves performance before deciding whether to scale it up. But the underlying logic is compelling: if a robot's AI system can learn not just what a human did, but what the human was thinking while doing it, the resulting model might be more robust and adaptable to novel situations.

What Other Sensor Technologies Are Being Explored?

Brain waves are just one of several new data modalities that Encord is developing to enhance robot training. The company is also experimenting with forearm sensors that detect electrical signals in muscles, a technique that could help overcome a major limitation of video-based training. Standard video footage often fails to capture the entire hand and all its subtle movements, but muscle sensors can provide a more complete picture of hand position and movement in three dimensions. This richer understanding of human dexterity could help robots learn fine motor skills more effectively.

Encord's datasets are also heavily annotated with physical descriptions of what each video contains, such as "right hand tightens bolt." These detailed labels help large language models understand what is happening in the footage, making the training data far more useful for teaching robots specific tasks. According to Vineeth Velmurugan, Encord's head of robot learning, this kind of dense annotation is worth approximately 100 times as much as raw, unannotated video for training specific tasks, and it only costs about 20 times more to produce.

How to Understand the Economics of Physical AI Data

• Cost Multiplier: Dense annotation of physical training data costs roughly 20 times more than collecting raw video, but provides 100 times more value for training specific robotic tasks.
• Scale Challenge: Unlike text-based AI models that scraped data from the internet at near-zero cost, physical training data must be manufactured through human labor, fundamentally changing the economics of building these systems.
• Industry Visibility: Encord's position between many robotics companies gives it a vantage point to spot which data techniques are gaining traction industrywide before any single customer can identify the trends.

This economic reality represents a sharp departure from how large language models were built. Companies like OpenAI trained their models by pulling text from Stack Overflow, Wikipedia, and other publicly available sources at minimal cost. Generating physical training data, by contrast, requires hiring workers, operating facilities, and investing in specialized equipment. That fundamental difference in cost structure will likely shape the competitive landscape of robotics for years to come.

Velmurugan, a veteran of OpenAI's robot lab and Berkshire Grey, a warehouse automation firm, joined Encord specifically to build the company's internal data-creation team. His visibility into programs across the industry allows him to see which approaches work and which don't, positioning Encord as a knowledge broker in the robotics ecosystem. The dozen or so pilots at Encord's facility, including workers like Andrew Ceja and Sofia Infante who previously worked at Scale, another AI data annotation firm, are essentially building the foundation for the next generation of physical AI systems.

The challenge ahead remains daunting. Robots still struggle with tasks that require human-level dexterity, such as manipulating ethernet cables with the precision required in data centers. Pincers lack the degrees of freedom that human fingers possess, and teaching AI systems to overcome these physical limitations will require far more training data than currently exists. But by adding brain waves, muscle sensors, and dense annotations to the mix, companies like Encord are betting that richer, more informative datasets will be the key to unlocking progress in physical AI.