The Real Bottleneck in Robotics Isn't Hardware,It's Data, Says Carnegie Mellon Professor
Robotics has barely progressed in seven decades because the industry focused on building better machines instead of smarter brains, according to Deepak Pathak, co-founder of Skild AI and a Carnegie Mellon University professor. His central argument challenges the current hype cycle around humanoid robots: the real bottleneck isn't mechanical design or actuators, but the lack of scalable training data and a fundamentally different approach to how robots learn.
Pathak's critique is blunt. If you can visually distinguish robotics videos from 30 years ago versus today, he says, he owes you dinner. The evidence he marshals spans decades: a robot from the 1960s that could view an image of blocks and arrange physical blocks to match the pattern; a leader-follower teleoperation system from 1957 that operated on the same principles still used by major labs today; and a Berkeley robot capable of cleaning an entire table and arranging items in a box, built by a single graduate student with one GPU machine.
The contrast is stark when compared to other technologies. Language models, computer vision, and mobile phone chips have all advanced dramatically over the same 70-year period. Yet robotics remains trapped in what Pathak calls "the same land from the last 70 years".
Why Is Scaling Robotics Data So Much Harder Than Scaling Language Models?
The language model recipe,massive datasets, massive models, and emergent capabilities,does not transfer directly to robotics because there is no "internet of robotics data." Manual teleoperation, the dominant data collection method for seven decades, yields roughly one example per minute. Pathak runs the arithmetic: even if the entire US population collected robot training data continuously, it would take more than a century to reach the scale of GPT-3, a model that is now considered outdated.
Current frontier language models train on hundreds of trillions of tokens. Robotics, by comparison, has never really scaled. The field is far from even discussing data at that magnitude. This gap explains why humanoid companies continue to rely on teleoperation despite its obvious limitations: there is no proven alternative at scale.
Pathak identifies three dimensions that define the data landscape for robotics:
- Scalability: How easily can you generate large volumes of training examples without human intervention or expensive physical infrastructure?
- Diversity: Does the data cover a wide range of environments, tasks, and scenarios, or does it repeat the same setup thousands of times?
- Closeness to Robot Joints: How directly does the data map to the robot's own actuators and sensors, versus requiring translation from human movements or simulated environments?
Each data source trades off these dimensions. Teleoperation offers high-quality, joint-aligned data but is extremely limited in scalability and diversity. Simulation is infinitely scalable but requires hand-engineering every scene. Human video from YouTube or egocentric cameras is highly scalable and diverse but far removed from actual robot joint angles.
What Is the Omnibody Brain, and Why Does It Matter?
Pathak's solution is an "omnibody" approach: a single end-to-end vision-based brain trained on under ten hours of data plus human video and deployed across many different robot bodies. The key insight is that one robot, one version, one shape is unlikely to be deployed forever across every task worldwide. By training a single brain that can work with multiple robot morphologies, Skild AI enables a data flywheel: as robots are deployed in the real world, they generate new data that feeds back into pre-training, continuously improving the model.
This two-stage recipe mirrors language models but adds a critical third bucket. First, pre-train on highly scalable, diverse, lower-quality data such as simulation and human video. Second, post-train on high-quality, low-volume teleoperation data. Third, feed deployment data back into pre-training via the flywheel. This approach has already been tested in live factory deployments, including GPU assembly work for NVIDIA in Houston.
How Does Pathak's Framework Overturn Conventional Intuitions About Robot Difficulty?
One of Pathak's most counterintuitive arguments involves what is actually hard for robots versus what humans assume is hard. Hans Moravec, a founding figure in artificial intelligence and fellow CMU professor, concluded after 30 years in robotics that "hard is easy, easy is hard." Whatever humans believe is difficult is often trivial for computers, and vice versa.
Consider these examples:
- Backflips and Dancing: Humans find these acrobatic feats impressive, but robots can learn them in roughly half an hour because the task is fully observed and fully simulable; the robot only needs to understand its own body.
- Climbing Stairs: Humans treat stairs as trivial, but robots struggle because they must perceive stair height and width, adjust to disturbances, and integrate vision with understanding. Skild's stair result, published about a year and a half before Pathak's talk, involved a robot operating from a torso camera with no mapping or planning, intentionally stepping over obstacles and handling fire-escape stairs, which are the hardest stairs in any building.
- Folding Laundry: Moderate difficulty for humans, but easy for deep learning because the task has extremely high tolerance for error; humans do not think about centimeter precision while folding.
- Inserting AirPods with a Gripper: Seems impossible without hands, but the real challenge is higher-level intelligence; a gripper can only close one way, so the arm must orient the object correctly before insertion.
Pathak is particularly scathing about the humanoid industry's approach to stairs. Even top humanoid companies have shown only one stair sample "just to check the tick box," and talk of a China-versus-US humanoid race is "sort of BS." His argument is direct: if humanoids cannot climb stairs, what is the point of having legs? That is the only reason to have legs.
How to Evaluate Robotics Progress Beyond Marketing Claims?
Pathak offers a framework for distinguishing genuine robotics progress from hype:
- Look for Diversity in Deployments: Has the robot been tested on multiple stair types, multiple environments, and multiple task variations, or just one carefully curated demo?
- Check for Scalability of Data Collection: Does the company rely on manual teleoperation, or have they demonstrated a path to scaling data collection through simulation, human video, or deployment flywheels?
- Assess Autonomy Claims Critically: Can the robot operate from onboard sensors without constant human intervention, or does it require a human operator or extensive pre-programming for each new environment?
- Examine Hardware Choices: Are design decisions driven by intelligence requirements, or are they borrowed from humanoid aesthetics without clear functional justification?
The broader implication is that robotics companies claiming imminent mass production should be pressed on their data strategy. Without a credible path to scaling training data beyond teleoperation, claims of autonomy and rapid deployment are likely overstated.
Pathak's 70-year historical frame serves as a reality check for an industry that often treats robotics as a solved problem waiting only for better hardware. The evidence suggests otherwise: intelligence, not hands, remains the real bottleneck.