Robotics has long struggled with a fundamental problem that natural language processing solved years ago: researchers working in isolated silos, unable to share training data and models across different robot types and labs. Now, a wave of new frameworks, including LeRobot from Hugging Face and RIO from Carnegie Mellon University, are attempting to create the standardized tooling and shared platforms that could finally unlock AI's potential in physical robotics.

What's Holding Robotics Back From AI Progress?

The gap between AI's success in language and vision versus robotics comes down to a simple but stubborn problem: data fragmentation. Unlike large language models (LLMs), which are trained on massive amounts of text freely available on the internet, robots generate training data one demonstration at a time, and that data typically stays locked within a single lab or company.

Robots also come in wildly different forms. Some are humanoid, others are quadrupeds, robotic arms, or grippers. Each embodiment requires different software, sensors, and control systems. Historically, this meant researchers spent enormous effort customizing their entire software stack for one specific robot, making it nearly impossible to transfer a trained model from one robot type to another.

The result is a field where the total amount of collected training data is actually less than the sum of its parts. Every demonstration recorded on a particular robot stays with that robot rather than accumulating into a shared pool that the entire field could benefit from.

How Are New Frameworks Solving the Fragmentation Problem?

The solution emerging from leading research institutions and companies mirrors what happened in natural language processing roughly a decade ago. When Hugging Face launched its Transformers library in 2016, it gave NLP researchers a common interface for models that had previously been reimplemented separately in each lab. The company then built a platform where researchers could host and share datasets, models, and tools.

Now, robotics is following a similar path. LeRobot, maintained by Steven Palma at Hugging Face, introduces a standardized dataset format that allows researchers to record robot demonstrations in a consistent way and share them on Hugging Face's platform. This means a researcher who records a robot performing a task in a supermarket can publish that data in LeRobot's format, and another researcher elsewhere doesn't need to record the same demonstration again.

"We introduced a dataset format that lets me go and record a robot in, say, a supermarket and put it out in the LeRobot dataset format. Then, someone else doesn't need to go and record the same thing. They can just use the data I recorded and put it in their training pipeline," said Steven Palma, robotics AI engineer at Hugging Face.

Steven Palma, Robotics AI Engineer at Hugging Face

Carnegie Mellon's RIO framework takes a slightly different technical approach but pursues the same goal. Instead of defining a shared dataset format for direct reuse, RIO defines an internal state representation optimized for speed. However, once data is collected, it can be exported to formats that other training pipelines expect, including LeRobot's.

Why Python and Accessibility Matter More Than You'd Think

A critical but often overlooked aspect of these new frameworks is their choice to build around Python rather than C++, the traditional language of robotics. This deliberate shift makes robotics accessible to machine learning researchers who may have no background in classical robotics.

Machine learning researchers are almost universally familiar with Python. By building frameworks around Python, the creators of LeRobot and RIO are lowering the barrier to entry for the broader AI research community. While the Robot Operating System (ROS) does support Python, the specifics of how different frameworks use it can become its own obstacle. The new generation of frameworks prioritizes ease of use for machine learning practitioners over the preferences of traditional roboticists.

Steps to Understanding How Robotics Frameworks Standardize AI Training

• Shared Dataset Formats: Frameworks like LeRobot define standardized ways to record and store robot demonstrations, including images, robot state information, and text, so data can be reused across different labs and projects without reformatting.
• Common Hosting Platforms: Hugging Face's model hub provides a central repository where researchers can upload and download robot training datasets and models, similar to how NLP researchers share language models.
• Accessible Programming Languages: Building frameworks in Python rather than C++ makes it possible for machine learning researchers without robotics expertise to contribute to and build upon robotics AI research.
• Cross-Robot Compatibility: New frameworks allow models trained on one type of robot to be adapted for different embodiments, reducing the need to start from scratch when deploying AI across a diverse fleet of robots.

Why This Matters Now More Than Ever

The timing of these frameworks is significant. Just a few years ago, the idea of building shared infrastructure for robotics AI was difficult to pitch to funders and researchers. But the explosive success of large language models and the surge of interest in AI across all domains have changed the conversation.

"I was pitching this idea two years ago, and no one was buying it. But now, everyone is talking about it," said Jean Oh, associate professor and head of Carnegie Mellon University's roBot Intelligence Group.

Jean Oh, Associate Professor and Head of roBot Intelligence Group, Carnegie Mellon University

The practical impact is already visible in research labs. Oh's lab at Carnegie Mellon had zero robots several years ago, then one, and now operates a fleet of ten. Most of these robots use different software, have different physical embodiments, and rely on different sensors. Without standardized frameworks, managing AI models across such a diverse fleet would be nearly impossible.

As robots become cheaper and more manufacturers enter the market, the fragmentation problem only grows. The lower cost of modern robots enables more companies to design and produce specialized robots for different tasks. Without shared frameworks and data repositories, each new robot type risks becoming another isolated silo, slowing progress across the entire field.

The race to build robotics' missing learning layer is ultimately about democratizing AI for physical systems the way Hugging Face and similar platforms democratized it for language. By creating shared standards, accessible tools, and centralized repositories, researchers hope to accelerate the pace at which robots can learn new skills and adapt to new environments, much as language models have transformed natural language processing.