Robots are abandoning hand-coded physics simulations in favor of learning from internet video, a shift that could finally unlock general-purpose physical AI. Instead of engineers manually programming every collision, friction coefficient, and contact model, neural networks called world models are watching millions of hours of cooking tutorials, factory floors, and traffic footage to develop intuitive understanding of how the physical world actually works.

What Exactly Is a World Model?

A world model is a neural network that learns physical intuition the way humans do: through observation rather than formal rules. Show it millions of hours of video, and it develops an internal picture of how objects behave under gravity, how materials change, and how physics works in real-world conditions. This approach mirrors how a child learns that a ball will roll off a table not by solving Newton's second law, but by watching it happen repeatedly.

The critical insight emerging from recent research is that physical knowledge splits into two distinct categories. World knowledge, such as how objects behave under gravity, is universal and has nothing to do with any specific robot. Action knowledge, how a particular robot translates commands into movement, is hardware-specific and must be learned from robot-specific data. The breakthrough is this: you need very little action knowledge once you have strong world knowledge underneath.

How Are Researchers Building World Models Right Now?

Five major architectural approaches have emerged, each built on different theories about how physical systems should be encoded in learned models:

• Video-Generative Models: NVIDIA's Cosmos and Runway's GWM-1 predict future frames given a specific action, working on the idea that video captures enough information to train robots. DeepMind's Genie 3 pushes this furthest, running at 24 frames per second as the first real-time world model functioning as a live, playable simulation.
• Latent Space Models: DeepMind's Dreamer series builds a simplified internal summary of the world's current state rather than predicting every pixel. Dreamer V2 was the first agent to reach human-level performance on Atari through a world model, while Dreamer V3 outperformed specialized methods across over 150 tasks with no task-specific tuning.
• Abstract Representation Models: JEPA (Joint Embedding Predictive Architecture), developed by Yann LeCun, predicts abstract representations instead of pixels. V-JEPA 2 achieved 80 percent zero-shot success on manipulation tasks using only internet video, demonstrating that conceptual understanding of the world is more effective for physical reasoning than raw visual information.

Why Does This Matter for Robotics?

Robotics today faces a data scarcity problem that language AI never had. When large language models (LLMs) like GPT-4 were trained, they had access to trillions of tokens from the internet, essentially free and already digitized. GPT-4 trained on around 13 trillion tokens. By contrast, the Open X-Embodiment dataset, the combined output of thirty-four robotics laboratories worldwide, contains around one million robot trajectories. That gap is enormous and cannot be closed with incremental investment.

World models solve this by leveraging internet video as a substitute for scarce robot data. Two recent results illustrate the practical impact. Meta's V-JEPA 2 was trained on over one million hours of internet video, then given just 62 hours of unlabeled robot video with no task-specific training. It achieved 80 percent zero-shot success on pick-and-place tasks across laboratories it had never seen before. DeepMind's Dreamer 4 learned to collect diamonds in the game Minecraft, which required 20,000 sequential decisions from raw pixels, without any environmental interaction whatsoever.

How Does This Compare to the Language AI Revolution?

Robotics and physical AI today occupy the exact position that language AI occupied in 2005. Every simulation is built on hand-coded physics, with engineers specifying every collision dynamic, friction coefficient, and contact model. A robot trained in one of these simulations performs well in the environment it was created for, but fails in unfamiliar settings like a kitchen or when asked to perform new tasks like grabbing an object.

The problem is structural, not incremental. You cannot hand-code your way to general physical intelligence any more than you could hand-code your way to GPT-4. When transformer models trained on internet text could generate fluent, coherent prose, the grammar rules that linguists spent decades writing down suddenly became obsolete. The gap between hand-built representations and learned ones marked a major structural shift. In under a decade, the entire edifice of rules-based natural language processing (NLP) collapsed. World models represent the same inflection point for robotics.

What Are the Key Advantages of Learning From Video?

Training robots on internet video represents what may be the most consequential architectural decision in the history of physical AI. The advantages are substantial. First, the internet is saturated with video demonstrating world knowledge: cooking tutorials, factory floors, traffic, construction, and countless other scenarios showing how objects behave and materials change. Second, implicit understanding gained through observation is often more reliable than hand-coded models because it is more flexible and holds up in situations no engineer could have anticipated.

The data scarcity challenge that has held robotics back for years suddenly becomes far more manageable. Once a robot has strong world knowledge from internet video, it needs only small amounts of robot-specific data to learn how that particular machine moves. This two-tier approach transforms the economics of robot training from a problem requiring massive amounts of scarce, expensive robot data to one where the expensive part is solved by freely available internet content.

The shift from hand-coded physics to learned world models represents a fundamental rethinking of how robots acquire knowledge. Rather than formalizing rules about the physical world, systems now learn intuitive understanding through observation, just as humans do. This approach has already produced robots that can perform manipulation tasks in unfamiliar environments with minimal robot-specific training, suggesting that general-purpose physical AI may finally be within reach.