NVIDIA has unveiled Cosmos 3, a breakthrough AI model designed to help robots, autonomous vehicles, and vision systems understand and predict the physical world, and Runway is among the founding members of a new global coalition accelerating this technology. The open model combines vision reasoning, world generation, and action prediction in a single system, reducing training cycles from months to days for physical AI applications.

What Is Cosmos 3 and Why Does It Matter for AI Video and Robotics?

Cosmos 3 represents a fundamental shift in how AI systems learn to interact with the real world. Unlike traditional AI models that process text or images in isolation, Cosmos 3 natively understands and generates text, images, video, ambient sound, and actions with what NVIDIA describes as leading physics accuracy. This matters because it gives developers a shared foundation for building physical AI systems without needing to train from scratch on fragmented tools and datasets.

The model uses a mixture-of-transformers architecture, which pairs a reasoning transformer with an expert generation transformer. In plain terms, this means the system first understands how objects interact and move in space, then generates realistic video and action sequences based on that understanding. The model was trained on billions of samples across multiple data types, giving it broad knowledge about how the physical world actually works.

How Can Developers Use Cosmos 3 in Their Projects?

• Vision Language Model: Developers can use Cosmos 3 to understand and reason across different types of data, similar to how a human might analyze a scene and predict what happens next.
• World Model or Video Foundation: The system can simulate physical environments and predict future states, which is invaluable for training and testing robots and autonomous vehicles without expensive real-world trials.
• Action Prediction Backbone: Cosmos 3 can serve as the core engine for training robots to perform specific tasks by learning from simulated environments.

NVIDIA offers three versions of Cosmos 3 to match different development needs. Cosmos 3 Super delivers the highest physics accuracy for robotics and autonomous vehicle models that demand precision. Cosmos 3 Nano provides high-quality video and action reasoning in fractions of a second, making it suitable for real-time applications. Cosmos 3 Edge, coming soon, will enable real-time inference directly on edge devices, meaning robots and systems can run the model locally without sending data to the cloud.

Who Is Part of the Cosmos Coalition and What Does It Do?

NVIDIA launched the Cosmos Coalition as a global collaboration bringing together world model builders and AI developers to advance open-source physical AI technology. Founding members include Agile Robots, Black Forest Labs, Generalist, LTX, Runway, and Skild AI. This coalition structure allows members to contribute their own models, research, and evaluation techniques while gaining access to Cosmos 3 technologies, training tools, and NVIDIA DGX Cloud infrastructure for large-scale model training.

The coalition's approach emphasizes building in the open rather than behind closed doors. By sharing a common ecosystem, members aim to accelerate innovation, improve interoperability between different systems, and speed up advances in physical AI across industries. This is particularly significant for Runway, a company known for video generation tools, as it positions the platform at the intersection of creative AI and physical world simulation.

What Real-World Applications Are Already Using Cosmos?

Physical AI developers across multiple industries are already building on the Cosmos platform. In robotics, companies like Agile Robots, Doosan Robotics, LG Electronics, Samsung Electronics, and Skild AI are leveraging Cosmos for manufacturing and automation tasks. Li Auto is using the technology for autonomous vehicle development. Vision AI agents from companies including Centific, Fogsphere, Linker Vision, Milestone Systems, and Yuan are deploying Cosmos for industrial AI and smart spaces applications.

The platform now includes new datasets specifically designed for robotics, physics, human motion, autonomous driving, warehouse safety, and spatial reasoning. NVIDIA also released new physical AI agent skills for neural scene reconstruction, defect-image generation, and video augmentation, expanding what developers can build.

How Do Cosmos 3's Performance Benchmarks Compare to Other Models?

Cosmos 3 ranks first among open-source models across multiple industry benchmarks. The model achieved top scores on Artificial Analysis, Physics-IQ, PAI-Bench, and R-Bench for world generation accuracy, which measures how realistically the model can simulate physical environments. It also leads on RoboLab and RoboArena for action policy, meaning it excels at predicting what actions a robot should take in a given situation. For vision understanding tasks, Cosmos 3 topped the VANTAGE-Bench and TAR leaderboards.

These benchmarks matter because they show that Cosmos 3 can reduce the time and cost of developing physical AI systems. By providing a strong pretrained foundation, developers need less custom training data and lower computational resources to build their own specialized models.

When and Where Can Developers Access Cosmos 3?

Cosmos 3 Super and Cosmos 3 Nano are available now. Developers can try the models on build.nvidia.com, download open models from Hugging Face, customize models using Hugging Face Diffusers, and access resources on GitHub. The models are also available as NVIDIA NIM microservices, which are containerized versions that simplify deployment in production environments.

For those needing cloud infrastructure, model builders and software providers can accelerate access and deployment through partners including Baseten, CoreWeave, Microsoft Azure, Nebius, Deep Infra, and Classmethod. Cosmos 3 Edge, designed for real-time inference at the edge, is coming soon.

The inclusion of Runway in the Cosmos Coalition signals that video generation and world modeling are converging. As physical AI systems become more sophisticated, the ability to generate realistic video simulations becomes essential for training and testing. This partnership positions Runway at the forefront of a broader shift toward AI systems that can both understand and create realistic simulations of the physical world.