Nvidia is fundamentally changing how robots and autonomous systems are trained by giving AI agents the ability to build and prepare realistic 3D simulation environments. CEO Jensen Huang announced Nvidia Omniverse Libraries, a collection of software components designed to help AI agents collaborate with developers inside existing 3D tools, preparing virtual worlds where physical systems can be tested and trained long before they operate in the real world.

What Are Omniverse Libraries and Why Do They Matter?

Omniverse Libraries represent a significant democratization of Nvidia's simulation platform, which has been in development for nearly a decade. The libraries break down Nvidia's complex Omniverse platform into smaller, more accessible building blocks that can be integrated directly into software tools developers already use daily. This approach makes simulation capabilities available to far more teams without requiring specialized, expensive hardware setups.

The libraries include three core components designed to handle different aspects of simulation preparation. These tools address a fundamental challenge in robotics and autonomous systems: preparing 3D content for simulation requires far more than just realistic visuals. Assets need proper structure, materials, accurate scale, labels, sensors, and physical properties to be useful for training AI systems.

How to Integrate Omniverse Libraries Into Your Development Workflow

• RTX Sensor Simulation (ovrtx): This tool helps applications generate camera, lidar, radar, and other sensor outputs from 3D scenes, allowing developers and AI agents to test how physical AI systems perceive virtual environments before deployment in the real world.
• Physical Behavior Simulation (ovphysx): Using GPU-accelerated physics, this component brings realistic behavior to 3D scenes by simulating collisions, mass, friction, and motion, enabling teams to test how objects and systems interact in simulation first.
• CAD-to-SimReady Conversion: These AI skills automatically convert computer-aided design data into simulation-ready assets built on OpenUSD, giving 3D content the properties needed for physical AI simulation and virtual testing without manual rework.

The libraries are openly available on GitHub, making them accessible to developers across industries. Nvidia is also providing a blueprint for integrating Omniverse libraries directly into Blender, one of the most widely used 3D creation tools, lowering barriers to adoption.

Which Companies Are Already Adopting This Technology?

Major software makers and startups are moving quickly to integrate Omniverse Libraries into their platforms. SideFX, the company behind Houdini, a leading procedural 3D content creation tool, is exploring how AI agents can help integrate Omniverse libraries into its workflows. This allows technical artists to generate, test physics, and prepare content for simulation while maintaining creative control.

"Procedural 3D creation is essential to building the complex, controllable worlds needed for simulation, robotics and industrial AI. With Nvidia Omniverse libraries and OpenUSD, SideFX is exploring how agent-ready tools can support Houdini workflows, helping technical artists review, test and prepare procedural content for simulation while staying in control of the creative process," said Kim Davidson, president and CEO of SideFX.

Kim Davidson, President and CEO of SideFX

PTC, which makes Onshape, a cloud-native computer-aided design platform, is using OpenUSD and the RTX sensor simulation library to connect design workflows with physical simulation. This integration helps product design teams keep their content connected across design, collaboration, and simulation workflows.

"Engineering teams are seeking more connected ways to design, collaborate and simulate throughout the development process. PTC's work with Nvidia supports that broader vision, while Nvidia Omniverse libraries help enable simulation-ready workflows that bring validation and testing closer to where products are designed," stated Neil Barua, president and CEO of PTC.

Neil Barua, President and CEO of PTC

Startups including Palatial, ForgeCAD, and MoonlakeAI are among the first to adopt Omniverse libraries for agent-assisted workflows. Palatial is using the CAD-to-SimReady skills to automate the creation and validation of simulation-ready assets at scale, directly from computer-aided design inputs.

Why This Represents a Turning Point for Physical AI

Jensen Huang emphasized the strategic importance of this shift, stating that "the physical AI era will be built in simulation first." This reflects a fundamental principle in robotics and autonomous systems: training and testing in virtual environments before real-world deployment reduces risk, saves time, and accelerates development cycles.

Jensen Huang

"The physical AI era will be built in simulation first. With Omniverse libraries, AI agents become collaborators inside the 3D tools developers already use, building the simulation-ready worlds where robots, factories and autonomous systems are trained and tested long before they reach the real world," said Jensen Huang, CEO of Nvidia.

Jensen Huang, CEO of Nvidia

Rev Lebaredian, vice president of physical AI simulation technology at Nvidia, explained that the company started working on Omniverse nearly a decade ago with the goal of simulating the physical world accurately enough to train AI systems that operate in real environments. Early versions were constrained by available technology, but advances in AI, GPU rendering, and physics simulation algorithms have made the vision feasible.

The shift toward disaggregating Omniverse into microservices and cloud-based components has already made the platform more accessible. Now, breaking it down further into agent-friendly libraries makes integration nearly automatic for partners and developers. This approach removes technical barriers that previously limited adoption to organizations with specialized expertise and expensive hardware.

What Hardware Will Support These New Capabilities?

Nvidia is making Omniverse Libraries available across a range of hardware options to ensure accessibility. RTX Spark systems, compact GPU-powered computers, will be available this fall from major manufacturers including ASUS, Dell Technologies, HP, Lenovo, Microsoft Surface, and MSI, with models from Acer and Gigabyte to follow. For more demanding workloads, DGX Station systems are available to order from ASUS, Dell, Gigabyte, HP, MSI, Supermicro, and Exxact.

Nvidia will showcase sample workflows at Siggraph, including a "SimReady" Blender demonstration built with Omniverse libraries and the Nvidia Nemotron Ultra open model. This blueprint shows how software makers can add agent-ready simulation capabilities, including sensor simulation, physics, and validation, into existing 3D applications while keeping creators in control.

The announcement reflects Nvidia's broader strategy to position itself at the intersection of computer graphics, physics simulation, and artificial intelligence. By making simulation tools more accessible and agent-friendly, Nvidia is enabling a new generation of developers to build the virtual training environments that will power the next wave of robotics and autonomous systems innovation.