Physical AI systems operate fundamentally differently from the language models that have dominated GPU demand for the past three years, requiring continuous processing rather than request-response cycles, and this structural difference is forcing enterprises to completely rethink how they plan, size, and cost their AI infrastructure. Robots, autonomous vehicles, surgical assistants, and warehouse automation systems must process sensor streams and produce control outputs within 10 to 100 milliseconds without interruption, creating a GPU demand profile that is predictable, steady, and immune to traffic spikes.

How Does Physical AI Create Different GPU Demand Than Language Models?

The distinction between physical AI and traditional language model inference comes down to operational constraints. Language models tolerate variable latency; if a response takes 800 milliseconds instead of 400, users notice but the application still functions. A robotic arm on an assembly line operating at 1,000 parts per hour cannot tolerate a 500-millisecond GPU scheduling delay. The compute requirement is not just continuous, it is deterministic.

Language model inference at hyperscale is driven by user query volume, spiking at peak hours and dropping overnight. Physical AI generates GPU demand with an entirely different structure: it is proportional to the number of deployed units multiplied by their operating hours. A warehouse with 3,000 autonomous mobile robots running 20 hours a day generates a GPU load that is fixed, predictable, and grows only when you add robots or operating shifts. There are no traffic spikes. There is no overnight lull. The GPU simply runs.

This distinction has two critical implications for enterprise GPU planning. First, the GPU economics of physical AI deployments are primarily a fleet size and utilization problem, not a peak-demand sizing problem. You do not need to provision for burst; you provision for the steady-state fleet. Second, the latency constraints of physical AI inference largely preclude centralized cloud inference. Round-trip latency from a factory floor to a cloud GPU cluster and back is typically 20 to 80 milliseconds under good conditions. For control loops that require responses within 50 milliseconds, that leaves essentially no margin. Physical AI inference runs at the edge, on the device itself or at a local edge cluster, not in a hyperscaler data center.

What Hardware Changes Enable Physical AI at Scale?

NVIDIA's Jetson platform has long been the reference architecture for edge AI in robotics, and the Jetson Thor, now in general availability as of August 2025, represents a generational step that materially changes what is possible at the edge. The Jetson Thor delivers 2,070 FP4 TFLOPS (floating-point operations per second) of AI compute with a 128 GB LPDDR5X memory pool, running within a power envelope of 40 to 130 watts. For reference, the Jetson AGX Orin it succeeds delivered 275 TOPS at up to 60 watts. That is a 7.5-fold compute improvement at roughly 2 times the maximum power, an efficiency gain of approximately 3.5 times.

What makes this relevant for enterprise planning is not the raw performance number; it is what the performance enables. Jetson Thor can run vision language models and vision language action models in real time at the edge, without cloud round-trips. This is the architectural shift that enables what industry analysts are calling "Level 3" physical AI: systems like Amazon's Vulcan manipulation robot that can handle approximately 75 percent of the one million unique items in Amazon's fulfillment catalog, grasping unfamiliar objects, extracting tightly packed items from cluttered bins, and choosing context-dependent grasp points without human intervention. That capability requires foundation model inference at the edge. The H100 in a 700-watt server form factor is irrelevant to this use case; the Jetson Thor at 130 watts is the enabling hardware.

Adoption is moving faster than many enterprises recognize. Boston Dynamics is integrating Jetson Thor into its humanoid Atlas robot. Agility Robotics is adopting Thor for the sixth generation of its Digit humanoid. NEURA Robotics launched a Gen 3 humanoid at CES 2026 powered by Jetson Thor. Amazon Robotics is running the NVIDIA Jetson platform across its manipulation systems and mobile robots and moved its BlueJay multi-arm manipulator from concept to production in just over a year using NVIDIA Omniverse simulation.

How Should Enterprises Structure Physical AI Infrastructure Across Three Compute Tiers?

A production physical AI deployment does not map to the infrastructure model of cloud AI. It spans three compute tiers with different hardware profiles, latency requirements, and cost structures, and all three must be planned simultaneously. The failure mode observed in enterprise physical AI programs is planning the edge hardware and the cloud training cluster in isolation, without accounting for the facility-level compute layer that connects them.

• Edge Compute Tier: Hardware includes NVIDIA Jetson Thor (2,070 FP4 TFLOPS, 40 to 130 watts) and Jetson AGX Orin (275 TOPS, 15 to 60 watts), with latency targets under 50 milliseconds for control loops. This tier handles sensor fusion from cameras, LiDAR, radar, and inertial measurement units, motion planning, collision avoidance, and real-time action selection. The key cost driver is hardware cost per deployed unit multiplied by fleet size. A fleet of 1,000 Jetson Thor-equipped robots represents a hardware line item that must be capitalized, maintained, and upgraded on a separate cycle from cloud infrastructure. Edge systems generate 1 to 10 terabytes of sensor data per operating vehicle per day, varying by sensor payload and resolution.
• Local GPU Cluster Tier: Hardware includes NVIDIA RTX PRO Server, H100, or L40S clusters with 4 to 16 GPUs and high-bandwidth local networking, with latency targets under 200 milliseconds for non-real-time tasks. This tier handles aggregated sensor data processing and indexing, local model versioning and over-the-air update serving to the edge fleet, digital twin synchronization, anomaly detection across the fleet, and short-horizon retraining on facility-specific data. The key cost driver is sized by the number of edge units the cluster must serve and the frequency of model updates. A 1,000-unit fleet generating 1 to 10 terabytes per unit per day will overwhelm an undersized facility cluster before it reaches the cloud training tier.
• Cloud Training Tier: Hardware includes H100, B200, or GB200 NVL72 clusters shared as a resource across multiple physical AI programs. This tier handles full model training and fine-tuning on aggregated real-world data, synthetic data generation via NVIDIA Isaac Sim and Cosmos world models, fleet-wide policy updates, and reinforcement learning from edge-collected trajectories. The key cost driver is data volume and the frequency of policy updates across the deployed fleet.

Why Is Cooling Architecture Becoming Critical for GPU-Dense Deployments?

Liquid cooling is no longer a specialty deployment; it is the default path for any data center designed to host GPU-dense AI workloads. The question facility engineers are now forced to answer during early site planning is not whether to use liquid but which kind, and what that decision implies for fluid chemistry, procurement complexity, and long-run operating cost.

Forced-air cooling, which remains the installed base majority worldwide, has a ceiling of approximately 20 to 30 kilowatts per rack under aggressive airflow conditions. A single H100 SXM5 node draws 10.2 kilowatts; a standard 8-GPU DGX H100 system draws approximately 10.2 kilowatts under sustained inference and up to 14 kilowatts under training. Four such nodes saturate what a well-designed air-cooled rack can handle, and rack densities for hyperscaler AI clusters are already landing at 40 to 80 kilowatts in production deployments. Air is architecturally dead for this workload class.

Direct-to-chip cold-plate cooling is the current mainstream choice for GPU-dense builds. Cold plates are brazed or diffusion-bonded aluminum or copper heat exchangers that mount directly to the CPU, GPU, and memory packages, replacing the conventional heatsink. Liquid is circulated through the cold plate, picks up heat directly from the die package, and returns to a facility-side heat rejection system. The server still has fans for residual component cooling, making direct-to-chip a hybrid system. Rack densities of 40 to 120 kilowatts are achievable depending on chassis configuration, and original equipment manufacturer support from NVIDIA, AMD, Intel, and the major original design manufacturers is now mature. This is the architecture that dominates new builds in 2026.

Single-phase immersion submerges complete servers in a dielectric fluid inside an open or sealed tank. The fluid does not change phase; it flows, absorbs heat by convection, and is pumped to an external heat exchanger. Densities above 100 kilowatts per rack are achievable, and power usage effectiveness can approach 1.03, meaning nearly all energy goes to computing rather than cooling overhead. The tradeoffs are structural, with tank weights running 2,000 to 4,000 kilograms fully loaded, and fluid cost, with dielectric fluids running $3 to $12 per liter versus $0.15 to $0.40 per liter for glycol-water blends.

Two-phase immersion submerges servers in a fluid that boils at low temperature at atmospheric or near-atmospheric pressure, transferring heat through the latent heat of vaporization rather than sensible heat. Theoretical thermal performance is excellent, but the practical supply chain for two-phase fluids collapsed significantly after 3M's 2022 announcement of the planned discontinuation of its Novec family. This architecture remains in production at select hyperscaler research sites but is not a mainstream recommendation for new builds today.

The fluid inside a direct-to-chip cooling loop is almost never pure water, and it is never untreated tap water. The dominant fluid class for direct-to-chip loops is inhibited glycol-water blends, with propylene glycol preferred over ethylene glycol in most data center applications. Propylene glycol's lower acute toxicity reduces leak response burden under occupational safety standards, and it is more compatible with incidental contact with aluminum heat exchangers without accelerating pitting corrosion. The typical direct-to-chip loop specification calls for a glycol concentration of 20 to 35 percent by volume, with the balance being deionized water.

As enterprises deploy physical AI at scale, the infrastructure decisions made today, from edge hardware selection to cooling architecture, will determine whether these systems can operate reliably and cost-effectively for years to come. The shift from cloud-centric AI to distributed, continuous-duty physical AI workloads represents not just a hardware upgrade but a fundamental rethinking of how AI infrastructure is planned, built, and maintained.