Perplexity AI has demonstrated that NVIDIA's newest Blackwell generation chips deliver dramatic speed improvements for running massive AI models in production, not just during training. The company published technical research showing that its deployment of a 235-billion-parameter Qwen3 mixture-of-experts model on NVIDIA's GB200 NVL72 racks reduced response latency by 46% compared to the previous Hopper generation hardware, while also lowering operational costs.

What Exactly Did Perplexity Build?

Perplexity's setup uses GB200 NVL72 racks, which pack 72 graphics processing units (GPUs) into a single system. Each GPU comes equipped with 180 gigabytes of high-bandwidth memory, and they're all connected via a technology called NVLink that delivers 1,800 gigabytes per second of bandwidth between chips. To put that in perspective, this interconnect speed is critical because it allows the GPUs to coordinate instantly when processing enormous models that don't fit on a single chip.

The performance gains are striking. When Perplexity measured the time it takes to perform NVLink all-reduce operations, a fundamental operation in distributed computing, the latency dropped from 586.1 microseconds on the older H200 Hopper chip to 313.3 microseconds on the GB200. That's a 46% reduction in response time. For the prefill combine operation, which processes the initial prompt before generating responses, the improvement was roughly 40%, falling from 730.1 microseconds to 438.5 microseconds. In some configurations, Perplexity achieved up to 30 times faster inference compared to the older H100 baseline.

How Does Perplexity Squeeze Extra Performance From Blackwell?

• Blackwell-native quantization: This technique reduces the precision of model weights, essentially using fewer bits to represent numbers, which speeds up computation without noticeably degrading the quality of AI responses.
• Prefill and decode disaggregation: Perplexity separates the initial processing of a user's prompt from the token-by-token generation phase, allowing each stage to be optimized independently for speed.
• Custom kernels: The team wrote specialized code tuned specifically for running a 235-billion-parameter model on this particular hardware configuration, extracting every ounce of performance from the Blackwell architecture.

The combination of hardware improvements and software optimization means the GB200 NVL72 setup significantly lowers the cost of serving massive AI models while improving response quality compared to Hopper-based systems.

Why Should You Care About This Inference Breakthrough?

Most conversations about NVIDIA's latest chips focus on training, the process of teaching AI models on massive datasets. But inference, the act of running a trained model to answer user questions or complete tasks, is where the real economic value lies. Companies like Perplexity, OpenAI, and Google serve billions of inference requests daily, and even small improvements in speed and cost directly impact their bottom line and user experience.

The Blackwell results matter because they show NVIDIA's newest hardware excels at inference on massive models, not just training. The 72-GPU NVLink topology delivering 1,800 gigabytes per second of bandwidth is particularly significant because competing solutions from AMD and Amazon Web Services often rely on slower interconnects between chips, which creates bottlenecks when serving models that need to coordinate across many GPUs simultaneously. This architectural advantage reinforces NVIDIA's dominance in the AI infrastructure race at a moment when competitors are aggressively pursuing alternatives.

For AI companies operating at scale, these performance gains translate directly into faster responses for end users and lower electricity and hardware costs per query. As AI models grow larger and more capable, the ability to serve them efficiently becomes a competitive moat. Perplexity's research demonstrates that Blackwell isn't just a marginal upgrade; it's a meaningful leap forward for the inference workloads that power the AI products millions of people use every day.