NVIDIA is shifting its inference strategy from one-size-fits-all GPUs to a specialized processor approach, pairing its Groq 3 LPX decode accelerator with d-Matrix's Raptor XPUs in a coordinated rack architecture designed to serve different stages of AI workloads. This move signals that the future of AI performance depends less on raw computing power and more on matching the right processor to the right task, whether that's generating tokens at lightning speed or handling massive language models with long context windows.

Why Is NVIDIA Suddenly Embracing Specialized Inference Hardware?

For years, NVIDIA dominated AI by offering powerful general-purpose graphics processing units (GPUs) that could handle nearly any workload. But as AI applications have evolved, particularly with the rise of agentic AI systems that need to reason, plan, and respond in real time, a single processor type has become a bottleneck. Different stages of AI inference have fundamentally different requirements. The prefill stage, which processes input context, demands raw compute power. The decode stage, which generates output tokens one at a time, is latency-sensitive and requires fast memory access rather than massive parallelism.

NVIDIA's $20 billion acquisition of Groq in late 2025 gave the company direct access to specialized decode technology. Now, the company is moving that technology into production while simultaneously integrating d-Matrix's decode accelerators into its ecosystem. This is not a sign of weakness; it is a strategic acknowledgment that heterogeneous computing, where different processors handle different tasks, will define the next generation of AI infrastructure.

What Do These New Specialized Processors Actually Do?

The Groq 3 LPX, which entered full production in September 2026, is purpose-built for one job: generating output tokens as fast as possible. The rack contains 256 specialized processing units with 128 gigabytes of aggregate static random-access memory (SRAM) and achieves approximately 11,000 tokens per second on compact models. In real-world testing, the system generated 3,431 output tokens per second on Gemma 4 31B, a 31-billion-parameter model, at 100,000 token context, delivering what NVIDIA claims is 4 times faster responsiveness than competing platforms.

The d-Matrix Raptor XPU takes a different approach. Rather than relying on SRAM, Raptor stacks a dynamic random-access memory (DRAM) die with a compute die into a single package, enabling much larger model support. A full Raptor rack contains 144 XPUs with 2.3 terabytes of 3D-DRAM capacity and 7.2 petabytes per second of memory bandwidth, projected to deliver approximately 1,000 tokens per second per user on 3-trillion-parameter models at 1-million-token context. The key difference: Groq 3 LPX excels at small, fast models; Raptor handles frontier-scale models that require massive memory capacity.

This segmentation is intentional. NVIDIA has created a decode portfolio where the choice of processor depends on the model size and latency requirements of the workload. For interactive applications requiring sub-100-millisecond response times on smaller models, Groq 3 LPX is the answer. For long-context reasoning tasks on massive models, Raptor provides the memory bandwidth and capacity that SRAM-based designs cannot match.

How Are These Processors Integrated Into NVIDIA's Ecosystem?

The integration happens through NVIDIA's MGX rack reference architecture and NVLink Fusion, a framework that allows specialized processors to connect directly to NVIDIA's infrastructure without relying on slower peripheral component interconnect (PCIe) connections. The Groq 3 LPX works alongside Vera Rubin NVL72 GPUs, with GPUs handling prefill and context processing while Groq handles decode. The d-Matrix Raptor XPUs integrate into the same MGX racks through NVLink Fusion, alongside Vera CPUs, BlueField-4 data processing units (DPUs), ConnectX-9 SuperNICs, and Spectrum-X networking.

The architectural pattern has already been validated in production. Gimlet Labs integrated d-Matrix's earlier Corsair platform alongside GPUs in March 2026 and measured a 2 to 5 times end-to-end speedup on interactive configurations, rising to 10 times on energy-optimized setups. Parasail deployed Corsair alongside NVIDIA Hopper and Blackwell GPUs in July 2026 and claimed up to 10 times faster interactive inference with up to 3 times better energy efficiency. These real-world deployments proved that heterogeneous inference works before NVIDIA committed to integrating d-Matrix at rack scale.

Steps to Understanding NVIDIA's Inference Strategy

• Recognize the workload split: Modern AI inference divides into prefill (processing input context) and decode (generating output tokens), with each stage having different performance characteristics and hardware requirements.
• Understand the processor trade-offs: SRAM-based designs like Groq 3 LPX deliver extreme speed on small models but cannot scale to frontier-size models due to memory constraints; DRAM-based designs like Raptor support massive models but at lower token-generation speeds.
• See the ecosystem lock-in: By integrating specialized processors through NVLink Fusion into MGX racks, NVIDIA makes it easier for customers to adopt multiple processor types while remaining within the NVIDIA infrastructure ecosystem.
• Track the timeline: Groq 3 LPX is in production now; Raptor is expected to tape out before the end of 2026, with initial MGX availability in Q4 2027, meaning the full heterogeneous stack will not be widely available for over a year.

What Does This Mean for Cloud Providers and AI Labs?

The shift to specialized inference creates both opportunity and complexity. Cloud providers can now offer differentiated services: ultra-low-latency token generation for interactive applications on Groq 3 LPX, and frontier-scale reasoning on Raptor. However, this requires sophisticated workload scheduling and data movement orchestration to avoid eroding the latency gains that specialization provides.

The commercial viability depends on cloud economics. NVIDIA argues that providers can charge premium prices for faster tokens, yet the Groq 3 LPX rack can be quoted as high as $1 million, a significant capital investment that must be justified through higher service pricing or dramatically improved agent completion times. Nebius, a cloud provider, plans to become the first production adopter of Groq 3 LPX in late 2026, offering it through an API that developers already use. This deployment will be the first real test of whether faster token generation translates into customer demand and profitable cloud services.

For AI labs and hyperscalers, the message is clear: the era of one processor for all inference workloads is ending. The companies that master heterogeneous compute, coordinating GPUs, specialized decode accelerators, and large-model inference processors across a single rack, will have a significant competitive advantage in serving agentic AI applications that demand both speed and reasoning capability.

When Will These Systems Actually Be Available?

Groq 3 LPX is in full production now, with Nebius planning deployment in late 2026. However, d-Matrix Raptor has not yet taped out, meaning the silicon has not been manufactured. Initial MGX availability with Raptor is expected in Q4 2027, more than a year away. This timeline gap is significant: Groq 3 LPX enters the market while Raptor remains in development, giving NVIDIA a near-term solution for small-model decode while the long-context frontier-scale solution matures.

The distance between announcement and revenue is equally visible in the competitive landscape. Cerebras projects up to 5,000 tokens per second on future wafer-scale systems, AMD acquired Taalas for hardcoded inference silicon, and OpenAI's Jalapeño serves internal demand. Yet none of these competitors plug into the MGX supply chain that hyperscaler procurement has already qualified, giving NVIDIA a structural advantage in ecosystem adoption even as specialized inference becomes a crowded market.

"Being integrated into NVIDIA's latest MGX rack-scale infrastructure with NVLink Fusion means our customers can deploy our inference XPUs alongside the broadly available NVIDIA AI factory platform," said Sid Sheth, Founder and Chief Executive Officer of d-Matrix.

Sid Sheth, Founder and Chief Executive Officer of d-Matrix

The inference market is undergoing a fundamental transformation. Futurum's analysis projects that agent- and reasoning-first inference silicon will grow from $35.9 billion in 2025 to $546 billion by 2030, passing pretraining as the largest workload segment in 2027. Within that market, the XPU sub-market that d-Matrix competes in is expected to expand from $37.4 billion to $237.2 billion over the same window. NVIDIA's strategy of assembling a portfolio of specialized processors, integrated through a common rack architecture and software ecosystem, positions the company to capture a significant share of this explosive growth.