NVIDIA announced that its Groq 3 LPX inference accelerator has entered mass production, marking the commercialization of technology acquired through a $17 billion deal signed in December 2025. The specialized hardware, which splits AI inference work between GPUs and custom chips, can generate approximately 3,400 tokens per second on dense language models. However, the eye-catching speed figure obscures important details about power consumption, cost, and whether the performance gains translate to faster real-world AI agent workflows.

What Is an Inference Chip, and Why Does NVIDIA Need One?

Inference chips are specialized processors designed to run trained AI models efficiently, rather than training them from scratch. Unlike general-purpose graphics processing units (GPUs), which handle both training and inference, inference chips optimize for speed and latency in the "decode" phase, where a model generates one token (roughly a word fragment) at a time in response to a user's request.

NVIDIA's approach differs from simply replacing GPUs with custom chips. Instead, the Vera Rubin platform splits inference work between two processor types: Rubin GPUs handle the "prefill" phase, which processes long input documents all at once, while Groq 3 LPU (Logical Processing Unit) chips handle the decode phase, where speed matters most for user experience. This division of labor is designed to reduce latency, the delay users perceive when waiting for an AI assistant to respond.

How Did NVIDIA Move from Acquisition to Mass Production in Eight Months?

The timeline reveals the urgency behind NVIDIA's investment. Groq and NVIDIA signed a non-exclusive license agreement on December 24, 2025. In March 2026, NVIDIA revealed the Groq 3 LPU and LPX designs at its GTC conference, announcing that mass production of all seven chips in the Vera Rubin platform had already begun. By August 2026, the company announced full-scale mass production and secured Nebius, a cloud infrastructure provider, as the first customer to deploy the systems.

The financial structure shows what NVIDIA prioritized. Of the $17 billion total consideration, $14.4 billion was allocated to goodwill, primarily attributed to talent acquisition. NVIDIA did not acquire Groq's existing products, customer contracts, or equity stakes, suggesting the deal was primarily about acquiring the engineering team and intellectual property behind the LPU architecture.

What Does the 3,400-Token Speed Actually Measure?

The benchmark that produced the 3,400-token-per-second figure came from Artificial Analysis, which ran Gemma 4, a 31-billion-parameter open-source model, on an NVIDIA-hosted Groq 3 LPX rack. The test used a 100,000-token input, simulating scenarios like coding agents reading hundreds of files or research assistants carrying forward long conversation histories. When the same test ran with a 10,000-token input, the output speed barely changed, reaching 3,382 tokens per second.

NVIDIA framed this as roughly 4x faster than the closest public endpoint, which ran at 870 tokens per second. However, this comparison involves a critical caveat: the competing system may have used only one or two chips, while the LPX side likely used approximately 64 chips or more. The perceived speed as a cloud service is comparable, but the silicon area, power, and infrastructure costs required are not equivalent.

Output token speed also does not equal task completion time. Real AI agent workflows involve reading files, running external tools, verifying results, and making multiple inference calls in sequence. Network latency, storage access, and tool-side processing are not captured in the 3,431-tokens-per-second figure. While faster token generation does add up across multiple calls, NVIDIA's claim that "hours of work become minutes" has not been verified through end-to-end task benchmarks.

How Does the Groq 3 LPX Hardware Actually Work?

A single Groq 3 LPX rack contains 32 liquid-cooled trays, each carrying 8 LP30 chips, for a total of 256 LPUs. These chips deliver 315 petaFLOPS of FP8 compute (a measure of floating-point calculations per second), 128 gigabytes of on-chip static random-access memory (SRAM), and 40 petabytes per second of SRAM bandwidth. The scale-up bandwidth linking chips across the rack reaches 640 terabytes per second.

The key architectural insight is that bandwidth matters more than capacity. Each individual LPU has only 500 megabytes of SRAM, far too little to hold a 31-billion-parameter model. Instead, the LPX splits large models across many chips and relies on extremely fast inter-chip communication to keep compute units fed with data. To run 256 chips with low latency, the LPU minimizes dynamic runtime decisions. A compiler pre-schedules the timing of compute, memory access, and communication in units of 320-byte vectors, with data timed to arrive on planned clock cycles.

Steps to Understanding Inference Hardware Trade-Offs

• Latency vs. Throughput: Inference chips optimize for low latency (fast response time) rather than high throughput (total requests processed). A 3,400-token-per-second figure measures single-user speed, not how many concurrent users a system can serve.
• Specialization vs. Flexibility: Custom inference chips excel at specific workloads like dense language models but may not adapt well to emerging model architectures or mixed workloads that combine training and inference.
• Cost per Unit of Work: Raw speed alone does not determine whether a system is economical. Price, power consumption, and the number of chips required to achieve that speed all factor into the true cost per token generated.
• Real-World Workflow Complexity: Benchmark speeds measure only the model inference phase, not the full time required for agents to read inputs, call external tools, and verify results.

What Remains Unknown About the Groq 3 LPX?

NVIDIA has not disclosed several critical details that would help cloud providers and enterprises evaluate the system. The full configuration used in the benchmark, including precision level, number of chips allocated, and how many concurrent users were served, remains undisclosed. Price, rack power consumption, and P95 or P99 latency (the speed at which 95% or 99% of requests complete) are also unknown.

NVIDIA's claim of "up to 35x throughput per megawatt" is a projection based on placing a 2-trillion-parameter mixture-of-experts model on the system, not a result measured in the Gemma 4 31B test. Mixture-of-experts models select different subsets of parameters for each token, which could change how the LPX performs compared to dense models.

Nebius, the first cloud provider to adopt the LPX, has not announced a launch date or pricing. The company stated it will make the LPX accessible through its existing Token Factory API, initially supporting a subset of models. A mass-production announcement is evidence that the hardware has advanced through manufacturing, but it does not yet mean developers have a cloud product they can choose today.

How Does This Fit Into NVIDIA's Broader AI Strategy?

NVIDIA CEO Jensen Huang stated in March that the company expects cumulative sales from its Blackwell and Vera Rubin platforms to reach $1 trillion through 2027. He also noted that a quarter of the data center capacity allocated to coding applications would use Groq chips, suggesting that inference acceleration is becoming a core part of NVIDIA's data center strategy.

The Groq 3 LPX is designed to complement rather than replace GPUs. While graphics processors can perform both AI model training and inference, Groq chips primarily target the latency-sensitive decode phase of running models. This complementary approach allows NVIDIA to serve different parts of the inference workload with specialized hardware.

Competition in specialized inference hardware has intensified. Advanced Micro Devices (AMD) has announced plans to integrate rack-scale systems with chips produced by Cerebras, another inference chip startup. The market for inference acceleration is becoming crowded, with multiple vendors pursuing different architectural approaches.

The eight-month sprint from acquisition to mass production demonstrates NVIDIA's confidence in the Groq technology and its ability to integrate acquired teams quickly. However, the real test will come when cloud providers deploy the systems at scale and customers can measure whether the speed gains justify the cost and complexity of managing a hybrid GPU-LPU infrastructure.