Groq's Groq 3 LPX inference accelerator has achieved a major performance milestone, delivering 3,431 output tokens per second on long-context AI tasks, according to independent benchmarks released today. The chip, integrated into NVIDIA's Vera Rubin platform, demonstrates how specialized hardware designed for inference can unlock new capabilities for AI agents that need to process massive amounts of context while maintaining interactive speed.

Why Does Long-Context Speed Matter for AI Agents?

To understand why this benchmark matters, consider how AI agents actually work. When an AI agent researches a company for an investment decision, it queries financial databases, searches news and filings, invokes sub-agents to run peer comparisons, and synthesizes everything into a recommendation. Each step adds to the context, the accumulated information the agent must process in the next step.

In agentic sessions, context can grow to hundreds of thousands of tokens across an entire task, especially as sessions exceed hundreds of turns. Without long context, agents can only consider a fraction of relevant information from earlier steps. The most capable agents must deliver both speed and long-context retention as sessions grow.

According to OpenRouter data, agentic AI workloads consume 15 times more tokens than a simple chat request. This fundamental difference means that infrastructure running agentic AI needs to handle token demand efficiently at scale.

How Does Groq 3 LPX Achieve This Speed?

The Groq 3 LPX uses a deterministic execution model that sets it apart from traditional GPU-based inference systems. The compiler has complete visibility into the individual compute units, memory capacity, and networking links before the workload begins. This allows it to produce a precise schedule, down to the clock cycle, of exactly how the workload will run.

This approach eliminates the need for real-time arbitration of data transfers between chips. Instead of multiple steps for each data transfer, the compiler-produced schedule means that local processing units (LPUs) can send data on the clock cycle when it is ready and consume data on the clock cycle when it arrives. The system's point-to-point networking between LPU pairs, with each LPU acting as both processor and router, drives down first-bit latency to an absolute minimum.

Steps to Understand Groq 3 LPX's Technical Advantages

• Deterministic Compiler Scheduling: The compiler plans all data transfers before execution begins, eliminating real-time arbitration overhead that slows down traditional systems at small batch sizes.
• Fine-Grained Computation-Communication Overlap: Groq 3 LPX overlaps compute operations with data transfers between chips, keeping processors active rather than waiting for data to arrive.
• Preplanned Chip-to-Chip Networking: The system uses 96 chip-to-chip links per LPU running at 112 gigabits per second, with a preplanned data transfer schedule that minimizes latency.
• Tensor Parallelism at Small Batch Sizes: The architecture enables effective tensor parallelism even at small batch sizes critical for agentic multiturn inference with large models.
• Distributed KV-Caching: The system extends memory across multiple LPUs while using KV-cache offloading to tier less-active context to host and storage, keeping previously processed context accessible without recomputation.

What Do the Benchmarks Show?

The Groq 3 LPX achieved a world-class interactivity score of 3,431 output tokens per second on the Artificial Analysis 100K context benchmark with the Gemma 4 31B model, according to independent third-party testing. This demonstrates leading high-interactivity performance at long context lengths without loss in precision or model quality.

The system also consistently delivers high throughput across both general agentic and coding-specific tasks, with a median output of 4,767 tokens per second on SPEED-Bench. It supports multiple co-execution configurations with Vera Rubin NVL72, including prefill-decode disaggregation, attention-FFN disaggregation, and speculative external-drafter decoding, scaling to multi-trillion parameter models.

Beyond raw speed, the Groq 3 LPX delivers significant efficiency gains. NVIDIA's Vera Rubin NVL72 systems, which incorporate the Groq 3 LPX, deliver up to 30 times higher throughput per megawatt than NVIDIA's GB300 NVL72 on agentic workloads. This translates directly into 30 times more agentic work for the same energy footprint, a critical metric for power-constrained AI factories.

The token cost advantage is equally striking. Vera Rubin NVL72 delivers up to 35 times lower cost per million tokens than GB300 NVL72, enabling agents to run continuously at scale across the full breadth of customer workloads.

What Changed in Groq's Business Strategy?

Groq's focus has shifted significantly in recent months. The company ran GroqCloud on its LPU architecture starting in 2024, but after licensing its technology to NVIDIA in December 2025, the independent Groq refocused entirely on inference infrastructure. This partnership represents a major validation of Groq's specialized inference approach, even as the company transitions its business model.

The integration of Groq 3 LPX into NVIDIA's Vera Rubin platform demonstrates how specialized inference accelerators can complement GPU-based systems. While NVIDIA's Vera Rubin NVL72 handles general-purpose throughput across diverse workloads, the Groq 3 LPX extends the platform's ability to address the highest-interactivity serving tiers, expanding Vera Rubin's power to serve multiagent systems with high interactivity and long context.

For AI infrastructure builders and enterprises deploying agentic AI at scale, these benchmarks signal that specialized inference hardware designed specifically for long-context, interactive workloads can deliver order-of-magnitude improvements in both speed and efficiency compared to general-purpose alternatives.