OpenAI has unveiled benchmark results for its first custom chip, the Jalapeño, which outperforms NVIDIA's most advanced inference processor on multiple metrics that directly affect how quickly AI models respond to users. The chip, co-developed with Broadcom, achieved 1.5 to 1.9 times higher throughput per kilowatt than NVIDIA's Blackwell system across three large language models, and delivered response speeds up to 4.9 times faster in low-latency scenarios.

On August 25 at the Hot Chips conference, OpenAI's Head of Hardware Richard Ho presented the first public benchmark results using SemiAnalysis's InferenceX tool. The tests compared Jalapeño against NVIDIA's GB300 processor on three publicly available models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. The results revealed a significant efficiency advantage; when running DeepSeek R1, Jalapeño achieved a maximum single-user generation speed of approximately 700 tokens per second, compared to NVIDIA's 169 tokens per second.

What Makes This Chip Different From Traditional Hardware?

The Jalapeño represents a fundamental shift in how AI hardware gets designed. For the past two decades, the industry followed a standard playbook: AI labs adapted their models to whatever chips NVIDIA released. OpenAI reversed that equation by building a chip specifically optimized for its own models and traffic patterns. This approach, called "model-centric chip design," prioritizes the unique demands of modern AI inference rather than generic computing tasks.

The chip's architecture reflects a single guiding principle: minimize data movement. In AI inference, the bottleneck is not computing power but rather the constant shuffling of massive model weights and context cache between memory and processors. Jalapeño addresses this by slicing computing cores and HBM4 memory into paired units, giving each processor direct, low-latency access to its own memory slice. This design choice eliminates the need for repeated remote data calls, keeping frequently accessed information local and ready for processing.

Another notable architectural decision was to handle both prefill (processing user input) and decode (generating tokens one by one) on the same accelerator, rather than splitting these tasks across dedicated chips as NVIDIA does. OpenAI's reasoning was practical: real-world traffic composition constantly shifts as users move from chat to inference models to AI agents. A fixed chip configuration optimized for today's traffic would leave half the hardware idle when traffic patterns change tomorrow.

How Did OpenAI Compress a Two-Year Process Into Nine Months?

The speed of Jalapeño's development challenges industry norms. Conventional high-performance chips typically require 18 to 24 months from architecture definition through tapeout, the final step before manufacturing. Google's TPU and Amazon's first-generation Trainium both took approximately two years. OpenAI compressed this cycle to just nine months on its first chip attempt, cutting the typical timeline by more than half.

The acceleration was powered by AI itself. OpenAI used its own AI models, including Codex and GPT-Astra, to generate optimized code kernels for attention and mixture-of-experts (MoE) modules. These AI-generated kernels ran 1.5 to 1.8 times faster than versions written by human experts. The same AI tools completed porting and optimization of DeepSeek R1 and Kimi K2.5 in approximately two months, a process that would typically require significantly longer.

"Jalapeño can process more AI tasks per unit of power consumption and return responses faster at the same time. With a single architecture, Jalapeño achieves higher throughput and lower latency, while existing hardware systems usually need to make trade-offs between the two," OpenAI stated in its official blog.

OpenAI, Official Blog

This acceleration is not a one-time sprint. The benchmark results released in August were based on the A0 version of the engineering chip. The improved B0 version has already entered manufacturing at TSMC's N3P process. OpenAI is simultaneously developing a second-generation chip and planning a third generation.

What Are the Real-World Performance Gains?

The benchmark improvements translate directly to user experience. When processing GPT-OSS 120B, Jalapeño achieved approximately 85,400 tokens per second per kilowatt, completing nearly twice as many tasks with the same power consumption compared to competing systems. In low-latency scenarios, the advantages become even more dramatic.

The chip specifications reflect its optimization for modern AI workloads. Jalapeño operates at a rated power consumption of 700 watts, includes HBM4 memory with a single-package memory bandwidth of approximately 15.4 terabytes per second, and features an independent input-output chip for rack-level communication. The B0 version delivers theoretical computing power of 13.4 petaflops using MXFP4 precision.

Steps to Understanding the Shift in AI Hardware Strategy

• Model-Centric Design: Instead of adapting models to existing chips, OpenAI designed Jalapeño around the specific computational patterns of its own AI models, prioritizing efficiency for real inference workloads rather than peak theoretical performance.
• Data Locality Architecture: The chip pairs computing cores with dedicated memory slices to minimize data movement, the primary bottleneck in AI inference, by keeping frequently accessed information local and immediately available.
• Flexible Task Handling: Jalapeño handles both input processing and token generation on the same accelerator, allowing the hardware to adapt to changing traffic patterns without leaving processors idle.
• AI-Accelerated Design: OpenAI used its own AI models to generate optimized code and complete hardware-software integration in months rather than years, compressing the traditional chip development cycle by more than half.

The implications extend beyond OpenAI's immediate infrastructure needs. When a company can iterate on chip designs every nine months instead of every 24 months, it can absorb the latest insights from model architecture improvements into each hardware generation. This acceleration path is now theoretically available to any organization with both cutting-edge AI models and chip design expertise, though few companies possess both capabilities at OpenAI's scale.

The Jalapeño deployment timeline reflects the experimental nature of this first iteration. OpenAI plans to deploy the chip at very small scale by the end of 2026, with mass production beginning in 2027. This phased approach allows the company to validate real-world performance before committing to large-scale manufacturing.