OpenAI has built its first custom inference chip, codenamed "Jalapeño," that outperforms Nvidia's top-tier GB200 and GB300 systems across multiple benchmarks. The chip is designed not for training AI models, but for running them efficiently once they're deployed, handling user requests faster while consuming less power. Sam Altman, OpenAI's CEO, confirmed the achievement on social media, stating simply: "We built a chip that's really fast."

Sam Altman, OpenAI's CEO

What Makes Jalapeño Different From Nvidia's Hardware?

Most AI hardware forces engineers to choose between two competing goals: processing more requests per unit of time (throughput) or responding to individual users faster (latency). Jalapeño attempts to do both simultaneously. The chip was co-designed with Cerebras, a hardware company specializing in AI accelerators, and represents OpenAI's effort to control the entire chain of AI production, from model development through software, chips, and data centers.

In real-world testing on three public AI models, Jalapeño achieved remarkable efficiency gains. On the largest model tested, Moonshot AI's Kimi K2.5 with 1 trillion parameters, Jalapeño delivered 1.5 times more AI work per watt at peak throughput while reducing end-to-end latency by 3.4 times compared to Nvidia's GB300. For highly interactive workloads, the performance advantage expanded to 2.1 to 4.1 times better than competing systems.

How Does Jalapeño Achieve These Performance Gains?

• Optimized Architecture: Jalapeño was designed from the ground up for large language models and interactive AI agents, rather than adapted from general-purpose hardware like traditional GPUs.
• Smart Memory Management: The chip explicitly places and keeps model states, including KV Cache data, locally on the chip to minimize data movement between cores and reduce latency during inference.
• Workload Adaptation: Different stages of AI inference have different bottlenecks; Jalapeño adjusts its performance based on whether the workload is compute-heavy (prefill) or memory-bandwidth-limited (decode).
• Power Efficiency: Despite a rated power consumption of 700 watts, Jalapeño typically runs at 550 watts or below during actual workloads, delivering more useful AI work per unit of energy.

The chip was tested using InferenceX, a public benchmark from SemiAnalysis, on three models: OpenAI's GPT-OSS 120B, DeepSeek's R1 670B, and Moonshot AI's Kimi K2.5 1T. The fact that Jalapeño performed well across externally developed models, not just OpenAI's own, suggests it functions as a more general-purpose inference architecture.

Why Should You Care About a Faster Inference Chip?

For everyday users, faster inference means quicker responses from AI assistants and smoother interactions with AI-powered applications. For OpenAI and other companies running massive AI services, efficiency gains translate directly to lower operating costs and the ability to serve more users without building expensive new data centers. This is particularly important as AI companies race to deploy AI agents, which are systems that perform multiple steps in sequence; even small latency improvements in each step compound into significantly faster overall performance.

"This capability is made possible by our deep partnership with Cerebras and their unique hardware architecture. In the future, this cooperation will further push the boundaries of the 'ultra-fast' experience. I am very much looking forward to our continued cooperation to continuously push the limits on the Cerebras platform, explore how to run our most capable models at the fastest speed, and bring this experience to customers with the highest performance requirements," stated Tibo, an OpenAI executive involved in the project.

Tibo, OpenAI

OpenAI is not the first major AI company to develop custom chips. Google created TPU (Tensor Processing Units), Amazon developed Trainium and Inferentia, Microsoft advanced Maia, and Meta has deployed self-developed AI accelerators. The underlying logic is consistent across the industry: general-purpose GPUs are flexible but not optimized for specific AI workloads. Companies that design chips tailored to their own models, software systems, and service methods can reduce costs and improve performance significantly.

The name "Jalapeño," Spanish for a type of chili pepper, sparked humor and discussion online when announced, but the technical achievement behind it represents a serious strategic move by OpenAI to reduce its dependence on Nvidia and gain more control over its infrastructure costs as AI models grow larger and more complex.