OpenAI has announced its first custom-designed inference chip, codenamed Jalapeño, which significantly outperforms Nvidia's leading GPU systems in speed and energy efficiency. CEO Sam Altman confirmed the breakthrough on social media, stating "we made a chip and it is fast." The chip represents a major step in OpenAI's effort to control its entire AI infrastructure stack, from models and software to hardware and data centers.

Sam Altman

What Makes Jalapeño Different From Existing AI Hardware?

Unlike training chips that help AI models learn, Jalapeño is specifically optimized for inference, the phase where AI models respond to user requests after training is complete. This distinction matters because inference requires different performance characteristics than training. The chip was designed to solve a fundamental engineering trade-off that has long plagued data center operators: systems typically must choose between processing many requests quickly or responding to individual requests with minimal delay.

Jalapeño breaks this constraint by achieving both high throughput and low latency simultaneously. In benchmark tests across three large language models, the chip delivered impressive gains. On OpenAI's GPT-OSS 120B model, DeepSeek's R1 670B model, and Moonshot AI's Kimi K2.5 1T model, Jalapeño achieved 1.5 to 1.9 times more AI work per watt compared to Nvidia's GB200 and GB300 systems, while reducing response times by 1.7 to 3.6 times.

For highly interactive workloads, where users expect near-instant responses, the performance advantage expanded dramatically to 2.1 to 4.1 times better than comparison systems. On the largest tested model, Kimi K2.5 1T, Jalapeño achieved approximately 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency.

How Did OpenAI Develop This Chip So Quickly?

OpenAI leveraged artificial intelligence itself to accelerate chip development. The company used AI-assisted design to reach the manufacturing stage in just nine months, a remarkably fast timeline for custom silicon. AI also optimized portions of the chip's arithmetic circuits, the computational core of the processor. Additionally, OpenAI used Codex, an AI code generation tool, combined with GPT-Astra to optimize three open-weight models within two months. The AI-generated implementations for attention mechanisms and mixture-of-experts blocks were 1.5 to 1.8 times faster than human-written versions.

The chip was developed in partnership with Cerebras, a hardware company specializing in AI accelerators. This collaboration leveraged Cerebras's unique hardware architecture to achieve the performance gains. The partnership demonstrates how specialized hardware companies and AI labs can combine expertise to create systems neither could build alone.

Key Performance Metrics and Real-World Implications

• Power Efficiency: Jalapeño is rated for 700 watts but consumed only 550 watts or less during testing, delivering exceptional performance within a tight power budget that data centers can easily accommodate.
• Latency Reduction: On DeepSeek R1 670B, Jalapeño achieved throughput of 12,258 mixed tokens per second per kilowatt, approximately 104.3 times higher than Nvidia's GB300 at comparable latency levels, critical for interactive AI agents.
• Model Compatibility: The chip was tested on both OpenAI's own models and external models from competitors like DeepSeek and Moonshot AI, proving Jalapeño works effectively across different AI architectures, not just OpenAI's proprietary systems.

The focus on latency matters especially for AI agents, which are AI systems that complete multi-step tasks autonomously. When an agent must perform 10 sequential steps, even small delays in each step accumulate into noticeable slowdowns. Jalapeño's architecture explicitly addresses this problem by keeping model state, including the KV cache (a data structure that stores information between processing steps), physically close to the computing resources handling inference, minimizing data movement delays.

Why Is OpenAI Building Its Own Chips?

OpenAI joins a growing list of major technology companies developing custom silicon. Google created TPUs (Tensor Processing Units), Amazon developed Trainium and Inferentia chips, Microsoft advanced Maia, and Meta deployed self-developed AI accelerators. The underlying logic is consistent across all these companies: general-purpose graphics processing units (GPUs) offer flexibility but are not optimized for specific AI workloads. For companies running massive numbers of AI requests daily, custom chips designed for their particular models, software systems, and service methods can dramatically reduce costs and improve performance.

"This capability is made possible by our deep partnership with Cerebras and their unique hardware architecture. In the future, this cooperation will further push the boundaries of the 'ultra-fast' experience," stated Tibo, an OpenAI representative.

Tibo, OpenAI

OpenAI emphasized that Jalapeño represents just the beginning of its chip strategy. The company plans to begin deploying Jalapeño in its compute infrastructure by the end of 2026, with a second generation already under development and a third generation taking shape. Importantly, OpenAI stated it will continue using accelerators from Nvidia and other partners for both training and inference alongside its own chips, indicating a hybrid approach rather than complete replacement of existing hardware.

What Does This Mean for the AI Industry?

Jalapeño signals that OpenAI is attempting to control the entire chain of AI infrastructure, from foundational models to software to custom chips to data center operations. This vertical integration strategy mirrors approaches used by other technology giants and reflects the competitive importance of inference efficiency. As AI models grow larger and more capable, the cost of running them at scale becomes increasingly important to profitability and market competitiveness.

The chip's performance on external models like DeepSeek R1 and Kimi K2.5 suggests that OpenAI may eventually offer Jalapeño-based inference services to other companies, creating a new revenue stream. However, OpenAI has not announced such plans. For now, the chip represents a significant technical achievement that strengthens OpenAI's operational independence and positions the company to serve users with faster, more efficient AI responses as models continue to grow in size and capability.