Inference chips, which handle real-time processing of trained AI models, are becoming the critical battleground in AI hardware as the industry recognizes that inference workloads require fundamentally different optimization than training. Cerebras Systems has raised $9.2 billion in total funding, making it the second-most funded deep tech company globally after SpaceX, with its May 2026 IPO bringing in $6.4 billion. This capital surge reflects a seismic shift in how the industry thinks about AI infrastructure, moving beyond the GPU-dominated training paradigm toward specialized processors optimized for inference.

Why Is the Industry Shifting from Training to Inference Hardware?

Inference is the process where a trained AI model processes new data to generate predictions or responses. Unlike training, which requires massive parallel processing power, inference often involves sequential operations where data moves through a model one step at a time. This fundamental difference means that the GPU-heavy approach optimized for training is inefficient for inference workloads.

The market is responding to this reality with substantial capital commitments. Groq, which specializes in inference chips, has raised $1.8 billion and is actively competing in this space. These companies are betting that inference will become the dominant use case as AI models move from research labs into production systems serving millions of users. A single large language model might be trained once, but it will serve millions of inference requests, making the economics of inference hardware fundamentally different from training hardware.

How Are Hardware Architectures Changing to Support Inference?

One of the clearest signals of this shift comes from hardware architecture recommendations. The previous standard for AI systems was one CPU (central processing unit) for every eight GPUs (graphics processing units). That ratio has now shifted to one-to-one, reflecting the sequential nature of inference tasks. This change is so significant that Nvidia announced plans to acquire Groq's intellectual property for $20 billion and is developing its own ARM-based CPU called Vera to address the inference bottleneck.

SiPearl, a European chip startup, is capitalizing on this trend with its Rhea1 processor. The chip features 80 ARM Neoverse V1 cores manufactured by TSMC using a 6-nanometer process, combined with high-bandwidth memory (HBM) technology traditionally reserved for GPUs. This hybrid approach reflects the industry's recognition that inference requires both CPU-like control and GPU-like memory bandwidth. The Rhea1 delivers 1.8 terabytes per second of total HBM memory bandwidth, designed specifically to address the data movement bottleneck that constrains inference performance.

"In SiPearl we always had a strong faith in CPUs, because CPUs are everywhere. Training and hype on AI pushed for GPUs everywhere, but when you have GPUs, you need to have CPUs to orchestrate the GPUs," said Philippe Notton.

Philippe Notton, Founder and CEO at SiPearl

How to Evaluate Inference Chip Opportunities

• Memory Bandwidth: Inference performance is constrained by how quickly data moves between memory and processors. The Rhea1's 1.8 terabytes per second of HBM bandwidth is designed to address this bottleneck, making memory architecture a key differentiator among competing chips.
• Power Efficiency: Inference chips must operate at scale in data centers, making power consumption critical. Chips optimized for sequential processing typically consume less power than GPUs optimized for parallel workloads, directly impacting operational costs and environmental footprint.
• Software Ecosystem: Hardware is only valuable if it can run existing AI frameworks. The Rhea1 supports TensorFlow and PyTorch, the dominant AI frameworks, ensuring compatibility with existing models and reducing developer friction when adopting new hardware.
• Deployment Timeline: SiPearl expects to ship Rhea1 in late 2026 or early 2027, while Athena1, a variant for defense and aerospace applications, is targeted for the second half of 2027. Timing matters in a market moving this quickly.

What Does This Capital Shift Mean for the Broader AI Market?

The inference chip revolution has immediate practical implications for companies building AI systems. Organizations that previously assumed they needed massive GPU clusters for all workloads can now consider hybrid approaches, using CPUs for inference tasks and reserving GPUs for training or other parallel workloads. This can significantly reduce both capital expenditure and operational costs.

The funding data tells a clear story about where the industry believes the real value lies. Cerebras has raised more capital than any other AI chip startup except SpaceX, which operates in an entirely different market. The company's IPO in May 2026 brought in $6.4 billion, suggesting institutional confidence in the inference chip thesis. This investment reflects a fundamental economic reality: inference workloads will vastly outnumber training workloads in production AI systems.

The European push for sovereign AI infrastructure also plays a significant role in this competition. SiPearl was founded in 2020 with explicit backing from the European Union to develop competitive AI chips entirely within Europe. The company closed a 130 million euro Series A round from European investors including the European Investment Bank and France 2030. This geopolitical dimension adds another layer to the inference chip competition, as countries seek to reduce dependence on US-dominated semiconductor supply chains.

The inference chip market is still in its early stages, but the capital flowing into companies like Cerebras and Groq, combined with major semiconductor players developing their own inference solutions, signals that this is where the next phase of AI infrastructure competition will unfold. Unlike the training chip market, which remains dominated by Nvidia, the inference space is genuinely competitive, with multiple viable approaches and significant room for specialized solutions tailored to specific use cases.