Google is reportedly working with AMD to design a next-generation AI accelerator that combines tensor processing with general-purpose CPU cores, marking a fundamental shift in how AI infrastructure handles reasoning workloads. The partnership suggests that as AI models become more focused on complex reasoning tasks, the traditional architecture of AI chips is no longer sufficient.

Why Are AI Labs Suddenly Prioritizing CPU Power Alongside Accelerators?

For years, AI chip design has centered almost entirely on maximizing tensor processing power, the specialized compute needed to train and run large language models (LLMs), which are AI systems trained on vast amounts of text data. But reasoning models, which spend extra compute time working through multi-step problems, require something different: they need general-purpose CPU cores running alongside the accelerators.

According to reports, Google and its customers are pushing for TPUs, or Tensor Processing Units (Google's custom AI accelerators), that include on-package CPU cores specifically for reinforcement learning and other CPU-intensive workloads. Reinforcement learning is a training technique where AI systems learn by trial and error, receiving rewards for correct actions. This process demands more traditional computing power than standard model inference.

Google has already begun increasing CPU resources in its latest systems. Its TPU 8i inference systems, designed for reasoning and reinforcement learning tasks, feature one Google Axion CPU for every two TPUs. By contrast, earlier systems running Google's 7th Generation TPUs used one Intel Xeon processor for every four TPUs, showing a clear trend toward more CPU-heavy architectures.

What Does This Partnership Mean for AI Infrastructure?

AMD's involvement marks a significant moment in AI chip design. The company brings expertise that Google lacks in-house: CPU architecture, advanced packaging techniques, and experience building hybrid chips that combine different types of processors. AMD already produces the Instinct MI300A, a data center accelerator that integrates both x86 CPU cores and AI accelerator chiplets on a single package.

The collaboration signals that Google is moving beyond incremental improvements to its TPU line. Rather than simply adding more tensor cores, the company is exploring an entirely new category of chip optimized for reasoning workloads. This represents a fundamental rethinking of AI infrastructure as the industry shifts from pure training and inference toward more complex reasoning tasks.

How Is the Economics of AI Inference Changing?

The push for CPU-heavy architectures reflects a broader economic shift in AI infrastructure. While per-token inference costs have fallen roughly 280-fold between late 2022 and late 2024, reasoning models consume tens to hundreds of times more tokens per answer than traditional models. This means the total cost per query can actually be higher, despite cheaper unit pricing.

Data centers are being rebuilt around this new reality. The capital expenditure line item growing fastest is gradually shifting from training to inference, driven by test-time compute, a technique where models spend extra processing time reasoning through problems to improve answer quality.

Steps to Evaluate Test-Time Compute Costs for Your Organization

• Classify Your Tasks by Difficulty: Low-difficulty, high-tolerance-for-error tasks like content classification typically don't need much test-time compute, while high-difficulty tasks like code debugging require significantly more reasoning compute and carry wider cost swings.
• Run Small-Scale Pilots: Rather than relying on vendor per-token pricing alone, test the actual task types you plan to use and observe how much compute the model consumes on average to reach acceptable quality levels.
• Plan for Tiered Pricing: Expect AI products to increasingly offer finer-grained pricing tiers or optional "deep reasoning" modes that users can activate when they need higher quality answers, accepting longer response times and higher costs.

This economic shift has practical implications for how AI products are designed. Several platforms have recently made "thinking" or "deep research" separate modes that users must actively opt into, rather than applying reasoning compute uniformly to every query. This design choice reflects the underlying cost structure of test-time compute, letting users decide whether the extra compute investment is worth the improved answer quality.

The Google-AMD partnership, if confirmed, would represent the hardware industry's response to this shift. By integrating CPU cores directly into the accelerator package, the companies could reduce latency between general-purpose and tensor compute, improving performance and reducing power consumption for reasoning workloads. This tight integration could become the standard architecture for the next generation of AI infrastructure.