Zyphra, a San Francisco-based AI research company, has demonstrated that smaller models can achieve competitive performance on advanced reasoning benchmarks by allocating extra computing power during inference, the moment when a model generates answers. Using its proprietary test-time compute method called Markovian RSA, the company boosted its ZAYA1-8B model to 91.9% accuracy on the AIME'25 benchmark and 89.6% on the HMMT'25 benchmark, making it competitive with substantially larger reasoning models.

What Is Test-Time Compute and Why Does It Matter?

Test-time compute refers to the strategy of spending additional computational resources during inference, when a model is actually answering questions or solving problems, rather than only during the training phase when the model learns from data. This approach flips conventional AI wisdom, which has historically focused on making models larger and more capable during training. By allowing models to "think longer" at test time, researchers can achieve better reasoning without necessarily building bigger models. Zyphra's success with this method suggests that the economics of AI development may be shifting away from pure model scale toward smarter inference strategies.

The practical implication is significant: organizations may be able to deploy smaller, more efficient models that still deliver high-quality reasoning by allocating compute resources where they're needed most, during actual use. This could reduce training costs and make advanced AI more accessible to companies that lack the resources to train massive models from scratch.

How Did Zyphra Build ZAYA1-8B at Scale?

Zyphra trained its reasoning model using a full-stack AMD computing platform, demonstrating that alternatives to NVIDIA's dominant GPU ecosystem can support large-scale AI development. The company pretrained ZAYA1-8B across 1,536 AMD Instinct MI300X GPUs organized into 192 nodes, each containing eight GPUs. This represents a significant validation of AMD's infrastructure for cutting-edge AI work.

ZAYA1-8B is built on Zyphra's MoE++ design, a variant of mixture-of-experts architecture that improves efficiency and scalability. The model features 0.7 billion active parameters and 8 billion total parameters, meaning it only activates a portion of its weights for each task, reducing computational overhead. The architecture includes two key innovations:

• Compressed Convolutional Attention (CCA): Reduces the computational cost of the attention mechanism, which is typically one of the most expensive components of transformer models.
• ZAYA1 Router: Manages which experts the model uses for each task, improving expert balancing and overall expressivity.
• Long Context Support: The model can process up to 16,000 tokens of context without requiring context parallelism, enabling efficient handling of longer documents and conversations.

The 192GB of high-bandwidth memory on each MI300X GPU proved crucial for this work. This memory capacity allowed Zyphra to fit larger models and longer context windows on individual accelerators without distributing them across multiple nodes, which would have introduced communication overhead and complexity.

How to Optimize AI Models for Efficient Inference at Scale

• Align Architecture to Hardware: Zyphra shaped ZAYA1-8B by aligning head dimensions, hidden dimensions, and attention configurations with the specific capabilities of AMD's GPU architecture, ensuring that the model's design matched the underlying hardware for maximum efficiency.
• Leverage High-Memory Accelerators: Using GPUs with 192GB of memory reduced the need for complex parallelism strategies, allowing the team to train larger models and longer context windows without splitting work across multiple nodes.
• Implement Efficient Attention Mechanisms: Compressed Convolutional Attention reduced both prefill FLOPs (floating-point operations) and KV-cache size, enabling faster inference and lower memory requirements during deployment.
• Invest in Stable Infrastructure: Zyphra achieved uninterrupted 48-hour training runs and even full weekend runs of approximately 72 hours by ensuring robust networking, driver settings, and hardware reliability across the entire cluster.

What Challenges Did Zyphra Face During Training?

Training a model across 1,536 GPUs introduces complexity that most AI labs rarely encounter. Zyphra had to ensure that every component of its software stack, from the deep learning framework to low-level GPU kernels, worked reliably at full scale. The company converted its training code from CUDA, NVIDIA's proprietary programming language, to HIP (Heterogeneous-compute Interface for Portability), AMD's open-source alternative. This conversion required careful optimization of custom kernels to match AMD's hardware characteristics.

"We trained about 20 trillion tokens on our AMD Instinct MI300X GPU cluster and achieved many uninterrupted 48-hour runs," explained Beren Millidge, co-founder and chief scientist at Zyphra. "Even saw a full weekend, which was about 72 hours. Twenty-four-hour runs now occur regularly, which is excellent."

Beren Millidge, Co-founder and Chief Scientist at Zyphra

One particularly instructive challenge involved intermittent data corruption on a single node. Rather than replacing the hardware immediately, Zyphra and AMD worked together to isolate and resolve the fault, demonstrating the value of having access to open-source software libraries that allow engineers to inspect code directly and troubleshoot at a deep level.

Why Does This Matter for the Future of AI Development?

Zyphra's achievement signals a broader shift in how AI labs approach model development. By demonstrating that test-time compute can deliver competitive reasoning performance on a smaller model, the company challenges the assumption that bigger is always better. This approach could democratize advanced AI capabilities, allowing smaller organizations and researchers to build competitive systems without access to the largest GPU clusters.

The use of AMD's full-stack platform also matters. For years, NVIDIA's GPUs have dominated AI training, creating a bottleneck for companies seeking alternatives. Zyphra's successful large-scale training on AMD infrastructure proves that the ecosystem is maturing and that organizations have viable choices for building frontier AI systems.

"AMD works for training at scale. We have good models coming out of it," stated Millidge. "The larger memory on AMD Instinct MI300X GPUs lets you fit larger models inside a single accelerator or node without spilling across nodes, which reduces the potential for communication problems."

Beren Millidge, Co-founder and Chief Scientist at Zyphra

As AI labs continue to explore inference scaling and test-time reasoning, Zyphra's work provides a concrete blueprint for how to build efficient, competitive models without necessarily pursuing ever-larger parameter counts. This shift could reshape AI economics, making advanced reasoning capabilities more accessible and sustainable for a broader range of organizations.