Storage is no longer just where data sits between compute operations; it's becoming a critical part of the AI pipeline itself. As artificial intelligence models grow larger and more complex, the infrastructure supporting them faces an unexpected challenge: the storage systems connecting GPUs and accelerators to data are struggling to keep pace. New benchmark results released this week highlight just how critical this bottleneck has become for organizations building cutting-edge AI infrastructure.

Why Is Storage Suddenly a Bottleneck for AI?

When AI training runs need to save their progress, or when inference systems need to retrieve cached data, storage moves from the background into the critical path. If storage can't deliver data fast enough, expensive GPUs and accelerators sit idle, wasting resources. This problem intensifies as models scale. The newly released MLPerf Storage v3.0 benchmark, which measures storage performance against real AI workloads rather than testing storage in isolation, reveals just how demanding modern AI infrastructure has become.

The benchmark tested storage systems handling two of the largest model sizes in production today: 405 billion parameter models and 1.25 trillion parameter models. These numbers represent some of the most computationally intensive AI systems deployed by hyperscalers and research institutions. The results show that predictable, scalable storage performance is now as important as raw speed.

What Do the New Benchmark Results Show?

The MLPerf Storage v3.0 results reveal the scale of the challenge. In testing with 1,024 simulated accelerators running a 1.25 trillion parameter model, storage systems needed to deliver write speeds of 877.52 gigabytes per second and read speeds of 588.28 gigabytes per second. To put that in perspective, that's equivalent to transferring the entire contents of a large external hard drive in a single second.

What matters more than peak performance, however, is how storage scales as AI clusters grow. The benchmark tested configurations ranging from 10 data nodes to 30 data nodes. Write throughput scaled predictably from 327.65 gigabytes per second with 10 nodes to 877.52 gigabytes per second with 30 nodes. This linear scaling is critical because AI workloads don't remain static; organizations continuously add compute power, increase model sizes, and expand capacity.

On the inference side, where AI models serve predictions to users, the challenge is equally pressing. Key-value caching, a technique that stores intermediate computation results to speed up responses, is becoming increasingly important as organizations move from training models to serving them at scale. The benchmark tested inference workloads with models ranging from 8 billion to 70 billion parameters, with storage-only configurations delivering between 33,403 and 85,736 tokens per second depending on model size.

How to Evaluate Storage for AI Infrastructure

• Scalability Testing: Verify that storage performance scales predictably as you add more nodes or capacity. Linear scaling ensures that adding infrastructure doesn't create new bottlenecks as your AI workloads grow.
• Real-World Workload Simulation: Benchmark storage against actual AI training and inference tasks, not generic storage tests. MLPerf-style benchmarks measure performance under conditions that match production AI environments.
• Data Movement Architecture: Examine how the storage system moves data across your network. Disaggregated architectures that separate metadata management from data transport can reduce bottlenecks compared to centralized storage paths.
• Checkpoint and Cache Performance: Test how quickly the system can save model checkpoints during training and retrieve key-value cache data during inference, since these operations directly impact GPU utilization rates.

The architecture underlying storage systems matters as much as the performance numbers themselves. Modern AI infrastructure increasingly uses disaggregated storage designs that separate metadata management from data transport. This approach allows data to move across network fabric rather than forcing every workload through a single centralized path, reducing congestion and improving predictability.

"These results yet again demonstrate that FlashBlade//EXA delivers the highest performance and scale on the market, staying ahead of the increasing demands of AI workloads. Organizations advancing the bleeding edge of AI development need infrastructure that can deliver the utmost performance and scalability, without making tradeoffs against reliability and operational simplicity," stated Rob Lee, Chief Technology and Growth Officer at Everpure.

Rob Lee, Chief Technology and Growth Officer, Everpure

Why This Matters for the Future of AI Infrastructure

The storage bottleneck represents a fundamental shift in how AI infrastructure must be designed. For years, the focus has been on accumulating GPUs and accelerators. But as models grow larger and more complex, the supporting infrastructure, particularly storage and networking, has become equally important. A GPU sitting idle because storage can't deliver data fast enough is just as wasteful as not having the GPU at all.

This challenge becomes more acute as organizations scale. A startup training a single large model might tolerate some storage latency. But hyperscalers running dozens of concurrent training jobs, each with thousands of GPUs, cannot afford inefficiency. Every millisecond of delay multiplies across thousands of accelerators, translating directly into wasted compute resources and higher infrastructure costs.

The broader implication is that AI infrastructure can no longer be treated as a collection of disconnected components. GPUs, networking, storage, and data management must work together as an integrated system. The performance of one increasingly depends on how well the others can keep pace. Organizations building AI data centers now need to think holistically about infrastructure, not just focus on the most visible component, the accelerators themselves.