Enterprise AI is no longer about training massive models in the cloud; it's about deploying intelligent agents that can execute complex tasks autonomously, and that shift is forcing companies to completely redesign their data centers, cost models, and workforce strategies.

For the past year, the conversation around enterprise AI has centered on securing graphics processing units (GPUs) for model training. But that assumption is now outdated. According to Dell Technologies' global chief technology officer John Roese, the maturation of agentic AI (systems that can independently pursue objectives rather than respond to single prompts) is rewriting the enterprise architecture playbook.

What's Actually Driving the Infrastructure Shift?

The fundamental difference lies in how AI agents work compared to traditional chatbots. When enterprises built chatbots, they needed powerful GPUs but relatively light CPU loads. Agents, however, operate differently. They use external tools, communication protocols, and knowledge graphs, components that don't naturally live on GPUs. "When you move to agentic, it's almost balanced," Roese explained. "The number of CPUs and GPUs are very similar, about maybe for every two GPUs you have a CPU".

This architectural shift has profound implications for enterprise spending. The prevailing myth that enterprises need thousands of GPUs is simply wrong. "Our biggest workload inside of Dell only sits on 16 GPUs and supports 40,000 people," Roese noted. "You don't need thousands of GPUs in an enterprise, because for each workload, agent or project, you only need a handful of GPUs, sometimes half a GPU". That's because most enterprise AI work focuses on inference (running models on existing data) rather than training (building new models from scratch).

Meanwhile, deployment options are expanding. A year ago, the most powerful AI models were locked behind cloud APIs (application programming interfaces). Now, hyperscalers like Google are enabling top-tier models to run on-premises through services like Google Distributed Cloud, giving enterprises options they didn't have before. "You can consume it in a virtual private cloud or your datacentre, and you can air-gap it from everything else," Roese said.

How Are Data Strategies Evolving to Support Agents?

Infrastructure alone isn't enough. The real bottleneck for AI agents is data delivery speed. Roese warned that bolting standard data storage systems onto AI compute clusters no longer meets performance demands. Instead, organizations need to build knowledge and context layers comprising vector databases (specialized systems for storing and searching semantic information), graph databases (systems that map relationships between data), and data annotation tools. These layers must be deeply integrated into compute infrastructure, not isolated.

The performance challenge is stark: "One of the performance bottlenecks is you can't get data fast enough to the GPUs to do the work," Roese explained. "The GPUs you're paying for are sitting idle, waiting for data." To reduce this latency, Dell's AI data platform is now integrated directly into Nvidia's Cuda-X interfaces, effectively running data layer services at GPU speed.

Steps to Optimize AI Agent Economics in Your Organization

• Implement Model Routing: Route complex planning tasks to expensive frontier models while sending routine tasks to smaller, on-premises open-source models where energy is the only operational cost. This approach can significantly reduce overall AI spending.
• Audit Job Containers: Analyze how AI agents will impact specific roles by identifying which tasks within each job can be automated, then determine whether to reduce headcount, expand responsibilities, or shift workers toward higher-value expert work.
• Integrate Data Infrastructure: Build vector databases, graph databases, and annotation tools directly into your compute layer rather than treating them as separate systems, ensuring data reaches GPUs at the speed needed for agent operations.

Cost management is becoming a competitive differentiator. With model deployment options now available at different pricing mechanisms, enterprises must treat AI workloads as an arbitrage game. Using specification-driven development (where AI writes software based on markdown documents) as an example, Roese noted that if an agentic framework spawns dozens of coding tasks and blindly sends them to top-tier models, costs spiral. But with intelligent model routing, enterprises can ensure complex planning tasks go to expensive frontier models while routine coding tasks route to smaller, cheaper alternatives.

Why the Human Element Is the Hardest Part

The most challenging aspect of operationalizing agentic AI isn't technical; it's organizational. Roese described the traditional human job as a "container of work" that includes hygiene tasks, productivity work, coordination, and expert tasks. Agents excel at specific types of work but cannot perform entire jobs. Dell audited 6,400 jobs across its own business to understand how AI agents would reshape roles.

"The first thing we realised is every single job in the company is going to change. I'm taking work out of the job and removing stuff from the container. If the container is now only half full, do I need half the number of people, or do I expand that by half? Am I able to do more expert work?" said John Roese.

John Roese, Global Chief Technology Officer at Dell Technologies

The impact is so profound that change management has become a core responsibility of IT leadership. "For the last four months, I've spent 50% of my time dealing with human dynamics," Roese remarked. "AI has ceased being a technology and an ROI (return on investment) discussion. It's now very much an organisational and human dynamic discussion. You simply can't use these things unless you fully understand how you're going to adapt the human population around them".

What Are Production Teams Learning About Agent Reliability?

Beyond infrastructure, the industry is also focusing on making agents more reliable in production environments. LangChain, a major framework for building AI agents, recently launched LangSmith Engine, a tool that watches production traces, clusters recurring failures into named issues, diagnoses root causes, and proposes fixes for review. This represents a shift toward treating agent development as a continuous lifecycle rather than a one-time deployment.

LangChain also made Sandboxes generally available, secure code execution environments built specifically for agent operations and integrated with the Deep Agents SDK (software development kit) and the LangSmith platform. These tools reflect an industry-wide recognition that agents need specialized infrastructure and monitoring to operate reliably at scale.

The convergence of these trends signals a fundamental transformation in how enterprises approach AI. It's no longer about acquiring the most powerful models or the most GPUs; it's about designing systems where agents can operate autonomously, efficiently, and safely, while ensuring the human workforce adapts to work alongside them. The companies that master this balance will gain significant competitive advantages in the coming years.