Network equipment in AI data centers is quietly consuming 15 to 25 percent of total rack power, a hidden efficiency challenge that chip makers are now racing to solve. While most attention has focused on the energy demands of graphics processing units (GPUs) and other AI accelerators, the networking layer that connects thousands of these systems together has become an equally critical bottleneck for data center operators managing massive training environments.

Why Is Network Power Consumption Suddenly a Major Concern?

The scale of modern AI clusters has exploded. Training environments now routinely consist of tens of thousands of accelerators working in tandem, all continuously exchanging data across the network. A single GPU rack can now approach power levels of approximately 120 kilowatts, and as clusters grow larger, the networking infrastructure required to connect them becomes proportionally more demanding.

This shift has forced hardware makers to rethink how they design network switches specifically for AI workloads. Traditional network switches, originally built for enterprise data centers and cloud environments, were never optimized for the unique demands of large-scale AI training. In these environments, even small delays in communication between accelerators can cascade across an entire training job, slowing down the entire operation and wasting energy in the process.

What Does Marvell's New Approach to AI Networking Look Like?

Marvell Technology announced the Teralynx T100, a network switch chip designed from the ground up for AI data centers. The chip delivers a throughput capacity of 102.4 terabits per second, meaning it can move massive amounts of data between systems at extraordinary speeds. More importantly, it consumes less than 1,000 watts of power, making it significantly more efficient than competing products in the same category.

The T100 represents a fundamentally different engineering approach compared to switches adapted from traditional cloud environments. Built using a 3-nanometer manufacturing process as a single integrated silicon design, the chip is optimized specifically for predictable performance in large-scale AI environments where network delays can impact the performance of an entire training operation.

The switch supports configurations with up to 512 ports, allowing data center operators to build larger networks with fewer intermediate links. This architectural advantage reduces both latency and the number of optical connections required, further cutting down on power consumption and operational complexity.

How Are Data Center Operators Adapting Their Network Architecture?

Marvell is explicitly designing the T100 to support multiple network architectures that have emerged specifically for AI applications. The chip supports Ethernet Scale-Up Networking (ESUN) and complies with specifications from the Ultra Ethernet Consortium, giving hyperscalers and cloud providers flexibility in how they design their infrastructure.

• Scale-Out Networks: Configurations that interconnect large numbers of systems horizontally, allowing data centers to expand capacity by adding more nodes to the network.
• Scale-Up Architectures: Designs that optimize performance within individual AI systems by reducing latency and improving communication efficiency between closely coupled accelerators.
• Flexible Interface Options: The T100 is offered in various packaging options with integrated copper or optical interfaces, allowing operators to choose the best connectivity solution for their specific deployment.

This flexibility is critical because different AI workloads and different hyperscalers have different networking requirements. Some prioritize raw throughput, while others need to minimize latency or reduce the total cost of ownership by cutting power consumption.

Why Should Data Center Operators Care About Network Efficiency Right Now?

As AI clusters continue to scale, network efficiency is becoming increasingly important to the total cost of ownership of AI data centers. The energy consumption of network equipment is no longer a minor factor that can be ignored; it is a significant operational expense that directly impacts profitability.

Modern GPU racks are approaching 120 kilowatts of power consumption, and with network components accounting for 15 to 25 percent of that total, optimizing switch efficiency can save substantial amounts of electricity and cooling costs. For large hyperscalers operating hundreds or thousands of racks, even small improvements in network power efficiency translate into millions of dollars in annual savings.

Marvell expects to begin shipping the first units of the Teralynx T100 to customers in the current quarter, making this technology available to data center operators who are actively building out their next-generation AI infrastructure. The introduction of the T100 underscores a broader shift in how the industry thinks about AI infrastructure optimization. While attention in recent years has primarily focused on GPUs from companies like NVIDIA and AI accelerators from AMD and others, competition in the network layer is also growing as the importance of efficient interconnects becomes undeniable.