Neural processing units (NPUs) have quietly become the most contested piece of silicon in computing, with major chipmakers and startups racing to embed AI acceleration directly into devices rather than relying on cloud servers. AMD's latest handheld chip includes a dedicated NPU delivering up to 50 trillion operations per second, while the company's new industrial processors target robotics and autonomous systems. Meanwhile, startups like Acrab are designing edge AI chips specifically to run 100-billion-parameter language models locally, signaling a fundamental shift in how and where artificial intelligence actually executes.

What's Driving the NPU Explosion Across Device Categories?

The push toward NPUs reflects a simple reality: cloud-based AI is becoming expensive and slow for real-time applications. Handheld gaming devices, industrial robots, and edge servers all need to make decisions instantly without waiting for a round trip to a distant data center. AMD's Ryzen AI Z2 Extreme processor, which launched in the ROG Xbox Ally X handheld in October 2025, includes an XDNA 2 neural processing unit alongside traditional CPU and GPU cores. This "tri-engine" architecture allows the chip to divide labor: the CPU handles system logic, the GPU manages graphics and vision processing, and the NPU specifically accelerates the mathematical operations that power artificial intelligence.

The NPU delivers up to 50 TOPS (trillions of operations per second) of dedicated AI performance, exceeding the requirements for Microsoft's Copilot+ PC program, which sets a baseline for AI-capable devices. This matters because it means handheld gaming devices and industrial systems can now run AI workloads locally without offloading to the cloud, reducing latency and protecting sensitive data.

Acrab's GΞLIX 1 processor takes this concept further, targeting large language models with 100-billion parameters running entirely on edge devices. The chip combines a 20-core ARM CPU, a 3 TFLOPS (teraflops, or trillions of floating-point operations per second) GPU, and a dedicated NPU, all connected by 768 gigabytes per second of shared memory bandwidth. This high-bandwidth memory architecture is critical because language models are notoriously "hungry" for data; without sufficient memory bandwidth, even a powerful NPU becomes a bottleneck.

How Are Chipmakers Competing in the NPU Space?

AMD's strategy differs markedly from Intel's approach. AMD is prioritizing raw performance and memory bandwidth, building larger, more powerful chips designed for "heavyweight" robotics such as humanoid robots and complex industrial arms. The company's Ryzen AI Embedded X100 series features up to 16 CPU cores, 40 graphics compute units, and the XDNA 2 NPU, all rated for industrial operation from minus 40 degrees Celsius to 105 degrees Celsius. These processors support up to 128 gigabytes of unified memory, enabling them to run large language models and complex computer vision algorithms simultaneously.

Intel, by contrast, has focused on smaller, highly efficient chips with integrated Xe3 graphics, targeting smaller drones and handheld industrial devices. This suggests a market segmentation where AMD aims for high-performance robotics while Intel pursues miniaturization and power efficiency.

The real competitive threat comes from Nvidia, which has dominated robotics for years through its Jetson and Thor platforms. Nvidia's advantage stems from CUDA, a software ecosystem that has become the de facto standard for AI development. AMD is attempting to disrupt this dominance by emphasizing that its x86-based processors use the same instruction set as desktop and server chips, potentially lowering the barrier to entry for developers. The company has also invested heavily in HIPIFY, a tool that automatically converts existing CUDA code into portable C++ code that runs on AMD hardware. In internal testing, AMD demonstrated that HIPIFY can automate 70 to 80 percent of the porting effort, significantly reducing the engineering cost for companies considering a switch from Nvidia.

Steps to Understand NPU Performance in Real-World Applications

• Memory Bandwidth Matters More Than Raw Compute: A processor with more NPU cores but insufficient memory bandwidth will underperform. Acrab's GΞLIX 1 delivers 768 gigabytes per second of shared L2 cache bandwidth specifically to feed its neural processing unit without creating a data bottleneck.
• Latency Reduction Drives Adoption: By integrating CPU, GPU, and NPU on a single die with unified memory, chips like AMD's X100 series eliminate the latency associated with data transfer between discrete components, a critical factor for machines that must react to physical environments in real-time.
• Software Compatibility Determines Market Success: Hardware alone is insufficient; developers need tools and frameworks to deploy AI models efficiently. AMD's ROCm platform and HIPIFY toolset are designed to lower the friction of switching from Nvidia's CUDA ecosystem.

What Performance Gains Are NPUs Actually Delivering?

The benchmarks reveal both impressive capabilities and important caveats. Acrab's GΞLIX 1 achieves 1,416.8 tokens per second for a Gemma 26B model with a 40,000-token key-value cache and 10,000-token input, reportedly outperforming similar desktop hardware in this specific application. This matters because "time to first token" (the latency before the model starts responding) and throughput (tokens per second) directly affect user experience in conversational AI applications.

AMD's Ryzen AI Embedded X199 showed a 1.4x improvement in time to first token and a 3.5x faster tokens-per-second rate compared to Intel's Core Ultra X7 358H when running Llama models. However, AMD acknowledged that some of these benchmarks involve "projected" performance, meaning they may not reflect real-world results across all applications.

The handheld gaming space illustrates a more sobering reality. AMD's Ryzen AI Z2 Extreme GPU includes 33 percent more compute units than its predecessor, but independent reviewers measured only a 12 percent improvement in actual gaming frame rates. The gap between spec-sheet math and real-world performance stems from a fundamental constraint: AMD gave the GPU more compute resources without proportionally increasing memory bandwidth, and handheld APUs are notoriously bandwidth-limited.

Where Are NPUs Being Deployed Today?

NPUs are appearing across three distinct market segments. In consumer devices, they enable handheld gaming systems and AI-capable laptops to run local AI workloads without cloud connectivity. AMD's Ryzen AI Z2 Extreme handheld processor, which launched at $999.99 in the ROG Xbox Ally X, represents this consumer push.

In industrial and robotics applications, NPUs are becoming essential infrastructure. AMD's Ryzen AI Embedded X100 series is designed for 24/7 operation with a guaranteed 10-year production lifecycle, catering to manufacturing, medical, and autonomous systems industries. The company is offering these processors as part of a Kria System on Module adhering to the standardized COM-HPC form factor, allowing industrial designers to integrate the compute unit into various carrier boards. For developers in the prototyping phase, AMD introduced the Kria AI Robotics Developer Platform, which combines the X100 processor with an AMD Spartan UltraScale+ FPGA (field-programmable gate array) for ultra-low latency communication with specialized robotic sensors and motor controllers.

Edge AI inference represents the third segment. Acrab's GΞLIX 1 processor is designed specifically to run large language models locally without cloud infrastructure, reducing latency, improving data privacy, and enabling real-time AI applications. The manufacturer offers a complete software stack, development tools, and reference designs, along with a proprietary Agent Box platform allowing developers to evaluate edge AI inference workloads without cloud resources.

What Challenges Remain for NPU Adoption?

Despite rapid progress, significant hurdles persist. The software ecosystem remains fragmented compared to Nvidia's CUDA dominance. While AMD's HIPIFY tool can automate 70 to 80 percent of code conversion, the remaining 20 to 30 percent still requires manual engineering effort. This friction slows adoption, particularly among companies with large existing codebases optimized for Nvidia hardware.

Memory bandwidth constraints continue to limit performance gains. Even with dedicated NPUs, processors must feed these accelerators with data fast enough to keep them fully utilized. This architectural challenge affects everything from handheld gaming devices to industrial robotics platforms.

Finally, the market is still defining what "good enough" performance looks like for different applications. Handheld gaming devices need different NPU capabilities than industrial robots or edge language model servers. As these markets mature, chipmakers will need to offer increasingly specialized silicon rather than one-size-fits-all solutions.