A new wave of edge computing hardware is bringing advanced AI processing directly to devices rather than relying on cloud connections, potentially reshaping how companies deploy artificial intelligence (AI) at the edge. Singapore-based startup Acrab has unveiled the Agent Box, a compact desktop system designed to run 100 billion parameter language models entirely on local hardware. The device represents a significant cost reduction compared to existing enterprise solutions, which could accelerate adoption of local AI inference across multiple industries.

What Is Edge AI Processing and Why Does It Matter?

Edge AI refers to running artificial intelligence models directly on local devices rather than sending data to remote cloud servers for processing. This approach offers several practical advantages: faster response times, reduced bandwidth costs, improved privacy since sensitive data stays local, and continued operation even when internet connectivity is limited or unavailable. For applications requiring real-time decision-making, edge processing eliminates the latency of sending data to distant data centers and waiting for responses.

The shift toward edge computing represents a departure from cloud-centric AI architectures that have dominated the past decade. As AI models become more capable and hardware becomes more efficient, processing can move closer to where data originates, whether in vehicles, factories, smart homes, or industrial equipment.

How Does Acrab's Hardware Compare to Existing Solutions?

Acrab's Agent Box is powered by a proprietary chip called GELIX 1, manufactured using a 5-nanometer process for efficient edge computing. The company claims the system delivers performance close to NVIDIA's DGX Spark, which costs approximately $5,000, while cutting costs to roughly one-fifth of that price and halving power consumption. The GELIX 1 processor combines a 20-core Arm central processing unit (CPU), a graphics processing unit (GPU) rated at 3 teraflops, and a specialized neural processing unit (NPU) tuned specifically for large language model inference.

In internal testing, Acrab recorded a prefill rate of 1,416.8 tokens per second under a Gemma 26B configuration with a 40,000 key-value cache and a 10,000 token input. Tokens are small units of text that AI models process; this metric measures how quickly the system can begin generating responses. The company claims this represents a 7.5 times improvement in prefill speed compared to an Apple Mac Mini M4 Pro, which achieved 188.9 tokens per second under the same test conditions.

"Our goal is to give device makers and developers the foundation to bring agentic intelligence into many different products and environments," said Dr. Ken Phua, CEO of Acrab.

Dr. Ken Phua, CEO of Acrab

What Practical Capabilities Does Local AI Processing Enable?

Acrab's demonstrations showed voice commands automatically creating and printing 3D models, operating robotic vacuum cleaners, and interacting with connected devices including smart lights, smart air conditioners, and smart locks. These examples illustrate how edge AI can power autonomous agents that understand natural language commands and control physical systems without relying on cloud services.

The benefits of keeping inference on the device extend across multiple dimensions:

• Privacy Protection: Sensitive data from cameras, microphones, and sensors remains on the device rather than being transmitted to external servers, addressing growing concerns about data collection and surveillance.
• Offline Operation: Devices can continue functioning in areas with limited or no internet connectivity, such as rural locations or underground spaces where cloud communication might fail.
• Reduced Latency: Processing happens in milliseconds rather than the seconds required for data to travel to a cloud server and back, critical for time-sensitive applications.
• Lower Operating Costs: Eliminating cloud API calls reduces ongoing expenses, particularly important for applications deployed at scale across thousands or millions of devices.
• Persistent Memory: Local systems can maintain continuous learning about a specific device's environment and user preferences, enabling personalized AI assistants.

What Makes Acrab's Backing Significant?

Although Acrab has disclosed limited corporate information, copyright notices accompanying its announcement reference CATL, the world's largest electric vehicle battery manufacturer. If that relationship accurately reflects financial backing, the announcement could attract additional attention because of CATL's scale and resources. The company plans to extend its computing platform into AI network-attached storage systems, industrial robots, service robots, and smart vehicles through partnerships with hardware manufacturers.

CATL's existing relationships with automakers and battery suppliers could accelerate adoption of edge AI hardware across multiple industries. The combination of specialized hardware, software platforms, and manufacturing partnerships suggests a comprehensive approach to bringing local AI processing to market.

How to Evaluate Edge AI Solutions for Your Use Case

Organizations considering edge AI deployment should evaluate several key factors:

• Performance Requirements: Determine the inference speed, accuracy, and model size needed for your specific application, then compare against available hardware specifications and benchmark results.
• Cost Structure: Calculate total cost of ownership including hardware, software licenses, power consumption, and ongoing maintenance, comparing against cloud-based alternatives over a multi-year deployment period.
• Connectivity Constraints: Assess whether your deployment environment has reliable internet access and sufficient bandwidth for cloud-based processing, or whether offline operation is essential.
• Data Sensitivity: Evaluate privacy and security requirements to determine whether keeping data local is necessary for compliance or competitive reasons.
• Ecosystem Support: Review available software frameworks, developer tools, and community support to ensure you can build and maintain applications on the chosen platform.

What Does This Trend Mean for the Broader AI Market?

The emergence of cost-effective edge AI hardware suggests the market is diversifying beyond centralized cloud computing. Recent market volatility in AI stocks has been driven primarily by margin unwinding rather than weak fundamentals, according to investment analysis. As the margin unwind calms down, companies offering specialized solutions for specific use cases, including edge AI hardware, may see renewed investor interest.

Acrab's Agent Box demonstrates that competitive alternatives to established players are emerging, offering comparable performance at significantly lower cost. This competition could accelerate innovation in edge computing hardware and software, ultimately benefiting organizations that need to deploy AI locally. The combination of specialized processors, optimized software stacks, and strategic partnerships suggests that edge AI is transitioning from a niche capability to a mainstream deployment option.