On-device artificial intelligence is no longer a single strategy; it's fragmenting into three distinct approaches, each solving a different problem. Medical wearables are running inference continuously on ultra-low-power silicon. Wearable robotics platforms are offloading computation to portable hubs. And compact computers are now capable of running massive language models entirely offline. Together, these developments reveal a fundamental shift: the cloud is no longer the default destination for AI processing.

Why Are Medical Devices Moving AI Processing Onto the Device Itself?

Medical devices face a unique constraint that cloud computing cannot solve: they must respond instantly, continuously, and without relying on network connectivity. A fall-detection pendant cannot wait for data to travel to a smartphone or server and back. A sleep-apnea monitor cannot afford to miss an event while the device is in low-power mode. An insole tracking gait patterns cannot stream raw sensor data all day without draining its battery.

Engineers from Ambient Scientific, Arduino, and SiMa.ai recently outlined why on-device inference has become essential for medical applications. The core insight is that medical devices now need three things simultaneously: continuous sensing, on-device inference, and low power consumption. Missing any one of those makes the product unreliable.

"Patients are very concerned about their medical data being tampered with. Being on device makes the solution more reliable, private, and secure," explained a representative from Ambient Scientific.

Ambient Scientific, Medical Device Engineering

Ambient Scientific has built a fall-detection pendant reference design that runs inference continuously on low-power silicon, with no phone or cloud connection required. The device can run for up to three months on a single charge while achieving 99% detection accuracy with few false positives. This approach extends to biomonitoring patches, sleep-apnea devices, wearable stethoscopes, medical alert bands, and smart insoles that track gait patterns associated with conditions like Parkinson's disease.

How Are Wearable Robotics Platforms Centralizing Edge AI?

Wetour Robotics introduced a different model for on-device AI: the portable hub. The company outlined Orchestra, a portable AI hub and operating system for wearable robotics that centralizes AI processing in a dedicated edge-computing unit powered by NVIDIA Jetson. Rather than distributing intelligence across multiple wearable endpoints, Orchestra concentrates computing power in a single portable device that coordinates multiple wearables.

Orchestra is designed to enable real-time visual perception, gesture recognition, and multi-device coordination by externalizing computing power from individual wearable endpoints into a dedicated portable hub. Smart glasses, gesture-control wristbands, and body-worn sensors remain lightweight and energy-efficient while the hub handles intensive AI processing, multi-device coordination, and real-time decision-making.

The platform includes two core technology modules. Vision-Link processes real-time visual input, performs scene understanding using NVIDIA-accelerated inference, and translates visual context into actionable commands for connected physical devices. Conductor reads electromyographic (EMG) signals from the wearer's wrist, uses proprietary algorithms running on NVIDIA Jetson to recognize continuous hand gestures in real time, and converts recognized gestures into precise control commands for connected devices such as exoskeletons, smart furniture, and robotic arms.

"Vision-Link turns what you see into machine action. Conductor turns how you move into machine commands. Both require low-latency, on-device AI processing enabled by NVIDIA edge computing. This is edge AI applied to the human body," said Nan Zheng, Chief Executive Officer of Wetour Robotics.

Nan Zheng, Chief Executive Officer, Wetour Robotics

Wetour Robotics is pursuing an open architecture approach, intending to explore open interface protocols that would allow third-party hardware manufacturers to build on the Orchestra platform while retaining proprietary capabilities in its core intelligence engine.

What Can Compact Computers Do With On-Device AI?

A third approach to edge AI focuses on raw computational capacity in a small form factor. MSI introduced the Pro Max Edge AI+, a mini PC designed for on-device AI inference with gobs of memory and high-end hardware housed in an ultra-compact 4-liter aluminum chassis. The system can run massive large language models (LLMs) with up to 120 billion parameters entirely offline, offering an alternative to cloud APIs or power-hungry server racks.

The Pro Max Edge AI+ is available with up to an AMD Ryzen AI Max+ 395 processor, a powerful chip based on Zen 5 featuring 16 cores, 32 threads, clock speeds from 3 gigahertz (base) to 5.1 gigahertz (maximum boost), and an onboard neural processing unit (NPU) capable of up to 50 TOPS (trillion operations per second). The chip as a whole is rated to deliver up to 126 TOPS of AI muscle.

The memory configuration is the real differentiator. The mini PC can be equipped with up to 128 gigabytes of fast LPDDR5X-8000 unified memory. Using AMD's Variable Graphics Memory technology, up to 96 gigabytes of that memory pool can be allocated as video RAM (VRAM). This unified memory approach sidesteps traditional hardware bottlenecks that might otherwise force buyers toward costlier workstation graphics processing units (GPUs), enabling the mini PC to generate roughly 15 tokens per second on a 109-billion-parameter model. For research teams handling massive datasets, multiple units can be strung together to run open-source models scaling up to 670 billion parameters.

How to Evaluate On-Device AI Solutions for Your Use Case

• Power Budget: Determine whether your application requires continuous operation on battery power, intermittent sensing with low-power duty cycles, or plugged-in operation. Medical wearables and portable hubs have vastly different power constraints than compact computers.
• Latency Requirements: Assess whether your application can tolerate cloud round-trip delays of hundreds of milliseconds or requires sub-50-millisecond responses. Fall detection, gesture recognition, and real-time robotics control all demand local inference.
• Privacy and Data Sensitivity: Evaluate whether your application involves sensitive personal data, medical records, or proprietary information that should never leave the device. On-device processing eliminates transmission risks entirely.
• Model Size and Complexity: Consider whether you need to run a small specialized model on a wearable or a massive 120-billion-parameter language model on a desktop system. The three approaches serve different computational scales.
• Connectivity Assumptions: Determine whether your application can assume reliable network access or must function in environments with spotty or no connectivity. Edge AI removes dependency on cloud availability.

What Are the Regulatory and Development Challenges?

Moving from prototype to production-ready on-device AI introduces regulatory and engineering complexity. Jim Bruges, who works in developer relations at Arduino, noted that a prototype proves an idea is technically feasible but does not prove the device is safe, manufacturable, repeatable, or ready for regulatory review. That gap is where many projects stall.

For medical devices, the clinical role matters as much as the operating environment. A system that makes or directs a diagnosis carries different regulatory risk than one that simply flags a possible concern for a clinician to review. Vaibhav Kale, director of customer application engineering and success at SiMa.ai, added that how results get communicated to patients matters too. An AI output based on available data should not be presented as an expert medical opinion, a distinction that gets harder to maintain once a model keeps learning after deployment.

"With AI, the whole thing changes because there is a possibility that it will evolve. Traditional software can be validated once, in a fixed state; a model that changes over time raises the question of when an update becomes significant enough to need re-approval," noted Vaibhav Kale.

Vaibhav Kale, Director of Customer Application Engineering and Success, SiMa.ai

The shift toward on-device inference reflects a maturation of edge computing hardware and a recognition that cloud processing, while powerful, introduces latency, privacy risks, and dependency on network availability. Medical devices, wearable robotics, and compact computers each represent a different answer to the same fundamental question: where should AI processing happen? The answer increasingly is: on the device itself.