A Caltech spinout just demonstrated a 2-billion-parameter AI model running entirely on smart glasses using 1-bit compression, cutting memory requirements by roughly 75% compared to standard 4-bit models. The breakthrough, shown at Qualcomm's Snapdragon Summit on September 24, suggests that on-device AI for wearables is finally becoming practical without sacrificing reasoning ability.

What Makes 1-Bit Compression Such a Game-Changer?

Traditional AI models store their weights (the numerical values that make a model "think") at high precision, typically 16 bits or 8 bits per weight. PrismML's Bonsai language model uses just 1 bit per weight, meaning each value is essentially a binary choice. This radical compression is not new in theory; Microsoft Research published similar ternary weight schemes in 2024. What changed is that hardware makers like Qualcomm finally built the necessary support directly into their chips.

The numbers tell the story. PrismML's 1.7-billion-parameter language model weighs roughly 0.43 gigabytes at 1-bit precision, versus about 1.66 gigabytes for the same model at 4-bit precision. That is a reduction of roughly 3.8 to 4 times smaller. On generation speed, the 1-bit version produced text at 15.36 tokens per second compared to 7.44 tokens per second for the 4-bit model, a roughly 2-fold speedup.

"Compressing a model without losing its reasoning ability was the multi-year engineering problem PrismML was built to solve," stated Babak Hassibi, CEO of PrismML.

Babak Hassibi, CEO at PrismML

The vision-language system demonstrated at the summit combined a 1.7-billion-parameter language model with a smaller vision encoder, allowing smart glasses to answer questions about what the wearer is looking at in real time. All processing happens locally on the Snapdragon AR1 Gen 1 platform, with no data sent to the cloud.

Why Does This Matter for Privacy and Performance?

The implications extend beyond raw speed. When AI runs entirely on-device, sensitive visual information never leaves the glasses. Users do not need Wi-Fi or cellular connectivity for the AI to function. The system responds instantly because there is no round-trip delay to a data center. For applications like real-time translation, object recognition, or accessibility features, this combination of privacy, speed, and independence is transformative.

Memory has become one of the most expensive components in consumer hardware. RAM now accounts for up to 60% of some phones' bill of materials, according to industry analysis. By shrinking AI models to a fraction of their original size, 1-bit compression directly addresses this cost pressure while enabling more capable wearables.

How Are Other Tech Giants Approaching On-Device AI?

PrismML is not alone in pursuing compact on-device models. Google deploys its Gemini Nano models directly on supported Android devices through its own software stack. Apple runs compact foundation models on-device as part of Apple Intelligence, offloading heavier tasks to Apple's Private Cloud Compute. Microsoft offers its Phi family of small, efficient models on Hugging Face. However, none of these competitors have yet published a 1-bit deployment on a wearable processor with the specific memory and speed figures PrismML demonstrated.

The broader on-device AI race has shifted from "can it run at all" to "how much can we shrink it without losing capability." Other recent examples include MiniCPM5-2B, which outscored larger rivals by 2.8 points on standard benchmarks, and Fastino's 340-million-parameter model, which achieved response times under 170 milliseconds on CPU hardware.

Steps to Understanding On-Device AI Deployment

• Model Quantization: Reducing the precision of weights from 16 bits to 8 bits, 4 bits, or even 1 bit shrinks file size dramatically while preserving reasoning ability through careful engineering.
• Hardware Acceleration: Chipmakers must build specialized support for low-bit operations into their processors; Qualcomm added 1-bit kernel support to its QNN (Qualcomm Neural Network) SDK to make this possible.
• System Architecture: On-device AI requires balancing memory constraints, power consumption, latency, and the ability to handle multiple tasks simultaneously without cloud connectivity.
• Benchmark Validation: PrismML tested its 1-bit model against standard evaluations including MMLU Redux, HumanEval+, and GSM8K to verify that compression did not degrade reasoning performance.

What Does This Mean for the Broader AI Infrastructure Shift?

The move toward on-device inference reflects a fundamental rethinking of where AI computation should happen. Rather than treating the cloud and edge as opposing choices, industry leaders increasingly see them as complementary layers. The cloud remains essential for training, large-scale reasoning, simulation, and fleet learning. The edge and device layer now handles real-time perception, decision-making, and action.

EdgeCortix, another company focused on edge AI, describes this emerging architecture as the "Thick Edge," a new infrastructure layer between embedded processors and hyperscale data centers. Physical AI systems like robots, autonomous vehicles, and industrial equipment need computing power and memory beyond what a typical embedded chip can provide, yet they require fast local responses and cannot depend on cloud latency. This middle layer is expanding as AI becomes more capable and more integrated into physical systems.

"Physical AI is not simply another AI model workload. It is a system workload. Once AI becomes part of a machine that must operate continuously in the real world, the optimization target changes," explained EdgeCortix in describing its RAIDEN chiplet-based platform.

EdgeCortix, Physical AI Computing Platform Developer

Memory and storage are becoming critical bottlenecks in this distributed AI future. As models grow larger and inference workloads scale, the industry is exploring heterogeneous storage architectures that combine high-bandwidth memory (HBM), NAND flash, and emerging technologies like resistive RAM (RRAM) to balance performance, capacity, and cost. Key-value cache management, which stores intermediate computation results during inference, is consuming enormous amounts of memory in large language models.

The convergence of these trends suggests that on-device AI will continue to improve in capability while shrinking in size and power consumption. PrismML's 1-bit breakthrough is one data point in a larger shift toward distributed, efficient, and privacy-preserving AI systems. As hardware makers optimize for low-bit operations and software engineers refine compression techniques, the barrier to running sophisticated AI directly on wearables, phones, and edge devices will continue to lower.