On-device AI is moving from experimental to practical, with three major model releases demonstrating that local inference can now match or exceed cloud-based performance while protecting user privacy and reducing costs. Liquid AI's LFM2.5-8B-A1B, Google's Gemma 4 family, and Google's experimental DiffusionGemma show that developers can build intelligent applications that run entirely on personal devices without sacrificing capability or speed.

Why Are Developers Suddenly Building AI That Stays Local?

The fundamental challenge has always been the same: smaller models that fit on personal devices were far less capable than massive cloud-based systems. But recent advances in model architecture are closing that gap. Liquid AI's LFM2.5-8B-A1B demonstrates this shift by using sparse activation, where only 1.5 billion of its 8.3 billion total parameters activate during inference. This technique dramatically reduces computational demands while maintaining strong reasoning and tool-calling abilities.

The practical motivation is straightforward. Cloud-based AI introduces latency, costs, and privacy risks. Local inference solves all three problems simultaneously. Google's Gemma 4 has been downloaded more than 150 million times since release, with developers building real-world applications that run entirely offline. The company also introduced DiffusionGemma, an experimental 26 billion parameter model that generates text up to 4 times faster than traditional approaches by processing entire blocks of text simultaneously rather than word-by-word.

What Specific Capabilities Make These Models Practical for Real Applications?

Liquid AI's LFM2.5 includes a 128,000 token context window, allowing it to process large documents and extended conversations without losing context. The model supports 10 languages natively, making it practical for global applications without requiring separate language-specific versions. More importantly, the model was specifically designed for real-world tasks rather than just benchmark optimization.

The model excels at tool calling, a capability essential for building autonomous assistants. It can chain multiple tool calls together, follow complex instructions, and execute workflows that resemble real assistant behavior. For example, an assistant could search a database, retrieve customer information, call another tool, and generate a final response all within a single conversation flow. This makes it particularly useful for enterprise automation systems and customer support applications.

Google's approach emphasizes accessibility across different hardware. The company released quantized versions of Gemma 4 that compress models to use less memory, making them practical even on mobile devices. HubX, an app building company, used the edge-optimized Gemma 4 E2B model to build BetterSpeak, an offline English tutoring platform that runs entirely on-device. The app handles grammar explanations and progress monitoring while processing all vocal and text data locally, protecting user privacy while reducing infrastructure costs.

Hallucination reduction represents another significant improvement. Liquid AI's LFM2.5 shows dramatic gains on the AA-Omniscience benchmark, with the non-hallucination rate improving from 7.46 to 63.47. For enterprise deployments where accuracy matters, reducing hallucinations is often more valuable than improving raw benchmark scores.

How to Choose and Deploy On-Device AI Models

• Match Model Size to Hardware: Liquid AI's LFM2.5 activates only 1.5 billion parameters despite having 8.3 billion total, making it practical for laptops and edge devices. Google's Gemma 4 E2B model offers even smaller footprints for mobile applications. Start by matching your hardware constraints to the model's active parameter count, not total parameters.
• Apply Quantization for Memory Efficiency: Google released 4-bit quantized versions of Gemma 4 that compress models without significant quality loss. Liquid AI supports multiple training approaches including continued pretraining, supervised fine-tuning, and LoRA-based training. These techniques allow you to adapt models to domain-specific tasks without training from scratch.
• Leverage Established Development Frameworks: Both Liquid AI and Google models support popular tools including Transformers, vLLM, SGLang, and llama.cpp. This broad ecosystem support makes adoption easier and allows deployment across platforms from Apple Silicon Macs to Windows laptops to edge devices.
• Evaluate Speed Versus Quality Trade-offs: DiffusionGemma generates text 4 times faster by processing 256 tokens in parallel, but produces lower quality output than standard Gemma 4. Choose based on your application's priorities: real-time interactive workflows benefit from speed, while high-stakes applications need maximum quality.

Performance metrics demonstrate the practical viability of local inference. Liquid AI's LFM2.5 can generate responses at speeds up to 18,500 output tokens per second on high-end hardware, or more than 1.6 billion tokens per day on a single H100 GPU. The model performs exceptionally well on CPUs too, making local deployment practical even without expensive specialized hardware. This matters for enterprise deployments, offline assistants, and privacy-focused applications where sending data to the cloud is not an option.

Google's DiffusionGemma takes a different approach to speed by shifting how models use available computing power. Most language models act like a typewriter, generating one token at a time from left to right. In cloud environments, this is efficient because servers batch thousands of user requests together. But when running locally for a single user, this word-by-word process leaves dedicated GPUs underutilized. DiffusionGemma reverses this inefficiency by drafting entire 256-token paragraphs simultaneously, giving the processor a larger chunk of work at once.

"DiffusionGemma is designed for researchers and developers exploring speed-critical, interactive local workflows such as in-line editing, rapid iteration, and generating non-linear text structures," noted Brendan O'Donoghue, Research Scientist at Google.

Brendan O'Donoghue, Research Scientist at Google

The performance gains are substantial. DiffusionGemma achieves over 1,000 tokens per second on a single NVIDIA H100 GPU and over 700 tokens per second on NVIDIA GeForce RTX 5090 consumer hardware. This speed comes with trade-offs; the model prioritizes velocity over output quality, making it better suited for interactive applications than high-stakes content generation. For applications demanding maximum quality, Google recommends deploying standard Gemma 4 instead.

Real-world applications already demonstrate the value of this shift. Beyond BetterSpeak's offline tutoring platform, developers are using Gemma 4 for vision-language tasks like object detection and visual question answering, while others are building applications that reimagine the real world as an adventure video game, leveraging the model's 256,000 token context window to maintain long interaction histories. These applications would be impractical or impossible with cloud-dependent systems due to latency and privacy constraints.

The broader implication is significant: developers now have practical options for building intelligent applications that run entirely locally, protecting user privacy, reducing latency, and eliminating cloud infrastructure costs. Whether you prioritize speed, efficiency, multilingual support, or tool-calling capabilities, there is now an on-device model designed for your specific use case. The question is no longer whether local AI is possible; it is which model best fits your application's needs.