A new generation of compact vision-language models is making it practical to run sophisticated AI directly on consumer devices, eliminating the need to send images and data to the cloud. Liquid AI released LFM2.5-VL-3B, a vision-language model with 3.1 billion parameters, that delivers performance comparable to models twice its size while maintaining fast inference speeds across CPUs, GPUs, and mobile hardware.

What Makes This Model Different From Larger Competitors?

LFM2.5-VL-3B improves significantly over its predecessor, LFM2-VL-3B, in four key areas. The model now understands digital screens and user interfaces with 80.7% accuracy on ScreenSpot-v2, outperforming the much larger Gemma-4-E4B model, which scored 51.2%. For function calling, a critical capability for AI agents that need to interact with tools and APIs, the model more than doubled its performance from 26.4 to 59.5 on ToolSandbox benchmarks. Object grounding, the ability to locate specific items in images and return their coordinates, improved by 30 percentage points, jumping from 57.1 to 87.9 precision on RefCOCO tasks.

The model also handles multiple images better than before, improving its ability to reason across sequences of photos or video frames. These improvements came from training on approximately 34 trillion tokens, a fourfold increase in vision pretraining data, combined with a doubled vocabulary to better support non-Latin scripts.

How Fast Does It Actually Run on Real Devices?

Speed matters for on-device AI because every millisecond of latency affects user experience. On an Apple M5 Max laptop, LFM2.5-VL-3B decodes 228 tokens per second while using about 3 gigabytes of memory, making it practical for everyday tasks like document analysis and image captioning. On an AMD Ryzen AI Max+ 395 processor, it achieves 116 tokens per second. Most impressively, the model runs on a Samsung Galaxy S26 Ultra smartphone at 20 tokens per second, proving that capable vision AI no longer requires a laptop or cloud connection.

For GPU-accelerated inference in data centers, LFM2.5-VL-3B returns the first token in about 34 milliseconds on a five-frame video clip when running on a single NVIDIA H100 SXM5 GPU, compared to around 200 milliseconds for larger Gemma models. At high concurrency levels, the model achieves approximately 11,000 output tokens per second on a single H100, roughly double the throughput of larger 4-billion-parameter models.

Steps to Deploy LFM2.5-VL-3B Across Different Platforms

• Edge Devices: Use llama.cpp with GGUF checkpoints for efficient inference on CPUs, or MLX for optimized performance on Apple Silicon devices like MacBooks and iPads.
• Production GPU Servers: Deploy with vLLM or SGLang for high-throughput serving on NVIDIA GPUs, supporting multiple concurrent requests in cloud or on-premises data centers.
• Cross-Platform Compatibility: Export to ONNX format to run the model across diverse hardware accelerators, ensuring flexibility if you need to switch processors or vendors.
• Browser-Based Testing: Try the WebGPU demo directly in a web browser with no installation required, allowing you to upload images or capture webcam stills to test the model's capabilities.

The availability across multiple inference frameworks reflects a broader industry shift toward standardized tooling for edge AI. Developers can now choose the deployment method that best fits their hardware and latency requirements without rewriting code.

Why Is On-Device Inference Becoming Critical for AI Applications?

Running AI models locally addresses privacy, latency, and reliability concerns that cloud-dependent systems cannot solve. When a model processes sensitive images, documents, or video on your device, that data never leaves your hardware. This matters for healthcare providers analyzing patient scans, manufacturers inspecting products on factory floors, and individuals using AI for personal tasks.

Latency also becomes negligible when inference happens locally. Cloud-based systems introduce network round-trip delays that can make real-time applications impractical. A robot navigating a warehouse, a smartphone app detecting objects in a photo, or a smart home device responding to voice commands all benefit from instant local processing.

The broader Physical AI Platforms Market reflects this trend. On-device deployment accounted for 58.96% of the market share in 2025, while cloud-based deployment is growing at a slower rate of 17.02% annually. This indicates that developers and enterprises increasingly prefer local inference for robotics, autonomous systems, and edge computing applications.

What Real-World Tasks Can This Model Handle?

LFM2.5-VL-3B excels at practical tasks that require understanding both physical and digital environments. The model can analyze an entire document page in a single pass, identifying regions and assigning layout labels for document processing workflows. It can navigate websites by visually analyzing each screen, identifying clickable elements, and determining the next action to reach a goal, making it useful for automating web-based tasks.

When given an image of a dish and asked for its recipe, the model can identify the dish and determine that it should call a search tool to find the recipe, demonstrating function calling in a practical context. The model also locates specified objects in images and returns their bounding box coordinates, enabling applications like inventory management, quality inspection, and accessibility tools.

How Does This Fit Into the Broader Edge AI Ecosystem?

Parallel developments in edge AI infrastructure are accelerating adoption. Google AI Edge and Raspberry Pi announced joint support for running language, vision, and speech models entirely locally on Raspberry Pi 5 using LiteRT, Google's on-device inference runtime, and the Gemma family of open models. The Reachy Mini robot demo showcases object detection running on the GPU while a Gemma 4 E2B language model runs on the CPU, with no cloud dependency and real-time performance.

Igalia contributed experimental WebGPU support in Mesa's V3DV Vulkan driver, enabling inference on the VideoCore VII GPU alongside CPU workloads. This allows continuous tasks to execute without competing for CPU cycles, a critical capability for robotics and autonomous systems. The improvements were developed upstream in Mesa, benefiting all Vulkan and WebGPU users on Raspberry Pi devices, not just AI workloads.

The Physical AI Platforms Market is projected to grow from USD 9.71 billion in 2026 to USD 20.23 billion by 2031, at a compound annual growth rate of 15.81%. Manufacturing and industrial automation account for 31.24% of the market, while aerospace, defense, and security is projected to expand at an 18.19% annual rate. These sectors depend on edge inference for safety-critical applications where cloud latency or connectivity failures are unacceptable.

The convergence of compact models like LFM2.5-VL-3B, improved hardware support from companies like Raspberry Pi and NVIDIA, and growing market demand suggests that on-device AI is transitioning from a niche capability to a standard expectation for consumer devices, industrial equipment, and autonomous systems.