Google has launched LiteRT.js, a new tool that lets artificial intelligence models run directly inside web browsers instead of requiring data to be sent to distant cloud servers. This shift moves AI inference, the process of using a trained model to make predictions, from centralized datacenters to individual devices, promising faster responses, lower costs, and better privacy for users.

What Is LiteRT.js and Why Does It Matter?

LiteRT.js is a JavaScript binding for Google's LiteRT on-device runtime, announced on July 9. It allows web developers to execute machine learning models entirely on client devices using local processors and graphics cards, rather than sending each request to a cloud service. Google stated that this approach delivers "enhanced user privacy, zero server costs, and ultra-low latency for real-time experiences".

Google

The traditional cloud AI architecture sends application data to a remote endpoint, where a model processes the request and returns a result. That approach gives developers centralized control and access to powerful hardware, but it also introduces network delays, server capacity requirements, API charges, and potential privacy concerns. LiteRT.js offers an alternative for workloads that can fit within browser memory and run efficiently on consumer hardware.

How Can Developers Use Browser-Based AI Today?

LiteRT.js supports several practical applications that developers can implement immediately. Once a model and required WebAssembly components are delivered to the browser, the client can perform subsequent inference locally without making a server request for every operation.

• Real-Time Vision Tasks: Running object detection or image classification against a webcam feed without continuously uploading video to a remote server.
• Audio and Speech Processing: Processing speech, audio, or images locally for real-time web applications that require immediate responses.
• Vector Search and Embeddings: Generating embeddings and performing vector search inside the browser for semantic search capabilities.
• Image Manipulation: Upscaling or manipulating images without transferring the original files to a remote service, preserving user privacy.
• Offline Functionality: Adding offline or intermittently connected AI features to web applications that work even without internet connectivity.
• Sensitive Data Protection: Keeping sensitive user inputs on the device rather than sending them to a cloud endpoint.
• Hybrid Cloud Optimization: Reducing token or API consumption by handling simple classification, extraction, embedding, or compact generative AI tasks locally while reserving cloud models for more difficult requests.

The hybrid approach is particularly valuable for cost-conscious applications. A local model could perform preliminary work such as classifying an input or determining whether a request requires a more capable cloud model. Only requests that exceed the local model's capabilities would be escalated to a paid cloud service.

What Hardware Acceleration Does LiteRT.js Support?

LiteRT.js delivers Google's native LiteRT runtime to browsers through WebAssembly, positioning it as an evolution from TensorFlow.js for developers deploying models in the.tflite format. The runtime supports multiple hardware acceleration pathways depending on what's available in the user's device.

Google claims LiteRT.js outperformed other web runtimes by up to 3 times across CPU and GPU tests involving classical computer vision and audio-processing models. The company also reported GPU or neural processing unit (NPU) performance ranging from five to 60 times faster than CPU execution for workloads such as object tracking, transcription, and image manipulation. These benchmark results were obtained in a controlled browser environment on a 2024 Apple MacBook Pro with M4 silicon, though Google cautioned that actual performance will vary according to local GPU capabilities, thermal throttling, and browser-driver optimization.

Is This Part of a Larger Edge AI Trend?

Browser-based AI inference aligns with a broader industry shift toward edge computing, where data processing happens closer to where information is generated. The global edge AI software market is projected to grow from USD 2.40 billion in 2025 to USD 8.89 billion by 2031, at a compound annual growth rate of 24.4 percent. This explosive growth reflects increasing demand for real-time AI-powered decision-making, localized data processing, and intelligent edge computing across connected devices.

The visual data segment is expected to account for the largest share of the edge AI software market during the forecast period, supported by growing demand for real-time analytics across cameras, sensors, and imaging devices. Edge AI enhances applications such as facial recognition, object detection, and video surveillance, enabling faster processing and improved decision-making across retail, healthcare, and security environments.

Advances in TinyML, a framework for deploying machine learning models on microcontrollers and embedded devices, combined with AI accelerators and edge computing technologies, are accelerating enterprise adoption. The need for lower latency, enhanced data privacy, and reduced reliance on cloud connectivity are driving this transition across consumer electronics, retail, and customer service applications.

What Are the Current Limitations?

LiteRT.js is not a complete replacement for cloud AI. The current implementation has constraints that developers should understand. Some model operations may not be supported by every backend, input and output tensors are currently limited to certain data types, partial delegation between CPU and GPU is not supported, and large models can exceed browser or WebAssembly memory limits.

In some cases, applications can fall back from GPU or WebNN execution to the CPU. Other compatibility problems may require developers to modify and reconvert the original model. Additionally, LiteRT.js would not automatically reduce charges from an existing cloud AI product, nor would it make cloud-hosted models run locally. Developers would need to design the application to route appropriate work to a compatible local model instead of making the corresponding remote API call.

What's Next for Generative AI in Browsers?

The initial announcement of LiteRT.js emphasizes established machine learning workloads such as object detection, depth estimation, audio processing, vector search, and image upscaling rather than presenting it as a browser replacement for cloud-scale large language models. However, Google lists highly optimized on-device generative AI as a major development priority and points developers to LiteRT-LM.js, the browser-facing component of the company's open-source LiteRT-LM framework for large language model inference.

As smaller language and multimodal models become more capable, web applications could divide work among three execution tiers: local browser models for frequent and privacy-sensitive operations, edge or regional infrastructure for moderate workloads, and cloud services for the most demanding tasks. This tiered approach could reshape how developers architect AI-powered applications over the next few years.