PrismML just released Bonsai 27B, a 27-billion parameter AI model compressed to 3.9GB that runs natively on iPhone 17 Pro devices, achieving a 14.2x compression ratio over standard models. The breakthrough uses 1-bit binary quantization, where each weight becomes either negative one or positive one, paired with a shared scale factor for every 128 weights. This delivers 1.125 effective bits per weight instead of the standard 16 bits.

The critical difference from most quantized models is that PrismML trained Bonsai 27B natively in 1-bit from the start, applying the constraint across embeddings, attention layers, and multilayer perceptrons end-to-end. Conventional post-training quantization tends to collapse on hard reasoning tasks like math, live code benchmarks, and agentic workflows. PrismML's argument is that native training degrades more gracefully, and the benchmarks mostly support that claim.

How Much Performance Do You Actually Lose?

The performance trade-offs are real but vary by task type. On math benchmarks (AIME), the 1-bit Bonsai model drops from 95.3 on the full-precision Qwen 3.6-27B base to 91.7, a 4.8% reduction. Coding performance falls from 88.7 to 81.9. However, the ternary variant (5.9GB, with weights at negative one, zero, or positive one) retains 94.6% of baseline performance. Both models support a 262,000 token context window and multimodal image input.

Speed on actual devices is modest. On the iPhone 17 Pro, the 1-bit variant runs at 11 tokens per second. On an M5 Max, that rises to 87 tokens per second. On an RTX 5090, 163 tokens per second. The phone speed is real but below the 15 to 20 tokens per second threshold that feels conversationally natural for live chat.

What Are the Real-World Limitations Developers Found?

The Hacker News discussion surfaced issues the marketing copy omitted. Tool-calling performance is reportedly worse than the base model, which matters considerably if you are building agentic workflows where the model must reliably invoke tools. Multiple developers flagged this concern. There are also reports of reasoning loops on specific tasks, particularly SQL queries. Initial toolchain support was rough; LM Studio and Unsloth did not support the format at launch, requiring custom llama.cpp forks.

One independent evaluation found benchmark irregularities on GSM8K, suggesting the degradation on certain mathematical reasoning tasks may exceed what the headline numbers indicate. Vision performance also came in below expectations given the multimodal claims. PrismML acknowledged the issues and directed users to Discord for support.

Steps to Evaluate Bonsai 27B for Your Use Case

• Validate Tool-Calling Performance: Test whether the model reliably invokes tools and functions in your specific agentic workflow before committing to production, since tool-calling regressed compared to the base model.
• Test SQL and Complex Reasoning: Run your actual SQL queries and mathematical reasoning tasks through the model to check for reasoning loops or unexpected degradation beyond the published benchmarks.
• Assess Context Window Needs: Confirm that the 262,000 token context window meets your document or codebase processing requirements, since most on-device models top out at 8,000 to 32,000 tokens.
• Check Toolchain Compatibility: Verify that your development environment supports the model format; initial support was limited, though llama.cpp and MLX are confirmed working options.

Where Bonsai 27B Actually Shines?

The strong case for Bonsai 27B is not speed; it is data residency. For healthcare applications, legal tooling, or any context where code and documents cannot leave the device, on-device inference is not a feature. It is the compliance path. Bonsai 27B gives you 27-billion parameter class reasoning for those workloads without a cloud dependency or a third-party processor in the inference path. This is worth more than any benchmark number in GDPR and HIPAA-regulated contexts.

The 262,000 token context is also significant. Most on-device models top out at 8,000 to 32,000 context tokens, which limits what you can process in a single pass. At 262,000 tokens, you can feed an entire codebase or document set into a single prompt on the device, offline, with no egress cost. For developers already using local AI inference for large models or privacy-first local AI in editors, Bonsai 27B slots naturally into the same workflow.

Bonsai 27B is available under Apache 2.0, meaning it is free with commercial use permitted. The model runs via MLX on Apple Silicon and iOS devices, via llama.cpp in GGUF format for cross-platform use, and through a free developer preview API hosted by Together AI for teams not ready to run inference locally. Model weights are on Hugging Face under Apache 2.0.

Apple is reportedly in early discussions evaluating PrismML's models, though nothing is confirmed and the conversations are described as preliminary. Even so, the direction is clear: the industry is pushing toward 27-billion parameter class capability at phone-native sizes. Bonsai 27B is the first model to actually deliver that, with known trade-offs that are worth testing before you ship to production.