AI agents are hemorrhaging compute resources by generating text for internal decisions that require only a simple choice. OpenAI's own researchers recently disclosed spending $7,000 a day running agent workloads, much of it wasted on having large language models (LLMs) produce written answers to questions that don't need words. A new approach called Kev, released by developer Jared Palmer, eliminates this inefficiency by routing certain decisions through specialized models that skip text generation entirely.

Why Are AI Agents Burning So Many Tokens?

The problem is fundamental to how most AI agents work today. When an agent needs to decide which tool to use, whether to escalate a request, or how to rank options, it typically sends that decision to a large generative model. The model then generates text describing the choice, the agent parses that output, and only then does the actual decision get made. This process wastes compute on intermediate steps that produce no user-facing value.

Consider a tool-routing decision in a customer service agent. The agent might ask its underlying model, "Which of these five tools should I use?" The model generates a full sentence or paragraph explaining its reasoning, the agent extracts the tool name from that text, and then executes it. All that text generation was overhead. The agent only needed a pointer to one of five options.

How Does Kev Change the Game?

Kev is a family of open decision models built on Qwen 3.5, released in three sizes: 0.8 billion, 4 billion, and 9 billion parameters. Unlike traditional generative models, Kev uses a "prefill-only" architecture that processes the state, questions, and candidate options in a single forward pass, then reads decisions from a pointer head without generating any text at all.

The model supports three decision types that mirror TypeSafe's System One API: Noul for yes/no questions, Choice for selecting among candidates, and Score for ordered rankings. Developers provide the state and available options, and Kev returns probabilities across those candidates. Critically, because Kev only scores the options it's given, it cannot invent new choices that weren't on the list.

What Are the Practical Benefits for Agent Builders?

The efficiency gains compound in complex agent loops. Multiple decisions can be made against the same context in a single forward pass, with a block-causal attention mask isolating each question while the pointer head scores candidates independently. This means routing, safety checks, escalation, and ranking can all move to the decision layer, leaving larger reasoning models to handle only the open-ended work that actually requires text generation.

Palmer's documentation shows the 4-billion-parameter model processing three questions in 277 milliseconds on an M5 machine, though without a controlled comparison against Qwen generating equivalent answers on identical hardware, the exact speed advantage remains unclear in practice. The largest model, Kev-9B, reached 83.7% accuracy on the project's locked out-of-domain test.

What Limitations Should Developers Know About?

Kev is not a silver bullet. The probabilities it returns don't always reflect how confident developers should actually be in the result. Palmer found that temperature calibration can drift on unseen data distributions, a critical problem for agents that use probability thresholds to decide whether to execute an action or escalate it. Even a high-probability choice from Kev can still be wrong.

Fine-tuning also changes some of the capabilities inherited from the underlying Qwen model. Palmer's evaluations show declines on general-knowledge and arithmetic tests, particularly among the smaller models. This is consistent with Kev's specialized role alongside a general-purpose model, though its real-world performance will depend heavily on how well it handles tools, choices, and labels it never encountered during training.

How to Integrate Decision Models Into Your Agent Architecture

• Identify Bounded Decisions: Map out which decisions in your agent loop require only a choice among predefined options, such as tool routing, safety escalation, or candidate ranking. These are the best candidates for a decision model.
• Separate Decision from Reasoning Layers: Move routing, ranking, and safety checks to the decision layer using Kev, leaving your larger generative model to handle open-ended reasoning and final user-facing responses.
• Test Probability Calibration: Before deploying Kev in production, evaluate how well its confidence scores align with actual accuracy on your specific data distribution. Use probability thresholds cautiously and monitor for drift over time.
• Plan for Fallback Behavior: Since Kev can still choose the wrong option from a list, design your agent to escalate or retry when confidence falls below a threshold, rather than blindly trusting high-probability outputs.

How Does Kev Compare to Existing Approaches?

The decision-model approach predates Kev. TypeSafe introduced Jev earlier in September 2026 as part of its System One platform, using the same Noul, Choice, and Score primitives. Kev implements TypeSafe's /v1/systemone request and response format, so applications built against the API can point to a local Kev server instead.

The biggest difference is licensing. Palmer released Kev under Apache 2.0 with the model weights, training code, and evaluation tooling, giving developers the option to run and train it on their own infrastructure. Jev's weights and training data aren't public, which makes direct performance comparisons difficult because differences between the models can't be isolated to architecture, size, or training.

For applications that make only a handful of bounded decisions, constrained decoding on a model that's already running may be simpler than adding another model to the stack. But agent loops make those decisions constantly, moving through routing, ranking, safety checks, tool selection, and escalation before generating much user-facing text. This pattern is showing up across model architectures: stripping out unnecessary computation when the task doesn't require it.

What Does This Mean for the Future of Agentic AI?

The rise of specialized decision models signals a broader shift in how agentic AI systems will be built. Rather than treating a single large language model as the brain of an agent, developers are learning to compose multiple specialized models for different tasks. When a decision only requires a choice or probability, Kev can handle it directly while leaving open-ended reasoning and final responses to the larger generative model.

This modular approach has real economic implications. If OpenAI's agent workloads cost $7,000 per day, and a significant portion of that is wasted on unnecessary text generation, switching to decision models for bounded choices could dramatically reduce operational costs at scale. As agent loops grow more complex and organizations deploy fleets of agents, the efficiency gains from skipping unnecessary generation will compound.