AI agents are entering production at scale, but most teams deploying them have no structured way to know which decisions actually worked. Between 70% and 95% of production agents fail depending on task complexity, according to Fiddler AI's July 2026 analysis, and 88% of agents that pass controlled demos fail in real workflows. The core problem is not that agents fail; it is that teams lack a practical system to find out which decisions failed, when, and why, without manually reviewing every single output themselves.

Why Can't Teams Just Review Everything?

The math does not work. When an AI agent handles thousands of decisions per hour, human review becomes the bottleneck. HubSpot's Customer Agent, for example, resolves approximately 65% of support conversations automatically. At that resolution rate, reviewing every closed ticket would require a team as large as the one the agent replaced. Teams face a hard choice: slow the agent down to match human capacity, or stop reviewing at all. Most choose the latter, which is where problems hide.

Real-world stakes illustrate why this matters. 1Password's research found that AI-generated security patches fail 74% of the time and cannot be deployed without expert human validation. New Orleans is testing Carbyne AI in live 911 dispatch, where misclassification could delay emergency response. In both contexts, exhaustive human review is either impractical at volume or structurally impossible in real time. A sampling and triage framework is not optional in these settings; it is the only path to safe operation.

How to Build a Sampling Framework That Actually Works?

The solution starts with defining what failure looks like for your specific agent. For a support agent, failure might mean a customer receives incorrect pricing information. For a code-generation agent, it might mean a test suite passes but the logic is wrong. For a security patch agent, it means a patch that introduces a new vulnerability. Write failure definitions as testable criteria. "The output is wrong" is not testable. "The output contains a price figure that differs from the product catalog by more than zero" is.

This is also where you set hard execution limits. An inventory reconciliation agent at an Anthropic logistics customer looped indefinitely on a non-existent product, consuming $4,000 in API costs within 90 minutes before a human noticed. A loop-detection rule and a $500 spend ceiling would have halted it automatically. Hard limits belong in your failure definition, not as an afterthought.

Once you know what failure looks like, use runtime signals to route outputs intelligently. Agent observability gives you the data you need to make sampling deliberate rather than random. Without it, you are choosing outputs to review at chance. With it, you can route based on risk signals including low confidence scores, unusual tool call sequences, outputs that fall near decision boundaries, or cases where the agent retried more than once.

Steps to Implement a Three-Tier Triage System

• Auto-Accept Outputs: Outputs with high confidence scores and no anomaly signals pass through automatically without human review, freeing reviewers to focus on riskier decisions.
• Human Review Tier: Outputs flagged by anomaly signals, including token counts significantly above baseline, tool calls in unexpected order, retries above a defined count, or confidence scores from a secondary evaluator model below a set floor, route to human reviewers.
• Auto-Reject Outputs: Outputs that violate hard limits, such as spending thresholds or loop-detection rules, are rejected automatically before they reach users or cause damage.

Not all outputs carry equal risk. A triage layer reads runtime signals and sorts outputs into these three buckets. Auto-accepted outputs are not guaranteed to be correct; they are outputs your triage layer did not flag. A stratified sample drawn weekly from that bucket tells you whether your triage rules are calibrated correctly.

Stratify by output type, by the tool sequence the agent used, and by time of day if your agent's context varies. A flat random sample will miss systematic failures concentrated in one output class. Doctolib deployed agents across 600 engineers and cut engineering cycle time from weeks to hours. At that scale, a stratified sample across code generation, test writing, and deployment tasks would surface failures in one category before they contaminate others.

What Happens After You Find a Failure?

Evaluation only improves quality if findings feed back into the system. Every failure pattern a human reviewer identifies should generate a new automated check in the triage layer. That check then catches the same pattern automatically in future outputs, freeing human reviewers to focus on genuinely novel failures. This loop is what separates a static quality gate from one that gets tighter over time. Track your triage layer's false-negative rate, which is failures that passed auto-accept, monthly. If it is not falling, your automated checks are not capturing what human reviewers are finding.

LLM evaluation tooling can run automated scorers against a defined rubric, which helps when failure criteria involve nuanced judgment rather than a simple rule. Secondary model scoring is not a substitute for human review of high-stakes outputs, but it scales well for volume tasks where the failure mode is consistent.

For decisions where a single error causes irreversible harm, the right design is a mandatory human checkpoint before the output is acted on, regardless of volume pressure. The 1Password security patch finding is a clear example: a 74% failure rate means automation cannot own the final deployment decision, only the generation step. Map your agent's output types and write one testable failure criterion for each before you instrument anything else. The difference between a runaway agent and a controlled one often comes down to whether someone defined what "failure" means before the agent went live.