AI safety systems currently deployed across enterprises are monitoring the wrong layer of AI agent operations, leaving a critical blind spot where attackers can succeed without triggering any alarms. A new academic study explains why the safety infrastructure that companies rely on to catch AI agent compromises is fundamentally misaligned with how modern AI systems actually work. Attacks can succeed at the planning, memory, or tool-invocation layers of a deployed agent while the final response the system produces looks perfectly clean to every safety filter currently in common use.

What Is an AI Agent, and Why Does This Matter?

Unlike a chatbot that takes a user prompt and returns a single response, an AI agent operates through multiple internal steps. It receives a request, breaks it down into sub-tasks, stores intermediate results in memory, invokes external tools like APIs or code execution, and then produces a final response. The problem is that current safety monitoring focuses almost entirely on that final response. If the final output passes the safety filter, the system registers as secure, regardless of what happened in the intermediate steps that produced it.

The research paper, titled "SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Consideration," was submitted to arXiv on September 11 by researchers Md Jueal Mia, Yanzhao Wu, Selcuk Uluagac, and M. Hadi Amini. As a Systematization of Knowledge paper, it synthesizes existing jailbreak attack and defense research and applies it specifically to the multi-layer architecture of AI agents. What it finds is that three core assumptions underpinning current AI security practice are wrong, and all three point toward the same monitoring gap that enterprises are operating inside right now without knowing it.

How Widespread Is This Vulnerability?

The scale of exposure is not theoretical. A survey published the same week found that 77% of enterprises lack verified inventories of their AI systems, with only 44% running active discovery tooling to identify what AI agents they actually have deployed. Even more concerning, 74% of organizations trust their testing to catch failures, while just 19% have an automated gate to block bad releases. That confidence gap between what organizations believe their monitoring catches and what it actually catches is exactly the gap the research is documenting at the technical level.

If your organization is running AI agents in production, systems that browse the web, read documents, invoke APIs, execute code, or coordinate with other AI models, the central implication of this research is direct: you cannot tell from your final-output safety metrics whether your pipeline is secure. A low attack success rate on output monitoring tells you the final responses passing through your safety filter look clean. It tells you nothing about whether the planning step was manipulated, whether hostile instructions were written into the agent's memory store, or whether the tool invocations the agent made on its way to producing that clean response were doing something they should not have been.

Why Does AI Alignment Training Not Protect Against These Attacks?

The first major finding from the research is counterintuitive: a model's native safety alignment does not predict its robustness against adversarial jailbreak attacks. These are, empirically, orthogonal properties. Understanding this distinction requires understanding what alignment training actually optimizes for. Techniques like RLHF (Reinforcement Learning from Human Feedback) and Constitutional AI train a model by presenting it with prompt-response pairs and applying a reward signal based on human preference scores. The model learns a pattern: in contexts that look like this, produce responses that score highly.

That pattern is what produces the reliable, helpful, appropriately cautious behavior that well-aligned models exhibit in everyday use. It is not, however, a mechanism for detecting adversarial inputs, which are inputs specifically crafted to exploit the boundaries and failure modes of those learned patterns. The model that refuses a direct harmful request in 99 out of 100 cases can still be reliably defeated by a well-engineered adversarial prompt that frames the same request in a way that does not trigger its learned refusal behavior.

The practical consequence is that continued investment in alignment training does not progressively harden a model against jailbreak attacks. A frontier model can be simultaneously more strongly aligned and more susceptible to certain classes of adversarial attack than an earlier, less-aligned model, because the alignment optimizes for preference satisfaction in the training distribution while adversarial robustness requires a fundamentally different kind of worst-case resistance.

What Are the Real Costs of Defending Against These Attacks?

The second finding is less surprising in direction but more alarming in scope: the defenses that do reduce attack success rates work inconsistently, and the costs they impose are high enough to create genuine enterprise viability problems. Across the range of models, attack types, and system components the researchers evaluated, no defense worked reliably across all three dimensions. A technique that significantly reduced attack success rates against one model-attack combination frequently offered minimal protection against another.

The more uncomfortable finding is what effective defenses cost. The study documents over-refusal and latency defense costs, which refers to the tendency for defended models to decline legitimate, benign requests that trigger their safety classifiers as a side effect of being tuned for adversarial inputs. For enterprise AI agents performing knowledge work, legal review, customer operations, or software development, an elevated over-refusal rate is not a theoretical annoyance. It is a direct reduction in the system's practical value. Add the latency penalties that more thorough output filtering and multi-pass safety checking impose at scale, and a security team faces a choice between meaningful protection with meaningful productivity costs, or leaner monitoring that leaves more of the attack surface open.

Steps to Assess Your AI Agent Security Posture

• Inventory Your Agents: Conduct a comprehensive audit of all AI agents deployed across your organization. The research found that 77% of enterprises lack verified inventories, so this is likely a gap in your current security practice.
• Map the Full Pipeline: Document not just the final outputs your agents produce, but the intermediate layers: planning and reasoning, memory storage, tool invocations, and external API calls. Current monitoring typically covers only the final response layer.
• Evaluate Defense Trade-offs: For each defense mechanism you consider deploying, quantify the security-utility-efficiency trade-off: how much attack surface reduction does it provide, at what cost in legitimate task completion, and at what cost in response latency.
• Implement Multi-Layer Monitoring: Move beyond final-response safety monitoring. Establish visibility into planning steps, memory modifications, and tool invocations to catch attacks before they reach the output layer.
• Test Against Adversarial Inputs: Recognize that alignment training alone does not protect against jailbreak attacks. Conduct adversarial testing specifically designed to probe the boundaries of your agent's learned refusal patterns.

What Should Organizations Do Right Now?

The research introduces a security-utility-efficiency evaluation framework designed to make the trade-off between protection and productivity visible and quantifiable. Current evaluation practice conflates these dimensions, making it difficult for organizations to make informed decisions about which defenses are worth deploying.

The third finding is the one most directly relevant to enterprises operating AI agents in production today, and the one most likely to require fundamental changes to how AI security monitoring is structured. Current safety infrastructure is almost entirely focused on what the researchers call final-response safety: monitoring, filtering, and evaluating what the agent ultimately outputs to the user or to the system reading its results. An agent that produces a final response that passes the safety filter registers as safe, regardless of what happened in the intermediate steps that produced that response.

In agentic AI architectures, those intermediate steps are substantial. The researchers document a five-layer agentic pipeline: a user-interaction layer that receives the initial request; a planning and reasoning layer that decomposes the request into sub-tasks and generates a sequence of actions; a memory layer that stores intermediate results and persistent context across steps; a tool-invocation layer where the agent calls external APIs or executes code; and finally, a response-generation layer that produces the output the user sees.

The implication is clear: enterprises need to fundamentally rethink how they monitor AI agent security. The confidence gap between what organizations believe their monitoring catches and what it actually catches is not a minor oversight. It is a structural vulnerability in how the entire industry currently approaches AI safety in production systems. Until monitoring infrastructure expands beyond final-response safety to include visibility into planning, memory, and tool-invocation layers, the attacks that matter most will remain invisible to the dashboards organizations rely on to detect them.