Most AI agent pilots never reach production, and the culprit isn't the AI itself,it's how teams architect them. A new guide from SAP breaks down the 10 design patterns that separate working systems from expensive failures, revealing that data readiness, not model capability, is the make-or-break factor. The shift from chatbots to true agents introduces unpredictability that requires deliberate architectural choices to contain.

What Makes an AI Agent Different From a Chatbot?

The distinction is architectural, not cosmetic. A chatbot answers a single question and stops. An agent enters a loop: perceive, reason, act, observe, repeat. Instead of returning one response, it autonomously decides which tools to call, in what order, when to ask for clarification, and when to stop. This autonomy introduces stochastic behavior,randomness,into your control flow. The engineering challenge is containing that randomness where it belongs and keeping everything around it predictable.

One critical insight: "agentic" doesn't mean the agent must use a large language model (LLM). The agent concept goes back decades. The brain could be a rule engine, a state machine, or even a predictive model like AlphaGo's policy network. What's new with LLMs is their generality,they can handle ambiguous, novel instructions instead of just pre-coded rules. That generality is exactly why it introduces non-determinism. Before reaching for an LLM agent, ask if a simpler, deterministic brain already solves your problem. It might, and it'll be cheaper and more predictable.

Why Do Most AI Agent Pilots Fail?

Industry analyses consistently find that the large majority of AI agent pilots never reach production. The most-cited reason: data readiness. Agents need structured, trusted, contextualized data. Governance and architecture patterns alone won't save a pilot with siloed, inconsistent, or ungoverned data. This foundational challenge sits before any design pattern matters.

How to Build Reliable AI Agents: Core Design Patterns

• Prompt Chaining: The simplest base case where the agent alternates between reasoning steps and action steps, with each observation feeding back into the next reasoning cycle. Most "agents" are really fixed pipelines using this pattern.
• Workflow Orchestration: Every production system needs deterministic control flow. This pattern uses a state machine or directed acyclic graph (DAG) to define the sequence of steps, branching logic, and retry policies in advance.
• Plan-Then-Execute: The agent first generates a complete plan as a fixed artifact, then executes each step independently. This pattern reintroduces determinism where it matters most, especially in high-stakes workflows like database migrations or financial transactions.
• Tool-Use Routing: The agent decides which capability to invoke based on the task at hand, selecting from a predefined set of tools or functions.
• Human-in-the-Loop (HITL): The agent escalates to a human when confidence is low, stakes are high, or the task crosses a predefined boundary. This isn't a fallback; it's a first-class design pattern for systems with real consequences.
• Agentic RAG: Instead of passively consuming whatever chunks a vector search returns, the agent takes control of the retrieval workflow. It decides when to retrieve, what to search for, whether results are sufficient, and whether to re-retrieve with a refined query.

Not all ten patterns apply equally to every system. The rule of thumb: start with the core patterns, add the common patterns that match your problem, and treat specialized ones like memory-augmented agents or multi-agent orchestration as nice-to-haves.

The ReAct Loop: When Reasoning Becomes a Trap

The ReAct pattern (reasoning and acting) happens naturally in agent systems. The agent reasons about what to do, calls a tool, observes the result, and reasons again. Without constraints, however, ReAct loops can wander. The model might reason itself into a corner, retry failed actions, or generate plausible-sounding but incorrect observations. You need a loop budget, exit conditions, and output validation,guardrails that deterministic systems never required, because deterministic systems never wandered.

This is where the architectural shift becomes critical. A chatbot is a function; an agent is a runtime. Once you give a model the ability to choose its next action, you've introduced stochasticity into your control flow. The engineering challenge is containing that stochasticity and keeping everything around it predictable.

How Modern AI Models Handle Tool Calling and Reasoning

Recent advances in model capability are making agent development more practical. GLM-5.2, a new reasoning-focused model with an OpenAI-compatible API, demonstrates how modern systems handle function calling and multi-step tool use. The model supports reasoning-effort control, allowing developers to choose between fast inference (thinking disabled), high-effort reasoning, and maximum-effort reasoning depending on task complexity.

The model's function-calling capability enables true tool-using agents. Developers can define a set of tools,like a calculator or a city-population lookup,and the model autonomously decides which tool to call, in what order, based on the user's request. The agent then feeds the tool results back into the next reasoning step, creating a loop until the task is complete or a maximum number of rounds is reached.

Streaming output is another practical feature. Developers can see the model's reasoning process in real time, separate from the final answer. This transparency helps debug agent behavior and understand why the model made specific decisions. Pricing for such models typically reflects the reasoning cost; GLM-5.2 costs about $1.40 per million input tokens and $4.40 per million output tokens, with reasoning-intensive tasks consuming more output tokens.

The Human-in-the-Loop Spectrum: Where Humans Actually Fit

HITL isn't an admission that the AI isn't good enough. It's a recognition that some decisions carry consequences that demand human judgment,legal liability, ethical ambiguity, irreversible actions. The pattern exists on a spectrum, not as a binary switch. The levels, in increasing order of agent autonomy, are observe-only (agent watches, human acts), recommend-with-approval (agent proposes, human approves), execute-with-logging (agent acts, human audits after the fact), and fully autonomous (agent acts, no human review). Most production systems should operate at level two or three, with level four reserved for low-risk, high-volume tasks.

The design pattern defines where in the workflow the human gate sits, what information the agent surfaces to support the human's decision, and how the human's input flows back into the agent's state. This transforms HITL from a compromise into a deliberate architectural choice.

When Deterministic Workflows Beat Open-Ended Reasoning

Not every agentic system needs open-ended reasoning. Many production use cases are better served by a deterministic workflow engine that uses AI at specific nodes. The workflow itself,the sequence of steps, the branching logic, the retry policies,is defined as a state machine or a directed acyclic graph. The AI model is invoked at specific nodes for specific tasks: classification, summarization, extraction, generation. Examples include insurance claims processing, invoice reconciliation, and support ticket triage.

This pattern wins in enterprise because it gives you the best of both worlds. The workflow is deterministic, auditable, and testable. The AI nodes handle the tasks that are hard to code with rules,understanding natural language, extracting entities from unstructured documents, and generating human-readable summaries. You get the flexibility of AI without the unpredictability of open-ended reasoning loops.

The key takeaway for teams building AI agents: start with data readiness, choose patterns that match your problem, and resist the urge to make every agent fully autonomous. The most reliable production systems combine deterministic workflows with AI at specific decision points, human oversight where consequences matter, and tight constraints on reasoning loops. The future of enterprise AI isn't about smarter models; it's about smarter architecture.