Most AI tutorials teach you how to build a single agent that answers questions or runs searches, but they skip the engineering layer that makes multi-agent systems reliable enough for production. A new handbook from FreeCodeCamp reveals the infrastructure challenges that separate hobby projects from systems that actually work in the real world: state recovery after crashes, standardized tool access across integrations, cross-framework agent coordination, and quality monitoring.

What Problems Actually Require Multiple Agents?

The first question most developers skip is whether their problem even needs multiple agents. Adding agents creates real costs: more moving parts, more failure points, shared state that can corrupt from multiple directions, and debugging that requires following execution across process boundaries. A single agent with good tools is often simpler, faster, and more reliable.

A problem warrants multiple agents when it has genuinely distinct specializations. This means subtasks so different in their tools, language model call patterns, temperature requirements, or failure modes that combining them into one agent creates more problems than it solves. The handbook identifies specific conditions that justify the coordination overhead:

• Tool Separation: One part of the workflow needs filesystem access, another needs database writes, and a third needs to call an external API, creating natural seams for agent separation.
• Temperature Requirements: Structured planning output wants low temperature for consistency, creative explanation wants slightly higher temperature for variety, and grading wants low temperature for analytical consistency.
• Failure Boundaries: One subtask can fail without stopping the others; an agent that plans a curriculum can succeed even if the quiz grading service is temporarily down.
• Deployment Independence: Different parts of the system might need to run at different scales, be updated independently, or be built by different teams using different frameworks.

None of these conditions by themselves mandate multi-agent architecture. Two of them probably do. All of them make a strong case.

How to Build Production-Grade Multi-Agent Systems?

The handbook demonstrates a complete architecture pattern using four technologies that tackle infrastructure problems at the protocol level:

• LangGraph: Handles stateful agent orchestration and manages the execution flow between multiple agents.
• Model Context Protocol (MCP): Provides standardized tool integration so agents can access external tools without proprietary adapters for every integration.
• Agent-to-Agent Protocol (A2A): Enables cross-framework agent coordination, allowing agents built with different frameworks like LangGraph and CrewAI to communicate.
• Ollama: Runs local language model inference without API keys or ongoing cloud costs.

The handbook walks through building a complete Learning Accelerator system that plans study roadmaps, explains topics from personal notes, runs quizzes, and adapts based on results. This architecture pattern runs in production today for sales enablement, compliance training, customer support, and engineering onboarding.

The system includes four agents coordinated by LangGraph, two MCP servers giving agents access to external tools, two A2A services allowing cross-framework agent delegation, Langfuse capturing full traces, and DeepEval running automated quality checks. Each layer incrementally builds on the previous one, and by the time the system is complete, developers understand not just how to wire these technologies together but why each one exists and what production failure mode it prevents.

What Model Size Actually Works for Tool Calling?

One critical detail that separates working systems from broken ones is model size. Agents call tools by generating structured JSON arguments. A model that hallucinates tool names or misformats arguments fails silently: the tool call doesn't execute, the agent loops, and you hit the iteration limit without a clear error.

Models under 7 billion parameters produce these JSON formatting errors frequently. The 7 to 9 billion parameter range is the minimum viable tier for reliable tool calling in production. The handbook recommends Qwen2.5 7B as the minimum for fully functional systems, though Qwen2.5-coder 32B provides the best tool-calling reliability. Even on CPU-only hardware, the 7B model works but runs 5 to 10 times slower than GPU-accelerated inference.

This matters because it means you can't just grab the smallest model available and expect it to work reliably in production. The infrastructure layer includes not just the orchestration framework but also the model selection that makes tool calling actually function.

Why Local Models Still Struggle With Real-World Tasks?

Despite the promise of running AI locally with Ollama, real-world testing reveals significant limitations. One developer installed Ollama on a Raspberry Pi 5 and tested three popular models: Llama, Gemma, and Deepseek. The models responded quickly, with Gemma taking only 2 to 3 seconds to start writing and about 30 seconds to finish a complete response.

However, accuracy was inconsistent. All three models correctly answered basic questions like the meaning of "IT" and "CISSP" abbreviations. But when asked about the 2013 Ylvis meme song "What Does the Fox Say," none of the models recognized the reference. When asked about Bill and Ted's Excellent Adventure, only Llama knew what the question referred to, and it mixed up the actors. When asked who the 60th president of the United States was, all three models hallucinated incorrect answers.

The developer concluded that while local AI on embedded devices represents the future, the technology isn't fully ready for production use yet. The models are still in their infancy, and expecting human-level quality from a model with only 1 billion parameters is unrealistic given where the technology currently stands.

What's the Difference Between Ollama and Orchestration Layers?

A common confusion in the local AI ecosystem is conflating Ollama with orchestration tools like OpenClaw. They occupy different layers of the stack. Ollama runs language models locally for inference without API keys. OpenClaw, by contrast, is an orchestration layer that coordinates AI tasks across 50 plus messaging channels like WhatsApp, Slack, Discord, Telegram, and iMessage.

OpenClaw doesn't run language models locally; it orchestrates them. The architecture is a hub-and-spoke system: a local WebSocket server called the Gateway running on port 18789 routes messages from your channels to Agent Runtimes that call model APIs from Anthropic, OpenAI, or Google and execute tool calls on your system. You can use both together: OpenClaw calls Ollama's API for local privacy while maintaining the unified interface across messaging platforms.

This distinction matters because it clarifies what problem each tool solves. Ollama handles the inference layer. OpenClaw handles the orchestration and multi-channel coordination layer. They're complementary, not competing.

Key Takeaways for Building Local AI Systems

The infrastructure layer that makes multi-agent systems production-ready involves far more than just choosing a language model. It requires standardized protocols for tool integration, cross-framework coordination, state management, and quality monitoring. Model selection matters: 7 billion parameters is the practical minimum for reliable tool calling. And understanding the difference between inference engines like Ollama and orchestration layers like OpenClaw prevents architectural confusion when building real systems.

The handbook demonstrates that the domain changes, but the infrastructure patterns don't. Whether you're building sales enablement agents, compliance training systems, customer support automation, or engineering onboarding tools, the same architectural principles apply. The key is solving the engineering layer that most tutorials skip entirely.