The Router-Planner Pattern: Architecture Behind Reliable Multi-Agent AI Systems

Why your AI system needs a traffic controller before it needs more intelligence

There is a crucial question to consider while designing AI agent systems: when do you add more agents and when do you just make one agent smarter?

The intuitive answer is usually "smarter agent." Give it better instructions, a bigger context window, more tools. Let it figure out what to do. This works fine for demos. It falls apart at scale not because the model is not capable, but because you have built a system where any failure is total. One confused agent and the whole thing breaks.

The pattern I have come to rely on is what I call the Router-Planner architecture. It's not new and when applied to LLM agents, it solves the right problems at the right layer.

Why a Single "Smart" Agent Breaks

Imagine a single agent handling all customer support for an e-commerce brand. It receives messages, decides what to do, calls APIs, writes to the CRM, communicates with the customer. One entity doing everything.

The failure modes:

Context pollution. A 30-turn conversation about a complicated exchange pollutes the context for everything the agent tries to do next. The longer the conversation, the noisier the reasoning.

Tool paralysis. When an agent has access to 12 tools, picking the right one requires reasoning over all 12. Studies consistently show that models get worse at tool selection as the tool set grows past about 6-8.

No separation of concerns. An agent optimizing for "sound empathetic to the customer" while simultaneously planning API calls and enforcing policy is doing three different jobs that pull in different directions.

Audit nightmares. When something goes wrong, e.g. wrong refund issued or wrong order returned, which part of the reasoning caused it? With one monolithic agent, you are looking at a long chain of thought and hoping you can trace it.

Enter the Router

The Router Agent has one job: understand what the customer wants and decide which specialized agent should handle it. It's not trying to resolve anything. It doesn't know the refund policy or API calls. It reads the message, classifies intent and routes. That's it.

What makes this powerful:

Narrow task, high accuracy: Intent classification is much easier than intent-classification-plus-planning-plus-tool-execution-plus-communication. An agent with a focused job does it better.

Failures are contained: If the Router misclassifies a request, the downstream specialist agent will quickly hit a dead end and either ask for clarification or kick back to the Router. The failure doesn't cascade into an incorrect action.

Fast: Intent classification can be done with a small model or even a fine-tuned classifier if your intent set is stable.

For an e-commerce context, the Router's routing table would look something like:

Return request → Return Planner Agent
Exchange request → Exchange Planner Agent
Order status / WISMO → WISMO Handler
Policy question → Q&A Agent (directly, no planner needed)
Emotional distress detected → Human escalation queue

The Planner: LLM-Driven vs. Deterministic?

Here's where the architecture gets interesting and requires a clarification: there is a real tradeoff and no clean answer. For example, Our Return Planner Agent is the specialist responsible for resolving return requests. It has to decide the sequence of steps to take. Does it: (a) Use an LLM to dynamically plan the sequence of tool calls, or (b) Follow a hardcoded deterministic workflow?

The case for agentic (LLM-driven) planning: It handles edge cases. What if a customer wants to return two items from the same order, one within 30 days and one outside? A deterministic workflow would need an explicit branch for this. An LLM planner might handle it gracefully.

The case for deterministic: Faster. Predictable. When you are targeting extremely fast responses and have a human approval flow in the loop, you cannot afford an LLM taking 800ms to decide it should check order status before checking return eligibility.

The pragmatic answer, refined after a few systems: deterministic for the happy path, agentic for exceptions.

In practice, this looks like:

1. Incoming request → Router → Return Planner
2. Planner attempts to match a deterministic workflow (e.g., "standard return within window")
3. If matched: execute the deterministic plan
4. If not matched (unusual case): invoke the LLM planner with a constrained tool set

The deterministic path covers maybe 75–80% of real requests. The agentic path covers the long tail. Both paths go through the same policy/guardrails layer. Neither can authorize actions they don't have tokens for.

The Q&A Agent: Always Talking

The Q&A Agent is the one constant in the system. While other agents are working, e.g., checking policies, calling APIs, waiting on human approval, the Q&A Agent is talking to the customer.

This matters a lot for an engaging conversation that needs to feel positive. A customer who waits 4 seconds in silence experiences that as "broken." A customer who waits 4 seconds while the agent says "Let me pull up your order details, give me just a moment" experiences it as "helpful and attentive." Same latency with very different perception.

The Q&A Agent has access to the real-time status of whatever the Planner Agent is doing. When the Planner finishes, the Q&A Agent communicates the outcome. It starts generating a response almost immediately, using whatever context it has. The Planner Agent's updates get injected into the Q&A Agent's context as they arrive.

The Summary Agent: Forgotten one

Every multi-agent architecture article I have read either buries the Summary Agent or skips it entirely. This is a mistake.

After a return is processed:

• The CRM needs a ticket update
• The customer history needs a note about what was resolved and how
• If there was a human escalation, the handoff notes need to be clean
• If the request was unusual, it should be flagged for policy review

None of this happens automatically. The Summary Agent takes the completed interaction, the tool call results, the customer conversation and synthesizes it into structured output that downstream systems can consume. Why is this a separate agent? Because the reasoning it does is different from planning or communication. It is retrospective, structured and needs to be consistent across interactions for your CRM data to be useful. Mixing this into another agent's job degrades the quality of both.

What This Looks Like in Code (Conceptually)

I won't paste a LangGraph tutorial here, there are plenty of those. But the structure in code mirrors the following architecture:

Each agent is a node. The Router Agent has conditional edges: one for each intent category. The Planner Agents each have their own subgraph: linear for the deterministic path, cyclic (allow backtracking and replanning) for the agentic path. The Q&A Agent is a parallel stream, not a sequential node. The Summary Agent is the final node before termination.

The state object flowing through the graph carries: the customer message, the classified intent, the order context fetched from the DB, the policy chunks retrieved from the Vector DB, the action log (what tools were called, what they returned) and the current conversation history.

Importantly: the state object is designed so that any agent can hand off to a human at any point and that human gets full context. It's the thing that makes the system actually trustworthy.

Why This Matters in every Agentic system

The Router-Planner pattern scales because the underlying problem is the same: you have complex incoming information, multiple possible actions, domain-specific rules that must be enforced and a need to explain what happened and why. Those requirements don't change based on whether you are handling a return request or an energy grid anomaly.

The lesson I keep coming back to: the best AI system design isn't about the AI. It is about how you structure the problem so that the AI has a narrow, focused, achievable job at every step. However, the more you treat the model as a general-purpose oracle, the more it fails you in general-purpose ways.