How I would Design an AI Agent System for an E-Commerce Customer Support (End-to-End)

A practical architecture walkthrough, from requirements to production.

There's a gap I keep noticing between how people talk about AI agents and how you actually build them. I first came across it while learning how to build AI systems all the way into production. I quickly realized the lack of structured guidance and a proper walkthrough on how to do it.

The talk is usually: "Just prompt the model and add some tools." The reality, once you are past the demo stage, looks nothing like that. It looks like distributed systems with latency budgets, guardrails enforced at multiple layers, policy engines you have to actually design and a routing problem that is harder than it sounds.

I spent some time recently working through a concrete design problem: an AI agent system for e-commerce customer support. The kind that handles returns, exchanges and "where is my order" queries which is scalable for a direct-to-consumer brand. What I want to do here is walk through the design decisions the way I found were the most effective.

Start With What You are Actually Trying to Do

Before drawing a single box on a whiteboard, you need constraints. For this system, I made some realistic goals such as:

• > 80% automation rate — the majority of tickets resolved without a human.
• Customer Satisfaction Rate (CSR) > 4/5 — customers need to feel helped.
• percentile_50 latency < 1.5 seconds, percentile_95 < 3.0 seconds — fast enough to feel like a real conversation

The scope is bounded: returns, exchanges and the classic "Where Is My Order". Then set the rules:

• Returns only within 30 days, item in good condition.
• Refunds under €50 get auto-approved; above that, a human reviews it.
• Exchanges follow product category logic.
• If a user explicitly asks for a human, they get one.
• Emotional escalation triggers a human handoff automatically

Encoding these rules correctly, enforcing them consistently and making sure an LLM doesn't hallucinate around them is one of the crucial parts of the Agentic System.

The Architecture: Don't Start With the AI

Edge/CDN → API Gateway → Services → Data Stores + Queues → Observability

That is the same basic backend structure you would use for any modern product. AI agents live inside that system. They do not replace it. The biggest mistake is treating the LLM like it is separate from normal software. It is not. It is just another service, with its own latency, costs and failure modes. Like any other service, it needs guardrails.

At the edge, you handle the usual infrastructure: caching, TLS termination, and security protection. The API gateway is where things start to become AI-specific. This is where you can:

• verify who the user is
• detect and remove sensitive data before sending anything to the model
• limit requests so one user cannot overload the system
• avoid duplicate actions when the same message is sent twice

Below that, you have your actual business services: Orders, Billing, Inventory, Support Tickets and so on. These are often already part of your stack. The AI layer simply talks to them.

That is the real pattern: AI is one part of the system, not the system itself.

The AI Components: Four Agents, Clear Responsibilities

The optimized system for this use case utilize four distinct agents with separated concerns:

1. Router Agent: The traffic controller. It receives incoming messages, understands user intent and decides which specialized agent should handle this. It doesn't try to resolve anything itself. It classifies and routes.

2. Q&A Agent: Always running, always speaking to the user. Think of it as the agent that holds the conversation. While other agents are doing work in the background (e.g. checking policy), this agent is keeping the customer informed and engaged.

3. Return/Exchange Planner Agent: This is the domain specialist. It decides the sequence of tool calls needed to resolve the request. "Should I check order status first? Validate the return window? Fetch exchange eligibility?" It plans and executes. Crucially, this is where you choose between a fully agentic LLM planner and a deterministic workflow.

4. Summary Agent: Writes the post-interaction summary & captures context for potential follow-ups.

The Return Flow in Detail

Let's trace through what happens when a customer says "I want to return my order."

1. Message hits the API Gateway: auth, rate limit, PII check
2. Routed to the Router Agent: intent classified as "return request"
3. Return Planner Agent activated
4. RAG query against Vector DB: what does the return policy actually say?
5. Policy/Guardrails layer checks: is this within 30 days? Is the item eligible?
6. If it passes: API call to create Return Authorization
7. If refund ≤€50: API call → automatic refund
8. If refund >€50: flag for human approval, pause flow
9. Status returned to Q&A Agent, which communicates the outcome to the customer
10. Summary Agent closes the ticket, logs it appropriately.

Ten steps. Multiple services. Two databases. One LLM you are trusting to do some of this reasoning. The places where things go wrong are predictable: policy retrieval fails (wrong chunk from Vector DB), business API times out, the model misreads the 30-day window, the human approval queue is backed up. You have to design for each of these.

Two Databases, Not One

The storage layer is one of the underestimated decisions. You need two fundamentally different types:

Relational DB: customer records, order history, return requests, audit logs. Structured, transactional and consistent.

Vector DB: FAQs, return policies, exchange rules, product categories. This is your RAG layer. When the Planner Agent needs to know if a specific product category is eligible for exchange, it retrieves the relevant policy chunk and reasons over it.

We are not using the Vector DB for memory. We are using it as a policy engine. The model is never the source of truth for the business rules but the Vector DB is. The model's job is to reason over what it retrieves, not to remember.

The Planner Agent Decision: Agentic vs. Deterministic

This is a choice that will affect your reliability and your latency budget.

A fully agentic planner uses an LLM to decide which tools to call and in what order. Flexible, handles edge cases gracefully, but adds latency and introduces the possibility of the model choosing a suboptimal path. A deterministic workflow hardcodes the decision tree: for return requests, always check eligibility first, then proceed with next steps. Faster, predictable, easier to audit.

For a production system hitting a percentile_95 latency target of 3.0 seconds, I would lean deterministic for the happy path and reserve the agentic planner for edge cases that don't fit the standard flow. The 80% of requests that are straightforward return requests get handled by the deterministic path. The weird cases like partial returns, international orders, items with exceptions should go to the planner.

Guardrails Aren't Optional

Policy enforcement has to be a layer, not a prompt. Capability tokens control what each agent can actually do. The Q&A Agent, the Router Agent should not do other tasks. These are enforced programmatically, not through instructions.

Role-Based Access Control means customers can only see and trigger actions on their own orders. The system should make it architecturally impossible to accidentally process someone else's return. When you require human sign-off on refunds above €50, that queue needs to be monitored and feed back into the agent flow gracefully.

Additionally, before executing an irreversible action (processing a refund), have the agent state what it's about to do and ask for explicit confirmation. This catches the cases where the model misread the request.

Observability: Measure Everything

Beyond the standard infra metrics, an agentic system needs:

Containment rate by journey: what % of return requests were fully resolved without human escalation?

Tool success rate: Business API calls should succeed ≥98% of the time. When they don't, what's the fallback?

Zero over-cap refunds: if the threshold is €50, zero €51 refunds should be auto-approved. Monitor this explicitly.

Emotion detection trigger rate: how often are customers escalating emotionally?

The eval layer is where you catch drift. LLMs change (even if the model version doesn't, prompts interact with context in unpredictable ways). Run regular evals on your golden test set. Track Customer Satisfaction Rate against your 4.0/5 target. Build the feedback loop before you need it.

Why This Matters

I work on multi-agent systems myself. The design decisions here aren't theoretical, they show up often during trial and error in every system that reaches production. The pattern generalizes further than e-commerce. Any system where an AI agent must operate within constraints that change faster than model weights can be updated needs a retrieval layer that's treated as an engineering problem, not an afterthought.

In the next posts, I plan to go deeper on three of these components: the Router-Planner orchestration pattern, RAG in production agent systems and the observability/evals layer that most tutorials skip entirely.

If you found this useful, follow along. I write about AI systems, ML engineering, and what it actually looks like to build things that work in production.