The night an agent invented a refund policy
Tuesday, 11:04pm. A customer emailed the support inbox asking about a partial refund. My shiny new Claude-powered agent — wired directly into the client’s help desk — answered in 40 seconds. Confident. Warm. Professional.
It also invented a 30-day money-back guarantee that did not exist.
By 8am the screenshot was circulating in a founder group chat I wasn’t in. By 10am I was on a call I did not want to be on. The retainer ended that Friday.
The setup was fine on paper: inbox in, LLM in the middle, draft or auto-send out. What was missing was the boring part — the trust layer. This article is that layer, spelled out, so you don’t repeat my Tuesday.
Why raw LLM agents fail in customer-facing flows
LLMs are next-token predictors dressed up as coworkers. When a customer asks something the docs don’t cover, the model doesn’t say I don’t know — it produces the most statistically plausible answer. Plausible is not true. Anthropic is fairly blunt about this in their own guidance on reducing hallucinations: give the model grounded context, force it to cite, and let it refuse.
The mistake almost every founder makes (I made it) is treating the model as the product. The model is a component. The system around the model is the product. If you’ve read our take on context engineering, this is the same argument applied to customer support: prompts don’t save you, structure does.
The three-layer trust setup I run now
This runs in n8n, but the pattern works in any orchestrator (Make, Zapier, LangGraph, a Python script — doesn’t matter). Three checkpoints. Each one makes the agent “dumber” and the business safer.
1. Retrieval first — no memory answers allowed
The agent is not allowed to answer from its own weights. Every reply must be grounded in a vetted document set.
- Store approved docs (refund policy, shipping, warranty, pricing tiers, SLAs) in a vector DB. I use Pinecone or Supabase pgvector — pick one.
- On every inbound message, run an embedding search against that store. Return the top 3–5 chunks.
- The system prompt says explicitly: You may only use facts present in the retrieved context. If the answer is not there, respond with the escalation phrase.
- Version the doc set. When legal updates the refund policy, the old chunks get purged, not layered on top.
This alone would have killed my Tuesday incident. There was no refund policy chunk to retrieve, so there was nothing to quote.
2. Citation check — no source pointer, no send
After the model drafts a reply, a second node inspects it. This is the boring gate that saves the retainer.
- Force the model to output structured JSON:
{ "reply": "...", "citations": ["doc_id_12", "doc_id_34"] }. - A validator node checks: is the citations array non-empty? Do those IDs actually exist in the retrieved chunks from step 1?
- If either check fails, the reply is discarded and the ticket is routed to a human. It never reaches the customer.
A useful mental model: the agent is a junior rep who is only allowed to speak while holding the policy binder open to the exact page. No binder page? Stay quiet.
3. Escalation node — sensitive topics route to a human
Even with retrieval and citations, some categories should never be fully automated. I run a lightweight classifier (a cheap model call is fine) that tags every inbound message. If the tag hits a sensitive list, it bypasses auto-send.
- Refunds — always human.
- Pricing changes or custom quotes — always human.
- Policy interpretation (“does my case count as…”) — always human.
- Legal, medical, safety — obviously human.
- Anything with an angry sentiment score — human, with the draft attached as a starting point.
The escalation node pings me in Telegram with the draft, the retrieved sources, and one-tap approve/reject buttons. If I approve, it sends. If I don’t touch it in 10 minutes, it defaults to “draft only, notify human agent.”
What the agent actually handles now
After the trust layer, the agent auto-sends maybe [40-60]% of tickets: order status, tracking, hours, product specs, integration questions, doc lookups. The rest gets drafted-for-human. Response times improved. Complaints dropped to zero on the automated bucket because it now refuses gracefully instead of hallucinating.
The tradeoff is real: the demo is less impressive. “Look, our agent handles 95% of tickets end to end!” sounds better than “our agent handles half, drafts the rest, and escalates anything with money attached.” But one is a magic trick and the other is a business.
The uncomfortable lesson for founders shipping AI
The founders getting burned right now are the ones who watched a Twitter demo, spun up an agent in a weekend, and pointed it at a live inbox. I was one of them. The gap between “works in the demo” and “safe in production” is entirely the trust layer — retrieval, citations, escalation, logging, evals.
This is the same reason a lot of agency AI work is quietly being pulled in-house and margins are compressing — clients can tell the difference between a wrapper and a system. If that dynamic is new to you, we wrote about it in the stealth retainer cut, and the operational side in the AI slop workflow teardown.
The checklist before any agent talks to a real human
- Vetted doc set in a vector store, versioned.
- System prompt that forbids answering without retrieved context.
- Structured output with mandatory citations.
- Validator node that kills uncited replies.
- Classifier that routes sensitive topics to human review.
- Telegram/Slack approval loop with a sensible timeout default.
- Full logging: every message, every retrieval, every send. Review weekly.
The agent got dumber on paper. The business got a lot safer. Boring is the entire job.