Building an agent that wows in a demo takes an afternoon. Building one you can put in front of real customers — and trust at 3am — takes engineering discipline. The gap between the two is where most agent projects quietly stall.
The demo trap
A demo runs on happy paths: clean inputs, a forgiving audience, and one chance to impress. Production runs on everything else — ambiguous requests, missing data, tool timeouts, adversarial users, and the long tail of edge cases no one scripted. An agent that's right 90% of the time feels magical in a demo and is a liability in production, because the 10% is where the refunds, the data leaks, and the angry tickets live.
The fix isn't a better model. It's treating the agent like any other piece of software that touches customers: bounded, tested, observed, and reversible.
“Ship the agent the way you'd ship a payments service — with limits, tests, logs, and a kill switch — not the way you'd ship a demo.”
Guardrails & policy
Guardrails define what the agent is allowed to do, independent of what the model decides to do. They sit between the agent's reasoning and the real world, and they are the single highest-leverage investment in agent safety.
- Tool scoping — give each agent the narrowest set of tools and permissions it needs — read-only by default, writes behind explicit allow-lists.
- Input & output filters — validate and sanitise both what comes in and what the agent emits, including prompt-injection and PII checks.
- Spending & rate limits — cap actions, tokens, and external calls per session so a runaway loop can't become a runaway bill.
- Policy layer — encode business rules (refund ceilings, approval thresholds) as code the agent cannot talk its way around.
Evals before you ship
You wouldn't ship a backend without tests. An agent is no different — except its “tests” are evals: a curated set of realistic tasks scored against expected outcomes. Evals turn “it felt better” into a number you can defend.
Golden tasks
A versioned set of real scenarios with known-good outcomes you run on every change.
Adversarial cases
Injections, jailbreaks, and malformed inputs that should fail safely, not loudly.
Regression gates
Block a deploy if the score drops — the same bar you hold normal code to.
LLM-as-judge
Automated grading for open-ended answers, spot-checked by humans to keep it honest.
Run evals in CI. A prompt tweak or a model upgrade that improves one task often silently breaks three others; only a standing eval suite catches that.
Human-in-the-loop
Autonomy is a dial, not a switch. The safest rollouts start with the agent proposing and a human approving, then graduate specific, well-understood actions to full autonomy once the data earns it. Reserve human review for the high-stakes, low-volume decisions — irreversible actions, large amounts, anything touching compliance — and let the agent own the high-volume, low-risk work outright.
Observability in production
When an agent misbehaves, “the model did something weird” is not a debuggable statement. You need the full trace: the prompt, the retrieved context, every tool call and its result, the reasoning steps, and the final action. Treat each agent run like a distributed-systems trace.
- Full-trace logging — capture inputs, context, tool I/O, and decisions for every run — not just the final output.
- Live metrics — track success rate, escalation rate, latency, and cost per task, and alert when they drift.
- Feedback capture — log thumbs-up/down and human corrections straight back into your eval set.
Rolling out safely
Launch the way you'd launch any risky change: shadow mode first (the agent runs but doesn't act, so you can compare its choices to reality), then a small percentage of traffic, then a staged ramp with a one-click rollback the whole time. Keep a human escalation path open on day one, and watch the dashboards you built in the previous step.
Done this way, “agent in production” stops being a leap of faith and becomes what it should be: a controlled, measured, reversible rollout — the same engineering rigour you already trust for everything else you ship.
