Key takeaways
Give an agent one narrow job before you give it more.
Ground every decision in sources it can cite.
Judge it by finished tasks, not by confident replies.
A demo proves an AI agent can do a task once. Production proves it can do that task a thousand times, on messy data, while your team sleeps. The gap between the two is rarely the model. It is everything around it.
Start with one job, not one assistant
The agents that reach production have a narrow brief. Refund requests under a set value are a better first job than customer support as a whole. A clear scope makes success measurable and failure easy to spot.
Write the job down the way you would for a new hire: what the agent receives, what it may change, and when it must stop and ask.
Give it context it can cite
Agents fail quietly when they guess. Ground every decision in your own policies, tickets and records, and require the agent to show its sources. If a person cannot trace an answer back to a document, the answer should not ship.
Keep permissions boring
Agents should inherit the access of the team they work for, nothing more. Read access is easy to grant. Write access deserves a limit, a log and a review path.
Measure the outcome, not the output
A confident reply is not a resolved task. Track whether the work actually finished, how often a person stepped in, and what each resolved task cost. Those three numbers tell you when an agent has earned more autonomy.
Design for the bad day
Every system fails eventually, so decide what failure should look like before launch. A tool times out, a record is missing, a customer asks something strange. In each case the agent should stop cleanly, explain what it knows and hand over with the context intact.
Run this as a drill. Pick the ten worst cases from last quarter and replay them. If the agent guesses its way through any of them, you have found your next piece of work.
Ship in small circles
Production is not a single switch. Start with a shadow run, where the agent works alongside your team and nothing it does reaches a customer. Compare its answers with what people actually did, then widen the rollout one group at a time.
Week one: shadow mode on real cases
Week two: live for a small internal group
Week three: live for a share of customers
Week four: full volume, with alerts in place
Each circle should have its own bar to clear. If the numbers slip, you step back one circle and fix the cause, with nobody paged at midnight.
Keep the humans in the loop you already have
Your best operators know where the edge cases live, so put them close to the work. Ask one experienced person to read a sample of runs each day during the first month. They will find problems no dashboard shows, such as a reply that is correct and still wrong for that customer.
Give them a simple way to flag a run and a promise that flags get fixed. People trust systems they can correct. Those flags also become your best test cases, because they come from the work itself and not from a list someone imagined in a meeting.
Name an owner
The last gap is rarely technical. Someone has to own the agent the way a manager owns a team: reading the failures, approving changes and deciding when it has earned more scope. When that name exists, agents improve every month. When it does not, they quietly decay.
A production ready agent, then, is a narrow job, cited context, honest metrics, a tested failure path and a person who cares how it performs next quarter. None of that depends on the newest model.
About the author
Mara Lind
Mara leads product at Falak. Before that she ran operations teams and spent years cleaning up after tools that stopped halfway.
