Building Durable AI Agents

Production AI agents still require the core disciplines of ML Ops: safe execution, reliability, retries, and recovery from failure, Hamza Tahir argues. Unlike a conventional DAG, an agent workflow can dynamically add steps, branch, preserve state, and loop through LLM and tool calls. The model does not take actions on its own: software around it interprets structured outputs and tool calls. That harness can also manage context, validate tool parameters, and support memory and indexing, so improving it can expand what a general-purpose agent can do without changing the model.

Durability depends on managing state, checkpointing work, and making retries and replay possible. ZenML users adapted its pipeline engine for agents with dynamic steps, conditional branching, and state, working around limitations in artifact storage; ZenML later added a mode for graphs defined at runtime rather than statically compiled. Recovery becomes harder when agents run in cloud sandboxes, execute arbitrary code, make MCP calls, or load skills. At enterprise scale, agents are moving beyond individual laptops toward fleets and swarms; 100,000 executions per hour was offered as a hypothetical scale, excluding token costs.

Deployment choices depend on the workload. Simple online agents can use managed services, while complex business-processing workflows at scale may justify owning more of the infrastructure. Enterprises expanding beyond one team may need an internal agent platform with a harness, a compute target such as AWS ECS or Kubernetes, and an API for starting work. Chatbot or voice agents, personal assistants, and always-on research agents impose different requirements. Long-running stateful work cannot reliably be treated as an ordinary short-lived API request: it needs workers and task queues. A durable message broker can preserve events as workers scale, whereas launching workers directly from an API leaves work exposed to network or compute failures. As workflows acquire dependent steps, observability requirements, and in-flight updates, a simple queue can grow into a DAG orchestration problem.

Long-running work also creates versioning and continuity questions. In Tahir’s hypothetical, an agent has been working for 30 days when its code is updated: should it continue on the old version or switch to the latest code? A hypothetical government ban on a model raises a similar concern for tasks already in progress. Coding agents add another failure mode: if a filesystem or pod fails after roughly 20,000 tool calls, near completion, the system must preserve the work even if it has not been committed. Developers should not have to write defensive code for every possible infrastructure failure, but reliable recovery requires infrastructure that captures what happened.

Non-deterministic outputs can be tested, but rigorous evaluation and safe production updates remain difficult. Agent behavior is stateful and high-entropy, so even a small system-prompt change can have unpredictable effects. Developers need to inspect prior executions, replay them, and experiment against production traces; replay is imperfect when the model changes, since the new model may not reach the same point and harness differences can complicate resumption. A practical review is to run agents in production for a week, filter for expensive successful customer traces, inspect their checkpoints, and look for recurring failure modes. Fixing one bottleneck can expose another, making checkpointing useful both for durability and for understanding system behavior.

Model and harness design are increasingly intertwined: providers optimize them together through reinforcement learning, and a model may perform less well in a harness it was not tuned for. Open-harness advocates favor model-independent standards, while integrated offerings such as Claude Code illustrate the alternative; Tahir expects both approaches to persist. The industry has yet to settle canonical boundaries between harnesses, infrastructure, and deployment, and he worries that provider choices could make interoperability harder. He said a recently released GLM model achieved about 95% of Opus 4.8’s performance, while open models are making it more economically viable for enterprises to consider their own systems. He expects open, general-purpose and domain-specific harnesses to grow alongside internal deployment platforms; if models become commoditized and token costs approach the cost of electricity, that infrastructure could become a competitive differentiator.