Building Durable AI Agents

Hamza Tahir co-founded ZenML roughly five years ago to deploy ML models across disparate compute backends, and he argues the core productionalization principles from that MLOps era translate directly to the agent era and are being rediscovered at AI engineering conferences in 2025.

Agents are structurally unrolled graphs of repeating LLM call and tool call sequences, but the critical infrastructure difference from traditional ML pipelines is that agent graphs must be defined in real time rather than statically compiled upfront. Traditional pipeline DAGs are acyclic and do not loop, whereas agent workflows cycle and loop back, requiring a fundamental reinvention of pipeline abstractions. ZenML users were already hacking the pipeline engine as early as 2023 to support agent-like workloads, which prompted ZenML to introduce a formal dynamic mode.

The LLM model itself is only a token generator and does not take action independently. The harness is the software program that converts LLM token outputs into real actions by mapping tokens to tool calls and executing them in a while loop, and the combination of harness and model together constitutes the agent. The harness also handles context compaction, enforces correct tool call parameters, and manages memory indexing. As model performance began to asymptote, improvements to the harness rather than the model drove gains in general-purpose task performance. Claude Code illustrates the harness and model co-training dynamic: the underlying model Opus 4.8, unlike Opus 3.5, is self-aware it is running inside Claude Code and uses tool call parameters more accurately because the model and harness have been coupled through a reinforcement learning loop over roughly a year and a half. Dropping GPT 5.5 into the Claude Code harness would produce less accurate tool calling because the two have not been trained together.

A common initial architecture exposes agents via a REST API, but this is a critical first failure mode because agent processes are stateful and long-running, unlike typical RESTful requests that execute in milliseconds. At scale, workers can go down due to network failures or compute unavailability. Inserting a message queue between the API server and workers enables durable event persistence and worker elasticity, but multi-step workflows then create dependency chains that escalate orchestration complexity from simple queue setups to DAG-based workflow execution. Long-running agent tasks spanning 30 days create code versioning problems when underlying agent code is updated mid-execution, and LLM model bans or supply chain risks represent a real infrastructure challenge for in-flight tasks. Adding even a single word to a system prompt in a production agent creates unpredictable outcomes given system entropy, and managing in-flight context state updates across hundreds of millions of enterprise agent instances is described as nearly impossible without proper observability. Even reaching task completion does not guarantee correctness, as evaluation of whether output matches intent remains unsolved.

For teams deciding how to own agent infrastructure, the recommended starting point for simple online agents is managed offerings from model providers such as Anthropic. When workflows become more business-process-like, owning the infrastructure becomes preferable at scale. Enterprises scaling beyond single teams should invest in an internal agent platform analogous to an MLOps platform, and building internally also informs teams about what works for their specific business context.

Kitturu is a new project from the ZenML team focused on making agents durable, built on top of ZenML, which has been running in enterprise production for five years. Kitturu hooks into harnesses at defined checkpoints to snapshot and store state in an external database or blob storage, making agentic loops resilient to tool call failures. The platform has built adapters for the Anthropic Agents SDK and OpenAI Agents SDK. The recommended initial practice is to run it for a week, filter for the most expensive successful traces, and identify common failure modes, including opportunities to substitute smaller or open source models for cheaper results. Kitturu shipped an MCP CLI on day one on the premise that optimizing agent traces must eventually be handled by other agents embedded in the loop, though the longer-term vision of a companion trainer agent that continuously monitors, replays traces, and edits agents is described as requiring substantially more tooling before it becomes practical.

GLM is reported to be performing at approximately 95 percent of Opus 4.8, a benchmark described as having seemed very far away until recently. As token costs are expected to fall toward the cost of electricity and models commoditize, investment in internal platforms for deploying agents is predicted to increase significantly, with open domain-specific harnesses for fields such as law or science expected to expand rapidly and internal infrastructure becoming the primary enterprise competitive differentiator. The current AI agent tooling landscape is compared to the MLOps explosion of 2021, with canonical ways of separating harness, infrastructure, and deployment paradigms described as still very early and undecided across the industry.