Durable Execution and the Infrastructure Powering AI Agents

Samar Abbas, CEO of Temporal, discusses durable execution as a method for managing state in applications with multiple microservices, likening it to the chaos of a busy restaurant kitchen. Temporal, an open-source platform, ensures durable execution by maintaining the state of functions during execution, allowing for seamless recovery after failures. As AI agents become more autonomous and long-running, durable execution has become a critical requirement in modern applications.

The complexity of state management in distributed applications can lead to unpredictability, especially during demand spikes. Temporal acts as the execution authority, ensuring that every order is processed exactly once, even in chaotic environments. The concept of durable execution originated at Uber in 2015 to address challenges in transitioning from a monolithic to a microservices architecture, where the orchestration of calls across numerous services was necessary.

Uber's experience highlighted the difficulties of building stateful applications, leading to the creation of Cadence, which evolved into Temporal. This platform allows developers to focus on functionality without worrying about durability or infrastructure failures. A notable incident at Uber involved a bug that reset users' loyalty points, but Temporal's event sourcing capabilities enabled recovery by resetting workflows to a previous state.

Temporal addresses challenges related to versioning and recovery of long-lived applications, emphasizing the economic impact of workflow failures, particularly in customer interactions. The platform boasts a five nines operational SLA, highlighting its reliability, especially in light of recent cloud provider outages. Organizations are increasingly prioritizing business continuity, and leveraging Temporal enhances reliability without additional effort.

With the rise of AI in applications, the stakes for reliability are higher. Temporal powers widely used AI agents and those in development, reflecting a shift towards more automated processes. The emergence of agentic apps, which utilize AI models to plan and execute tasks, represents a new class of applications. Temporal aims to engage developers, recognizing that enthusiasm is crucial for increasing the number of applications created.

The focus is shifting from short-lived interactive agent applications to long-duration agents that require durability and long-running state management. As agents evolve from human-prompted LLMs to autonomous operators, they face significant state management challenges. Organizations are deploying hundreds of agents using Temporal, moving towards a multi-agent technology landscape with specialized agents performing specific tasks.

Context plays a crucial role in AI performance; more context leads to better outcomes. Developers achieve better results when focusing on specific tasks rather than managing everything at once. The future envisions engineers managing multiple tasks simultaneously, with AI agents enhancing productivity. Temporal maintains context across multiple human interactions, ensuring continuity in deep research use cases.

The focus has shifted to managing internet-scale transactions, with Temporal's cloud system capable of handling spikes of 150,000 actions per second. The rise of AI agents has increased the demand for scalability, prompting discussions on enhancing performance. Temporal keeps detailed execution traces for analysis and troubleshooting, facilitating collaboration by allowing teams to share workflow execution histories.

The conversation emphasizes the importance of state management and visibility in business transactions, with analytics derived from agents' execution history leading to insights into tool performance and failure rates. However, challenges persist in maintaining operational agents and ensuring observability. The market dynamics indicate a shift in value towards APIs, with the demand for automation suggesting that SaaS is evolving alongside agent technology.

Temporal's approach focuses on solving state management issues to support long-lived, mission-critical, and asynchronous agents. The speaker discusses integrating with various platforms while simplifying the developer experience. Observability remains critical, with enterprises exploring a mix of in-house and external tools to address challenges.

The introduction of coding agents offers the potential to generate strongly typed applications for business processes at a lower cost. Context management is crucial, with many startups claiming to provide context layers, but traditional data orchestration solutions often fall short. Temporal's role as an orchestrator for real-time data processing is highlighted, showcasing its high throughput capabilities.

The conversation advocates for a shift towards a swarm of agents, with a key opportunity in developing a durable Remote Procedure Call (RPC) to manage state across these agents. The emergence of specialized agents focuses on specific tasks, with notable applications demonstrating impactful uses of this technology. The leadership structure of the company reflects a transition to address execution risk, emphasizing the importance of solving customer needs and maintaining clarity on value provided to customers. Both speakers express excitement about future opportunities and a commitment to investing in R&D.