Harrison Chase of LangChain on Deep Agents, LangSmith, and Earning Trust | NVIDIA AI Podcast Ep. 297

Harrison Chase, CEO and co-founder of LangChain, discusses the founding vision of LangChain, which aims to equip developers with tools for building systems and agents around large language models (LLMs). He introduces "deep agents," a library that simplifies agent development while remaining model-agnostic and open-source. Chase notes that not all applications require autonomous agents; some enterprises prefer Langgraph, which combines LLM autonomy with directed workflows.

He emphasizes the importance of observability and evaluation for LLMs and agents, highlighting that agents have a more open-ended interaction space than traditional software, necessitating thorough evaluation. LangSmith is introduced as a platform for observability and evaluation, supporting the agent development life cycle of "build, test, run, manage." Chase explains that agents consist of a model, a harness (deep agents), and an execution environment, with NVIDIA's OpenShell providing a secure runtime.

Chase stresses the significance of auditability and traceability in establishing trust for agents in enterprise settings. Key components for building trust include observability of agent actions and running predefined scenarios for performance evaluation. He advocates for evaluation-driven development, suggesting that starting with a few scenarios is sufficient for effective assessment, with the evaluation dataset evolving based on user interactions.

While experimentation is common in AI and open-source communities, enterprises tend to be more cautious, often testing agents with a limited audience. Chase warns against lengthy development cycles and advises enterprises to be prepared for regular updates, as advancements in the field occur rapidly. He highlights the growing importance of AI agents, particularly their ability to perform larger tasks, and emphasizes the need to continually reassess AI capabilities, especially in integrating frontier and open models.

Chase discusses his collaboration with NVIDIA on a blueprint incorporating subagents that utilize various model types based on cost and performance needs. He notes that open-source models are becoming increasingly capable, with intelligence and coding proficiency essential for effectively driving AI harnesses. The example of Quen Coder is provided, showcasing its general-purpose capabilities due to its coding skills.

The conversation also delves into Open Claw, which operates continuously, raising cost considerations for frequent use of coding agents. The cost-effectiveness of open models in always-on scenarios is highlighted, alongside the need for models that can handle sensitive data. The NVIDIA Neymar Tron Coalition is introduced, focusing on developing an open runtime that integrates well with open harnesses.

Looking ahead, Chase predicts the emergence of asynchronous subagents managed by an orchestrator agent, as well as proactive background agents that draft responses for human approval. The concept of agent memory is introduced, where agents retain interactions and update their skills, while human interaction remains vital for their development. The discussion touches on agent identity, with a shift towards agents having their own identities rather than merely acting on behalf of users. Chase notes that the success of Open Claw has increased enterprise demand for similar solutions, leading to considerations around control and security in enterprise applications.