The Agentic Gap: What Enterprises Think vs. What Actually Works With Jeff Dalton (Head of AI and Chief Scientist at Valence)

Jeff Dalton, Head of AI and Chief Scientist at Valence, argues that the fundamental definition of an agent has not changed in roughly 50 years: an agent plans, takes actions, observes the environment, and maintains a state representation of the world. What has changed is implementation, specifically that agents now operate in language rather than small finite state trees. He marks GPT-2 as a visible inflection point where problems expected to take years were solved within six months to a year. The agentic landscape evolved through deep learning classifiers, generative foundation models, instruction-tuned models, tool-using models, reasoning models, and now multi-agent systems, with reasoning models like the O series using reflective self-talk rather than formal reasoning.

Valence is building an AI coach named Nadia, which Dalton distinguishes from an assistant on the basis of objective function. An assistant completes tasks; Nadia is designed to optimize for value over time, helping users learn, grow, and improve performance within their specific organizational context. Nadia introduces what Dalton calls productive friction, for example stopping a user from sending an email and suggesting a direct conversation instead. It is connected to organizational values while learning each user's personality, profile, and working relationships. Individual chat conversations remain private to the user and are not visible to the organization, because trust requires users to be emotionally vulnerable and transparent to achieve growth.

Memory is treated as a first-class object at Valence because long-term coaching requires remembering the right things and identifying patterns across past conversations. Valence performs coreference resolution so Nadia can distinguish between different people sharing the same name in a user's network. Users can view and edit their memory and people records, and the system surfaces what it knows about them, what it is learning, and which growth areas they can choose to work on. Dalton argues memory does not need to be present at all times and that information never used by any outputs can be pruned, with Nadia using selective memory triggering such as looking up a person only when that person is mentioned.

Valence built Nadia as a single-threaded agent with workflows controlled by code rather than as a complex multi-agent system from the start, pulling in sub-agents only where appropriate. The system is a hybrid of large language model capabilities and deterministic scaffolding designed by domain experts, using deterministic state machine management and structured reasoning traces so that when an agent gets stuck the cause is inspectable and fixable through code changes. Valence moved away from open-ended natural language prompts toward treating prompts as semi-structured objects closer to code, which allows failures to be traced back to specific lines and corrected. Dalton draws a direct parallel to software engineering, arguing that building good agent systems requires detailed logging, tracing, and component-level instrumentation equivalent to unit tests.

Valence built Nadia to be eval-first from the start. Evaluation uses human-labeled datasets to assess offline whether coaching is flowing as expected, including whether the model transitions between coaching phases correctly. Rubric pillars include quality of coaching, facilitation of change, whether users learned new things, flow of conversation, and user satisfaction and retention. Dalton warns that bad evaluation is worse than no evaluation because it causes teams to measure the wrong thing and move in the wrong direction without knowing whether they are improving. He recommends examining even ten examples with human judgment to reach roughly 90 percent confidence about what is going wrong in a system, and treats LLM-as-a-judge as useful for scale but not a replacement for direct human inspection.

Real deployment exposed issues that could not be anticipated in advance. When a large legal firm joined as a client, uploading documents triggered legal advice guardrails inappropriately. The same domain-shift problem arose when rolling out to health companies around medical advice, and after elections users came in expressing depression, requiring a response that handled emotions tactfully without triggering political guardrails. Dalton argues guardrails require defense in depth across multiple layers: foundation model guardrails, input-level rejection of prompt injection and personally identifiable information before content reaches a model, LLM-driven input guardrails, and LLM-driven output guardrails. Each component has different operating points of precision and recall and must be measured individually and in aggregate. His general principle is that if a capability really matters to the business, teams should own and build it rather than rely on off-the-shelf frameworks that become difficult to adapt when they do not behave exactly as needed.

Dalton acknowledges he does not have a good solution today for federating personal knowledge across agents in a deterministically enforceable way, and notes there are no clear standards for making personal knowledge bases portable across agents or for scoping how agents can use and share that knowledge. He points to work by Tim Berners-Lee recognizing that privacy will require new foundational layers and standards for personal knowledge bases, and predicts these standards will become increasingly important as agentic systems grow more complex. Measurement for coaching agents operates at both the conversation level and the system level, covering process quality, outcome quality, and session success, and he identifies longitudinal measurement of change over a user's journey as a highly custom and specific challenge in agentic systems.