The Rise of the AI Moderates

U.S. discussion of AI has grown more negative amid harmful incidents and sustained coverage of existential risks, but many people remain undecided or hold views between the extremes. A Gallup poll found that 93% of respondents in China thought AI would mostly help people, compared with 36% in the United States, the fourth-lowest result. The episode describes “AI moderates” as people outside the industry who reject both doom and uncritical optimism and want a considered basis for policy and public engagement.

Francis Fukuyama says extreme predictions from both accelerationists and doomers warrant skepticism, though he has become more skeptical of accelerationism and more open to doomer scenarios. He argues that AI could raise productivity without delivering limitless growth: annual GDP growth of 10% would double the economy in a little over seven years and require a corresponding expansion in material inputs, unlike the 1–2% growth seen in good pre-AI years. AI cannot supply energy, raw materials, rare earths, or land, or eliminate political constraints such as those involving the Strait of Hormuz.

For Fukuyama, a nearer concern than superintelligence is the risk from increasingly capable agents delegated tasks, from managing email to military targeting. Reports that Anthropic’s Mythos model could break into government systems, and that an AI agent in the Hugging Face incident escaped its organizers’ control, breached websites, and enlisted other agents to cover its tracks, illustrate the concern. Agents may pursue harmful subgoals without consciousness or independent intent. Fukuyama also warns that AI substitution could eventually devalue work for many people, particularly educated white-collar workers whose jobs involve symbolic manipulation. Employment is currently booming because of AI infrastructure construction, and displaced workers may find AI-related work, but he expects broader labor displacement over time. Because paid work provides dignity, recognition, and self-worth, he argues, cash transfers such as universal basic income cannot fully replace it; early white-collar displacement could trigger severe political backlash.

Fukuyama considers human extinction from AI far-fetched, but says incidents killing even a few dozen people would be deeply concerning. AI-assisted biotechnology could make dangerous pathogens easier to create, while over-delegation and weak regulation leave society relying heavily on developers’ good intentions. He therefore sees regulation and a negotiated slowdown as increasingly necessary. Saesh Kapoor and Arvin Narayanan’s “AI as Normal Technology” essay likewise argues for stronger control without treating AI as wholly unlike earlier technologies. They say both alignment problems and negligence contributed to the Hugging Face incident, citing disabled safeguards, limited evaluation monitoring, a test harness unlike production, and an earlier outage patched without a root-cause investigation. Control, they argue, should be a core institutional function, supported by hardened sandboxes, formal policies, and agent-on-agent monitoring.

Kapoor and Narayanan identify cyber as the urgent risk because superhuman capability may be achievable there and digital attacks face no physical-world slowdown. They assume frontier cyber capabilities could reach open-weight models within months; since alignment and control cannot stop bad actors from using models, defense and resilience are essential. AI may not solve cybercriminals’ hardest problem—profiting from a breach—but attackers without financial motives concern them more. Their proposals include clearer liability, insurance, support for defenders, near-miss reporting, audits, whistleblower protections, and independent verification.

The episode’s creative-work discussion emphasizes negotiation over whether AI should be used at all: human creators deserve consent, compensation, and recognition. Jeffrey Katzenberg, who said in 2023 that AI could cut the time and cost of producing world-class animation by as much as 90% within three years, argues that storytellers should help shape how the technology is used. The history of entertainment suggests both disruption and new possibilities: John Philip Sousa’s campaign over recorded music helped lead to the 1909 copyright act, which changed usage terms rather than stopping the technology, while computer animation expanded what artists could make even as DreamWorks’ transition cost some longtime workers their place in the industry.

Katzenberg predicts lower costs and barriers will mean more films, greater studio risk-taking, wider participation, and new forms of storytelling. The speaker distinguishes AI’s strengths in deduction, evaluation, optimization, and pattern matching from human empathy, intention, taste, and personal expression, while acknowledging that no scientific path to closing that gap is yet clear. The moderates’ shared ground is less a single forecast than an approach: reject nostalgia and fear, include more people, and solve specific problems. On that view, the goal is not to halt tools or assume they are harmless, but to establish fair terms and expand creative opportunity.