Mike McCormick: AI Acceleration vs Risks, Funding Global Resilience, AGI scenarios, U.S. vs China

Mike McCormick shares his transition from policy to venture capital, driven by a deep interest in AI and the potential for achieving AGI within his lifetime. He highlights the transformative impact AI could have on humanity, contrasting immediate applications in Silicon Valley with broader societal implications. The COVID pandemic has shifted perspectives, emphasizing the need for significant change. After leaving his previous firm, he founded Halcyon, a nonprofit and fund focused on AI security, safety, and biosecurity, addressing risks like misuse by authoritarian governments and the dangers of superintelligence.

Halcyon has incubated 16 organizations and raised around $250 million, with a recent addition of a VC fund to support mission-aligned companies. Mike's unconventional approach starts with a nonprofit to build relationships with researchers and lawmakers. He discusses the importance of "defense in depth" in AI security, which includes careful dataset selection and real-time cybersecurity measures. The conversation also touches on the need for resilience strategies beyond AI, such as stockpiling PPE and developing medical countermeasures.

The tension between rapid AI development and safety concerns is a central theme, with accelerationists advocating for swift advancements to maintain a competitive edge over China, while others call for a pause to address risks. A two-by-two matrix is introduced to evaluate AI strategies, contrasting development speed with control centralization, raising concerns about the implications of centralized superintelligence.

The discussion highlights the challenges faced by early-stage founders in the AI safety community, particularly those motivated by societal impact. The conversation also addresses the cybersecurity risks Fortune 500 companies face when integrating AI, emphasizing the role of the insurance industry in enhancing safety through partnerships. Founders are prioritized based on their genuine commitment to societal impact, with a focus on governance and mission orientation.

Concerns about the geopolitical implications of AI development, particularly between the U.S. and China, are raised, including the strategic importance of Taiwan. The risks of overshooting or undershooting in AI development are discussed, with potential outcomes ranging from military power shifts to economic instability. The conversation emphasizes the uncertainties in AI development, including data availability and operational challenges in sectors like healthcare.

Participants express skepticism about Universal Basic Income as a solution to potential mass unemployment due to AI advancements. They discuss the rapid obsolescence of technology and the need for faster advancements, while also recognizing the transformative potential of AI. The unpredictability of which AI company will succeed leads to suggestions for diversifying investments.

The future of AI technologies is debated, with a belief that scaling existing models could lead to AGI, while new architectures may also be necessary. The conversation concludes with a call for increased investment in AI safety and alignment, urging individuals to engage with initiatives that address major global challenges.