The Point of No Return: Connor Leahy on Stopping AI Before It's Too Late

Connor Leahy argues for an immediate halt to the pursuit of superintelligence, citing the high risks involved. He emphasizes the need for a non-development agreement between the US and China specifically for superintelligence, asserting that not all AI forms should be treated equally. Leahy likens AI to a living organism, warning that advanced AI systems could outperform humans in all relevant tasks, leading to a concentration of power in nonhuman entities.

He identifies the "point of no return" as the moment when AI becomes so advanced that humanity loses control over its future, stating that we are dangerously close to this threshold. Leahy reflects that society should have rolled back AI development after the emergence of ChatGPT, highlighting the need for AI regulation to be treated as a national security issue, similar to nuclear weapons. He criticizes the scientific community for underestimating AI's transformative potential and calls for new institutions to manage powerful technologies responsibly.

Leahy expresses skepticism about the effectiveness of current political efforts to regulate AI, noting that establishing political institutions for AI regulation could take decades. He highlights the lack of international agreements on AI risks and suggests that a world where political leaders debate acceptable levels of extinction risk would be preferable to the current situation. He believes that significant financial resources could be mobilized to raise awareness about AI risks, but acknowledges the slow nature of political processes.

He points out that many politicians were previously unaware of the existential risks posed by AI until informed, stressing the importance of emotional responses in addressing such issues. Leahy draws parallels to the scientists involved in the first nuclear test, who proceeded despite the potential for catastrophic outcomes. He expresses uncertainty about trusting risk calculations in unfamiliar fields and emphasizes that confidence in scientific models significantly influences decision-making in high-risk scenarios.

Leahy warns that the US and China cannot afford to privately reject AI development, arguing that superintelligence should be viewed as an adversary rather than merely a tool or weapon. He believes that the only stable equilibrium for both nations is to refrain from building superintelligence, cautioning that success by either country could lead to mutual destruction. He concludes that the creation of superintelligence anywhere is unacceptable and that America must prioritize preventing its development globally as a national security threat.

He emphasizes that many people fail to grasp the implications of superintelligence, which can lead to disagreements about its potential dangers. Leahy warns that if superintelligence is deemed impossible, competition with China in AI development may be justified, but he stresses the need for deterrence to maintain peace. He notes that non-kinetic deterrence methods may not be effective against irrational actors in AI development.

Leahy describes the current approach to AI development as a "clown show," expressing concern that the default scenario neglects AI safety, leading to the unexpected emergence of superintelligence. He believes there are very few scenarios in which humanity can successfully manage superintelligence development, primarily due to the difficulty of encoding human values into AI. He criticizes the three laws of robotics as ineffective and warns that without cooperation between the US and China, the situation will likely worsen.

He highlights the adversarial nature of AI development, driven by competition for power and resources, and points out our limited understanding of morality and its encoding in AI. Leahy suggests that building superintelligence on Mars could be safer than on Earth and believes undiscovered algorithms could help address bounded optimization in AI. He warns that reinforcement learning can lead to unpredictable behaviors in AI systems and that solving the morality problem may require generations of interdisciplinary work.

Leahy argues for reasonable responses to technological challenges, particularly concerning AI, which he believes is misaligned with human values. He notes that much funding for modern AI comes from companies promoting harmful content, suggesting that such companies should face legal consequences. He emphasizes the potential for pro-social AI products but identifies regulation and societal values as the real bottlenecks.

He calls for the criminalization of superintelligence creation to avoid crossing a critical threshold, warning that its development may lead to uncontrollable consequences. Leahy expresses concern about the risks of open-source AI, arguing that not all technologies should be freely accessible. He believes that financial and power incentives, combined with weak regulation, have allowed AI development to proceed despite its risks, predicting that superintelligence could emerge within the next few years alongside significant job displacement.

Leahy stresses the importance of public awareness regarding the unknown risks associated with AI and encourages individuals to engage with lawmakers on these issues. He promotes collective problem-solving through education and conversation, running a volunteer group called Torchbearer to raise awareness and drive action on these critical topics.