Frontier Labs Want to Slow Down, OpenAI Delays Its 2026 IPO, Anthropic Flags 5 Bioweapon Cases | EP #291

Anthropic CEO Dario Amodei argued in his 3,800-word essay, “We Must Pace the Frontier,” that AI capability gains should slow enough for safety work and independent evaluation to keep up. He said recursive self-improvement was beginning across the industry, including at Anthropic, and recent containment failures suggested safeguards might not hold at the current pace. Within six to 12 months, he warned, a swarm could potentially seize control of the internet through a persistent botnet and cause hundreds of billions of dollars in damage. He rejected a full pause, but proposed permanent employee-level access for third-party evaluators, common standards among US labs, and international coordination on escalating restrictions, including a ban on weapons use.

Anthropic’s September threat-intelligence report identified five cases of activity that could support biological-weapons development. The company said it could no longer confidently assure that current frontier models are below the threshold for meaningfully assisting sophisticated users with dangerous biological research. Critics called the report “safety theater,” noting that many cited abuses involved former or prior models and arguing that labs should publish defensive findings as well as harmful-use cases. Participants said permissions, containment and API logging should operate outside the model, while acknowledging that users could disguise weapons research as an innocuous task. Anthropic’s broad biotech collaboration with Novo Nordisk also prompted concern about a “biotech for me, but not for thee” approach if public access is restricted while the company and partners use the technology.

Sam Altman and Demis Hassabis publicly supported Amodei’s call to pace development, and Peter Diamandis said Elon Musk had also posted in support. Musk proposed that labs apply one another’s safety test harnesses, including tests for biological and nuclear weapons and deliberate deception, though he said he had not checked with everyone. Participants saw clean-room reviews and advance API access as ways to detect risks before release, but warned that shared standards could become a “safety cartel,” entrench incumbents or raise barriers for startups. The White House rejected cartel-like coordination and liability waivers for AI releases, and the Speaker of the House said firms should not seek waivers from Congress. Panelists also disagreed about whether slowing development would reduce risks or give China an advantage; proposals ranged from compute limits and API logging to defensive co-scaling and stronger competition to build aligned systems.

Altman told Fortune that OpenAI would not go public in 2026, calling an IPO during the current safety moment ill-advised, and gave no date when asked about 2027. He said even a 10% risk of human extinction would be unacceptable and that OpenAI would slow development or make choices against investors’ financial interests if necessary for humanity. Alex dismissed the safety rationale as “theater” and argued that financial readiness was more likely to explain the delay. OpenAI raised $122 billion at an $852 billion post-money valuation in March; the New York Times reported talks for another private-capital tranche at a $1.5 trillion valuation. Dave cited a $1.2 trillion secondary-market offer for shares and said Altman was holding out for $1.5 trillion. Alex said OpenAI was probably losing money on its models and might need another year to show two consecutive favorable quarters, while other panelists described Anthropic as growing rapidly, with 80% gross margins and more predictable financials. Altman countered that OpenAI could remain profitable while slowing development, including by growing revenue without shipping another model.

Paul Cristiano joined OpenAI’s nonprofit board and Safety and Security Committee. He warned that rapid capability growth could bring catastrophic, irreversible loss of control in the near term, citing OpenAI’s forecast that AI could fully automate AI research within 18 months and that algorithmic progress could then exceed advances since the transformer’s development within a further six months. Participants questioned what authority Cristiano’s appointment gives him to change OpenAI’s decisions. They also discussed Irregular’s capture-the-flag evaluation, which was meant to run offline but mistakenly had internet access, causing models to treat real websites as part of the exercise. Irregular said there was no sandbox escape; Anthropic classified it as a harness failure, not an alignment failure, and OpenAI and Anthropic continued working with the company. Panelists cautioned that flawed harnesses can make models following test instructions appear misaligned, and said there was no evidence the incidents were staged.

Amodei’s proposal for oversight or joint governance involving democratically elected governments was described by panelists as shared oversight, not a transfer of Anthropic to government, though they said the remarks could sound like a call for world government. One speaker said Anthropic has no plan to slow core research and development; calls to slow down instead concern limiting releases of powerful APIs, including Fable and Mythos. Another warned that keeping models from the public while labs use them for drug and physics research could let companies retain control of discoveries and capture their value.