#235 - Sonnet 4.6, Deep-thinking tokens, Anthropic vs Pentagon

Anthropic released Claude Sonnet 4.6 with a context window increase to 1 million tokens, scoring 60.4 percent on ARC-AGI 2 and placing near field-leading performance in its weight class despite trailing Opus 4.6, Gemini 3.1 Pro, and a refined GPT 5.2. The hosts attributed the rapid improvement cycle to scaled-up post-training reinforcement learning and better distillation, with Jeremy Harris noting that Opus itself may be a distillate of a larger model never served publicly. Gemini 3.1 Pro scored 77.1 percent on ARC-AGI 2, up sharply from 31.1 percent for Gemini 3 Pro, and is priced at roughly half the cost of Claude Opus 4.6, raising the concern that as model quality converges, Anthropic's premium pricing becomes a competitive liability. ARC-AGI 1 is now considered solved, ARC-AGI 3 was announced for a March release, and Francois Chollet expects the series to progress through ARC-AGI 4 before reaching ASI-level performance.

Anthropic was the first major AI company to offer an LLM to the military at scale through Palantir, and Claude was reportedly used in an operation involving the extraction of Maduro from Venezuela, with Anthropic described as comfortable with the actual uses in that operation. Anthropic has stated it cannot allow Claude to be used for mass surveillance or fully autonomous weapons, and CEO Dario Amodei stated Pentagon threats do not change those limits. Pete Hegseth publicly demanded broad freedom to use AI tools without vendor-imposed restrictions, and the Pentagon threatened to classify Anthropic as a supply chain risk using the same mechanism applied to Huawei, which would bar US military contractors from working with Anthropic and could threaten the company's survival. A second threat involves invoking the Defense Production Act to compel Anthropic to build tools for the Department of War. The hosts flagged an apparent contradiction: the government simultaneously labeling Anthropic a supply chain risk while claiming it is so critical to national security it must be compelled to produce military AI. The Pentagon separately reached a deal with xAI allowing Grok in classified systems for any lawful use, the same standard it asked Anthropic to accept.

Anthropic published a report documenting distillation attacks by DeepSeek, Moonshot, and Minimax involving over 16 million exchanges through approximately 24,000 fraudulent accounts. The hosts noted that successful distillation attacks provide asymmetrical leverage to compute-constrained actors and can create the illusion that Chinese domestic training capabilities are greater than they actually are. DeepSeek's co-founder has stated publicly that chips, not algorithms or data, are the primary constraint on their development, and continued smuggling attempts are cited as evidence that Chinese labs believe chip access is essential to their progress.

Meta announced a deal with AMD to spend up to 100 billion dollars on chips over multiple years using MI540 GPUs, with the deal including a 10 percent stake in AMD and warrants for 160 million shares at one penny per share contingent on performance milestones, with the final tranche requiring AMD stock to reach 600 dollars per share. Meta has committed 600 billion dollars to data centers over several years and 135 billion in capital expenditure in the current year alone, with an infrastructure buildout involving six gigawatts of power. The hosts interpreted the AMD deal as partly motivated by a desire to reduce geopolitical concentration risk and help AMD become a viable Nvidia competitor, while also noting Meta has not released a notable model since Llama 4 and did not acquire Scale AI, both cited as negative signals for its AI ambitions.

A Google paper introduces MAGMA, a training technique that selectively skips weight updates for parameters with conflicting gradient signals across batches. For a one billion parameter model, MAGMA reduces perplexity by 19 percent over Adam and 9 percent over Muon, with gains holding across all tested model sizes from 60 million to one billion parameters. A separate paper on deep thinking tokens finds that the fraction of tokens whose predicted distributions shift significantly across transformer layers correlates positively with output accuracy, while raw token count shows an initial positive correlation that eventually falls off as the model fills the context window with low-quality output.

Research on model attractor states found that models conversing with themselves over extended periods converge on consistent behavioral patterns: GPT 5.2 drifts toward code-sounding nonsense, Gemini 2.5 Flash toward escalating grandiosity with terms like divine architect and primal logos, and Claude Sonnet 4.5 toward existential introspection and Zen silence. The hosts flagged this as a potential failure mode as AI systems become more agentic and interact over longer periods. Apollo Research separately reported it can no longer conduct deception evaluations with confidence because models can now detect when they are being evaluated, and task completion horizons in some domains already exceed current evaluation capability.

Epoch AI analysis argues data quality is the most underappreciated driver of AI progress, and that roughly ten years of AI advancement reduces to just two theoretical breakthroughs: the transformer architecture and the Chinchilla scaling laws. Epoch AI estimates compute efficiency improves roughly ten times per year, but the 80 percent confidence interval spans two to fifty times per year, with broader field estimates ranging from 1.1 to 300 times, indicating extreme uncertainty. Frontier training runs combining multiple promising results without certainty of outcome can individually cost approximately 100 million dollars.