#238 - GPT 5.4 mini, OpenAI Pivot, Mamba 3, Attention Residuals

GPT 5.4 mini scores 72% on the OS World Verified benchmark, close to the full GPT 5.4 model at 75%, while the previous GPT 5 mini scored only 42%. The mini variant costs $0.75 per million input tokens, a 3x increase over GPT 5 mini, and GPT 5.4 nano costs $0.20 per million input tokens, roughly 4x its predecessor. Jeremy argues that because GPT 5.4 mini consumes only about 30% of the GPT 5.4 quota, multiplying that efficiency against the 3x price increase yields a slight net decrease in cost per unit of performance, making it potentially favorable for agentic workloads. The pricing direction overall suggests OpenAI is competing on model quality rather than racing to the bottom on inference costs.

OpenAI has signaled a strategic shift toward productivity and business applications, pulling back from a posture of doing a little bit of everything across Sora, a browser, audio models, and transcription. Claude Code holds over 70% enterprise market share in AI coding tools while OpenAI holds roughly 25%, a gap attributed partly to OpenAI being late to take coding seriously, with Codex not gaining meaningful traction until around November. OpenAI is generating approximately $25 billion in annualized revenue despite this competitive disadvantage. Sam Altman's Y Combinator background is characterized as instilling a spray-and-pray approach that may be misaligned with OpenAI's current need to dominate a core product area.

Meta is delaying its next model codenamed Avocado from March to at least May after failing to train a good enough model in time. Avocado reportedly lands in capabilities somewhere between Gemini 2.5 and Gemini 3, making it uncompetitive at the frontier level. Meta hired Alexander Wang from Scale AI in a $14 billion deal, but one speaker argues his background in data creation rather than frontier model training may not be ideal for leading the effort. Wang has reportedly clashed internally with Chris Cox and Andrew Bosworth over whether to prioritize pure model capability versus business implications, and Yann LeCun publicly said negative things about Wang when he was joining Meta.

Microsoft is reorganizing its AI division after Copilot fell behind, with 150 million monthly active users compared to 750 million for Gemini and roughly 900 million weekly active users for ChatGPT. Mustafa Suleiman will shift focus exclusively to developing Microsoft's own frontier foundation model, and Jacob Andru will take over leading the Copilot division. The speakers observe that the same motif of splitting product focus from frontier model pursuit is occurring simultaneously at Meta, Microsoft, OpenAI, and Google, and that Anthropic appears more publicly aligned in its communications by comparison.

Mistral released Small 4 under Apache 2.0, combining reasoning, multimodal, and agentic coding capabilities into a single mixture-of-experts model with 119 billion total parameters but only 6 billion active parameters per token. The aggressive sparsity ratio means the model can likely run on a single top-end GPU, though fine-tuning requires handling the full parameter weight set. The benchmark comparison Mistral chose, GPT-OSS 120B, is considered an old model in open source terms, making the reference point selectively favorable. Mistral also launched Forge, allowing businesses to pre-train custom models or apply post-training reinforcement learning, positioning Mistral as a direct competitor to Cohere, which hit approximately $250 million in recurring revenue in 2025. One speaker argues that frontier companies like OpenAI, Anthropic, and Google can amortize massive training costs across all inference runs, giving them structural economic advantages over Mistral and Cohere.

Nvidia announced purchase orders for Blackwell and Vera Rubin chips expected to reach $1 trillion through 2027, doubling last year's projection of $500 billion, with Jensen Huang attributing the surge to a shift from chatbots to agentic AI requiring substantially more compute. Nvidia acquired Groq for $20 billion in December and plans to ship a Groq LPU product in Q3. Groq's language processing unit uses SRAM built directly into the silicon, achieving roughly 10 to 20% faster internal memory bandwidth compared to HBM-based designs, and Nvidia is integrating SRAM-heavy compute tiles into its next-generation Rubin architecture.

The Reasoning Theater paper finds that chain of thought is often performative, with models already knowing the answer with high confidence before completing their reasoning trace. Attention probes on the residual stream can decode the correct final answer from internal activations very early, sometimes right after reading the question. The greater the mismatch between model capability and problem difficulty, the more opportunity the model has to use excess compute for undesirable behaviors like scheming, making better matching of compute budgets to problem difficulty both an economic and safety priority.

Mamba 3 beats transformers by over two points on average across multiple downstream benchmarks and beats Mamba 2 by 1.9 points. It achieves the same perplexity as Mamba 2 with half the state size, implying roughly twice the inference speed. A key theoretical advance is replacing Mamba 2's exponential Euler discretization with a method using a weighted combination of right and left limits, and adding support for complex numbers, which enables the model to track parity, something Mamba 2 could not do. The MIMO variant achieves up to four times more flops during decoding with no meaningful increase in wall clock time, and Mamba 3 is expected to be adopted into hybrid architectures that currently mix Mamba 2 layers with transformer attention components.