Chips, Memory, and Power | Pat Gelsinger

Gelsinger’s central argument is that AI infrastructure is increasingly constrained by the whole system, not just the chip: manufacturing, power, connectivity, packaging, and integration all matter, making the rack a more useful unit of analysis. He estimates about three months for design, at least three months for fabrication, and another three months for advanced packaging before a chip can work in a rack-scale system. Reaching scale and running software can take a year and a half, by which time assumptions about the workload may have changed. He cited Graphcore as a design that was not bad but was overtaken as the field moved on.

AI tools can assist with logic design, kernels, and translating high-level goals, but Gelsinger says analog design remains difficult to automate and depends on extensive silicon data. Conservative design can make automation more tractable. He compared the shift with Intel’s work around the 486, when the company developed a hardware description language and compiler and helped establish automated placement, routing, and timing practices. Current design flows can require $50 million in mask costs before prototyping, increasing the appeal of new lithography and lower-cost design methods.

Gelsinger estimates that roughly 100 specialized AI-chip efforts are underway, but expects the field to narrow to a few winners because success requires capital, scale, workloads, and software development. He questions whether fine-grained specialization for prefill, decode, or mid-fill will remain durable as workloads evolve. Reasoning models could increase demand for CPU-like compute, and some applications may return to HPC-style workloads where 64-bit precision matters. The host argued that AI agents could make specialized hardware easier to program and sustain more architectural variety. Gelsinger said large AI users will help choose platforms and that software abstractions may hide hardware differences, while warning that specialization can make scaling, upgrades, connectivity, and fleet management more difficult.

Gelsinger has worked on at least five new memory architectures, including Optane, and says nearly 100 have been attempted across the industry over 30 years without a major new type achieving broad adoption. He attributes some of the difficulty to the memory industry’s cycles of one profitable year followed by four losing years, though he sees AI’s capital and technology demands as a possible opening for innovation. He expects advances in materials such as ferroelectrics and in high-density, high-performance stackable memory. He estimates that memory-industry market capitalization rose by about $2.5 trillion over the past four years, while acknowledging the figure may be approximate. He considers HBM a poor fit for AI’s memory-compute workloads because of low bit density, bandwidth limits, and power and thermal challenges. He is skeptical that low-cost flash or MRAM can meet the speed and performance requirements, and says taller stacks worsen yield: three-, four-, or five-layer stacks may be a manufacturing sweet spot, while memory attached to a compute tile is more likely to use two or four layers than eight. He favors high-bandwidth, high-density memory close to compute over workload-specific processing-in-memory designs.

Advanced packages, in his view, must bring together compute, memory, power delivery, signal distribution, thermal management, and potentially optical connectivity. Modeling tools remain inadequate for 3D hotspots and heat dissipation, while guard banding consumes about 40% of the power envelope. Stacked systems need better heat spreading, cooling, voltage regulation, and power delivery. Gelsinger puts communication energy per bit at roughly 1,000 times compute energy per operation, and is wary of off-package memory links because optical-electrical conversion adds energy cost and loss. He favors optical I/O but not optics inside the core compute-memory complex; copper scale-up links are getting shorter-range and more expensive, with five-meter copper costing more than 100-meter optical. He expects NPO, CPO, or similar approaches to become an industry conversion point in 2028–29, but says adoption depends on an immature supply chain, laser capacity, thermal solutions, and package integration. AI’s large, predictable traffic flows could favor optical circuit switching or another optical switching approach over conventional packet networks.

Gelsinger says energy capacity is becoming a limit on economic capacity and may constrain AI data-center projects and capital commitments, though he expects a headwind rather than a halt to construction. He said U.S. coal capacity was taken offline for 10–15 years at roughly the rate renewables were added, leaving overall capacity flat, while national energy capacity has recently grown by about 4% per year. He called for more U.S. capacity and domestic energy supply chains, citing nuclear power as a desirable baseload source, Chinese dependencies in renewables, and roughly eight-year lead times for gas turbines. He sees 800-volt DC data centers as a way to remove power-conversion steps retained for historical and standardization reasons, with vertical GaN potentially converting 800 volts to 48, 12, or 5 volts in one step. Solid-state transformers and switches, along with cooling technologies such as turbines and refrigerants, are further areas for innovation.