Powering the AI Inference Wave with EPRI's Ben Sooter - Ep. 292
Wednesday, 4 March 2026 · 2 min read · Listen to the episode ↗
In Episode 292, Ben Sooter discusses the synergy between AI inference demands and energy grids, emphasizing the role of micro data centers in enhancing efficiency and reducing latency. He outlines challenges with energy load management and proposes a Distributed Inference model that capitalizes on clean energy. The conversation also highlights AI applications in optimizing battery storage and data analysis, signaling a transformative impact on energy reliability while addressing the unique power needs of distributed data centers.
Noah Kravitz hosts Ben Sooter, Director of R&D at EPRI, discussing the intersection of AI data centers and energy grids, with a focus on micro data centers. EPRI collaborates with over 400 companies to foster innovation in energy reliability and affordability. Ben highlights advancements in energy technology, including solar energy, battery storage, and electric vehicles, while addressing challenges in managing electric vehicle loads and solar capacity alongside rapid AI developments.
Ben outlines the distinction between large multi-gigawatt data centers for AI model training and the increasing power demands for inference applications, noting that 80% of compute capacity is consumed during inference. This shift complicates energy load balancing due to differing consumption patterns. He discusses the trend towards micro data centers, which are smaller and geographically distributed to enhance efficiency and reduce latency, drawing parallels to early Netflix distribution strategies.
The integration of micro data centers into the power grid presents challenges due to their high power demands. Ben suggests utilizing underutilized substations as potential sites, considering their excess capacity and necessary infrastructure for optimal operation. The conversation highlights the demand for data centers near populated areas, with available real estate in metropolitan environments suggesting potential for distributed data centers.
The concept of Distributed Inference is proposed, with a model of 25 megawatts distributed across five data centers to align with utility grid capabilities. This model allows for quicker deployment of services and integrates clean energy resources to manage peak demand. The role of AI in the energy sector is explored, particularly in battery storage design and the analysis of historical data for new insights.
Ben discusses two layers of AI's impact: knowledge work applicable across industries and industry-specific applications that integrate old data into current models. Examples of innovative AI applications, such as digitizing microfiche, illustrate the technology's potential in data extraction. Looking ahead, Ben expresses optimism for the development of micro data centers and the evolving landscape of AI-powered services in the energy sector.
This summary was generated from the episode transcript and can contain mistakes.