The Future of AI Infrastructure with CoreWeave

CoreWeave is building an AI-specific cloud platform from the ground up rather than adapting a general-purpose cloud, and Corey Sanders, who spent 20 years at Microsoft on Azure before joining CoreWeave as Senior Vice President of Product, argues that a decade of inherited assumptions from general-purpose cloud design is an active limiter for incumbents. His hiring strategy at CoreWeave centers on finding people willing to challenge those Azure-era assumptions, which he frames as a deliberate response to the innovator's dilemma facing large cloud providers.

Training workloads differ fundamentally from standard public cloud compute because they require large tranches of deeply interconnected GPUs linked via specialized networks such as InfiniBand or RoCE, along with specialized storage, and cannot be deployed on-demand the way fungible general-purpose compute can. CoreWeave customizes caching, Kubernetes orchestration, and bare-metal GPU deployment specifically for these workloads, with the economic justification being that customers will pay a premium to maximize utilization of expensive GPUs. GPU failures and storage loading speeds are the key bottlenecks that slow training jobs and increase cost, and a single slow GPU within a cluster spanning hundreds or tens of thousands of GPUs can degrade overall job performance while being difficult to locate. CoreWeave recently launched a GPU straggler detection capability to address this, and Sanders describes diagnosing these failures as partly science and partly an art practiced by experienced specialists.

CoreWeave launched a product called ARIA, which stands for AI Research and Iteration Agent, designed to perform continuous analysis on training experiments and recommend next steps. The current industry standard for evaluating training runs is visually inspecting line charts across multiple runs, a workflow Sanders predicts will become antiquated as agent-led analysis replaces it. ARIA is aimed primarily at enterprises one step behind the frontier labs, where historically trained agents can reduce trial and error and accelerate the path from experimentation to production. Frontier labs are considered less addressable because they operate at the cutting edge where such tools are less applicable.

Sanders describes a CoreWeave AI loop concept in which AI applications are never finished being improved, with inference results continuously feeding back into training. A complex AI application may involve 30 to 50 different models interacting with each other, with frontier models handling deep analytical problems and smaller, cheaper models handling tasks like language translation. Specialized workloads such as gene folding for pharmaceutical companies may require specialized models rather than general-purpose frontier models. CoreWeave's integrated platform connects Weights and Biases for experiment tracking, the Weave platform for production traces, and evaluations to identify where models go wrong or are too slow. Most modifications in the agentic loop are small changes such as prompt edits or model swaps rather than full fine-tuning or reinforcement learning workflows. CoreWeave offers a model registry called Cori that tracks lineage including every updated model and parameter version for rollback and comparison, and integrates inference, tracing, evaluations, fine-tuning, and reinforcement learning services in one place.

CoreWeave's stated position is that multi-cloud is the expected reality and the company will not be the only cloud most customers use. Its differentiation strategy is delivering the best implementations of open-source and multi-cloud capabilities rather than enforcing lock-in. CoreWeave built Sunk, short for Slurm on Kubernetes, to combine Slurm job scheduling with Kubernetes orchestration and address the difficulty AI researchers face managing Slurm across unreliable infrastructure. CoreWeave subsequently announced Sunk Anywhere, which allows customers to deploy Sunk on infrastructure other than CoreWeave's own, though the company believes Sunk performs best on its own infrastructure. Sanders traces his multi-cloud conviction to personally deploying the first Linux infrastructure on Microsoft's Windows Azure.

CoreWeave added robotics-specific functionality to its Weights and Biases platform, including visual experiment tracking that displays robot movement rather than standard line charts, and acquired a company called Monolith to support industrials and manufacturing workloads. Storage platform caching and other software services built on Kubernetes operators are being enabled to run on edge and on-premises environments.

Sanders predicted that AI engineering teams will eventually exist at every enterprise, surpassing cloud engineering teams in size, with cloud engineering becoming a legacy supporting function. He also predicted that AI-driven interfaces responding to visual cues such as facial expressions will become the norm within five to seven years, and that the transition to pervasive AI will happen faster than cloud adoption, which took roughly 15 to 20 years to fully mature, while acknowledging his directional estimates could prove wrong.