The Shape of Compute (Chris Lattner of Modular)

Chris Lattner from Mojo Modular discusses the evolution and goals of Modular, focusing on simplifying AI programming and addressing complex issues related to heterogeneous compute. The team aims to meet high performance standards, particularly in comparison to Nvidia GPUs, while avoiding CUDA and tackling fundamental problems that others deemed impossible. Modular's significant release in December marked a transition from research to practical application, leading to an update that included function calling features and support for 500 models, as well as compatibility with H100 and upcoming AMD MI300 and MI325.

Lattner reflects on the team's journey, noting the shift from CPU optimization to GPU work and the challenges faced. The first year was dedicated to proving their compilation philosophy, while the second year focused on enhancing usability through the development of Mojo and an AI framework for CPUs. He emphasizes the importance of perseverance and belief in one's vision, challenging the notion of "impossible" in competing against established technologies like CUDA. Lattner introduces Max, a platform designed for efficient model serving that does not rely on CUDA, highlighting its advantages such as faster server startup and better horizontal auto-scaling.

The necessity for a new programming language to address the limitations of existing languages like C++ and OpenCL in accelerated computing environments is discussed. Mojo aims to leverage various hardware capabilities, support advanced compilers, and maintain high performance while being user-friendly. It is designed for high-performance applications, particularly those utilizing GPUs, and integrates seamlessly with existing Python code, simplifying performance enhancements.

The conversation also touches on the integration of AI technology within the Python ecosystem, noting the advantages of GPUs for faster execution and scalability. Lattner expresses excitement about upcoming technology releases, particularly in generative AI. He emphasizes the importance of a platform team managing a shared compute pool of GPUs and the need for product teams to track workloads effectively.

The discussion highlights the advantages of modular design, advocating for faster progress and adaptability in AI systems. Lattner reflects on the motivation behind Modular, driven by the disparity in capabilities between large corporations and smaller teams, and the belief that simpler, composable solutions are essential for innovation. The evolution of AI is noted, with a focus on the complexities of inference and the challenges businesses face in managing GPU resources compared to CPUs.

Lattner emphasizes the importance of simplification in AI software to allow users to focus on core tasks and proposes their endpoint as a solution that empowers teams to integrate AI while maintaining control over proprietary data. The conversation also addresses the challenges of deploying custom models and highlights the platform's scalability and ownership of AI solutions.

The pricing model for the Max framework and Mojo is discussed, noting that while the technology is free, enterprise support and cluster management are available for a fee. Lattner contrasts the launch of Mojo with his experience with Swift, emphasizing the importance of grounding the language in real use cases. The state of AI initiatives at Apple is mentioned, with concerns about adaptability, while Google is recognized for its early innovations in AI.

The conversation concludes with a focus on the importance of delivering tangible results in AI, contrasting this with the prevalence of vaporware in the industry. Lattner highlights the challenges in GPU technology and the need for solutions like Mojo to mitigate adaptation costs, fostering an exciting landscape for innovation. He expresses optimism about future discussions and user feedback, reflecting on the emotional challenges of managing a startup and the importance of resilience in overcoming skepticism.