Autonomous Vehicle Research at Waymo

Drago Engelov, Vice President and Head of the AI Foundations team at Waymo, shared updates on the company's progress in autonomous vehicle research since September 2020. Waymo One, operational in five major cities—San Francisco, Los Angeles, Phoenix, Atlanta, and Austin—serves hundreds of thousands of rides weekly, with plans for expansion. The company has driven over 100 million autonomous miles, demonstrating a safety performance that shows Waymo vehicles are significantly less likely to be involved in accidents.

Engelov discussed the selection of testing cities based on their impact on operations, emphasizing the transition from suburban to dense urban environments and the focus on highway testing. The architecture of the driverless car system planned for 2025 includes advanced sensors, substantial onboard computing power, and a commitment to electric vehicles. He noted that public perception remains a challenge, as many people lack confidence in autonomous technology, highlighting the importance of transparency and community engagement to build trust.

Waymo's approach to driverless car technology involves unique modeling strategies, with a shift towards larger AI models that integrate perception and planning functions. The company is also exploring large vision language models (VLMs) that combine visual and linguistic inputs, addressing safety concerns related to prediction hallucinations. The validation of onboard models is a significant challenge, often taking longer than initial development.

The conversation highlighted the complexities of autonomy in AI, particularly in generating commands for robots and interacting within multi-agent environments. Recent advancements include tokenizing motions to model agent interactions, inspired by large language models. However, simulating realistic environments remains a challenge, as accurate simulations are necessary for predicting responses to actions.

Engineers at Waymo are utilizing advanced tools for developing multi-agent software, collaborating with major companies to drive the next generation of AI technology. Engelov emphasized the importance of scene editing, forecasting, and planning in simulations for autonomous vehicles, noting advancements in AI and machine learning that have improved understanding of model behavior.

Research findings indicate that scaling laws in motion can enhance prediction performance, although the motion space for AVs is less diverse than language models. Current research is exploring new architectures and the development of machine learning-based simulators. Innovations like Google's Genie model enable real-time controllable video generation, capturing real-world behavior.

The discussion also touched on swarming behaviors in AVs, suggesting that coordinated groups could enhance traffic flow. As AVs become more prevalent, sharing information about complex situations will be crucial. Engelov expressed enthusiasm for safety studies showing significant improvements in accident prevention, emphasizing the potential to save lives through expanded deployment of AV technologies.