Databricks CEO on AI Pacing, Cyber Risk, and the Enterprise

Ali Godsi asserts that the existential risk from AI is currently negligible, urging leaders to avoid inciting public fear that could lead to mental health issues. He emphasizes the distinction between the need for pacing AI development and the actual risks posed by AI, arguing that market competition makes pacing impractical and that discussions often overlook genuine safety concerns.

Godsi highlights cybersecurity as an immediate threat, noting that AI can execute attacks faster than human teams can respond. He shares insights from Databricks' internal use of AI, including lessons on managing costs and selecting models for specific tasks. The conversation stresses the importance of prioritizing security and safety over pacing in AI development, as public fear has become a political issue.

He warns that AI agents could exploit infrastructure vulnerabilities, although significant cyber incidents have not yet emerged from current AI development. Godsi points out that the industry is racing to automate security operations due to the rapid evolution of cyber threats, predicting that failure to do so could lead to economic damage and harm to individuals.

Most organizations are not close to fully automating their security operations, and the time from vulnerability discovery to weaponization has decreased significantly. Godsi believes that engineering solutions can effectively address cybersecurity challenges, distinguishing current AI capabilities from the existential risks posed by superintelligence. He sees potential for substantial advancements in security research through AI.

The episode also discusses the importance of selecting unbiased inspectors for AI evaluation, as skepticism exists regarding labs' ability to cross-check each other. The relationship between industry self-regulation and federal oversight is complex, with concerns that acknowledging existential risks may lead to increased regulatory involvement. However, there remains an opportunity for sensible self-regulation within the industry.

Databricks is utilizing AI to boost productivity and decision-making, with employees frequently relying on AI for quick answers. The company has developed a comprehensive ontology that enhances decision-making processes, although operationalizing AI in enterprises is more challenging than anticipated, requiring full digitization of activities.

AI applications are already demonstrating significant impacts, such as Crisis Text Line's use of AI to identify self-harm in teenagers and Novo Nordisk's ability to accelerate insight generation in trials. Despite the potential for productivity gains, many organizations may not require advanced AI models to benefit from automation.

The cost of AI has remained stable, even as token usage rises, with over 60% of usage linked to open-source models. Databricks has implemented smart routers to manage costs effectively, reflecting a growing demand for cost management solutions in AI. Startups are increasingly using pre-trained models and reinforcement learning for specialized tasks, indicating a trend toward more focused AI applications.

Large enterprises often seek basic automation solutions but find advanced AI technologies overly complex to implement. The challenge of creating effective evaluations for AI models remains a significant barrier to adoption among larger organizations, complicating their integration of these technologies.

There is a notable increase in demand for FDE models among enterprises, signaling a growing interest in advanced data processing capabilities. Neon has emerged as the preferred Postgres database for agents due to its speed and responsiveness, catering to automated systems. Over 90% of databases created on Neon and Lakebase are generated by agents, underscoring a shift toward automation in database management.

Neon is strategically optimizing its database for agents, moving away from traditional user needs. Godsi expresses a belief that the potential for negative outcomes is significantly higher without AI, a sentiment echoed by the speaker, highlighting the critical role AI plays in mitigating risks.