Building the “See Something, Say Something” AI for Every Camera

Sheikhar Shrestha emphasizes the importance of recognizing suspicious precursor behaviors to prevent incidents before they occur. He aims to transform existing cameras into intelligent systems capable of detecting and reporting suspicious activities through a vision language model that analyzes images and videos. Drawing from his experience as a victim of armed robbery, he highlights the potential of AI forensics to create detailed incident accounts and enable automated responses, such as alerting security personnel when a weapon is brandished.

Sheikhar founded Ambient to integrate AI into physical security, focusing on proactive measures rather than reactive responses. The company enhances existing security systems by analyzing real-time camera feeds to identify suspicious activities and facilitate timely interventions. His early interest in security systems, influenced by childhood experiences, led him and his colleague Vakesh to recognize the potential of image captioning models in 2016 for deploying technology across all cameras.

Vakesh discusses the critical role of AI in enhancing physical security through surveillance cameras, noting the impracticality of human monitoring in large spaces. The "see something, say something" initiative underscores that incidents often have identifiable precursor behaviors. The solution lies in transforming every camera into a smart node that detects suspicious activity and alerts security teams for timely responses. He shares a past incident where security footage helped recover a stolen server, reinforcing the need for effective surveillance.

AI applications in this context include prevention, where suspicious behaviors are identified and authorities alerted, and forensics, where AI tracks movements and builds timelines to expedite investigations. The conversation shifts to technological advancements, particularly the concept of deep captioning, which connects image understanding with language models. Recent advancements in Generative AI have improved product capabilities, allowing for more effective detections and automation. Vision language models (VLMs) now utilize transformer-based architectures to better understand images, achieving a milestone where they can interpret images more accurately than humans.

Ambient has begun using open-source VLMs in their pipeline and recently announced their own VLM, Pulsar, which features built-in reasoning capabilities. The company has deployed tens of thousands of cameras and developed proprietary data to create a VLM that is 50 times more compute-efficient than existing models, significantly improving threat detection in security feeds. Privacy concerns are paramount, particularly in the enterprise sector, and the product adopts a privacy-first approach, focusing on identifying suspicious behaviors while avoiding facial recognition.

The company employs a 24/7 operations team to review alerts with low AI confidence before they reach customers, a human-in-the-loop approach that has become standard in the industry. An edge GPU server is utilized at large sites, with the company retrofitting organizations for proactive physical security. AI plays a crucial role in assessing situations, automating responses, and acting as a real-time assistant for operators, with proposed actions including lockdowns and law enforcement notifications.

The company targets large, complex organizations with significant security needs, such as corporate campuses, hospitals, and data centers. The growth in security solutions for high net worth individuals has become a significant revenue stream. The speaker recounts a significant event involving a large aerospace company where the camera system detected a fire, enabling a swift response, and another incident where a major e-commerce company successfully apprehended a perpetrator after updating their product to include breach detection capabilities.

The company adapted during the COVID-19 pandemic by repositioning its technology to serve active sectors like museums, which continued to adopt new security solutions. Physical security spending in the U.S. exceeds $100 billion, primarily on manual labor, with technology accounting for less than 10%. The company offers subscription software that monitors cameras for threats and suspicious events, facilitating automated incident responses.

Selling software in the physical security sector is challenging due to limited budgets and a lack of AI investment. The sales strategy focuses on educating prospects about the current state of physical security and presenting a vision for automation. The funding landscape for physical security has shifted, with growing investor interest and recognition of the sector's potential for AI applications. The speaker reflects on their success in a challenging environment, emphasizing the importance of perseverance and maintaining a positive mindset while navigating business complexities.