One Brain, Any Robot: Skild AI's Skild Brain Explained - Ep. 295

Noah Kravitz hosts a discussion with Deepak Pathak and Avinab Gupta from Skilled about the development of OmniBrain, a general-purpose brain for robots. Deepak emphasizes Skilled's mission to create OmniBored Intelligence, which aims to address the data challenges in robotics, contrasting it with the rapid advancements in language models like ChatGPT. He highlights the need for a general approach to enhance robotic capabilities across various scenarios.

Avinab critiques the traditional vertical approach in robotics, which focuses on specific tasks and struggles with real-world applications. He advocates for a horizontal platform, similar to large language models, to enable broader applications and better handle corner cases through data integration. The speakers discuss the synergy in their collaboration and the inspiration behind OmniBrain, stressing the shift from programming behaviors to learning from data.

They identify three main data sources for robotics: robot data, video data, and simulation data, each with its strengths and limitations. The training process involves pre-training on video and simulation data to build robustness, followed by post-training on real-world data for precision. They draw parallels with language models, noting that general models are trained on diverse data while specialized models are fine-tuned with high-quality, task-specific data.

The conversation also touches on the challenges of testing OmniBrain, particularly in balancing general and specialized knowledge, with key requirements including accuracy, efficiency, and safety. Looking ahead, they anticipate the automation of human actions in physical environments, progressing from structured tasks to more complex scenarios. While safety concerns remain a barrier to humanoid robots in domestic settings, the rapid advancements in AI and hardware offer an optimistic outlook for the future of robotics.