Verifiable Databases with Shyam & Emanuele from Provably

Anna Rose hosts a discussion with Sham and Emanuele, co-founders of Provably, focusing on verifiable databases. They explore tools that allow for proving database aspects without revealing underlying data, emphasizing real-world applications for SQL database correctness and future uses in multi-agent AI systems and on-chain data markets. Sham shares his extensive experience at IBM, highlighting the shift from centralized to distributed data integrity, while Emanuele discusses his journey from IBM Research to exploring blockchain technology.

The conversation delves into the concept of verifiable databases, particularly MiracleTree, which aims to efficiently manage large data volumes by prioritizing query complexity. They discuss the limitations of processing recursive queries and the specialization of their system for basic relational SQL, which covers a significant portion of verifiable operations. The current approach allows for direct SQL query execution, facilitating faster proof generation without circuit compilation.

They introduce a specific case AG protocol that utilizes cryptographic techniques to avoid writing specialized circuits for every query, enabling users to prove financial performance through a single SQL query in a blockchain context. The importance of query integrity for accurate business reporting is emphasized, along with the potential for users to prove facts about their private data without disclosing sensitive information.

The speakers address the convergence of traditional finance with stable coin transactions to create verifiable data and propose the idea of private data oracles for better loan rates. They also discuss the emergence of always-on devices capable of running proofs locally, enhancing the client-side proving process.

Currently, proofs are generated by Provably using a server, with plans for data owners to run their own prover or middleware in the future. The efficiency of proof sizes and verification times is highlighted, with a focus on the linear complexity of the prover relative to query complexity. They draw insights from other proof systems while noting their system's unique characteristics.

Challenges in proving transaction validity over time and the importance of privacy in database commitments are discussed. Emanuele emphasizes control over query capabilities, while concerns about data quality and the necessity of data signing by devices are raised. They stress the importance of provenance guarantees for data queries and security within cryptographic systems.

Looking ahead, Provably plans to transition to multi-agent frameworks, focusing on data integrity in agent communication. They acknowledge the current limitations in verifiability and the potential for improvement through complex game theory techniques. Insights from their findings will be shared in upcoming articles and blogs, expressing gratitude for the opportunity to discuss their innovations.