Do Less: The Power of History Expiry on Ethereum

Christine welcomes listeners to the Infinite Jungle podcast and introduces Piper Miriam, a researcher from the Ethereum Foundation. Piper has been involved in the Ethereum ecosystem since its inception in 2015, initially serving as a CTO before focusing on Ethereum development. She created tools for Ethereum, including PyEVM, a Python implementation of the Ethereum Virtual Machine aimed at enhancing usability.

The discussion highlights the challenges of client diversity in Ethereum, particularly during 2016-2017 when GETH and Parity dominated. Piper's attempt to develop a lightweight Ethereum client called Trinity ultimately failed, leading to insights about the limitations of lightweight clients and the subsequent development of the portal network. This network addresses fundamental issues with existing peer-to-peer infrastructure and raises questions about who will manage Ethereum's history as clients begin to drop it. Alternatives like Aero files and archival formats are mentioned, but concerns about their reliability, especially when hosted by large companies, are noted.

The conversation shifts to the evolving nature of Ethereum nodes, with current execution nodes being homogenous. Future nodes may specialize in different functions, leading to a fragmentation of node types. This fragmentation could redefine what constitutes an Ethereum node, allowing for various operational choices. History expiry plays a crucial role in this redefinition, enabling some nodes to maintain full copies of history while others may not need to store all state data.

Portal is presented as the only known decentralized option for peer-to-peer storage of Ethereum history, contrasting with centralized options, which, while fast, come with trade-offs. The potential fragmentation of execution nodes raises concerns about uniformity in node behavior, complicating the Ethereum protocol and increasing testing burdens as operators manage different operational modes. The term "fragmenting" is discussed, with a search for a more neutral term to describe the phenomenon. The need for changes to reduce node size is acknowledged, which will require adjustments in assumptions and increase the testing load for operators.

Speaker 1 expresses skepticism about the long-term fragility of Ethereum nodes, emphasizing that most operations depend on current chain data rather than historical data, which is mainly for archival purposes. They note that the lack of incentives for running nodes is unlikely to change, but smaller nodes could emerge, potentially increasing network size and reducing resource requirements. The speaker highlights the need for a diverse network with various operators, including super nodes and those maintaining full blockchain copies, to ensure health and stability.

Speaker 2 points out that many Ethereum nodes are operated by validators focused on staking rewards, which limits client diversity. They question whether history expiry will encourage a shift towards lighter clients. Speaker 1 shares optimism about increased client diversity and the emergence of lightweight clients, which could provide more options for node operators. They also reflect on the challenges of collective action in the Ethereum ecosystem, stressing the importance of understanding motivations to foster collaboration.

The discussion on client history reveals concerns about whether other clients will adopt history expiry, which could enhance client diversity and allow validators to upgrade their hardware. Speaker 1 notes that most nodes exist due to validator requirements and expresses a desire for Ethereum nodes to be beneficial for developers, not just for infrastructure. They highlight that developers often rely on centralized RPC providers instead of running full nodes, and they are curious if a new local portal-based RPC provider could incentivize more developers to operate their own nodes.

The speaker believes that creating user-friendly node options could diminish reliance on incentives, which can lead to issues like Miner Extractable Value (MEV). They emphasize the need for tailored nodes for specific use cases, marking a significant shift in Ethereum's structure that has not been widely discussed. Speaker 1 expresses interest in the governance process and timeline for implementing history expiry, questioning the delays and lack of discussion around it.

History expiry is characterized as a critical yet underappreciated topic, aimed at reducing client workload by eliminating excessive data retention. The implementation faced delays due to the need for alternatives to mature and the necessity of a forcing function, such as the two-terabyte hard drive limit. Execution client teams prioritize protocol development over application-focused use cases, while the newly introduced Portal Network aims to address application needs more effectively.

The Portal Network is envisioned as a decentralized, peer-to-peer storage layer that focuses on robust data management and serves APIs for data access without being hindered by protocol changes. The Portal team, consisting of around 20 engineers, is developing multiple clients, with funding primarily from the Ethereum Foundation, though a more diverse funding model is needed for sustainability. The project is designed to be a multi-client effort rather than a single implementation.

Concerns are raised about meeting the May 1st deadline due to EIP-6110's impact on validator deposit transactions, leading developers to suggest postponing the history expiry deadline until after Pectra's Mainnet launch. There are significant issues regarding the lack of testing on the Sepulia Testnet and questions surrounding the portal network's audits, contributing to uncertainty about consensus on the Ethereum wire protocol versioning.

Despite these challenges, there is confidence in achieving history expiry this year, with an understanding that rushing to meet deadlines could compromise product readiness. The consensus is that delaying the Mainnet Go Live until after Pectra is a sensible approach, and there is strong support for conducting thorough trials on testnets prior to the Mainnet launch. Each client team is assured to conduct their own testing to ensure proper functionality, with most clients opting for snap sync rather than deep history for syncing.

The discussion highlights the need for a rational approach to protocol development, moving away from arbitrary deadlines. Developers express feelings of burnout from the pressures of code changes and governance processes, and there is an inquiry into what might drive a developer away from Ethereum protocol development, referencing past uncertainties in the portal network. The emotional weight of potential failure is acknowledged, with concerns about losing social capital and credibility.

The speaker reflects on their commitment to Ethereum's core mission of contributing positively to the world, noting that a lack of credible opportunities to work on public goods within the ecosystem could diminish their interest. They draw a comparison between the appeal of Ethereum and traditional finance, suggesting that without a meaningful purpose, the latter may seem more attractive. The hope is expressed that Ethereum will maintain its core values and mission throughout its evolution.