Summary
- Chain Integration Process (1:40)
- Introduction to Chain Integration Process
- Steps for Integration
- Initiation in The Graph’s Forum
- Integration and Testing Phase
- Mainnet Activation
- Benefits of the Chain Integration Process
- Starting the Chain Integration Process
- GGP Updates (8:25)
- Introduction to Graph Governance Proposals
- Overview of GGP-0031
- On-chain Dispute Status and Resolution
- Removal of Owner Tax on Subgraph Upgrades
- Overview of GGP-0034
- Permissionless Payer Functionality
- Overview of GGP-0035
- Elimination of Cooldown Period for Delegation Parameters
- Further Reading on GGPs
- Graph Node Update (15:44)
- Recap of release 0.33.0
- New Features and Enhancements
- Recent Patch (Release 0.34.1)
- Bug Fixes and Substreams Triggers
- Indexer Hints and Quality of Life Improvements
- Upcoming Features
- Aggregation
- eth_getBalance
- Other Updates
- EIP-4844 Support
- EIP-4844 Support (25:52)
- Understanding EIP-4844
- Pinax’s Work on Blob Storage
- Firehose Implementation for Blob Storage
- Beta Version of Firehose for the Beacon Chain
- Substreams Development
- Next Steps and Community Feedback
Chain Integration Process (1:40)
The Chain Integration Process (CIP) serves as an onboarding mechanism for new chains seeking integration with The Graph, aiming to leverage a network of over 125 performant Indexers for production. The process unfolds in several steps:
- Begin by initiating a thread in The Graph’s Forum, facilitated by the Protocol Team, and utilizing the provided template to gather all necessary information for commencement.
- Integration and Testing phase involves collaboration among The Graph Foundation, Protocol Team, Core Devs, and Indexers:
- A set of Indexers conducts testing of the Graph Node integration.
- Integration and testing of Subgraph Studio ensue to ensure a seamless protocol experience.
- Mainnet activation is the final step, overseen by the Foundation:
Assuming no issues arise in the previous phases, The Graph Council proceeds to vote on granting full protocol support.
The Chain Integration Process (CIP) benefits The Graph Protocol in multiple ways. It expands the network’s reach and utility by integrating new chains, attracting more developers and users, while ensuring high-quality indexing services. Additionally, the integration of new chains contributes to the protocol’s growth and adaptability. Moreover, the transparent and collaborative nature of the CIP fosters community engagement, driving collective investment in the protocol’s success.
Overall, the CIP streamlines the integration process, ensuring new chains can seamlessly join The Graph Network and benefit from its robust infrastructure.
GGP Updates (8:25)
Graph Governance Proposal (GGP) 0031, GGP-0034 and GGP-0035 have recently been approved and implemented on Arbitrum. These GGPs introduce significant enhancements to the protocol, with implications for users across the ecosystem.
GGP-0031 encompasses two Graph Improvement Proposals (GIPs):
-
Firstly, it introduces the capability to store dispute status and resolution on-chain, providing users with greater transparency and confidence in protocol operations.
-
Additionally, GGP-0031 removes the owner tax on subgraph upgrades, simplifying the process for developers and potentially encouraging more frequent updates to subgraphs. Previously, when upgrading subgraphs, developers had to allocate additional GRT to cover the owner tax, which was a larger percentage of the curation tax. With the tax change, approved by the GGP, the owner tax on subgraph upgrades has been removed. Now, when a subgraph is upgraded, all Curators will pay their corresponding fraction of the curation tax without the need for the subgraph developer to add additional GRT.
The permissionless payer functionality introduced by GGP-0034 removes restrictions on specific addresses injecting fees into the protocol, allowing for broader user participation and a more permissionless ecosystem.
GGP-0035, which addresses GIP-0062, has been implemented to eliminate the cooldown period for delegation parameters among Indexers. This change was prompted by the realization that the cooldown feature, originally intended to provide confidence to Delegators by preventing Indexers from making sudden changes to parameters, was seldom utilized. Moreover, it was observed that the existing relationship between Delegators and Indexers already provided a mechanism for ensuring responsible parameter management. Therefore, the removal of the cooldown feature simplifies the protocol and allows Indexers greater flexibility in adjusting delegation parameters, while also resolving issues such as unexpected cooldown durations experienced by an Indexer on L2 due to a UI bug.
Read more on each GGP:
GGP-0031
GGP-0034
GGP-0035
Graph Node Update (15:44)
Release 33, deployed at the end of the last year, introduced significant enhancements including Arweave file data sources, polling block handler, initialization handler and data source context. This release also brought forth support for pure Substreams chains, expanding chain compatibility by enabling support without the need to have an RPC.
Recent Patch
Patch Release 34.1 addressed existing bugs and introduced additional Substreams triggers. This enhancement increases the integration flexibility between Substreams and graph-node, allowing for any map module output in Substreams packages to be processed in subgraph handlers, thereby facilitating the integration of new chains more seamlessly.
Indexer hints enable subgraph developers to signal their readiness to prune archival data to optimize disk space usage and query speeds, with the decision to prune being left to the Indexer. Additionally, the introduction of endBlock and auto-generating IDs have been quality of life improvements, streamlining the development process.
This release also provides initial Starknet support. This development is particularly noteworthy because it was spearheaded by a member of the Starknet community, illustrating a strong example of community-driven innovation. Starknet, being a non-EVM supported chain, required a unique approach for integration. The contributing community member built a Firehose, which is a baseline requirement for integration, and also built a native graph-node integration specifically for Starknet. While this integration has been included in the recent release and is currently being tested to ensure its functionality, it is important to note that it hasn’t been fully deployed on the network yet. Despite this, the integration is already being used in production by the contributor, showcasing the potential and effectiveness of open-source contributions to expanding The Graph’s support for various blockchain ecosystems.
Coming Soon
Aggregation
Currently work is being done to facilitate declarative, time-based calculations directly within the indexing and query layer. This allows for more efficient, less error-prone, and performance-optimized data handling, particularly for use cases that require the analysis of time series data. Aggregation has been tailor made for known decentralized exchanges (DEXs) use cases, where aggregating trading data over specific time intervals (e.g., hourly, daily) is crucial for generating insights into trading volume, price movements, liquidity trends, and more. By simplifying the process of time-based data aggregation, these features can significantly enhance the performance and reliability of DEX analytics and other similar applications.
A call for beta testers will be included in The Graph Builders Newsletter!
Subscribe here.
eth_getBalance
This feature aims to simplify the process of tracking and querying the balances of smart contracts or addresses directly within a blockchain’s data layer. This is particularly useful for applications that need to monitor token holdings, payment flows, or liquidity provisioning across different entities within a blockchain network.
And more…
In addition, several other updates are underway to further enhance the functionality and performance of blockchain data indexing platforms. These include significant dependency updates, such as upgrades to Diesel and Wasmtime, which are crucial for improving the overall stability, security, and performance of the system.
The introduction of a parent hash in block metadata offers improved data integrity and traceability within the blockchain, enabling more sophisticated data analysis and verification processes. This feature is particularly important for applications requiring a high degree of security and accuracy in transaction processing and block verification.
Filtering based on indexed arguments represents a significant enhancement in query optimization, allowing for more precise and efficient data retrieval. This feature enables developers to build more responsive and scalable applications by reducing the overhead associated with processing large volumes of irrelevant data.
The experimental feature of parallel trigger fetching in prefiltered subgraphs aims to drastically improve indexing speed and efficiency. By enabling parallel processing of data triggers based on predefined filtering criteria, this feature can significantly reduce the time required to index and update subgraphs, enhancing the timeliness and responsiveness of applications built on top of these platforms.
EIP-4844 Support (25:52)
What is EIP-4844?
EIP-4844, also known as proto-danksharding, introduces a significant evolution in Ethereum’s scalability efforts through the implementation of blob-carrying transactions. These transactions enable rollup sequencers and potentially other users to post large amounts of data to the Ethereum mainnet more affordably than current methods. By limiting the size and quantity of these “blobs” within each block, EIP-4844 maintains decentralization, ensuring the computational and storage demands on Ethereum nodes remain manageable. This approach not only improves scalability by reducing costs for layer 2 solutions but also lays the groundwork for further scalability enhancements, including the eventual adoption of full Danksharding.
Blobs, central to EIP-4844’s strategy, serve as auxiliary data carriers that accompany standard Ethereum transactions within blocks. These blobs are designed to contain batched transactions, primarily from Ethereum Layer 2 rollups, facilitating cheaper data posting to the mainnet. Blob-carrying transactions introduce new fields for specifying transaction bids and blob references, emphasizing that the actual blob data does not reside within the block but is instead referenced and stored temporarily by Ethereum’s consensus layer for about 18 days. This novel structure reduces the data processing workload on the Ethereum execution layer, supporting more efficient and scalable blockchain operations while preparing for future enhancements in Ethereum’s scalability and functionality.
Pinax’s Work on Blob Storage
The Graph plans to handle blob storage through Firehose implementation, which stores blobs in flat files. This approach is noted for its flexibility and potential to accommodate various use cases for blobs. A beta version of a Firehose for the beacon chain has been developed by the Pinax team. It utilizes the StreamingFast RPC Poller for efficient data retrieval and integration.
The codebase for this implementation and related Substreams is available here:
Next Steps and Community Feedback
Immediate next steps involve refining the block model, exploring potential data insights from blobs, and integrating beacon Substreams into The Graph’s infrastructure more seamlessly. Long-term plans for blob storage and processing will be shaped by the community’s needs and creative use cases for blobs. The Pinax team encourages feedback and dialogue with the community to guide future developments and ensure that The Graph’s tools and services meet evolving demands in the Web3 space.
Reach out to Pinax by contacting “matthewdarwin” on Discord or X.
Stay Tuned!
Join us next month for Core Devs Call #29!
Keep up to date by joining discussions in the forum, following The Graph on X or joining the Discord server!
The 28th Core Dev call is happening next Thursday 2024-02-22T16:00:00Z.
Please subscribe to The Graph Foundation’s Ecosystem Calendar, so you don’t miss any of the upcoming calls!
Tentative agenda:
Got something to present and discuss?
The recording will be uploaded here.
The Graph Core R&D Workspace
Monthly Core Dev updates
