We are calling on the Graph Foundation to relaunch the MIPs program due to its numerous flaws, as outlined in our previous post. In general, the Foundation should
- Clearly define the goals and objectives of the program, ensuring that they align with The Graph’s overall mission and vision.
- Simplify the program’s structure and requirements, making it easier for indexers to participate and contribute, and reduce the workload of the program organizers.
- Increase transparency and communication with program participants, including regular updates on progress and any changes to the program.
- Establish effective channels for feedback and collaboration between organizers, developers, and indexers.
- Review and address any issues with the scoring and evaluation process to ensure fairness and transparency.
We are specifically proposing a new rating system and program structure that will:
- Prioritize openness and transparency.
- Simplify the MIPs program to reduce the workload of organizers.
- Accelerate the integration of new chains.
- Incentivize high performance and quality of service.
- Foster collaboration and leverage the expertise of indexers.
Our proposed rating system has three key criteria, corresponding to the three primary objectives of the MIPs:
- C1. Community Participation
- C2. Chain and subgraph coverage
- C3. Performance and Quality of Service
The program is structured to allow for maximum parallelization of the integration of chains. It is flexible and removes dependencies between chains, so delays with one chain do not significantly impact the others. Multiple chains can be worked on simultaneously during testnet and mainnet stages, and indexers are encouraged to take initiatives on other chains.
A rough roadmap listing potential future chains is shared with indexers, allowing them to properly evaluate hardware requirements and sync the chains in advance. This roadmap is not guaranteed, and indexers should expect that some chains may not make the final cut. Technically capable and motivated indexers can experiment with offchain indexation of subgraphs and identify pain points and raise issues to blockchain node developers and Graph core teams.
This stage is constantly active and indexers are free to participate or not, without impacting their score for Stage 1 and 2. However, indexers who engage in Stage 0 may be advantaged in future stages by achieving a greater level of readiness. Contributions to this stage will only be directly rewarded through the Community Participation criterion (C1).
Once the core developers determine that the maturity of the integration of the indexer stack with the new chain is acceptable, a formal testnet stage is launched. Indexers are given a two-week notice for integrating a new chain. The notice will indicate which subgraphs will be used for testing, giving indexers ample time to index them.
During the testnet stage, scoring will evaluate chain and subgraph coverage (C2), performance and quality of service (C3.1), and potentially an indexation speed and throughput competition may be held (C3.2). Shortly after the end of the stage, another testnet stage is launched for a different chain.
Once the testnet stage is completed, the Graph Council adopts the necessary GIPs to bring the chain to mainnet. To incentivize mainnet indexing, a supplemental GRT grant is distributed to indexers.
Only graph coverage is evaluated at this stage (C2).
C1. Community Participation:
- Grant GRT for notable contributions to the community made during the program. These grants are allocated by a committee of Indexers, Core devs and Foundation representatives on the following criteria:
- Quality of the github issues raised (well-written, well-researched, and relevant issues that contribute to the improvement of the MIPs program)
- Quality of PRs: Indexers who submit high-quality pull requests that are well-written, well-documented, and adhere to coding standards may be rewarded.
- Participation on Discord and forums (active participation in Discord or forum discussions, provide helpful feedback, and contribute to the community
- Documentation and guides published
- Tooling shared with the community (scripts, dashboards, indexer software distribution, etc.)
- Support provided to subgraph developers
- Contributions to blockchain node development (e.g. working with node developers to fix issues affecting indexers, making snapshots available, etc.)
- The committee meets on a regular basis (e.g. every month) to review contributions and award GRT.
C2. Chain and subgraph coverage
- The same methodology should be used for both testnet and mainnet, with different objectives:
- Testnet: Encourage broad participation
- Mainnet: Ensure continuous support from indexers and guarantee proper quality of service to subgraph developers looking to migrate to the decentralized network.
- Grants are proportional to the allocation period of the indexer, rather than allocation size, to level the playing field for new and smaller indexers. Allocation period is calculated after the first successful allocation closure with a valid Proof of Indexing (POI) to incentivize indexing speed. Indexers will be invited to perform a “reallocate” action to start the accumulation of MIPs rewards.
- Grants are calculated using an exponential curve on the proportion of chain subgraphs that the indexer is supporting, to incentivize maximum coverage while subgraph developers are testing their subgraphs on testnet and migrating to mainnet.
- The calculation formula is made public, allowing anyone to independently calculate the subgraph coverage score using only on-chain data, with the exception of POIs, which must be verified off-chain.
- For mainnet, a longer evaluation period should be considered (months instead of days for testnet) to ensure proper migration support.
C3. Performance and Quality of Service:
- C3.1: Data freshness and base latency
- A simple testing harness will send a moderate volume of queries (1-5 qps) directly to indexers, bypassing the gateways.
- Testing will happen over several days, for every subgraph deployment supported by the program.
- The testing harness will calculate average query latency and block delay, and cross-check query results for accuracy.
- The testing harness will run simultaneously from several worldwide locations, averaging latency from different geographic areas.
- The code will be published, so indexers can run their own instance to test their setup before the official test. This will allow indexers to optimize their indexer and contribute to the improvement of the testing harness.
- The harness is designed to mimic Gateway behavior as much as possible, minus the Indexer Selection Algorithm, so any performance optimization made by indexers will also improve regular mainnet usage.
- Indexers will provide an authorization token to the MIPs organizers to grant them free queries directly through their indexer service endpoint.
- Near-realtime metrics will be provided for transparency and to allow indexers to optimize and improve their setup on a continuous basis.
- C3.2: High-performance and scale
- Optional competitions will be organized to incentivize indexers to invest in high-performance and scalable architecture.
- Competition C3.2.1: Throughput:
- Using the testing harness of C3.1, a very high volume of queries will be sent to the indexers on a few moderately large subgraphs. Indexers will be ranked on the query volume they are able to sustain for a 15-minute period without generating an elevated error rate or hitting a cliff in query latency.
- Competition C3.2.2: Indexation speed
- A very complex and hard to index subgraph will be deployed at a predetermined time, and indexers will be ranked on their ability to rapidly index the new subgraph.
- The first indexers to post their POI for a predetermined block in a Discord channel will win.
We hope that these recommendations will help to improve the MIPs program and foster a more collaborative and productive relationship between indexers and the Graph Foundation. We look forward to working with the Foundation to make these changes and create a more successful program for everyone involved.