POIFIER - The Graph protocol infrastructure project made by Indexers for Indexers, which aims to help with verification of indexing data consistency.
Indexer is a node operator who provides indexing and query processing services for The Graph protocol consumers.
As the proof of data indexing Indexer submits POI (Proof Of Indexing) to the protocol and gets indexing rewards.
Submitted POI for a block is a digest for all entity store transactions for a specific subgraph deployment up to and including that block.
Since fair Indexer strives to provide service by delivering consistent and up-to-date data, the main challenge is verification of indexing data.
Data verification can be achieved by comparing the current POI generated by Indexer for a particular subgraph and ethereum block to POIs generated by other Indexers if they would generate POI for the same subgraph and ethereum block using referenced indexer ID.
We aim to build an end-to-end solution to implement this idea which comprises client and server (GUI) applications to help Indexer with verification of their indexing data.
For XYZ reason indexer’s node may experience an issue when node would produce “invalid” POI while no visible issue detected by node operator.
Invalid POI is a symptom of indexing data inconsistency and would lead to a case when the indexer would serve a query request with “wrong” data.
So far there is very little visibility and no tool to verify whether generated POI is valid or not, but it was supposed by design that POI generated by any indexer having referenced indexer ID for a particular subgraph and block would match.
There are could be multiple approaches, but we would focus on next 2:
Centralized, aka Oracle: where all POIs generated by indexers would test against POIs generated by single Oracle
Decentralized, aka consensus: where indexer’s POI would test against referenced POI which is being selected as a majority of submitted POIs for given subgraph/epoch/block
POIFIER is a client-server tool with deep analytics which aims to implement Decentralized approach.
On a regular basis Indexers would submit POIs generated for all subgraphs being indexed by their graph-node for a range of Epochs and Ethereum Blocks using referenced indexer ID (0x0…0)
POIfier-server would select POI based on major number of appearance for each subgraph/block/epoch and name it as a “Valid POI”
POIs would be assessed against “Valid POI”
For Graph Developers:
- On early stage to detect, identify and fix indexing issues
- To research and investigate multiple indexer / cross ethereum nodes issue
- On early stage to detect, identify and fix single indexer issue
- On early stage to detect, identify and flag developer on multiple indexer issue
- Add visibility into indexing data consistency
- Improve protocol indexing data consistency
POIFIER infrastructure consists of:
Poifier-client - python3 client which is deployed within Indexer infrastructure, queries indexer’s graph-node against subgraphs and POI per epoch/block and uploads report on poifier-server on regular basis
Poifier-server - user friendly GUI with analytics data, where registered indexer is able to submit to and analyze POIs
- uses qraphql queries and no db integration needed
- does require API TOKEN to get authorized on POIfier server
- POI is being generated based in referenced indexer id (0x0…0) and can not be used for closing allocations
Step by step guide
Get API Token
Go to https://poifier.io, connect with Metamask (Metamask account must be linked to the address of the indexer, i.e. it must be the operator or the beneficiary or the indexer itself)
Provide index-node id and click “Get Token”
Copy provided API TOKEN
Do not share your API TOKEN with anyone
Read README carefully
Run docker-compose or single docker container with all required keys
View submitted data on portal via GUI
- Go to https://poifier.io, connect with Metamask (Node operator account)
Now you are able to see POIs submitted by your node along with other Indexer’s POIs
Worth to note, this is beta version, where some features may not be available
Grassets Tech & Ryabina