The Arbitrators are contacting Indexer address 0x2f09092aacd80196fc984908c5a9a7ab3ee4f1ce and fisherman 0x0a015d9e17d3978164486473a2ddbb654418db07 for a new dispute filed in the protocol.
Fisherman, could you share the insights or data you gathered that led to you filing the dispute? Please provide all relevant information and records about the open dispute. This will likely include POIs generated for the affected subgraph(s).
About the Procedure
The Arbitration Charter regulates the arbitration process. You can find it in Radicle project ID rad:git:hnrkrhnth6afcc6mnmtokbp4h9575fgrhzbay or at GIP-0009: Arbitration Charter.
Hello, fisherman here, firstly thank you for approving my sign up request, I was not able to reply to this forum post until this was processed. This is not ideal as a fisherman as you’re unable to contribute to the discussion, is it possible that this dispute could have been closed as a fisherman loss because I was unable to reply? If so, then I would encourage an automated system to process account onboarding.
Beyond that, to provide info on what data I collected before filing this dispute, please refer to the following dispute summary, note that if you need any additional information I am happy to collaborate in whatever way I can:
The subgraph can be correctly indexed to block 75,174,859 (as proven by another indexer submitting a matching POI)
The disputed indexer’s POI does not match what other indexers and my graph-node computed
This suggests the mismatch is not caused by a determinism issue in the subgraph
Additional Evidence
I verified all other Horizon closed allocations on this subgraph QmdDSA73QkFAvRZqtRoHgLvxonuSZTuJhoWkmBtEdVuTmz against my graph-node. The disputed indexer is the only one with a mismatch and the subgraph matched correctly for all other indexers, even at higher block heights than the disputed POI, see the following table sorted by block height:
Indexer
Block
Submitted POI
My Calculated POI
Result
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59
75,811,870
0xfd9d1ceb...07ee6d
0xfd9d1ceb...07ee6d
Match
0xd9819426c82e2b8fc58b9b62a78efe93f78077c6
75,811,870
0x093d0d11...d185b2
0x093d0d11...d185b2
Match
0xe48b586eeb81bde60f14b0b8d80ddd06c7a24720
75,811,870
0xfe0637fc...7b6d40
0xfe0637fc...7b6d40
Match
0xa6ff993e0f6253f1b7f55c873577a2f0f0ceb325
75,811,870
0xee4a4c0e...ee4d9f
0xee4a4c0e...ee4d9f
Match
0x2b3c7d1ef5fdfc0557934019c531d3e70d6200ae
75,738,330
0x4ef17d77...837718
0x4ef17d77...837718
Match
0x090f7382f9ea85c733cd501f4d87f16cb5b83ed3
75,738,330
0x9b4c4087...ee97b7
0x9b4c4087...ee97b7
Match
0xe91273727203bcc827521fc8b0c762d435c3c5d0
75,309,887
0x33ae9581...f3f0a0
0x33ae9581...f3f0a0
Match
0xe48b586eeb81bde60f14b0b8d80ddd06c7a24720
75,309,887
0x733ae005...8902ad
0x733ae005...8902ad
Match
0xd9819426c82e2b8fc58b9b62a78efe93f78077c6
75,309,887
0xf54a0d18...5db817
0xf54a0d18...5db817
Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59
75,309,887
0x67cd3b2b...7e4604
0x67cd3b2b...7e4604
Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0
75,239,491
0xbb7415d6...bf38ef
0xbb7415d6...bf38ef
Match
0x090f7382f9ea85c733cd501f4d87f16cb5b83ed3
75,239,491
0xc0641ab5...be5bd1
0xc0641ab5...be5bd1
Match
0x2f09092aacd80196fc984908c5a9a7ab3ee4f1ce
75,174,859
0xcf490b57...f539a4
0x7cbe229c...5c7e4e
Mismatch
0xe13840a2e92e0cb17a246609b432d0fa2e418774
75,174,859
0x3e50d118...8e3ab1
0x3e50d118...8e3ab1
Match
0x2b3c7d1ef5fdfc0557934019c531d3e70d6200ae
74,795,540
0xde2b83bc...62efa7
0xde2b83bc...62efa7
Match
0x090f7382f9ea85c733cd501f4d87f16cb5b83ed3
74,666,332
0xc22489c9...797539
0xc22489c9...797539
Match
0xe91273727203bcc827521fc8b0c762d435c3c5d0
74,666,332
0x9f72246d...5ee65b
0x9f72246d...5ee65b
Match
0xe48b586eeb81bde60f14b0b8d80ddd06c7a24720
74,666,332
0x43e54443...7d4f6d
0x43e54443...7d4f6d
Match
0xd9819426c82e2b8fc58b9b62a78efe93f78077c6
74,666,332
0x5348289f...b0d08d
0x5348289f...b0d08d
Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59
74,472,074
0x381cbd3c...e5fc84
0x381cbd3c...e5fc84
Match
0xe91273727203bcc827521fc8b0c762d435c3c5d0
74,472,074
0x3dfec087...0261b9
0x3dfec087...0261b9
Match
0xe48b586eeb81bde60f14b0b8d80ddd06c7a24720
74,218,531
0xd868e8e2...60b7fa
0xd868e8e2...60b7fa
Match
0x2b3c7d1ef5fdfc0557934019c531d3e70d6200ae
74,031,480
0x7d1d7ffc...e24415
0x7d1d7ffc...e24415
Match
0xd9819426c82e2b8fc58b9b62a78efe93f78077c6
74,031,480
0xb6381ac7...53ca07
0xb6381ac7...53ca07
Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59
73,969,878
0xee0060f3...4c38b5
0xee0060f3...4c38b5
Match
0xe48b586eeb81bde60f14b0b8d80ddd06c7a24720
73,905,660
0xfc56199b...a963ad
0xfc56199b...a963ad
Match
0xe91273727203bcc827521fc8b0c762d435c3c5d0
73,905,660
0x4fbf5cdc...7e155c
0x4fbf5cdc...7e155c
Match
0x2b3c7d1ef5fdfc0557934019c531d3e70d6200ae
73,780,735
0xc53f51b7...02d246
0xc53f51b7...02d246
Match
Closing Transaction
The allocation was closed in Arbitrum block: 418,828,128
Final note
I am happy to provide any additional information the Arbitration Team requires.
Hi @badcoffee, thanks for the thorough report. The information you provide checks out on our end. We are looking into a few other things before coming to a decision, while also giving the indexer a couple more days to respond. Stay tuned for the resolution.
Hi! Sergey from P2P.org here, I’m responsible for the operations of the indexer in question.
The allocation in question was closed using standard indexer-agent behaviour, POI was obtained automatically by indexer-agent. We are using official images of indexer stack (indexer-agent and graph-node). Database has not been tampered with and we do not do any modifications to the databases manually.
I’ve double checked the POI of the subgraph deployment and still received 0xcf490b576f194303816d691570c674a22c4632ccd978d2f31aa1b23a07f539a4 as the result.
We’ve raised issues with POI consistency in the past and we are not the only indexer experiencing issues with subgraphs suddenly starting to diverge in POIs. We’ve got confirmation from Ørjan on IOH that the closing of the allocations with diverging POI caused by graph-node issue should not be slashed.
I am happy to provide any additional information needed for further investigation.
Hey @sk_p2p@badcoffee, thanks both for reaching out and contributing to this investigation.
We’ve tracked down the POI divergence to epoch906, after that P2P indexer started producing POIs that do not match the consensus from at least other 5 indexers, see:
Given the information available it’s unclear if this is a bug on graph-node or an input data integrity issue (RPCs can sometimes miss blocks for instance). The widespread consensus on a POI from several other indexers would indicate it’s likely the latter.
In light of this, Arbitration Council has resolved to:
Resolve the dispute as a DRAW (no slashing).
Recommend the disputed indexer to take measures to rectify the discrepancy:
Ensure the blockchain client being used is producing accurate data (client node is up to date and running production versions).
Rewind the affected subgraph, forcing to re-index history at least as back as epoch 905.
Close any open allocations for this subgraph with POI 0x00.
As per section 13 of the Arbitration Charter in cases of discrepancies like this one the disputed indexer has 7 days to remediate the issue, after which any new POIs CAN be disputed and result in slashing.
p2p-org-arbitrum.ethindexer 0x2f09092aacd80196fc984908c5a9a7ab3ee4f1ce is well synced for this subgraph, and the subgraph was fully synced at the time the allocation was closed.
All these time POI divergence is not slashable when a subgraph is properly closed via indexer-agent, unless it’s closed through a forced contract interaction or a copied POI. If POI divergence alone were slashable under these conditions, it would imply that many established indexers could be exposed to slashing risk, as such divergences are typically resolved through routine remediation steps (e.g.clearing call cache, clearing block cache and rewinding).
An indexer that serves 0 queries and only claims indexing rewards initiating a slashing dispute against a long-term indexer that serves the network, what a cliche.
An indexer that serves 0 queries and only claims indexing rewards initiating a slashing dispute against a long-term indexer that serves the network, what a cliche.
Not collecting query fees and serving 0 queries is not the same thing.
