Request for Information about Disputes #GDR-51

The Arbitrators are contacting Indexer address 0xf00f7157fa8fd0420b87956d46058a16b2f23adc and fisherman 0x0a015d9e17d3978164486473a2ddbb654418db07 for a new dispute filed in the protocol.

Dispute ID: 0x36f44f45f4e337d0b584f19a2f1a0a6d4ac6b21c187a2d3956ac237e9c7e104d
Allocation: 0x6730b328d97c936f0383376e7f83bb001b052ffc
Subgraph: QmdDSA73QkFAvRZqtRoHgLvxonuSZTuJhoWkmBtEdVuTmz

Fisherman, could you share the insights or data you gathered that led to you filing the dispute? Please provide all relevant information and records about the open dispute. This will likely include POIs generated for the affected subgraph(s).


About the Procedure

The Arbitration Charter regulates the arbitration process. You can find it in Radicle project ID rad:git:hnrkrhnth6afcc6mnmtokbp4h9575fgrhzbay or at GIP-0009: Arbitration Charter.

Please use this forum for all communications.

Arbitration Team

Dispute Summary

Field Value
Indexer 0xf00f7157fa8fd0420b87956d46058a16b2f23adc
ENS p2p-org-booster.eth
Operator 0xb0cccab26193c6df7d9997a88e146b0f0a7d3e36
Allocation ID 0x6730b328d97c936f0383376e7f83bb001b052ffc
Subgraph QmdDSA73QkFAvRZqtRoHgLvxonuSZTuJhoWkmBtEdVuTmz
Submitted POI 0x7ac7e69c8693f2854fb35a5e599846e9bf5d50c717ab760a3f8af971b7f06a21
Expected POI 0xe743d870cf110daf447d0a9afa197a7618ed412db278df585b1d8d4d7891c188
Result Mismatch
Arbitrum Block Hash 0xc57fd663a935d96307088007c7dd2dab879a3cc240d1b556bc8c2dabb32132b0
Arbiscan Transaction Hash 0x6f69511b5de11c22d81e471dc4e167dabd8cd34e35607a8c3ecd176516b8d803
Network avalanche
POI Block Number 83,806,796
Closed At Epoch 1240
Charter Type Horizon (GIP-0085)

Note to arbitrators

The disputed indexer P2P also runs indexer address 0x2f09092aacd80196fc984908c5a9a7ab3ee4f1ce which was previously disputed in GDR #47. In GDR #47 the arbitration council determined that the cause of the POI mismatch on subgraph QmdDSA73QkFAvRZqtRoHgLvxonuSZTuJhoWkmBtEdVuTmz was likely due to an input data integrity issue from the RPC and resolved the dispute as a draw. The indexer was given 7 days notice to remediate the issue, after which new POIs produced by this indexer could be disputed again and can be resolved at the Arbitrator’s discretion. Before anyone argues that GDR #47 was opened against p2p-org-arbitrum.eth 0x2f09092aacd80196fc984908c5a9a7ab3ee4f1ce while this GDR is opened against p2p-org-booster.eth 0xf00f7157fa8fd0420b87956d46058a16b2f23adc, I would like to note that sybiling one’s identity should not constitute a reasonable defense for data integrity issues. If it did, then indexers would be encouraged to sybil as soon as they have been put on 7-day notice by arbitrators, rather than fixing their infrastructure. In GDR #47 Sergey confirmed that P2P’s indexer stack deterministically generates the bad POI, while my evidence showed that other indexers had no issues with POI production on that subgraph. The pattern observed in this dispute is the same, P2P’s indexer is the only one of many that is unable to generate a correct POI. The section 13 7-day remediation window for P2P from GDR-47 expired more than three months ago (2026-01-27).

Evidence

I verified all other closed allocations on this subgraph against my graph-node. The disputed indexer is the only one with a mismatch:

Indexer Block Submitted POI My Calculated POI Result
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 84,058,676 0x4a2df53e...9426c2 0x4a2df53e...9426c2 Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 83,974,400 0xed2b97c9...3aa0a1 0xed2b97c9...3aa0a1 Match
0x090f7382f9ea85c733cd501f4d87f16cb5b83ed3 83,974,400 0x6aca1b61...463f11 0x6aca1b61...463f11 Match
0xf00f7157fa8fd0420b87956d46058a16b2f23adc 83,806,796 0x7ac7e69c...f06a21 0xe743d870...91c188 Mismatch
0xdc53e62df89fd07b31ed4ff886397b9e7ae4625e 83,723,358 0x67ff56f6...d84c79 0x67ff56f6...d84c79 Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 83,723,358 0xce63c36d...756fa3 0xce63c36d...756fa3 Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 83,639,230 0x4fa8461b...0444a8 0x4fa8461b...0444a8 Match
0x090f7382f9ea85c733cd501f4d87f16cb5b83ed3 83,554,522 0x6efe0ade...d716fa 0x6efe0ade...d716fa Match
0xdc53e62df89fd07b31ed4ff886397b9e7ae4625e 83,469,634 0x5fea4631...55e0fe 0x5fea4631...55e0fe Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 83,469,634 0xf056b41e...8d843c 0xf056b41e...8d843c Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 83,384,830 0xadafe301...dd7c83 0xadafe301...dd7c83 Match
0xf92f430dd8567b0d466358c79594ab58d919a6d4 83,384,830 0x6739c9a3...db20aa 0x6739c9a3...db20aa Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 83,129,078 0x76f7dedf...a78132 0x76f7dedf...a78132 Match
0x090f7382f9ea85c733cd501f4d87f16cb5b83ed3 83,129,078 0x2c21ce14...36b902 0x2c21ce14...36b902 Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 83,044,378 0x51cd43db...49b061 0x51cd43db...49b061 Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 83,044,378 0xc5538f29...55136a 0xc5538f29...55136a Match
0xdc53e62df89fd07b31ed4ff886397b9e7ae4625e 83,044,378 0x7aba0f66...b853ec 0x7aba0f66...b853ec Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 82,960,755 0x645808da...8d44e7 0x645808da...8d44e7 Match
0x2b3c7d1ef5fdfc0557934019c531d3e70d6200ae 82,960,755 0x3cc5fd8f...e37c59 0x3cc5fd8f...e37c59 Match
0xe6de2325ef1aac1f058fae59d3c38a472f569846 82,876,529 0xb54cf772...24f444 0xb54cf772...24f444 Match
0x090f7382f9ea85c733cd501f4d87f16cb5b83ed3 82,876,529 0x5c950ec5...929a74 0x5c950ec5...929a74 Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 82,792,962 0x16e304e5...47cf93 0x16e304e5...47cf93 Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 82,792,962 0xa53851fd...ab253e 0xa53851fd...ab253e Match
0xe13840a2e92e0cb17a246609b432d0fa2e418774 82,792,962 0xbcd2684f...b25181 0xbcd2684f...b25181 Match
0xdc53e62df89fd07b31ed4ff886397b9e7ae4625e 82,543,422 0x035074b8...75603c 0x035074b8...75603c Match
0xe6de2325ef1aac1f058fae59d3c38a472f569846 82,462,206 0x5daac220...61648e 0x5daac220...61648e Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 82,378,151 0xd4158301...6f6122 0xd4158301...6f6122 Match
0x090f7382f9ea85c733cd501f4d87f16cb5b83ed3 82,378,151 0x8f21e883...1ac168 0x8f21e883...1ac168 Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 82,294,647 0x7108948a...d7a3bc 0x7108948a...d7a3bc Match
0xdc53e62df89fd07b31ed4ff886397b9e7ae4625e 82,294,647 0xf9ae1e27...cb3efc 0xf9ae1e27...cb3efc Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 82,210,071 0xb14dd3d6...ddbd8c 0xb14dd3d6...ddbd8c Match
0x2b3c7d1ef5fdfc0557934019c531d3e70d6200ae 81,955,640 0x961225aa...8dc77e 0x961225aa...8dc77e Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 81,870,336 0x9f5a3a23...395375 0x9f5a3a23...395375 Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 81,786,076 0xe8b1bf5d...a86bce 0xe8b1bf5d...a86bce Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 81,701,082 0x561bde88...e634f6 0x561bde88...e634f6 Match
0x2b3c7d1ef5fdfc0557934019c531d3e70d6200ae 81,701,082 0xe4823668...62e889 0xe4823668...62e889 Match
0x090f7382f9ea85c733cd501f4d87f16cb5b83ed3 81,616,978 0xd746dfd3...e7709a 0xd746dfd3...e7709a Match
0xe6de2325ef1aac1f058fae59d3c38a472f569846 81,616,978 0x342b35b7...42f70d 0x342b35b7...42f70d Match
0xf92f430dd8567b0d466358c79594ab58d919a6d4 81,616,978 0xab79d1f6...139fa4 0xab79d1f6...139fa4 Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 81,279,195 0x50a41cd2...8d39f4 0x50a41cd2...8d39f4 Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 81,110,836 0xe304e7b4...2ff4f5 0xe304e7b4...2ff4f5 Match
0x2b3c7d1ef5fdfc0557934019c531d3e70d6200ae 81,110,836 0x6225104b...481453 0x6225104b...481453 Match
0xe6de2325ef1aac1f058fae59d3c38a472f569846 81,026,064 0xa2e6fd26...e97945 0xa2e6fd26...e97945 Match
0xe13840a2e92e0cb17a246609b432d0fa2e418774 80,857,868 0x73cce1a7...8316c2 0x73cce1a7...8316c2 Match
0xdc53e62df89fd07b31ed4ff886397b9e7ae4625e 80,773,943 0xe7114b1f...b6906d 0xe7114b1f...b6906d Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 80,773,943 0x72d59875...c940d7 0x72d59875...c940d7 Match
0xa6ff993e0f6253f1b7f55c873577a2f0f0ceb325 80,773,943 0x0a25ec32...e2fdbf 0x0a25ec32...e2fdbf Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 80,690,733 0xeb04f653...d44fff 0xeb04f653...d44fff Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 80,690,733 0x612c41b2...7a72e9 0x612c41b2...7a72e9 Match
0x2b3c7d1ef5fdfc0557934019c531d3e70d6200ae 80,608,014 0x448cbe36...4e2786 0x448cbe36...4e2786 Match
0xdc53e62df89fd07b31ed4ff886397b9e7ae4625e 80,532,498 0x8eec7728...215935 0x8eec7728...215935 Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 80,458,211 0xe8beb416...c40592 0xe8beb416...c40592 Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 80,255,716 0x2c0eb3ce...e488f9 0x2c0eb3ce...e488f9 Match
0x2b3c7d1ef5fdfc0557934019c531d3e70d6200ae 80,153,980 0xb408694d...fb331c 0xb408694d...fb331c Match
0xe6de2325ef1aac1f058fae59d3c38a472f569846 80,070,510 0xf9d5af78...fea275 0xf9d5af78...fea275 Match
0x2f09092aacd80196fc984908c5a9a7ab3ee4f1ce 80,070,510 0x1a286f65...1b5a37 0x1a286f65...1b5a37 Match
0xdc53e62df89fd07b31ed4ff886397b9e7ae4625e 79,986,995 0xc79ba016...15c272 0xc79ba016...15c272 Match
0xa6ff993e0f6253f1b7f55c873577a2f0f0ceb325 79,986,995 0x0ff813ea...7cfef5 0x0ff813ea...7cfef5 Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 79,903,733 0xbad8c1b3...208001 0xbad8c1b3...208001 Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 79,903,733 0x7ee19c6e...b64ddb 0x7ee19c6e...b64ddb Match
0x2b3c7d1ef5fdfc0557934019c531d3e70d6200ae 79,903,733 0xf78aed2f...5c08df 0xf78aed2f...5c08df Match
0xf92f430dd8567b0d466358c79594ab58d919a6d4 79,903,733 0x709363a7...d13eed 0x709363a7...d13eed Match
0xf92f430dd8567b0d466358c79594ab58d919a6d4 79,819,761 0x366de47f...34a1fa 0x366de47f...34a1fa Match
0xf92f430dd8567b0d466358c79594ab58d919a6d4 79,735,005 0x2291b1ee...7cfc78 0x2291b1ee...7cfc78 Match
0xf92f430dd8567b0d466358c79594ab58d919a6d4 79,482,197 0x8b506259...b7efaf 0x8b506259...b7efaf Match
0xe48b586eeb81bde60f14b0b8d80ddd06c7a24720 79,482,197 0x7aadce85...40b093 0x7aadce85...40b093 Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 79,399,775 0x67f7f044...cb8cf4 0x67f7f044...cb8cf4 Match
0x090f7382f9ea85c733cd501f4d87f16cb5b83ed3 79,317,123 0x0cd03a52...c840aa 0x0cd03a52...c840aa Match
0xe13840a2e92e0cb17a246609b432d0fa2e418774 78,985,197 0x05b03492...4ccc4d 0x05b03492...4ccc4d Match
0xe48b586eeb81bde60f14b0b8d80ddd06c7a24720 78,900,752 0xd8cdef04...925971 0xd8cdef04...925971 Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 78,816,330 0xfe432ac3...e95f60 0xfe432ac3...e95f60 Match
0x090f7382f9ea85c733cd501f4d87f16cb5b83ed3 78,316,337 0xf4badaad...598cb7 0xf4badaad...598cb7 Match
0xe48b586eeb81bde60f14b0b8d80ddd06c7a24720 77,908,073 0xd4887375...ac157a 0xd4887375...ac157a Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 77,908,073 0xf70a1673...689d5e 0xf70a1673...689d5e Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 77,662,078 0xf6b90364...2983f0 0xf6b90364...2983f0 Match
0xe48b586eeb81bde60f14b0b8d80ddd06c7a24720 77,348,773 0xf96797d3...ff57c9 0xf96797d3...ff57c9 Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 77,289,237 0x21f375af...7adb61 0x21f375af...7adb61 Match
0xd9819426c82e2b8fc58b9b62a78efe93f78077c6 77,190,912 0x33824352...1c1899 0x33824352...1c1899 Match
0xa6ff993e0f6253f1b7f55c873577a2f0f0ceb325 77,114,939 0x6bb2f452...198b64 0x6bb2f452...198b64 Match
0xe13840a2e92e0cb17a246609b432d0fa2e418774 77,114,939 0x43f31a51...809879 0x43f31a51...809879 Match
0x2b3c7d1ef5fdfc0557934019c531d3e70d6200ae 77,039,126 0x9f8f9044...557b14 0x9f8f9044...557b14 Match
0xe6de2325ef1aac1f058fae59d3c38a472f569846 76,884,614 0x66d34003...12582f 0x66d34003...12582f Match
0xdc53e62df89fd07b31ed4ff886397b9e7ae4625e 76,884,614 0xe913d18d...8a1eea 0xe913d18d...8a1eea Match
0x090f7382f9ea85c733cd501f4d87f16cb5b83ed3 76,807,834 0x9787979b...9d64e9 0x9787979b...9d64e9 Match
0x9af3fc811a66dbbca44acce94906d8743f9cf0d0 76,807,834 0x511992e2...bfce82 0x511992e2...bfce82 Match
0xe48b586eeb81bde60f14b0b8d80ddd06c7a24720 76,807,834 0x3df4f4b7...a9a458 0x3df4f4b7...a9a458 Match
0xe9e284277648fcdb09b8efc1832c73c09b5ecf59 76,807,834 0xe3749176...c584f3 0xe3749176...c584f3 Match
0xe91273727203bcc827521fc8b0c762d435c3c5d0 76,807,834 0x9b1a5929...fff17c 0x9b1a5929...fff17c Match
0xd9819426c82e2b8fc58b9b62a78efe93f78077c6 76,807,834 0x0d46730f...e4bf59 0x0d46730f...e4bf59 Match
0xa6ff993e0f6253f1b7f55c873577a2f0f0ceb325 76,732,196 0x8afeadaa...1cba30 0x8afeadaa...1cba30 Match

Corroborating Evidence

The disputed indexer also submitted bad POIs on at least 4 other subgraphs at the same 83,806,796 block height. See the following:

Field Value
Allocation ID 0xaa1f76fec4ffa8efc9380e35fd089f027d84f169
Subgraph QmZyoRwhXKY2o9Yxqur5hNufKPPSwhZANtf4QQaiqC7Qms
Network avalanche
POI Block Number 83,806,796
Indexer 0xf00f7157fa8fd0420b87956d46058a16b2f23adc
Operator 0xb0cccab26193c6df7d9997a88e146b0f0a7d3e36
Submitted POI 0x8247a79df85fd8da0ddf5ea1f03f7a5a376fdb5ab5a7126af1a59e557fd6b32b
Expected POI 0x748ff5faf6bb3a1b709d2087328fd5ad51218fa467f41ddba1732a25789f0913
Result Mismatch
Field Value
Allocation ID 0xad5e4519ff62c41c81ad8774e81e6e767a08962c
Subgraph QmNmddMQAaLL1zpwpA69tHjCom4udHcFJ5a9PZPxU5Tkew
Network avalanche
POI Block Number 83,806,796
Indexer 0xf00f7157fa8fd0420b87956d46058a16b2f23adc
Operator 0xb0cccab26193c6df7d9997a88e146b0f0a7d3e36
Submitted POI 0xf5210098c4571d85de54c0ca2fa268b6776e2697d4ce94c4bb11c513ad4b796b
Expected POI 0xca500f3ea90a933b1c7ab4f45df815c5c0a9fb15dd194cf2f95f81be39872baa
Result Mismatch
Field Value
Allocation ID 0x8ac053fb7bb151aad97eddc6fc3efe673d1558f5
Subgraph QmY1133fWsE94ZDtgJyTYjzxwZNdCJKXX3j6gVs4mxNNUb
Network avalanche
POI Block Number 83,806,796
Indexer 0xf00f7157fa8fd0420b87956d46058a16b2f23adc
Operator 0xb0cccab26193c6df7d9997a88e146b0f0a7d3e36
Submitted POI 0x2aad5b966c2528e29838923592f62c0e324f374b31e4caf1edd574ce3f8df8c3
Expected POI 0x723f0e29cc1d14753c60e8444de2d11c8e65588ccb9ac2fa7d3b07d09f7a01dc
Result Mismatch
Field Value
Allocation ID 0x7efe913839f9e6e8b4c5fc6100bd8e45d53d652f
Subgraph QmTbmXhUr67gAt5eLy5zUo7dRqATG8S8QgPEcdVhmenRNU
Network avalanche
POI Block Number 83,806,796
Indexer 0xf00f7157fa8fd0420b87956d46058a16b2f23adc
Operator 0xb0cccab26193c6df7d9997a88e146b0f0a7d3e36
Submitted POI 0x7a40592f88c4f5068905c7228cb4bd8e249f1bd23d58330d0d016bf9c57528d4
Expected POI 0xd98acf12d569ea322a16fbd5353df1d6250f1ac016f565530a7df2158c6561cb
Result Mismatch
1 Like

Hi! Sergey from P2P.org here, I’m responsible for the indexer in question. POIs from these subgraphs were generated by indexer-agent automatically. I’ve ran our POI bisection tool and found out that POIs are matching until block 82212058 which was created 1 month ago, which is 2 and a half months after the dispute was resolved.

After the previous dispute was received, we performed following steps:

  • Closed current allocations to the subgraph with 0x0 POI
  • Cleared chain call caches for the subgraph’s chain, block caches are not used since network uses firehose
  • Rolled back subgraph to the block where the POI has matched the other indexers
  • Ensured that POI on the subgraph aligns with other indexers before opening new allocations
  • This can be proven by querying public POIs from our indexers - divergence was resolved and occured the second time.

We are using Pinax as our provider for firehose on avalanche chain. We are willing to assist the investigation and provide all necessary information that is needed.

We are currently in the process of developing internal automation tools to detect and resolve POI divergences, however it’s not in production yet. We are actively responding to data divergence issues in graphprotocol’s discord.

1 Like

Hi all, Hau from Pinax here :waving_hand: . Just sharing my perspective.

Based on input from Fisherman and discussion in #47, P2P indexer has already resolved the POI issue for subgraph QmdDSA73QkFAvRZqtRoHgLvxonuSZTuJhoWkmBtEdVuTmz at block 75,174,859.

Disputes #50, #51, #52, #53 and #54 appear to be new issues at later blocks, and should be investigated further to determine the root cause of the discrepancies.

From experience, subgraphs can still exhibit discrepancies at later blocks despite rewinding, clearing block and call caches, or performing a full resync.

It’s also worth noting that several of these discrepancies may occur at the same block height across multiple subgraphs. This pattern may indicate a shared upstream data inconsistency (e.g. RPC or Firehose input), rather than isolated subgraph or indexer-specific issues.

We are currently performing off-chain syncing for the following five subgraphs related to disputes #50#54:

  • QmY1133fWsE94ZDtgJyTYjzxwZNdCJKXX3j6gVs4mxNNUb

  • QmdDSA73QkFAvRZqtRoHgLvxonuSZTuJhoWkmBtEdVuTmz

  • QmZyoRwhXKY2o9Yxqur5hNufKPPSwhZANtf4QQaiqC7Qms

  • QmNmddMQAaLL1zpwpA69tHjCom4udHcFJ5a9PZPxU5Tkew

  • QmTbmXhUr67gAt5eLy5zUo7dRqATG8S8QgPEcdVhmenRNU

This process may take approximately 2–3 days at best to complete verification. We’re available to assist with any Firehose-related questions if needed.

In our view, based on past disputes, indexers who used the indexer-agent as intended to close allocations should not be considered disputable, provided the subgraph was fully synced or only encountered deterministic errors.

Conversely, indexers who manually submitted POIs via direct contract calls without achieving full sync, or who bypassed non-deterministic errors, may be subject to slashing.

P2P indexer also mentioned :

1 Like