# Request for information about disputes #GDR-14 **URL:** \ **Category:** Arbitration\ **Created:** [July 19, 2022, 5:03pm UTC](https://forum.thegraph.com/t/request-for-information-about-disputes-gdr-14/3478 "2022-07-19T17:03:31Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![86b](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.thegraph.com/86b/32/752_2.png) [@86b](https://forum.thegraph.com/u/86b)\ **Post date:** [July 19, 2022, 5:03pm UTC](https://forum.thegraph.com/t/request-for-information-about-disputes-gdr-14/3478/1 "2022-07-19T17:03:31Z")
The arbitrators are contacting indexer address `0xd75c4dbcb215a6cf9097cfbcc70aab2596b96a9c` **(semiotic-indexer)** about disputes filed in the protocol. **To make the investigation as brief as possible, please provide the following information and any other relevant records about the open disputes:** - Version of graph-node used. - Graph-node configuration in full. - Type and version of Ethereum node. - Table of PoIs generated by the affected subgraphs. - A dump of the call cache data from the database for the affected subgraphs. - Entity values as of the divergent block once identified. **This is not an all-inclusive list. Request for additional information may be made if considered necessary.** ### How to Get Relevant Data You can use the following queries to extract information from the indexer software to be used in the evaluation. ```auto # Get subgraphs schema SELECT name FROM public.deployment_schemas where subgraph = 'QmZDfRpakj4BTNaRUDiCAknzE5rbXQcQm2hRYzX8kNSpx6'; # Move to the subgraphs schema SET SEARCH_PATH TO ; # Dump PoI TABLE - (from outside of psql console but on server) pg_dump --dbname=" --port=5432 --username= --table='."poi2$"' --file='' # Dump call cache (on some setups this table may be in a different schema, check it with `select * from public.chains`) pg_dump --dbname=" --port=5432 --username= --table='public.eth_call_cache"' --file='' ``` Once a divergent block is identified: ```auto # loop through all entity tables and get changes for that block # for each entity table in subgraph deployment schema: select * from where lower(); ``` ### Purpose of the Requirement **This requirement is related to the following disputes:** ```auto Dispute (0xc2f035f63b01216acb12d66e9720b0242c4c678575ab9ec201b25d0d2b1ac72e) ├─ Type: Indexing ├─ Status: Undecided (2.97 days ago) ├─ Indexer: semiotic-indexer (0xd75c4dbcb215a6cf9097cfbcc70aab2596b96a9c) ├─ Fisherman: 0xc56961836857210e256d71c91a62e90865075380 ├─ SubgraphDeployment │ └─ id: 0xa1a64dc25cf8e6bb6adc4072b3027cd12efca9d323421f91859e1bc796cf2def (QmZDfRpakj4BTNaRUDiCAknzE5rbXQcQm2hRYzX8kNSpx6) ├─ Economics │ ├─ indexerSlashableStake: 250000.0025 GRT │ └─ indexingRewardsCollected: 34.652382949899126087 GRT ├─ Allocation │ ├─ id: 0x55c86a7725d187bd79ac82d93271d7752d63fc64 │ ├─ createdAtEpoch: 554 │ ├─ createdAtBlock: 0x2d8f762a19658256cd2d9412bc0091dca449f03cfd3fb31aea2794de0243698a │ ├─ closedAtEpoch │ │ ├─ id: 555 │ │ └─ startBlock: 0x13459e090c49bdb217e9609b160685b9a1db155181bdfb3db262a33122c3f39f (#15135298) │ └─ closedAtBlock: 0xf64c1a25f755d744f744fdef917eb5760235e6436f7f4f6d35d1142a598bb6e4 (#15136959) └─ POI ├─ submitted: 0x646f45f2e0d2b4a838dcbf8c5030a0505c3b7b2bb734bc6d16d708f4d91fc07d ├─ match: Not-Found ├─ previousEpochPOI: Not-Found └─ lastEpochPOI: Not-Found ``` ### About the Procedure The Arbitration Charter regulates the arbitration process. You can find it in Radicle project ID rad:git:hnrkrhnth6afcc6mnmtokbp4h9575fgrhzbay For communications, please use this forum. Additionally, please monitor _Graph Protocol’s Discord #arbitration_ ([https://discord.gg/9gg7XvfggW](https://discord.gg/9gg7XvfggW)) for any arbitrator reaching out for more information. ---
**Author:** ![kaiw](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.thegraph.com/kaiw/32/1487_2.png) [@kaiw](https://forum.thegraph.com/u/kaiw)\ **Post date:** [July 21, 2022, 4:13pm UTC](https://forum.thegraph.com/t/request-for-information-about-disputes-gdr-14/3478/2 "2022-07-21T16:13:35Z")
Hi @86b , I’ve received the dispute in regards to the Europa subgraph (ID QmZDfRpakj4BTNaRUDiCAknzE5rbXQcQm2hRYzX8kNSpx6) and am in the process of reviewing the request. Unfortunately, I no longer have the subgraph deployment schema for this subgraph, as I removed the schema from the database after its failure: ```auto indexer@indexer-shell-0 [~] $ history | grep graphman | grep QmZDfRpakj4BTNaRUDiCAknzE5rbXQcQm2hRYzX8kNSpx6 | grep -v grep 436 graphman info QmZDfRpakj4BTNaRUDiCAknzE5rbXQcQm2hRYzX8kNSpx6 437 ./graphman info QmZDfRpakj4BTNaRUDiCAknzE5rbXQcQm2hRYzX8kNSpx6 439 ./graphman info QmZDfRpakj4BTNaRUDiCAknzE5rbXQcQm2hRYzX8kNSpx6 indexer@indexer-shell-0 [~] $ ``` I understand this was a mistake on my part. However, I will collect the eth\_call\_cache from around the time period requested. The call cache will be up until 14 July 2022. According to my shell’s history file, the POI was generated for block get\_manual\_poi.sh QmZDfRpakj4BTNaRUDiCAknzE5rbXQcQm2hRYzX8kNSpx6 14762100, one block prior to the reported failure. I did attempt to close this allocation with a 0x0 POI, but was unable to due to an error message in the contract which I did not understand. (See linked transactions below.) I now understand that the error message indicated that I was to wait more than some number of epochs. - [Tx 0xe03fa0c946ce5c4214cf13cdcd849825aec0ba9e3c086368db6c26c92cf911cc](https://etherscan.io/tx/0xe03fa0c946ce5c4214cf13cdcd849825aec0ba9e3c086368db6c26c92cf911cc) - [Tx 0x08ad015c68ebd6f3f5d75fb95d27d39113512a7380b1bf0a04037f038a45d6c2](https://etherscan.io/tx/0x08ad015c68ebd6f3f5d75fb95d27d39113512a7380b1bf0a04037f038a45d6c2) - [Tx 0x466d524166bb442b9302999f92b93bb3e26f8c0a7f31b0fab6e8e2485dc47412](https://etherscan.io/tx/0x466d524166bb442b9302999f92b93bb3e26f8c0a7f31b0fab6e8e2485dc47412) Our indexer is using Erigon Alpha v2022.06.06, graph-node version is v0.26.0, indexer-agent 0.19.3, and indexer-service 0.19.3 with the auto-agora patch. Some background: this subgraph came up as a result of our testing the allocation optimiser tool which is one on the core developers’ task list. We’re working through a number of different use cases and patterns to try and reflect the wide variety of practices amongst the indexers. Regards, Kai ---
**Author:** ![86b](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.thegraph.com/86b/32/752_2.png) [@86b](https://forum.thegraph.com/u/86b)\ **Post date:** [July 23, 2022, 6:08am UTC](https://forum.thegraph.com/t/request-for-information-about-disputes-gdr-14/3478/4 "2022-07-23T06:08:58Z")
Thanks for the reply Kai! Just to clarify here: You ended up closing out the allocation using `14762100` as your reference block for the manual POI calculation? Also, would you mind sharing your `get_manual_poi.sh` to better help understand how it’s being calculated on your end? Thanks! ---
**Author:** ![kaiw](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.thegraph.com/kaiw/32/1487_2.png) [@kaiw](https://forum.thegraph.com/u/kaiw)\ **Post date:** [July 25, 2022, 9:20pm UTC](https://forum.thegraph.com/t/request-for-information-about-disputes-gdr-14/3478/5 "2022-07-25T21:20:25Z")
Sure thing, I’ve off-chained Europa to review for this RFP, and it failed right on the start block. I’m not certain how our nodes made it all the way to 14762100. I suspect that our nodes somehow skipped over the current failing block of 14762089, possibly because of an indexer node crash. I recall us having a number of problems with the index nodes crashing because of some subgraphs failing non-deterministically at the time. I’ll upload the schema as it currently sits on our DB now. As far as the POI calculator goes, what it really does is queries the administrative endpoint for a POI then reports back with that. Sadly, I’m not familiar enough yet with the graph node code base to know how that GQL query is answered under the hood. See the code below: ```auto #!/bin/sh # Find all subgraph health: #query { # indexingStatuses { # subgraph # synced # health # } #} INDEXER_ADDRESS='0xd75C4Dbcb215A6cf9097cfbCc70aAB2596B96a9C' SUBGRAPH_DEPLOYMENT=$1 BLOCK_NUMBER=$2 if [-z "$INDEXER_ADDRESS"]; then echo 'Environment variable INDEXER_ADDRESS must be set to a valid indexer address' exit 1 fi if [-z "$SUBGRAPH_DEPLOYMENT"] || [-z "$BLOCK_NUMBER"]; then echo "Usage: $0 [subgraph deployment IPFS hash] [last good block number]" exit 1 fi BLOCK_HEX=$(echo "obase=16; $BLOCK_NUMBER" | bc | xargs printf "0x%s\n") BLOCK_HASH=$(curl \ -s -X POST -H 'Content-Type: application/json' \ --data '{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["'$BLOCK_HEX'", false],"id":1}' \ $INDEXER_EVM_RPC_URL | \ jq -r '.result.hash') if [-z "$BLOCK_HASH"]; then echo "Block $BLOCK_NUMBER not found" exit 2 fi gql_query='query { proofOfIndexing(subgraph: \"'$SUBGRAPH_DEPLOYMENT'\", blockHash: \"'$BLOCK_HASH'\", blockNumber: '$BLOCK_NUMBER', indexer: \"'$INDEXER_ADDRESS'\") }' json="{\"query\":\"$gql_query\"}" POI=$(curl \ -s -L -X POST -H 'Content-Type: application/json' \ -d "$json" http://query-node:8030/graphql | \ jq -r '.data.proofOfIndexing') echo $POI ``` ---
**Author:** ![kaiw](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.thegraph.com/kaiw/32/1487_2.png) [@kaiw](https://forum.thegraph.com/u/kaiw)\ **Post date:** [July 25, 2022, 10:22pm UTC](https://forum.thegraph.com/t/request-for-information-about-disputes-gdr-14/3478/6 "2022-07-25T22:22:28Z")
Just confirmed, our index node is skipping blocks… somehow. It seems that a restart triggered it to jump up to 14762092. See below: ![image](https://us1.discourse-cdn.com/flex020/uploads/thegraph1/original/2X/4/4f8e381de726e60bc5b478d5f1018280a58107a2.png) ---
**Author:** ![kaiw](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.thegraph.com/kaiw/32/1487_2.png) [@kaiw](https://forum.thegraph.com/u/kaiw)\ **Post date:** [July 25, 2022, 10:33pm UTC](https://forum.thegraph.com/t/request-for-information-about-disputes-gdr-14/3478/7 "2022-07-25T22:33:56Z")
For the record, all non-pasteable documents will be posted to this [public folder](https://drive.google.com/drive/folders/1e3Ytul2PsZW-jXbWQyo9HYo9rWDum5ey). Our pod config is in the tar file called `rfp-gdr14-podcfg.tgz`. ---
**Author:** ![indexer\_payne](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.thegraph.com/indexer_payne/32/44_2.png) [@indexer\_payne](https://forum.thegraph.com/u/indexer_payne)\ **Post date:** [July 26, 2022, 10:10am UTC](https://forum.thegraph.com/t/request-for-information-about-disputes-gdr-14/3478/8 "2022-07-26T10:10:50Z")
I saw this happening to one of my subgraphs too. If you search in discord you’ll see me posting about it. I’m on mobile right now, so I don’t know which one it is, but it’s a problem for sure. ---
**Author:** ![kaiw](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.thegraph.com/kaiw/32/1487_2.png) [@kaiw](https://forum.thegraph.com/u/kaiw)\ **Post date:** [July 26, 2022, 6:54pm UTC](https://forum.thegraph.com/t/request-for-information-about-disputes-gdr-14/3478/9 "2022-07-26T18:54:12Z")
Thanks for the heads up @indexer_payne ! It seemed similar to [Bug 3236](https://github.com/graphprotocol/graph-node/issues/3236) reported by @alexis some time back. To better understand the behaviour, I simulated the same conditions by restarting the index node offchaining Europa, but that restart does not seem to be the jump-to-next-block trigger point. I’ve seen this happen on a couple of other subgraphs in the past, however they behaved similar to 3236, not this. Any hypotheses are welcome. ---
**Author:** ![86b](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.thegraph.com/86b/32/752_2.png) [@86b](https://forum.thegraph.com/u/86b)\ **Post date:** [December 22, 2022, 9:21pm UTC](https://forum.thegraph.com/t/request-for-information-about-disputes-gdr-14/3478/11 "2022-12-22T21:21:03Z")
**Previous Notes:** This dispute was made for the same reasons as [GDR-13](https://www.notion.so/GDR-13-9c280619f3a4421985bd26c75cd5f0ad) by the same Fisherman where the reasons for submitting the dispute are the same. Like GDR-13, this indexer also wasn’t using a pre-release version of `graph-node` which allows Europa to be fully indexed, but it appears after feedback with Kai from Semiotic that they were operating in good faith and going off a POI script they use to close out failed subgraphs. However, there also seems to be some confusion on Semiotic’s behalf (like Protofire’s in GDR-13) regarding which block hash and number to use when calculating a failed subgraph’s POI. Based on the findings, it appears that both parties were operating in good faith, but clarity will need to be made with regards to which block to use in POI calculations so that we can establish rock-solid guidelines for failed subgraph POIs. Due to this, the Arbiters settled this dispute as a draw: > **[Ethereum Transaction Hash (Txhash) Details | Etherscan](https://etherscan.io/tx/0x51dfbd174377be4b0d087c9058d2bca93aad186e6a8134250af82687aeb950c3)** > > Ethereum (ETH) detailed transaction info for txhash 0x51dfbd174377be4b0d087c9058d2bca93aad186e6a8134250af82687aeb950c3. The transaction status, block confirmation, gas fee, Ether (ETH), and token transfer are shown.