Request for information about disputes #GDR-7

Governance & GIPs Arbitration
4

Following we provide the requested Information for this Dispute.

We are just not really sure what output is expected on the “Entity values as of the divergent block once identified” data. We would appreciate further help and information on that part.

Version of graph-node used.

graphprotocol/graph-node:v0.23.1

Graph-node configuration in full.

thegraph:
    image: graphprotocol/graph-node:v0.23.1
    environment:
      BLOCK_INGESTOR: thegraph-index-0001
      ethereum: mainnet:https://api.anyblock.tools/ethereum/ethereum/mainnet/rpc/******/
      ETHEREUM_RPC_MAX_PARALLEL_REQUESTS: 256
      GRAPH_ETHEREUM_TARGET_TRIGGERS_PER_BLOCK_RANGE: 256
      ETHEREUM_BLOCK_BATCH_SIZE: 100
      GRAPH_ETHEREUM_MAX_BLOCK_RANGE_SIZE: 2048
      GRAPH_ETHEREUM_MAX_EVENT_ONLY_RANGE: 1024
      GRAPH_ETHEREUM_MAX_PARALLEL_TRIGGERS_PER_BLOCK_RANGE: 800
      GRAPH_ALLOW_NON_DETERMINISTIC_FULLTEXT_SEARCH: "true"
      RUST_LOG: warn
      GRAPH_KILL_IF_UNRESPONSIVE: "true"
      GRAPH_IPFS_SUBGRAPH_LOADING_TIMEOUT: 180
      GRAPH_IPFS_TIMEOUT: 180
      GRAPH_MAX_IPFS_CACHE_SIZE: 256
      GRAPH_ENTITY_CACHE_SIZE: 40000
      GRAPH_QUERY_CACHE_BLOCKS: 2
      GRAPH_QUERY_CACHE_MAX_MEM: 5000
      GRAPH_GRAPHQL_MAX_FIRST: 5000
      ELASTICSEARCH_URL: http://****
      EXPERIMENTAL_SUBGRAPH_VERSION_SWITCHING_MODE: synced
      postgres_host: ****
      postgres_user: postgres
      postgres_pass: ****
      postgres_db: thegraph
      STORE_CONNECTION_POOL_SIZE: 128
      node_role: index-node
      node_id: thegraph-index-0001
      ipfs: https://ipfs.network.thegraph.com
      ETHEREUM_POLLING_INTERVAL: 125

Type and version of Ethereum node.

Anyblock RPC-API

Table of PoIs generated by the affected subgraphs.

A dump of the call cache data from the database for the affected subgraphs.

Entity values as of the divergent block once identified.

We are not entirely sure what output is expected here and how to identify a divergent block and how to run the expected steps. We would appreciate some help here.

Statement

First of all, I would like to thank you for implementing this system and the role of the fishermen and arbitrators. They serve an important purpose and enable the decentralized system to function reasonably.

To clarify this situation satisfactorily, I would like to give some information and background. We at Anyblock Analytics are currently in the process of automating and optimizing the allocation process. To this end, we are in the process of implementing this project with the help of a grant. (See GitHub - anyblockanalytics/thegraph-allocation-optimization: Allocation Optimization The Graph).

We decided on a strategy for indexing subgraphs where we perform a small allocation on all new subgraphs that show up. We had a global rule to allocate 10 (later 2) GRT to ALL subgraphs with minSignal of 500 (minStake did not work reliably). We have since changed the global rule to ‘never’ and hand pick subgraphs, which ultimately hurts the overall ecosystem in our opinion. This served the purpose of providing new subgraphs directly with an indexer that is in sync and can process queries. However, over time we realized that there were some subgraphs that were either broken or created with bad intentions and then deleted (see also Request for information about disputes #GDR-3 - #5 by indexer_payne). We were not prepared for this case at the time. I.e. we were “baited” and indexed all subgraphs that were defective (now we have developed a script that blacklists most subgraphs that are defective and we have also shared this tool with the other indexers in Discord).

In the course of this, at some point it was no longer possible through the indexer agent to place new manual allocations, or to clear these old allocations for which we had no valid POI for the current epoch.

We came across two sources:

  1. https://hackmd.io/MMHefHW5TxOOLtqqutP-VA?both#9-Valid-Proofs-of-Indexing-for-a-given-epoch which mentions that an indexer must submit a valid POI to the FIRST BLOCK of an epoch. If a POI is valid for previous blocks, then a Dispute is a Draw. It is also mentioned that an indexer can submit the last valid POI if the subgraph contains a bug.

  2. Failed subgraphs - Manually Closing Allocations - The Graph Academy it is mentioned that in order to maximize the rewards of a failed subgraph, every epoch should be checked to find out the last valid POI.

For this purpose, we wrote a tool that for each subgraph we have to clear manually because the indexer agent was blocked, iterates through each allocation and looks for the last valid POI (thegraph-allocation-optimization/poi.py at develop · anyblockanalytics/thegraph-allocation-optimization · GitHub). If none is found, a 0x0 POI is output. If a last valid POI is found, this POI is used to close the allocation. Through research in the last two sources, as well as in Discord, we were convinced that it is OK to specify the last valid POI when manually closing an allocation.

Let us now move on to the first dispute:

This dispute is about allocation 0x7bb9cc693a6e1523fee00b99f4f405dd3df4f876 . This allocation was created in epoch 204 and manually closed in epoch 219. The various sources said that for the epoch in which the allocation is closed, one must specify a valid POI for the FIRST BLOCK. If we check Graph Explorer we see that for epoch 219 the starting block is “12902242”.

If we now query the indexer server for the POI for this block in the respective epoch, using this query:

http -b post http://localhost:8030/graphql query='query poi { proofOfIndexing(subgraph: "QmQKU61VgfxvD1zsRKXydV1cDrirRJnjfmz6jnP3fonYZw", blockNumber: 12902242,blockHash: "0x985c628817706e2b14fdba75b47c5794d4484ec4c593dda76719e02291f730fe", indexer: "0x453b5e165cf98ff60167ccd3560ebf8d436ca86c")}‘

Then you get this POI as result: 0x5cc307ecd069223429de491e4d6a5a91e6d630fa3590dcfa1f5e3a99210bfe3b

We also submitted this POI to close the allocation. Since the closing of the allocation has been done manually and not through the agent, it was sadly re-allocated automatically and continued to sync.

We can also show a valid POI for the block in which the allocation was closed. The allocation was closed at block 12908408. For this block we get the POI 0x5d6a290ea0e8d23ce55e85a7e6ed7de883611b38be8630cb9eec6b43796a5f7b . This can be done with the query:

http -b post http://localhost:8030/graphql query='query poi { proofOfIndexing(subgraph: "QmQKU61VgfxvD1zsRKXydV1cDrirRJnjfmz6jnP3fonYZw", blockNumber: 12908408,blockHash: "0xcd47ca957cda0bc46434ae8e5e05ab1abd06645ccb089bdfb565f9e6829623a1", indexer: "0x453b5e165cf98ff60167ccd3560ebf8d436ca86c")}'

This block is also in epoch 219.

Let us now turn to Dispute 2:

This is the Allocation Id 0x26a964d7fd08092c4b5088e196e6519e3e9f4880 which was created in epoch 219 and closed in epoch 226. It is true in this case that we have no POI for epoch 226. However, we have proceeded here as recommended by the link in The Graph Academy. We found out the last valid POI for an epoch. The last valid POI was in epoch 219. Here we have the same POI as in the previous case because it is the same block and the same epoch. However, we can also provide a POI for the block where the allocation was created:

http -b post http://localhost:8030/graphql query='query poi { proofOfIndexing(subgraph: "QmQKU61VgfxvD1zsRKXydV1cDrirRJnjfmz6jnP3fonYZw", blockNumber: 12953001,blockHash: "0x41cdd1a51905c1e8611ec86604b26fb4111d82034ff5dc1b162c7054c755c6ce", indexer: "0x453b5e165cf98ff60167ccd3560ebf8d436ca86c")}‘

The poi for this block would be 0xd1ddb0d49b62d6a678daa636969cb3ac081d9a89e5a535a6fc1241336c8d4096. With this we can show that we have a valid POI at all. We also had to close the allocation manually because of the same problem with the clogged indexer and therefore took the last valid POI.

Technically, we might have made a mistake, but not with a bad intention. It is very important to us to give something back to the community and that is why we develop all these tools (Blacklisting Tool, Automatic Allocation Tooling, Last Valid POI Generation tooling…) to help the indexers to avoid as much manual work as possible and to automate it without running into malicious bait subgraphs.

Ultimately, if you have a look at the transaction history of our operator account (https://etherscan.io/txs?a=0x6bb16952cf5754651a5b1334422c95e3f55e2c95) you can see that to close all pending and broken allocations on 2021-08-03, we spent over 1 ETH to claim roughly 60 GRT, which is not the best strategy to get rich quick :wink:

2 Likes