Subgraphs management

Subgraph management

The Graph is spreading across the Internet and it is used with a lot of different projects.

At some point it may be required to manage deployed subgraphs. It is generally related to the cases:

  1. Deploy subgraph
  2. Deploy new version of subgraph - the old one will remain in the list and database
  3. Reassign subgraph to a different index node
  4. Pause indexing subgraph - subgraph will remain in the list and database

For this, we can use the following approaches

  1. Graph CLI
  2. Graph node API
  3. Graphman

Graph CLI

The Graph CLI (graph-cli) is a tool for subgraph build and deployment

# Install graph-cli
yarn global add @graphprotocol/graph-cli

# Install dependencies
yarn install

# Codegen
yarn codegen

# Build
yarn build

# Graph create
graph create ${SUBGRAPH_NAME} \
  --node ${INDEX_NODE_ENDPOINT} \

# Graph deploy
graph deploy ${SUBGRAPH_NAME} \
  --version-label ${GITHUB_REF_NAME} \
  --node ${INDEX_NODE_ENDPOINT} \
  --ipfs ${IPFS_ENDPOINT}

Graph node API

We can find in graph-node/server/json-rpc/src/lib.rs defined 4 methods

subgraph_create
subgraph_deploy
subgraph_remove
subgraph_reassign

And we can interact with the Graph node JSON-RPC using the port set by --admin-port option which is by default 8020 and use different HTTP clients like httpie and curl

# Debian
apt install httpie curl

# RHEL
yum install httpie curl

# Python
pip install httpie

Create subgraph deployment

# httpie
http post index-node:8020 \
  jsonrpc=2.0 \
  id=2 \
  method=subgraph_create \
  params:='{"name": "indexer-agent/ke3y2c7Kai"}'

# curl
curl -X POST index-node:8020 \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc": "2.0", "id": "2", "method": "subgraph_create", "params": {"name": "indexer-agent/ke3y2c7Kai"}}'

Deploy subgraph

# httpie
http post index-node:8020 \
  jsonrpc=2.0 \
  id=2 \
  method=subgraph_deploy \
  params:='{"name": "indexer-agent/ke3y2c7Kai", "ipfs_hash": "QmWK7Phe4L6H3dzXHh48G1TrYDugeAVaFroUke3y2c7Kai"}'

# curl
curl -X POST index-node:8020 \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc": "2.0", "id": "2", "method": "subgraph_deploy", "params": {"name": "indexer-agent/ke3y2c7Kai", "ipfs_hash": "QmWK7Phe4L6H3dzXHh48G1TrYDugeAVaFroUke3y2c7Kai"}}'

Reassign subgraph

# httpie
http post index-node:8020 \
  jsonrpc=2.0 \
  id=2 \
  method=subgraph_reassign \
  params:='{"ipfs_hash": "QmWK7Phe4L6H3dzXHh48G1TrYDugeAVaFroUke3y2c7Kai", "node_id": "index_node_2"}'

# curl
curl -X POST index-node:8020 \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc": "2.0", "id": "2", "method": "subgraph_reassign", "params": {"ipfs_hash": "QmWK7Phe4L6H3dzXHh48G1TrYDugeAVaFroUke3y2c7Kai", "node_id": "index_node_2"}}'

Pause subgraph

Assign to non-existing node id

# httpie
http post index-node:8020 \
  jsonrpc=2.0 \
  id=2 \
  method=subgraph_reassign \
  params:='{"ipfs_hash": "QmWK7Phe4L6H3dzXHh48G1TrYDugeAVaFroUke3y2c7Kai", "node_id": "paused"}'

# curl
curl -X POST index-node:8020 \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc": "2.0", "id": "2", "method": "subgraph_reassign", "params": {"ipfs_hash": "QmWK7Phe4L6H3dzXHh48G1TrYDugeAVaFroUke3y2c7Kai", "node_id": "paused"}}'

Resume subgraph

Assign to an existing node id

# httpie
http post index-node:8020 \
  jsonrpc=2.0 \
  id=2 \
  method=subgraph_reassign \
  params:='{"ipfs_hash": "QmWK7Phe4L6H3dzXHh48G1TrYDugeAVaFroUke3y2c7Kai", "node_id": "index_node_2"}'

# curl
curl -X POST index-node:8020 \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc": "2.0", "id": "2", "method": "subgraph_reassign", "params": {"ipfs_hash": "QmWK7Phe4L6H3dzXHh48G1TrYDugeAVaFroUke3y2c7Kai", "node_id": "index_node_2"}}'

Remove subgraph

# httpie
http post index-node:8020 \
  jsonrpc=2.0 \
  id=2 \
  method=subgraph_remove \
  params:='{"name": "indexer-agent/ke3y2c7Kai", "ipfs_hash": "QmWK7Phe4L6H3dzXHh48G1TrYDugeAVaFroUke3y2c7Kai"}'

# curl
curl -X POST index-node:8020 \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc": "2.0", "id": "2", "method": "subgraph_remove", "params": {"name": "indexer-agent/ke3y2c7Kai", "ipfs_hash": "QmWK7Phe4L6H3dzXHh48G1TrYDugeAVaFroUke3y2c7Kai"}}'

Will remove the mapping from the given name to the underlying deployment. If no other subgraph name uses that deployment, it becomes eligible for removal, and the steps for removing unused deployments will delete its data.

Graphman

The graphman command is included in the official containers, and we can docker exec into our graph-node container to run it. It requires a configuration file. For more information please see Common maintenance tasks.

  1. Login to the graph node container

  2. Create config file

    cat << EOF > graph-node.toml
    [store]
    [store.primary]
    connection = "postgresql://$postgres_user:$postgres_pass@$postgres_host:$postgres_port/$postgres_db"
    pool_size = 3
    
    [deployment]
    [[deployment.rule]]
    store = "primary"
    indexers = [ "default" ]
    
    [chains]
    ingestor = "default"
    EOF
    
  3. Get information about subgraph

    graphman --config graph-node.toml info indexer-agent/ke3y2c7Kai
    
  4. Unregister a subgraph name with a Graph node

    graphman --config graph-node.toml remove indexer-agent/ke3y2c7Kai
    
  5. Compiles a list of unused deployments

    graphman --config graph-node.toml unused record
    
  6. Inspect the list

    graphman --config graph-node.toml unused list -e
    
  7. Remove the deployments that have previously marked for removal

    graphman --config graph-node.toml unused remove
    
  8. Reassign subgraph

    graphman --config graph-node.toml reassign QmWK7Phe4L6H3dzXHh48G1TrYDugeAVaFroUke3y2c7Kai index_node_2
    
  9. Pause subgraph

    graphman --config graph-node.toml reassign QmWK7Phe4L6H3dzXHh48G1TrYDugeAVaFroUke3y2c7Kai paused
    
  10. Resume subgraph

    graphman --config graph-node.toml reassign QmWK7Phe4L6H3dzXHh48G1TrYDugeAVaFroUke3y2c7Kai index_node_1
    
3 Likes

Very nice post!
I forwarded this to #indexers-software channel in Discord and pinned it!

1 Like

Thank you! As an inspiration I used your and other members notes from the mentioned channel - just to have all in one place and easy accessible in the web.

2 Likes

Very helpful commands for all the Graph indexers out there.
The commands will be great help for new indexers joining the network.

1 Like