GRP-0001 Decorator Hooks, Cron Jobs and Initialisation Handlers

by @Mack and @Slimchance

Abstract

Subgraph developers would like a way to hydrate the store when the subgraph is deployed. They would also like ways to run a function at a certain block/timestamp or block/time interval.

  • The suggestions have a “BlockNumber” and a “TimeStamp” variant. The TimeStamp variants would trigger on the first block following the specified timestamp.


Motivation


Initialisation Handler

Subgraph developers may wish to run a handler on startup:

  • Hydrate the store with data found in one of the mapping files, json or on IPFS. This would be useful in several cases. This could be contant data like enums and other entity types which need to exist before other entities are created. (The current solution to this is to put a function that only runs once in the eventHandler of the first eventType to be emitted.)

  • If a subgraph developer wants to index a list of similar contracts, they would have to create a separate datasource for each one. An initialisation handler allows for a much cleaner pattern: Create a template with the common ABI/ handlers. Then, within the initialisation handler create a dynamic data source for each of the contracts.


initialisationHandler:
    handler: myInitialisationHandler



Cron Jobs & Filters

One use case we have thought about for Cron Jobs is mapping code that would only need to be run every x blocks.

Handlers are run on every matching trigger;

  • Blockhandlers are run every block
  • Event- and callhandlers are run on every matching event/call - This can sometimes be multiple times in a single block.

For some usecases it is redundant to run the mapping code this often. Allowing subgraph developers to define an interval for which to run the mapping logic, would be a massive performance increases for these subgraphs.

  • This would be very useful when creating timeseries data
  • This would be useful when utilizing blockhandlers

Two types of Cron Jobs:

  • On a blockhandler (“Run this handler every x blocks/seconds”)
  • On other handlers (“If this handler have been run within x blocks/seconds => Skip”)
blockHandlers:
    - handler: myCronHandler
        filter:
            kind: cron



Decorator => Hook

Subgraph developers may wish to add hooks at certain blocknumbers/timestamps. This would run a piece of code at a certain blocknumber/timestamp.

  • Update the state of an entity when an entity attribute blocknumber/timestamp is reached. Runs once.
  • Run hook logic when declared blockNumber/timestamp is reached. Runs once.
  • Hooks receive the entity that triggered the block- or timetrigger.
  • In our example below, Hooks need to be unique to entities.

---- schema.graphql

type Auction @entity {
  id: ID!
  item: Artwork!
  seller: Account!
  currency: Bytes! 
  isLive: Boolean!
  startTime: BigInt! @blockTrigger => startAuction() // Trigger on startTime blockNumber
  endTime: BigInt! @blockTrigger => endAuction()
  winner: Account
}

type Artwork @entity {
  id: ID!
  isRevealed: Boolean!
  revealTimestamp: BigInt! @timeTrigger => revealed() // Trigger at revealTimestamp
}

---- src/hooks/index.ts

function startAuction(entity: Auction) {
    entity.isLive = true;
    entity.save()
}

function endAuction(entity: Auction) {
    entity.isLive = false;
    entity.save()
    
function revealed(entity: Artwork) {
    entity.isRevealed = true;
    // Update URI from contract tokenURI function and call ipfs for new metadata 
    entity.save()
}


Suggestions on how to run the Blocktriggers


Our initial suggestion would be to run an internal job in graph-node that could register each entity that has a blockTrigger with a non-null blockNumber (or store this in a lookup table), attaching the blockHandler to call the required hook, only at registered blocks.

This would in theory reduce the amount of times that the blockHandler would need to run, and would, in theory, reduce the impact blockHandlers have on index and sync time.

The same features could also allow for a timestamp instead of a blocknumber. This could be more intuitive to subgraph developers. It could be especially important on networks with variable blocktime. Similarly, there could be a lookup table with all the timestamps that has a trigger or hook. When a new block has a timestamp equal or higher than the smallest timestamp in the lookup table, the handler/code would be run.


# BlockTrigger Lookup Table / Queue

# Could be used for Cron Queue as well

type BlockTriggerLookup @entity {
    id: ID                
    nameOfEntity: String! # Used to Read Entity from Store
    entityId: String!     # EntityID
    blockNumber: BigInt!  # When to Trigger Hook
    timestamp: BigInt!    # Alt. When to Trigger Hook
    hookToRun: String!    # Which Hook to Run
}



Building on this concept

This could also lay some of the groundwork for dynamic handlers - as this is a dynamic blockHandler that would only map and trigger on certain blocks (or ranges of blocks).

In addition it could also have a skipHook for transactions and events the user would want to skip, and a endHook/endBlock, to stop listening at that block to an event or trigger.




Made with :green_heart: by Graphrica :earth_africa:

11 Likes

:rocket: This would make subgraph development a bit more flexible.

2 Likes

Very cool idea, thanks for motivating

2 Likes

Let us know if you would like a walkthrough of the thinking :eagle: :up:

1 Like

That would be great. I’m curious as to some of the use case examples that come to mind initially for the Init Handlers that can help contextualise this further.

2 Likes

Thanks for asking :slight_smile:

  • Subgraph developers may wish to hydrate the store with data.
    • One example would be metadata that is not found on-chain, and that is needed in the mapping code.

    • Another example would be storing static data that is needed in the frontend to make their application serverless.

    • A common workaround is to create an onStartup function in the mapping code. You would then identify which handler is triggered first. Inside this handler, you would check if onStartup has been run. (typically by loading and doing a nullcheck on a specific entity from the store). If the entity is null, you would run your onStartup mapping code.

      1. This workaround is not very elegant or intuitive
      2. If a handler runs for too long, it will time out and throw an error. With the workaround described above, a lot of work is done within a single (regular) handler. It is not uncommon to see timeouts when this workaround is used. An initialisation handler could have a higher “gas limit” (See GIP-0005: Gas Costing for handlers)

  • Subgraph developers may wish to index a list of similar contracts that have a common set of triggers.

    • For simplicity sake, let’s take an example where a subgraph developer wants to index 200 ERC-20 tokens. The apparent pattern is to create 200 separate datasources.
    • An initialisation handler allows for a much cleaner pattern: Create a template with the (common) ABI/handlers. Then, inside the initialisation handler, loop through an array with the 200 contract addresses. For each address, they would instantiate a dynamic data source using the template.

This GRP would be incredibly useful for Soulbound Labs! We’ve wanted to award badges with built-in expiration dates, but haven’t found a clean paradigm we were happy with.![:crossed_fingers:]

2 Likes

I’m definitely in favor of this. I have a couple pretty immediate places I would implement this if it were available.

1 Like

I look forward to seeing Cron Jobs and Initialization Handlers become true :slight_smile:

1 Like

Hi! A lot of great ideas here - I think there are actually a few separate features being proposed. Some are ideas which we have explored previously:

The Cron Jobs and Decorators feel like bigger ideas which warrant separate discussion - I think it’s worth understanding the use cases a little better, for example auction liveness seems like it could be handled by a query time resolver (which aren’t currently supported, but have also been discussed for some time)

I think for the purposes of this discussion it makes sense to focus first on initialisation handlers - I wonder if they could be work as a blockHandler which runs only once, i.e.

blockHandlers:
    - handler: myInitialisationHandler
        filter:
            kind: once
3 Likes