Hi! This is Dennison from WithTally.com, we’re building on top of the Graph using Subgraphs to power deep, rich, data about governance ecosystems and their accompanying token ecosystems.
Our subgraphs are quite large and take many days to sync, but provide us with a fantastic insight into what is happening inside governance ecosystems.
The problem is however because our subgraphs go beyond the simple “indexing events” use case, there are plenty of opportunities to introduce bugs and we frequently find ourselves stuck in the rather long development cycle of developing, deploy, sync, test.
We are sensitive to the accuracy of data we have developed an internal tool to monitor our subgraphs in production and inform us if we spot any anomalies in the live data.
However, this solution is not perfect and introduces its own problems because we need to compare the methods we used to aggregate data in subgraphs to a different formula to aggregate data in the production monitoring when in both cases the source of truth is actually the blockchain: not either “view” of the data we generate.
At its core the problem is this:
The blockchain is the source of truth, a subgraph is a “view” on this source of truth which applications downstream interpret as a source of truth.
Verifying that the subgraphs “view” generated by the mappings is correct is problematic because it requires a comparison tool which also needs to generate a “view” on the underlying source of truth (the blockchain) to test.
Unfortunately, The Graph ecosystem (and assembly script at large) lacks a comprehensive unit testing tool. This means the test cycle is currently either very cumbersome (creating entire demo ecosystems- local subgraph nodes, local smart contract deployments, etc…) or it is exceptionally long- deploying to production, waiting (in our case DAYS) before a bug becomes evident for a new deployment, making a fix, and then proceeding to resync.
What is needed is the ability to test mappings in isolation and in concert to understand, prior to deployment, if they are doing what we think they are doing, and to test them against deterministic data sources/events.
We already have an idea of how this would be done and have been speaking with another team that would work with us on the technical capacity to tackle this. We’ve also spoken with a number of other individuals in the space who also feel that Unit Testing is key for building a more robust Graph Ecosystem.
Unit testing is necessary for The Graph to be the solution for teams in production with mission-critical subgraphs. We would be happy to spearhead a grant proposal but we also recognize this is a project that would probably involve a larger number of participants. Would be interested to see what others think.