Another request:
State of Contract Calls
I know that contract calls are not considered best practice, but not all contracts are written such that events give a complete picture of the smart contract state.
It would be interesting if we could mock contract calls to see how our subgraphs behave on reverted or failed calls.
Ian from Uniswap here. Just wanted to jump in and share that we’d greatly benefit from a testing solution like this.
The Uniswap v2 subgraph takes over 30 days to sync now. Yes you can read that again its true.
Its very difficult to be confident in new versions of the mappings without isolated testing.
@dennison Super interesting point; we’ll need fixtures for not only events but contract calls. I’ve added that to the spec.
@ianlapham A 30 day sync time is insane. That’s like Dennison’s WETH subgraph. The latest iteration of the spec is on Github here. It would be great to have your input so that the spec will meet your needs as well. Please comment or open a Github PR so that we have your input!
Finding a time to meet in person might be challenging, as Dennison is available later and Sebastian is on CET. How flexible are you guys? The earlier the better Dennison, and the later the better Sebastian.
We found a time to meet this Friday at 11am PST. If you @ianlapham or anyone else would like to join, let me know and I’ll shoot you an invite.
Hi everyone! This is Seba from Protofire. It has been about two years of building 20+ subgraphs for our team and as the first member of that team I would like to join to the discussion to put my 2 cents on this topic.
We have also faced challenges from both small subgraphs with high traffic, as well as large subgraphs that take weeks to index the last block, in which we experienced the need of having a more strong testing environment and a granular way to isolate certain scenarios to reproduce a bug or to avoid waiting a few hours/days of indexing to test a fix or validate an improvement. We cannot agree more that unit testing is an increasingly imperative need to provide high-quality subgraphs.
I also wanted to mention that we have been exploring the idea of developing a linter for subgraphs
which also seems like a brilliant idea to leverage the quality of community-developed subgraphs. We wanna capitalize the experience gained during this time developing subgraphs (in which we identify some patterns) and the development of Solhint.
Our team will carefully analyze the proposals made in this thread and get back to you with feedback.
Great call today everyone! I’m really pleased to see people come together like this.
Our big decisions today were:
The Scope of Work has been updated.
We realized that there are many things we want to build, but we should start simple so that it can be shipped quickly. The learning from this tool will inform the next iteration.
@sistemico It would be great to have your input!
Next steps are to find a suitable team to build it.
I want to follow up with the latest news: Limechain is going to build it! It’ll be great to have a professional dev shop work on the project.
Hello! I wanted to provide an update on this, Limechain have been working on this as Brendan mentioned. There were some iterations in terms of approach (many thanks to @fubhy and @Leo for providing guidance & feedback), and we are now at the point where it would be great to get some wider feedback on the framework and developer experience. Please do respond this thread, or DM me here or on Discord if you would be interested!
Fantastic! We’d love to give it a try. I’ll dm you
Further update - please see this Loom video for a demo of the progress so far on Subtest, by Petko from Limechain. The repository is available at Subtest - this is not currently ready for usage, but includes documentation on the overall approach and the user stories from the initial scoping document. It would be great to get feedback on the approach and implementation to inform the ongoing work!
This is really cool! I have a couple of questions:
Petko from Limechain here 
Thanks Brandon! We appreciate all the feedback we can get on this one. I’ll try to answer your questions given the way I see things:
as-pect does an amazing job at encapsulated, general-purpose unit testing.as-pect is great for unit testing. Subtest targets scenarios specific to The Graph, and more precisely - event handler functions and the store. It spins up a mock store and makes assertions on it. That’s it. It doesn’t add a whole lot of new testing capabilities in general to AS. That would be its own project.Thanks again to you and everyone else for the feedback 
Hi Petko! I watched the Loom video and really like what I saw. The interface is exactly as I imagined in terms of setting up the store, running handlers, and asserting the results. Great job!
I have a couple of questions:
as-pect perhaps we could even plug it into existing tools.Hey Brendan, thanks a ton for the feedback, it really means a lot! 
Also - great questions! Here goes:
Just to reiterate, the Subtest framework is split in two parts - the npm module for state methods and helper functions, which is what the subgraph developers will be importing and using in their test files. It’s written in AssemblyScript, because that way we can easily pass data around between the subgraph and the testing library, and also because subgraph devs are already used to AS. The second part is the Rust wrapper, which is used to inject all of the other host functions - type conversions, logging, etc. It’s also what will be used to run the tests and to log their results. Initially we tried to have that bit written in TypeScript, because it would be faster to develop, but we hit a snag, because that would’ve meant we’d need to rewrite all of the type conversions and the other host functions to TS (They are implemented in Rust in the graph-node repo). We figured we could try to port them from Rust to AS, but still that wasn’t any easier (it was actually worse in terms of effort). So we settled on just using them directly as an import in a Rust project. The reason was mostly practical. Along with that, there is a bit of a personal preference involved, since I am an avid Rust fan. And also Rust is a more mature language with a wider ecosystem of tools. Furthermore, we discovered that when importing straight from graph-node, we get a lot of other things for free - methods for calling functions from the WASM module, wrapper types for all sorts of WASM-related data, etc. Got a bit derailed here, but I hope that answers your question. 
To be honest, I haven’t looked at reporters in as-pect much, but they seem very useful and a great addition to the framework in a post-MVP stage (here’s the MVP board for reference).
Interesting! I’m 100% on-board with code reuse. Perhaps down the road the community will create a TypeScript toolchain, but clearly it’s not worth the effort right now. Thank you for the explanation!
One more question: if a test fails does the test runner return a non-zero exit code? That would be a must have for the MVP! Our CI system would appreciate it
No worries! A TypeScript version of the wrapper would indeed be very interesting, and I’d be up for the challenge, downside is it’s going to take a lot longer to build. But if it’s what the community wants it’s what the community wants.
Again, awesome question! Currently it’s not doing that but it’s a breeze to plug it in, as you pointed out, it’ll be very valuable feedback.
Hey guys! Great work so far on the framework. Took it for a test spin (even though I know its not ready yet) and ran into a snag. It is common in mappings to check and see if an entity exists by calling Entity.load() and checking if null. When trying to do this with the mock store, the load() function throws an error rather than returning null if the entity doesn’t exist.
I looked at the issues and project board and didn’t see this mentioned explicitly so i thought I would bring it up. Otherwise this is a fantastic development and way more smooth than our old testing implementation
Thank you! That’s actually a pretty good catch and we will look into it ASAP. This is precisely why we appreciate feedback from subgraph devs!
