StreamingFast May 2024 Update

Research & Development Core Team Updates
1

Good morning citizens of the world!

:tada: Looking back (what was delivered)

  • A major milestone was achieved in the Ethereum ecosystem, with the release of geth v1.14.0, which includes a native tracer for the Firehose! The battle is still on for the best Ethereum client!
    • See our article covering the news, and what it means for The Graph.
  • The 1.6.0 release of Substreams adds a brand new kind of modules, the blockIndex modules, which are Rust modules that output a list of tags or labels.
    • You can then use the blockFilter syntax on a downstream module, and query for certain tags. The engine will then skip execution of any blocks where those tags are not present, and is also capable of skipping huge amounts of blocks, saving valuable resources for those sparse Substreams.
    • Imagine labels like contract:0x12312312301231231230123121230123123210 or event:topic.0:0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef or method:0x3593564c or uniswap:swap (meaning there’s at least one swap from uniswap in that block).
    • The caching layer recently released for internal mappers outputs also means that much less data is consumed when using pre-built blockIndex modules.
    • We’ve also written a bunch of general purpose indexes, starting with Ethereum, and we’ll continue this month.
  • https://substreams.dev is getting a constant refresh, with Jeremy hard at work. New features are lined up for this app.
  • We started work on a new Beta Payment Gateway!
    • We’ve brought query fees on-chain for all of our customers’ consumption of Substreams and Firehose for the month of April. This is real usage of Substreams/Firehose, payed for by real customers.
    • We’re still tooling things around to bring more transparency on-chain, but we wanted to hit the ground rolling with a real impact on query volume.
    • We’ve also been testing the Auth and Billing Protocol that we’ve designed, with a few indexers (thanks to the Pinax guys in particular), and internally.
  • I secretly merged Go support for building Substreams modules. It’s beta, don’t ask for support, but do provide feedback… it’s just for the very curious of you all… but it seems to work! See an example here: substreams/wasm/bench/tinygo/main.go at develop · streamingfast/substreams · GitHub (kudos to Charles and Colin for lot of the work in there)

We’ve also made the impressive feat of augmenting by the log of 5x the number of square bits per seconds processed, every minute! Let us know if you want to know more!

:rocket: Looking ahead

  • Still continuing our work on no-code (code-generation) for Substreams modules.
    • substreams init will be revamped to use a new remote code generation framework, that will provide pre-built spkg instead of requiring the end user to install all the dependencies to build modules.
    • The same UI will be brought to substreams.dev, as an interface conversational UI.
    • One of the first modules will be one for Cosmos-based chains (Injective in particular), to bring them right into the heart of graph-node, using the newly supported Substreams to mappings released 0.35
  • We’re continuing work on the new Payments Gateway, that will live at https://thegraph.market and will be bringing more players aboard, and bringing more query fees on-chain.
    • This is the short-term path to bringing Firehose and Substreams to the network. There are other avenues that we’re keeping tabs on and will follow-up with the right teams to make sure we have the best experience for these new products and a bunch of others, on The Graph network.
    • That’s quite exciting and hopefully will have a great impact on the economy.
5 Likes
2

Thanks @abourget, For sharing this information.

1 Like