StreamingFast July 2024 Update

Research & Development Core Team Updates
1

Hey everybody,

Here’s a quick update from StreamingFast.

Perhaps a quick primer on our objectives and key results for the quarter:

  • We’re looking to optimize the onboarding time for someone new to Substreams, and measure that with better precision. We’ll be doing customer interviews and test what works, and where things stall.
    • Any feedback on that front will be highly welcome.
  • We want to see the number of Substreams-powered-subgraphs deployed on the network rise, so we’ll be making sure all the tools, and the experience there is flawless (for new chains, we want to polish the experience, to make it on par with Ethereum).
  • We’re putting efforts to clarify and understand who our Substreams users are (personas). We’ll be conducting interviews to gather quantitative data about the stacks, contexts, programming languages, expectations of customers and refining personas, with the goal of accelerating decision making, and hitting the mark more often than not, with future developments.

We’re also continuing to integrate new chains and protocols, and are building great expertise in that domain, which helps with any new integration.

Looking back :tada:

  • Our DevRel folks presented to Injective developers, our newly introduced stack.
  • We found a bug in the Substreams store merging operations, and fixed it. We analyzed affected caches, rebuilt them, and we notified those affected. Please upgrade your Substreams instances to the latest versions to make sure this doesn’t affect you.
  • We shipped new code generation tools, with remote building capabilities, to smooth onboarding. Substreams Codegen — No-code Tool to Bootstrap Your Project | by StreamingFast | Jun, 2024 | Medium
  • We’ve now sync’d the Base blockchain, with Extended trace data (some had rolled out the poller, but now we have the most complete data).
  • Substreams.dev saw many improvements, like the Protobuf Docs Explorer on the side of pages like this: Substreams Registry and Developer Portal
    • https://thegraph.market also saw a few important bugs squashed too; please always let us know if you see issues on these sites!
  • We’ve been working on optimizing storage costs, and digging into resource usage for a Substreams cluster.
    • We’ll be able to publish more numbers on what is the true cost of running Substreams.
    • We are doing this in a cloud environment, and would appreciate input by those running in different environments to gather a more holistic view of the different trade offs (thinking bare-metal providers here); thanks in advance for your contribution!
  • We’re been gearing up to structure ourselves more like a product company, to make sure Substreams succeeds and brings its full potential to The Graph Network.
  • We had spoken about bringing the no-code tools to the Web UI, but have chosen instead to make the onboarding flow an integrated experience within Dev Containers, which solves many more issues than just simplifying the first package creation. See below.
  • We collected a few more performance metrics on Firehose as a drop-in replacement to JSON-RPC in graph-node, re-confirming it’s still always faster/cheaper, and up to 3x faster.

Looking ahead :rocket:

  • We’re working on improving the onboarding flow, and making the developer experience sicker than everywhere else in the blockchain space.
    • Hint: we’ll be releasing something based on https://containers.dev with an integrated experience. GitHub Codespaces, VSCode Desktop, DevPod, let the fun begin.
  • Quite a bit of improvements are coming for substreams, like an improved gui experience, a new substreams build command to abstract a few things away, an authentication flow (with substreams auth or otherwise), to thegraph.market to start streaming fast, some new substreams codegen tools to scaffold subgraphs and SQL sinks out of a Substreams package. Those things should ship soon, stay tuned.
  • Finalize the work on bringing two new chains to The Graph Network, one of which is the first supported Polkadot/Substrate chain.
  • We’re also finalizing the patterns of Foundational Modules for the different chains, as a way to quickly get running, with higher abstractions on chains with different data models.

We’re always happy to hear more about you, in here, or on our Discord! Come on, join us!

1 Like