Pinax August 2023 Update

Research & Development Core Team Updates
1

:woman_astronaut: Summary

The Pinax team is excited to present our first Core Dev monthly update for The Graph community! :partying_face: The themes for the past month and months ahead are scaling Firehose and Substreams, and enabling indexers and developers to leverage the technologies for themselves with tooling and educational content. We’ve made great progress over the past months tuning our infrastructure to optimize performance for the chains we’ve been supporting over the past year, and adding infrastructure to support additional chains. We’ve delivered Substreams tooling and educational content and have expanded the team to accelerate these efforts. We’re preparing to offer new free and paid Firehose and Substreams services and are thrilled to continue collaborating with The Graph Core Devs and community to bring this technology to the mainstream.

:tada: Looking back (what was delivered)

  • Provide infrastructure for chains currently supported on The Graph’s hosted service & Subgraph Studio
    • In order to prepare adding more blockchains to the decentralized networks, Pinax is working to understand the operational requirements of blockchains currently available on studio and the hosted service, as well as new blockchains that may be added in the future. This includes RPC and/or Firehose/Substreams support. Over the last month, we worked on the following blockchains:
      • Near and Near testnet FIrehose/Substreams
      • BSC Erigon
      • Chapel Erigon
      • Polygon Erigon
      • Optimism
  • Education, onboarding & community support for Indexers and Developers working with new data services
  • Collaboration with core developers and new consumers on designing and testing new data services while augmenting the Indexer experience with new tooling
    • Provided some suggestions for improvements to Near Firehose to the StreamingFast team
    • Ported many of of the Pinax-developed substreams sinks to new substreams-is
    • Added more features to substreams init CLI
    • Ported Antelope Firehose from firehose-acme to firehose-core
    • Did performance testing on the Messari Uniswap v2 substream which synced in around 12 hours
    • Split our Firehose and Substreams endpoints to improve performance
    • Completed a successful benchmark using new hardware: 1000 parallel processes, 10 TiB merged blocks in 3.5 hrs on the EOS blockchain
  • Provide operational support
    • Ongoing support for indexers with Firehose and Substream related questions in Discord

:rocket: Looking ahead (upcoming priorities)

  • Provide infrastructure for chains currently supported on The Graph’s hosted service & Subgraph Studio
  • Run RPC nodes and Firehose for additional blockchains. We will continue to investigate the currently supported blockchains on studio and the hosted service
  • Education, onboarding & community support for Indexers and Developers working with new data services
  • Collaboration with core developers and new consumers on designing and testing new data services while augmenting the Indexer experience with new tooling
    • Launch new website with free public Firehose and Substreams endpoints
    • Paid plans for Firehose and Substream endpoints with increased rate limits and connections
    • Improve Substreams documentation so indexers know better how to deploy and scale it
    • Start getting familiar with Launchpad v2
    • Clickhouse and webhook sink testing
    • Improve our internal networking for Substreams scalability testing
  • Provide operational support
    • Join with other Core Dev teams to discuss how we can improve support processes via the support working group
7 Likes