GraphOps Update April 2024

Research & Development Core Team Updates
1

:astronaut: Executive Summary

The GraphOps team is excited to share updates across our workstreams this month.

GraphSeer, our new explorer for The Graph Network, was shared with the community at Indexer Office Hours (IOH) #150 and during the Core Dev Call #29. GraphSeer brings data together from a wide range of sources, including the Gateway QoS Oracle, providing a valuable view of Indexer query performance, and where query demand in the network is coming from. We appreciate your feedback greatly (please continue giving it!), and look forward to evolving GraphSeer into the best place to understand The Graph’s Decentralised Network.

Subgraph Radio’s POI crosschecking feature now covers 97% of all subgraph deployments on Arbitrum One! Subgraph Radio has seen a release of version 1.0.2, focusing on POI message frequency fixes. We’ve also integrated version 0.7.1 of Graphcast SDK, introducing enhanced error handling capabilities and addressing a timestamp underflow error.

Within Launchpad, we’ve hosted the 14th edition of Launchpad Office Hours (LOH), covering Helm Charts Basics. We also introduced support for different scaling factors per release and improved labeling support for Launchpad Namespaces. Our Arbitrum Namespace now features separate proxy instances for Arbitrum Classic and Nitro.

For Indexer Components, we continue progressing on File Hosting Service development, focusing on enabling object storage for both data providers and consumers, along with several refinements aimed at improving performance and usability.

Additionally, for Network Operations we added L2 test support to the Core Network Subgraph and introduced the startBlock parameter for manifest entities to improve subgraph indexing speed.

Happy April everyone :sparkles:

:tada: Looking Back (what was delivered)

GraphSeer (Explorer)

Subgraph Radio (Documentation, Docker Image, Radio Crate, Radio Source)

  • Subgraph Radio
    • Fixed an issue that was causing inconsistent messaging intervals
    • Release 1.0.2 fixing message frequency
  • Listener Radio
    • Release 0.0.4
      • Enabled CORS for serving Graphseer queries, added aggregate metrics and queries
      • Fixed database pool timeout
Subgraph Radio Development Activity

Graphcast SDK (Documentation, SDK Crate, SDK Source)

  • Released version 0.7.1 of the SDK
  • Enhanced error handling robustness by addressing an underflow error
Graphcast SDK Development Activity
  • Graphcast SDK pull requests merged:
    • PR #322: Addresses an underflow error to enhance the robustness of error handling.
    • PR #323: Released version 0.7.1.

Launchpad (Documentation, Starter, Charts, Namespaces)

  • Hosted LOH #14 covering Helm charts basics
  • Launchpad Namespaces now has support for different scaling factors per release
  • Launchpad Namespaces has improved labeling support to also cover volumeClaimTemplates
  • Arbitrum Namespace now has separate proxy instances for Classic and Nitro
  • A few small improvements to both Charts and Namespaces featuring contributions from the community
  • Many releases across the board keeping up with upstream new versions
  • Enhanced flexibility in handling different versions of listener-radio and subgraph-radio, including alpha releases
Launchpad Development Activity
  • Launchpad Charts pull requests merged:
    • PR #251, PR #252, PR #253, PR #254: Improved flexibility and handling of listener-radio versions along with accepting alpha releases of subgraph-radio and listener-radio.
    • PR #259, PR #263, PR #255, PR #271: Dependency updates for Lighthouse, progressively enhancing stability and performance.
    • PR #256, PR #262, PR #270: Erigon dependency updates, ensuring compatibility with latest versions.
    • PR #264: Updated proxyd dependencies to improve interoperability.
    • PR #260: Celo dependency update to correct minor version constraints.
    • PR #250, PR #273: Updates to subgraphradio dependencies to address latest operational requirements.
    • PR #261, PR #267: Enhancement of Arbitrum dependencies for improved blockchain interactions.
    • PR #265: Added paused_at to indexing-status dashboard to enhance monitoring capabilities.
    • PR #266, PR #275: Release and update of graph-node chart versions, aligning with the latest developments.
    • PR #268: Fixed a merge bug in graph-node configuration to honor graphNodeGroups.enabled: false settings.
  • Launchpad Namespaces pull requests merged:
    • Sealed-Secrets, with PR #611 and PR #558 upgrading to versions 2.15.3 and 2.15.2 respectively.
    • Continuous integration (CI) improvements for monitoring, polygon, ethereum, gnosis, and arbitrum-one dependencies across numerous pull requests like PR #609, PR #608, PR #607, and more, ensuring the software stack remains up-to-date and resilient.
    • Several key package updates within the graph dependencies demonstrated by PR #600, PR #599, and PR #543, pointing to a dedicated effort in keeping core technology elements at their peak performance level.
    • Operational enhancements and feature development are highlighted by PR #574 introducing per-release scaling and PR #559 adding volumeClaimTemplate labels, underscoring the team’s ongoing efforts to improve scalability and manageability.

Indexer Components (Indexer Components, Indexer Service Rewrite, Allocation Optimizer)

  • File Hosting Service
    • New features:
      • For both data providers and consumers - enable object storage for read/write
      • Consumer - optionally resume download from failed sessions
      • Provider - individual bundle price management in admin endpoint
      • Provider - basic metrics
    • Refactors: Slim down docker image size, new configuration parameter naming, documentation updates, small performance improvements
      Please provide the input for summarization.

Network Operations (Graph Network Subgraph, Graph Network Substreams)

  • Added L2 test support
  • Added startBlock for manifest entity
    I’m ready to summarize your GitHub PR activity when you’re ready to share details.

Ecosystem Relations

:rocket: Looking Ahead (upcoming priorities)

  • Subgraph Radio
    • GraphSeer integration
  • Graphcast SDK
    • Listener radio may be adding new metrics/queries to support better Graphseer analytics
  • GraphSeer
    • Adding message and Subgraph coverage stats from Graphcast
    • Store and component refactors for sustainable development
    • Reviewing feedback gathered from the community and addressing them
  • Launchpad
    • Documentation improvements in general, with a particular focus on the getting started with Launchpad flow
    • Bring more consistency across Helm chars by the introduction of a library chart, as well as bug fixes
    • Support Polygon
  • Indexer Components
    • File Hosting Service
      • Support both file and bundle levels of service (currently bundles only)
      • Deploy to GraphOps cluster and test internally with direct p2p free query path
      • Once Horizon new data service contract interface becomes available, File Hosting Service can be deployed to the network for discovery
      • Once Scalar TAP gets validated, File hosting service can be used with TAP payments
  • Network Operations
    • Adding DataSource yaml parsing (full parse of each data source for a given manifest)
  • Ecosystem Relations
5 Likes