GraphOps Update October 2023

Research & Development Core Team Updates
1

:astronaut: Executive Summary

GraphOps is excited to announce our latest development updates. In Subgraph Radio, we have made significant improvements to the notification system, fixed comparison result ratios, and made major improvements to Radio peering. Version 0.1.7 of Subgraph Radio has been released, along with version 0.5.0 of Graphcast SDK. We have made several updates to the SDK, including fixing shutdown signal handling and refactoring message decoding. In Launchpad, we have added an exciting new scaling interface, as well as support for the Ethereum Holesky testnet. Numerous bug fixes, improvements, and added features have been implemented in both Launchpad Charts and Launchpad Namespaces. We continue to press forward on Indexer Service in Rust in collaboration with Semiotic and E&N, with the latest focus being testing and validating functionality as compared to Indexer Service V1.

Happy October everyone :sparkles:

:tada: Looking Back (what was delivered)

Subgraph Radio (Documentation, Docker Image, Radio Crate, Radio Source)

  • Added waku discovery network ENR by default, increasing average peer count from 3 to 20
  • Updated grafana.json to include total messages received, number of connected peers and number of gossip peers
  • Reworked the notification system and introduced notification modes. Users can now choose how often to receive divergence notifications and what the notification message format is, the available options are periodic_report, periodic_update and live
  • Fixed comparison result ratios in the GraphQL response as well as the corresponding Prometheus metrics
  • Versions 0.1.5, 0.1.6, and 0.1.7 of Subgraph Radio have been released
  • Improved accuracy of comparison result ratios for local attestation
  • Added divergence summary notification on a given interval
  • Various bug fixes and improvements in performance and code simplicity.
Subgraph Radio Development Activity
  • Subgraph Radio pull requests merged:
    • PR #57, PR #68: Released versions 0.1.5 and 0.1.6
    • PR #74: Updated comparison result ratios to accurately reflect local attestation
    • PR #75: Fixed notification on result types
    • PR #76: Fixed ratios
    • PR #77: Improved invalid message log
    • PR #79: Released version 0.1.7
    • PR #81: Removed local_ppois_to_compare and refactored diverged_subgraph
    • PR #82: Removed labels workflow
    • PR #85: Added divergence summary notification on a given interval
    • PR #87: Updated grafana.json
    • PR #88: Added waku discovery network enr to bootstrapping nodes by default
    • PR #65: Added network field in offchain request
    • PR #66: Fixed p2pNodePort in Kubernetes deployment
  • Graphcast CLI pull requests merged:
    • PR #14: Removed labels workflow.

Graphcast SDK (Documentation, SDK Crate, SDK Source)

  • Updated Waku bindings to version 0.3.1, which includes discv5 fix
  • Added waku discovery network ENR to our boot nodes
  • Fixed shutdown signal handling in example Ping Pong Radio
  • Refactored inbound message decoding
  • Versions 0.4.3 and 0.5.0 of the SDK have been released
  • Various performance improvements, including refactoring message decoding
  • Updated bindings version for improved compatibility and stability
  • Fixed boot node connections issue
Graphcast SDK Development Activity

Launchpad (Documentation, Starter, Charts, Namespaces)

  • Added an exciting new scaling interface to Launchpad’s blockchain Namespaces. This preconfigures placement rules, proxyd backend configuration and p2p node port passthrough for any number of blockchain node instances!
  • Added support for the Ethereum Holesky testnet
  • Added the ability to override Chart version and Chart URL in Launchpad Namespaces
  • We added more documentation covering the stable and canary release channels
  • We ran two more Launchpad Office Hours sessions with the Indexer community, discussing and demoing the new interfaces and what to expect from the release channels
  • Kept up to date with a regular churn of version bumps tracking upstream
  • Added a new Grafana Dashboard to the Proxyd chart
  • Many charts now better support multiple deployments in the same cluster by having different ClusterRole names
  • Improved automatic labeling for Launchpad Namespaces
  • Removed labels workflow from Launchpad Starter
  • Various updates and bug fixes for Launchpad Charts, including fixed RBAC and p2p functionality, updated dependencies, and added features
  • Various updates and bug fixes for Launchpad Namespaces, including updated dependencies, fixed typos, improved handling of labels, and added new features
Launchpad Development Activity

Indexer Components (Indexer Components, Indexer Service Rewrite, Allocation Optimizer)

  • Added indexer management client to track NetworkSubgraph instance and postgres connection status
  • Added cost models APIs at /cost endpoint
  • Added rate limited API at /status endpoint
  • Define cost model schema and resolvers with postgres and graphQL types, with global cost model fallback when specific deployments are queried
  • Added configuration protocol_network field to Allocation Optimizer to support multi-network indexer software
  • Added configuration syncing_network in Allocation Optimizer to filter subgraph selection to the network subgraph
  • Added migration files without embedding to binaries
  • Added TAP receipts handling and Escrow monitor functionality
  • Improved handling of network subgraph query errors
  • Various code cleanup and bug fixes
Indexer Components Development Activity
  • Indexer Service RS pull requests merged:
    • PR #43: Added migration files without embedding to binaries
    • PR #46: Reduced duplicated code and cleaned up the project
    • PR #47: Added TAP receipts handling and Escrow monitor functionality
    • PR #48: Enabled SQLX/PG tests
    • PR #52: Added missing /cost GraphQL API
    • PR #53: Improved handling of network subgraph query errors
    • PR #55: Limited functions exposed in indexing status API
    • PR #56: Removed unused dependencies
    • PR #58: Fixed malformed indexing status response and allowed required status fields by gateway
    • PR #59: Resolved clippy errors introduced recently
    • PR #60: Fixed formatting
    • PR #62: Removed name label from QUERY_DURATION metrics

Network Operations (Graph Network Subgraph, Graph Network Substreams)

  • Performance improvements in Analytics subgraph to solve arbitrum one slowdown
  • Fixed network subgraph subgraph owner curation missing in L2 when transferring from L1
  • Network Subgraph tests merged and added to CI
  • Fixed GraphNetwork totals in Network Subgraph after L2 transfer tools rollout
  • Added new Network Subgraph GraphNetwork totals to reflect L2 transfers
  • Added support for L2 → L1 transfer (i.e. going back to L1 after transferring to L2) in Network Subgraph
  • Graph Network Substream has SubgraphDeployment feature parity with Network Subgraph for indexer-agent
  • Added subgraph deployment feature parity
  • Fixed a keyword issue in the substreams graph
Network Operations Development Activity

Ecosystem Relations

:rocket: Looking Ahead (upcoming priorities)

  • Subgraph Radio
    • Release 0.1.8 to unlock the new features for users
    • Subgraph Upgrade pre-sync feature demo
    • Migrate from using JSON to SQLite DB for persisted state
    • Bigger Subgraph Radio demo, focusing on POI cross-checking feature
    • Initial work on embedded frontend
  • Launchpad
    • Advance Firehose and Substreams support
    • Add the new scaling interface to all supported chains
  • Indexer Components
    • Increase test coverage and validate parity with indexer-service
  • Network Operations
    • Continue pushing graph network substreams forward, with L2 Transfer tools feature parity a key objective
  • Ecosystem Relations
    • Please join the community for Indexer Office Hours on Tuesday every week at 6pm UTC in the Graph Protocol Discord
    • Please join the community for Launchpad Office Hours on Wednesday every second week at 5pm UTC in the Graph Protocol Discord
6 Likes