GraphOps Update June 2023

Research & Development Core Team Updates
1

:woman_astronaut: Exec summary

In May, GraphOps has continued to push forward Graphcast and the POI Radio! A Graphcast Registry has been deployed to Arbitrum Goerli and Arbitrum One, enabling support for these networks. POI Radio now also supports storage persistence, making the Radio more resilient across restarts. Smarter notifications and support for Telegram, two features Indexers have been asking for, have also landed. We continue to make progress toward our Network Operations goals, including greater entity coverage in the Core Network Substream, and critical work supporting the upcoming L2 migration helpers.

Happy June everyone!

:tada: Looking back (what was delivered)

Graphcast (Documentation, SDK Crate, Code, GRC-001)

  • Version 0.3.2 has landed
  • Added support for Graphcast on Arbitrum Goerli and Arbitrum One
  • Configuration variables generalised and abstracted away from the SDK, leaving implementation to the individual Radios
  • Major refactor to reduce the risk of deadlocks, improve performance and modularise the codebase
  • Introduced different logger formats (pretty, json, full, compact)
  • Added support for Telegram bot notifications
  • Added support for discv5
Development Activity
  • Graphcast SDK pull requests merged:
    • PR #230: Released version 0.3.2
    • PR #228: Enabled discv5 configs
    • PR #229: Fixed callbook input order
    • PR #226: Refactored callbook and graphcast_id structs, cleaned query functions
    • PR #225: Refactored radio_name to be a String
    • PR #222: Added telegram notifications
    • PR #221: Updated query and added configurations
    • PR #219: Released version 0.3.1
    • PR #218: Added options to toggle logger format
    • PR #214: Switched to git dep for Waku
    • PR #212: Released version v0.2.0, updated release process and changelog script, refactored config parsing and ping-pong Radio
    • PR #209: Released version v0.2.0
    • PR #207: Released version v0.1.2, updated release process and script, fixed content topic lock, refactored config parsing and ping-pong Radio
    • PR #208: Fixed disabling of unsubscribe until waku dep update
    • PR #206: Released version v0.1.2
    • PR #205: Updated ping-pong Radio
    • PR #202: Updated release process and script
  • Graphcast Registry pull requests merged:
    • PR #14: Disabled initializer for upgradable contract v0.
    • PR #13: Updated code for Arbitrum, eliminating the need for v2 from the beginning.
  • Graphcast Registry Subgraph pull requests merged:
    • PR #7: Updated template and readme as part of the chore.
    • PR #6: Fixed issue related to handling indexer loads.
    • PR #3: Modified event handlers to allow operators.
    • PR #2: Completed repository checklist.

POI Radio (Documentation, Docker Image, Radio Crate, Code)

  • Version 0.3.4 has landed
  • Major refactor to reduce the risk of deadlocks, improve performance and modularise codebase
  • Improved config variables parsing and handling
  • Added state persistence, now the Radio can periodically store its state in a JSON file. See PERSISTENCE_FILE_PATH config var
  • Added Telegram divergence notifications option, see TELEGRAM_TOKEN and TELEGRAM_CHAT_ID config vars
  • Improved divergence notifications, they now don’t send duplicated notifications for known and already reported divergences
  • Added configuration option for different logger formats (pretty, json, full, compact), see LOG_FORMAT config var
  • Added basic end-to-end testing suite
  • Restructured repository into a Cargo workspace
  • Utilising discv5 to allow running Radios to directly connect to each other, further decentralising the system. See DISCV5_ENRS and DISCV5_PORT config vars
Development Activity
  • POI Radio pull requests merged:
    • PR #183: Released version 0.3.4
    • PR #182: Fixed server context types
    • PR #181: Released version 0.3.3
    • PR #179: Released version 0.3.2
    • PR #176: Added end-to-end tests
    • PR #173: Refactored release tag by removing ‘v’ prefix
    • PR #172: Added discv5 configs
    • PR #174: Refactored interval constants
    • PR #171: Refactored operator struct to replace global mutex
    • PR #167: Added telegram notifications
    • PR #165: Added option to disable notifications for known divergences
    • PR #163: Released version 0.3.1
    • PR #164: Released version 0.3.1
    • PR #158: Updated logging level
    • PR #148: Released version 0.3.0
    • PR #146: Added options to toggle logger format
    • PR #141: Added file persistence for local attestations and remote messages
    • PR #147: Removed e2e test suite from the repository
    • PR #145: Reverted CI workflow
    • PR #144: Updated CI workflow and added docs
    • PR #139: Split comparison and gossip of POI
    • PR #140: Added ratio queries, refactored logs
    • PR #138: Updated dependencies without conflicts
    • PR #135: Released version 0.2.0, fixed disable unsubscribe and use timeout
    • PR #134: Released version 0.2.0
    • PR #130: Fixed disable unsubscribe and use timeout
    • PR #129: Released version 1.0.3, moved integration tests to main repo, introduced new release process and changelog script
    • PR #127: Released version 0.1.3
    • PR #128: Fixed radio name defined in main
    • PR #123: Refactored gossip poi, added benches
    • PR #122: Updated release process and script
    • PR #119: Moved config parsing to Radio

Network Operations (Core Network Subgraph, Core Network Substreams)

Core Network Substream activity:

  • Added staked and signalled tokens for each subgraph deployment
  • Indexing (total, indexer, and delegator) rewards for each subgraph deployment
  • Added store for subgraph deployment to track allocation id relationship

Core Network Subgraph activity:

  • Released new production version (v0.19.1)
  • L2 transfer tools:
    • Implemented support for all currently required transfer tools (Subgraph, Curation, Delegation, Staking)
    • Deployed multiple different testing environments
    • Lots of manual testing
    • Worked alongside product engineering team to help integrate the tooling
    • Worked alongside indexer tooling team to help integrate it on the indexer-agent
    • Rolled out production testnet L2 transfer tool compliant subgraphs
    • Fixed many minor issues with the implementations along the way
  • Added SbS/SpS detection mechanism
  • Fixed deployment workflows for GH
  • EBO:
  • Activity feed:
Development Activity

Ecosystem Relations

  • “Substream-backed-subgraphs” workshop at ETHLisbon by Sahra!
  • “Subgraphs and more…” workshop at ETHUruguay by Juan!
  • Recordings and agendas for this month’s Indexer Office Hours are available. Follow updates on Twitter @TheGraphIOH.

:rocket: Looking ahead (upcoming priorities)

  • Graphcast & POI Radio
    • Fix Grafana table deadlock issue and provide better Grafana documentation
    • Release a walkthrough workshop for Indexers to add POI Radio to their stack
  • Network Operations
    • Continue work on the Core Network Substreams, with our current milestone defined here
    • Continue work on the L2 migration, Core Network Subgraphs, and the Epoch Block Oracle
  • Launchpad and Kubernetes Tooling
    • Make our ongoing work on Launchpad V2 public
  • Ecosystem Relations
    • Please join the community for Indexer Office Hours (on Tuesday) and Launchpad Office Hours (on Wednesday) every week at 6pm UTC in the Graph Protocol Discord
5 Likes