GraphOps is excited to share our progress this month! We spent a wonderful week together with other Core Developers and aligned across many initiatives, set Q4 goals and progressed key decisions and lines of work.
We recorded three new Subgraph Radio workshops this month, covering operating the Radio as an Indexer (video 1, video 2), and using Graphcast CLI to signal Subgraph Upgrades to Indexers (video 3) as a Subgraph Developer. We also improved Radio shutdown signal handling and we released version
0.1.10 of Subgraph Radio. For Graphcast SDK, we have released version
0.5.2, with a number of performance improvements and bug fixes. We have also updated the ping pong example, and made Radio peer data available via GraphQL.
For Launchpad, we ran another session of Launchpad Office Hours and released new step-by-step guides for provisioning and upgrading a Kubernetes cluster using kubeadm. Additionally, OpenEBS Rawfile compatibility with newer Kubernetes versions was fixed and dependencies across many components have been updated.
For the Indexer stack,
graphops/indexer-service-rs has been migrated to
graphprotocol/indexer-rs. Indexer service component standardisation, testing and payment flow integration are progressing well.
Lastly, within Network Operations, Subgraph Network Substream
SubgraphVersion entities have reached feature parity with the Graph Network Subgraph. We have also begun working on a new File Data Service under The World of Data Services.
Happy November everyone
Subgraph Radio (Documentation, Docker Image, Radio Crate, Radio Source)
- 3 workshops recorded covering Subgraph Radio, aimed at Indexers and Subgraph Developers
- Further improved shutdown signalling within Radios, every spawned task now shuts down gracefully
- Added network peer logs
- GraphQL endpoint now exposes peerData and localPeerData
- Made a start on developing a self-hosted frontend within Subgraph Radio, with all required assets bundled in the Radio binary/docker image
0.1.10of Subgraph Radio have been released
- Introduced fresh styles and new server frontend functionality
- Implemented default addition of Waku discovery network enr to bootstrapping nodes
- Now have access to divergence summary notifications at set intervals.
Subgraph Radio Development Activity
- Subgraph Radio pull requests merged:
- PR #109, PR #96: Updated a few config names and shutdown handlers for better control flow struct
- PR #107, PR #87: Updated test packages waku dep and grafana.json
- PR #97, PR #108: Released versions 0.1.8 and 0.1.9
- PR #104, PR #103: Introduced new styles and server frontend functionality
- PR #98: Updated changelog
- PR #88: Waku discovery network enr now added to bootstrapping nodes by default
- PR #85: Divergence summary notification now available at a given interval
Graphcast SDK (Documentation, SDK Crate, SDK Source)
- Default waku protocol switched from filter to relay on all Radios
- Research on how to revise our Waku node configuration to improve connectivity, stability and scalability
- Collaborated with the Waku team and validated our discv5 and peer management strategy, and message flow when using filter protocol
0.5.2of the SDK have been released
- Refactored decode function and updated example
- Introduced new GraphQL type ‘peerData’ and refactored peer helpers
- Various performance improvements and bug fixes
Graphcast SDK Development Activity
- Graphcast SDK pull requests merged:
- PR #293, PR #302, PR #308: Released versions 0.5.0, 0.5.1, and 0.5.2
- PR #303: Refactored decode function and updated example
- PR #297, PR #307: Added content topic check at message handle, Removed filter subscription from content topic update
- PR #296: Fixed empty set_waku_event_callback at shutdown and refactored mpsc sender
- PR #299: Introduced peerData GraphQL type and refactored peer helpers
Launchpad (Documentation, Starter, Charts, Namespaces)
- Fixed OpenEBS compatiblity with newer Kubernetes versions
- Updated Graph Node dashboards
- Added new documentation with step-by-step guides to provision and upgrade a Kubernetes cluster
- Ran another Launchpad Office Hours session (recording) with the Indexer community, demoing how to create a Kubernetes cluster with
- Launchpad Charts: Enhanced handling for Arbitrum versioning, including in the CI, and added default values in the Generic App
- Added support for topologyAwareRouting in the Indexer service/agent
- Updated dependencies across various components including Subgraphradio, Proxyd, and OpenEBS
- Launchpad Namespaces: Adjustments made to CI configuration, including improved regex handling, and allowed primary shard connections from any host
- Various components, such as Gnosis, Celo, Polygon, Monitoring, Ethereum, Arbitrum, Graph, Avalanche, Ingress and sealed-secrets dependencies have been updated in the CI
Launchpad Development Activity
- Launchpad Charts pull requests merged:
- PR #169, PR #182, PR #180: Improved handling of Arbitrum versioning, within the CI, and added default values in the Generic App.
- PR #173: Added support for topologyAwareRouting for the indexer service/agent.
- PR #171, PR #172: Updated the Subgraphradio and Proxyd dependencies.
- PR #175: Updated OpenEBS images to address csistorage API issues.
- PR #176: New sections added to the Proxyd config interface.
- PR #177: Removed P2P from Arbitrum charts.
- PR #170: Updated Graph-Node dashboard.
- Launchpad Namespaces pull requests merged:
- PR #165, PR #177, PR #153: Adjusted the CI configuration to fix regex for the latest stable and allow primary shard connections from any host, as well as fixing canary version tags
- PR #171, PR #147: Updated Gnosis dependencies in the CI
- PR #170, PR #145: Updated Celo dependencies in the CI
- PR #169, PR #157, PR #148: Updated Polygon dependencies in the CI
- PR #168, PR #141: Updated monitoring dependencies in the CI
- PR #167, PR #155, PR #146: Updated Ethereum dependencies in the CI
- PR #166, PR #132: Updated Arbitrum dependencies in the CI
- PR #156, PR #137: Updated Graph dependencies in the CI
- PR #154, PR #144: Updated Avalanche dependencies in the CI
- PR #139: Updated the ingress dependencies in the CI
- PR #138: Updated the sealed-secrets dependencies in the CI.
Indexer Components (Indexer Components, Indexer Service Rewrite, Allocation Optimizer)
- Collaborative efforts with E&N and Semiotic on standardising indexer-service and integrating payment flow
indexer-service-rshas been tested in local network environments (by E&N)
- Research and early discussions on File Data Service, specified general project goals and design approaches
No PR activity reported.
Indexer Components Development Activity
Network Operations (Graph Network Subgraph, Graph Network Substreams)
- Fixed issues with the File Data Sources refactors for all related subgraphs. Subgraphs are ready to be deployed as soon as UI changes are ready.
- Subscriptions subgraph has been updated to allow for trusted partners through the gateway registry.
- Graph Network Substream
SubgraphVersionentities feature parity is reached
- Introduced feature parity for subgraph and subgraph version entities in Graph Network Substreams.
Network Operations Development Activity
- Recordings and agendas for Indexer Office Hours are available. Follow updates on Twitter @TheGraphIOH.
- Subgraph Radio
- Prepare for
- Stricter validation and better error messages relating to configuration variables
- Improve periodic report log/summary + add state persistence for that
- Move waku discovery ENR to SDK and use it if discv5 is enabled
- Switch to using DB instead of a JSON file for state persistence
- Prepare for
- Adding Firehose support is our top priority
- Indexer Components
- Finish standardisation and payments for
- Setup a library for
indexer-serviceand move current indexer-service to
- Finish standardisation and payments for
- Network Operations
- Develop a proof of concept for File Data Service
- Ecosystem Relations