GraphOps Update January 2024

:astronaut: Executive Summary

The GraphOps team is excited to share our updates and achievements this month!

We launched the Subgraph Radio Stable Release 1.0.0 :tada: featuring improved state persistence, enhanced Indexer address validation, updated Slack notification logic, and improved end-to-end tests.

The File Hosting Service repository is now public! This means the community can now track all developments on File Hosting Service β€”a decentralized, peer-to-peer data sharing platform optimized for efficient and secure file transfers. Our recent efforts focus on enabling file sharing, client-side file discovery, adding a server cost endpoint, and improving server-side local file system methods.

Moving onto Graphcast SDK, we rolled out the version 0.6.0. This release include updated nonce types, reintroduced discovery ENR and refactored the Slack bot for seamless notifications setup.

Within Launchpad, we deprecated Arbitrum Goerli and Graph Goerli namespaces. We also updated Arbitrum namespace, Lighthouse chart and graph-toolbox image build. Additionally, we upgraded various dependencies to bolster reliability. Our team also hosted another Launchpad Office Hours session to recap 2023 progress and outline Launchpad’s roadmap for 2024.

Happy 2024 everyone :sparkles:

:tada: Looking Back (what was delivered)

Subgraph Radio (Documentation, Docker Image, Radio Crate, Radio Source)

  • Subgraph Radio Stable Release 1.0.0
    • We hosted a live demo and Q&A session on Indexer Office Hours #138
    • Upgraded state persistence to use a SQLite database, instead of a JSON file
    • Improved Indexer address validation logic
    • Removed Waku discovery ENR since it’s now handled by SDK
    • Updated Slack notification logic. Slack bot now utilizes webhook.
    • Updated e2e tests:
      • Added new scenarios
      • Improved performance of existing tests (test sender instances are sending messages less frequently, which leads to reduce network load)
Subgraph Radio Development Activity
  • Subgraph Radio pull requests merged:
    • PR #117: Switched to SQLite for state persistence
    • PR #119: Added validation for indexer address config variable
    • PR #120: Removed Waku discovery enr from Radio
    • PR #121: Switched Slack bot to use webhook instead of bot token
    • PR #126: Updated e2e tests

Graphcast SDK (Documentation, SDK Crate, SDK Source)

  • Updated type of nonce field on GraphcastMessage
  • Added Waku ENR back to SDK, now all Radios will connect to it by default
  • Refactor SlackBot to use webhook, making Slack notifications setup easier
  • Released Graphcast SDK 0.6.0
  • Updated nonce type in the SDK
  • Reintroduced discovery ENR back to the SDK
  • Slack bot has been refactored to use webhook instead of bot token
Graphcast SDK Development Activity
  • Graphcast SDK pull requests merged:
    • PR #310: Updated nonce type in the SDK
    • PR #311: Added discovery enr back to the SDK
    • PR #312: Refactored Slack bot to use webhook instead of bot token
    • PR #313: Release 0.6.0

Launchpad (Documentation, Starter, Charts, Namespaces)

  • Deprecated Arbitrum Goerli namespace
  • Deprecated Graph Goerli namespace
  • Updated Arbitrum namespace to address Network Subgraph is not Fresh error
  • Updated and released new versions of Launchpad Charts and Launchpad Namespaces as per latest upstream dependencies
  • Updated Lighthouse chart to use hostPort instead of nodePort for working P2P connectivity on mainnet
  • Updated graph-toolbox image build to fix integration issues with graph cli and graphman
  • Launchpad Charts and Launchpad Namespaces saw updates to dependencies improving reliability
  • Introduced a Launchpad Charts Contributor Guide
  • Adjustments to nodePort and hostPort configurations
  • Launchpad Office Hours #9: We reviewed all the progress that Launchpad had during 2024 and delved into an open discussion of the road ahead
Launchpad Development Activity
  • Launchpad Charts pull requests merged:
    • PR #203: Sets replicas for indexer-service in graph-networks-indexer
    • PR #204: Adds a contributing guide to the documentation
    • PR #205: Updates lighthouse to use hostPort instead of nodePort
    • PR #208: Updates proxyd dependencies
    • PR #209, PR #210: Updates subgraphradio and arbitrum dependencies
    • PR #211: Upgrades a missed config flag for arb-nitro
    • PR #212: Updates storage environment for the 1.0.0 release of subgraph-radio
  • Launchpad Namespaces pull requests merged:

Indexer Components (Indexer Components, Indexer Service Rewrite, Allocation Optimizer)

  • File Hosting Service repository is now public
  • File sharing:
    • Basic client-side file discovery
    • client-side failure mode handling - track missing receipts in cache
    • Custom errors, added unit and integration tests, added benchmarks
    • Added server cost endpoint and management
    • Updated server side local file system access methods
    • Added documentation and user guides
  • Indexer Agent:
    • TAP redemption PR open for review - database migration, redeem cycle, TAP contracts
Indexer Components Development Activity

Ecosystem Relations

:rocket: Looking Ahead (upcoming priorities)

  • Subgraph Radio
    • Enrich Prometheus metrics
    • Serve more data on the graphql endpoint
    • Work on a dedicated frontend view/interface
  • Graphcast SDK
    • Export functions and other items in a way that graphcast-js can easily import and use
  • Launchpad
    • Adding Firehose and Substreams support
    • Make schemas available (as previously demo-ed in LOH)
    • Adding back Celo and Polygon, and bring in Optimism support
    • Improve documentation, FCOS guides
    • Better secrets management
    • Improve the observability layer (dashboards, alert rules)
    • Bring in some data backup / moving capabilities
  • Indexer Components
    • File Sharing Data Service:
      • Add on-chain components to interact with network contracts
      • Support generic storage methods for File Sharing server
      • Work with first key users for early feedback
      • Port file sharing server to indexer-service framework
      • Support work on TAP receipt integration into indexer-agent, participate in scoping allocation optimizer for query fees
  • Ecosystem Relations
    • Please join the community for Indexer Office Hours on Tuesday every week at 6pm UTC in the Graph Protocol Discord
    • Please join the community for Launchpad Office Hours on Wednesday every second week at 5pm UTC in the Graph Protocol Discord