The Graph Core Devs Call #15

:information_source: The 15th Core R&D call is happening this Thursday (2022-08-31T23:00:00Z).

This is a recurring call where different contributors from various teams gather around to discuss major updates from different working groups and brainstorm around active R&D tracks.

:calendar: Please subscribe to The Graph Foundation’s Ecosystem Calendar so you don’t miss any of the upcoming calls!


:notebook: Agenda:

  • [The Graph Foundation]
    • MIPs Program Overview
  • [Data & APIs WG]
    • Graph Node and Graph Client updates
    • Building a Cosmos Subgraph and other Tendermint-based Chains
    • Update on Substreams in Subgraphs and DevEx
  • [Network and Protocol Operations WG]
    • Introduction to Graphcast - The Graph’s Gossip Network

:tv: The recording will be uploaded here.


:rocket: The Graph Core R&D Workspace

As The Graph is being built in public and in a decentralized way, we’ve put together a new publicly accessible The Graph Core R&D Workspace . You can resort to this page to follow along with each working group’s progress on significant workstreams, access meeting notes, and recordings.


:crystal_ball: Roadmap

The Graph Foundation has recently released a blog post presenting the Core R&D roadmap. Please feel free to reach out if you believe you can contribute to any of the listed workstreams!


See you this Thursday! :astronaut:

3 Likes

Hey everyone! Thanks for all the feedback, comments and questions on the Core Dev Call yesterday. As promised, here’s a recording of a demo of the proof of concept, recorded about a month ago - the “frontend”, workflow and logs are more or less the same since then, but we’ve also added a bunch of new features that will be laid out in the README of the proof of concept repo soon. We will record another demo once the proof of concept is finished, where we will dive deeper into how things work.

Summary

In The Graph’s Core Devs Call #15 we learn about the MIPs program, view demos on Graphcast, @Live Queries and Building Subgraphs for Tendermint chains and receive a development update on Substreams.

Topics Include:

  • Migration Infrastructure Providers Program Overview
  • Intro to Graphcast
  • @Live Queries w/ Dotan
  • Building Subgraphs for Tendermint Chains w/ Bartosz
  • Update on Substreams

Detailed notes regarding each topic with timestamps can be found below.

MIPs (Migration Infrastructure Providers) Program Overview (03:05)

Focus

The focus of the MIPs program is to bootstrap Indexers to support new chains on the decentralized network, allowing for multi-chain subgraphs to migrate.

Incentives

50M GRT for Indexers (0.5% GRT supply)

25M GRT for subgraph migration grants (0.25% GRT supply)

Breakdown for Distribution

Chain allocation will be defined mid-program. GRT will be allocated towards bootstrapping specific chains for the program.

Criteria for Performance

The core objectives of the program are to test network infrastructure, expand the decentralized network to new chains and support the migration of subgraphs. Indexers will participate in a gamified system of scoring and a leaderboard will be used to distribute awards at the end of each cycle. Points will be gained based on an Indexers participation in three core areas:

  • Subgraph Coverage
  • Quality of Service
  • Community Support

Chains

Each new chain will be announced with 2 weeks’ notice to allow for Indexers to prepare for the chain. Gnosis chain is the first chain announced to be part of the program.

Indexer Journey

Lastly, check out The Graph’s official blog post covering the MIPs program.

Intro to Graphcast (11:20)

Graphcast | The Graph’s Gossip Nework

Petko, a team member at GraphOps, gave a presentation on Graphcast. Graphcast is a decentralized, distributed peer-to-peer communication tool that allows Indexers across the decentralized network to exchange information in real time. Graphcast is an optional off-chain layer, which allows Indexers to exchange P2P messages without paying extravagant gas costs. There is work being done to ensure message integrity and can be read more about in the GRC.

What does Graphcast unlock?

  • New forms of healthy communication among Indexers
  • Easier experience for tooling developers
  • Democratizing access to underlying data

Concerns

  • No strong guarantees of message integrity
  • Difficulties in testing Radios/ a network of radios would need to be deployed to test messages passed between them
  • Lack of multi-language support

Use Cases

  • Conducting auctions for warp syncing purposes
  • Self-reporting on query analytics
  • Self-reporting on Indexing analytics
  • Real-time cross checking of subgraph data integrity
  • AutoAgora cross-indexer signals/negotiations for improved automated query pricing

Tech Stack

Waku v2 protocol for decentralized messaging features (open to discussion for alternatives)

TypeScript SDK that abstracts over Waku, provides identity resolution which improves message integrity and exposes a rich interface for building Radios

The project is open Source and built publicy.

Next Steps

  • Collect feedback from Indexers
  • Publish Graphcast SD

@Live Queries (24:00)

Dotan Simha, of The Guild core development team, presented on their work to introduce @live queries into the GraphQL interface. @live queries allow dApp developers query real-time data from their subgraphs. A GIP was recently introduced on The Graph’s forum. View Dotan’s demo below!

Building Subgraphs for Tendermint Chains (32:13)

The Figment Core Development Team has been working on integrating the Cosmos Ecosystem with The Graph. Bartosz Pienkowski presented on how to build subgraphs for the Cosmos Ecosystem.

Substreams Development Update (42:05)

Substreams Core

  • Teams internally and externally are experimenting with Substreams.
  • In draft: helpers for simpler event extraction

Substreams in Subgraphs

GIP Published

Initial simple implementation in Graph Node

  • Imports a Substeams package
  • Persists the data in Postgres, queryable via the usual subgraph API

Continuing Discussion

  • Validating the Initial Graph Node Integration
    • Importing more complex Substreams
      • AMMs
  • Ensuring correctness during live operation, how does the system react when things fail or connection is lost
  • Testing determinism, do Substreams output the same data when utilized by alternate Indexers?
  • Performance benchmarking
  • Scalability benchmarking

Tooling to Builld and Deploy

  • Graph CLI will need an update for the Substream data source

  • Could take a more rust native approach via a Cargo plugin

Schema & Graph out Definition

  • The schema has information that isn’t relevant to substreams
  • Relationships between entities
  • Derived fields
  • Interfaces
  • The schema might only be a subset of output
  • Some entities might only be relevant internally

Getting Involved

Documentation can be found at substreams.streamingfast.io

Stay Tuned!

Join us next month for Core R&D call #16!

Keep up to date by joining discussions in the forum, following The Graph on Twitter or joining the Discord server.

Notes will be updated with YouTube video when available.

Hey folks!
Adding a reference to Chris’ response, from GraphOps, to Oliver’s question regarding regulation and price fixing risks associated with GraphCast: GRC-001: Graphcast - a Gossip Network for Indexers - #6 by chris

1 Like