The Graph’s Core Devs Meeting #10

Join The Graph’s Core Devs meeting #10, happening tomorrow on Wednesday, February 2nd @ 7am PST. We will be covering a number of discussion topics that will be moderated by The Graph’s Core Dev teams to generate open community dialogues. The meeting link is: Launch Zoom Meeting


Discussion Topics:

  • Multi-chain integration with Firehose
  • Graph Node debug tool e2e demo with LimeChain
  • GraphQL API: new features & versioning impact
  • Indexer Software updates
  • GIP-0023: Subgraph ownership transfer using external NFT contract

The Core Devs call not only covers graph node related topics, but every research & development work in the Graph ecosystem. Everyone involved in that space is welcome to propose a topic for the Core Devs calls that you can introduce for discussions. This is a chance to connect and reflect on your work with other Core Devs in order to inform the community and gather feedback. If there is any content you would like us to add as discussion topics for next week, please feel to suggest them as a response in this thread. Thank you!

6 Likes

Summary

In Core Devs Meeting #10 we receive updates from the core development teams, learn about the Firehose SDK, explore new GraphQL API features, watch a demo from LimeChain and hear about the progress on a development starter toolkit for Graph Node.

Topics Include:

  • Developer Updates
    • Edge & Node
    • Figment
    • The Guild
    • Streaming Fast
  • Firehose SDK
  • LimeChain Subgraph Debug Tool
  • Graph Node Testing Experience
  • New GraphQL API Features
    • GraphQL validations have been merged
    • Upcoming GraphQL API features
  • LimeChain Demo | Subgraph Debug Tool
  • Graph Node Testing Experience
  • GIP–023

Detailed notes regarding each topic with timestamps can be found below.

Developer Updates (2:17)

Next week, 50 people from seven teams will attend an R&D developer retreat. Working groups between different teams are being formalized around a set of common focus areas. During the next core developers call, time will be dedicated to sharing information on some of these working groups.

Read below for the updates from each development team:

Edge & Node (2:17)

Risks, Problems, Help

  • Many PRs waiting to be deployed/merged (Waiting on POI investigations)
  • Many action items work in progress
  • Lots of open discussions (availability chain, architecture)
    • Availability Chain - Files from sources like IPFS may or may not actually be available. An availability chain is a chain that tracks if specific files are available for indexing, acting as a source of truth. This is required to have consistent, deterministic indexing of those sources by different indexers.

Achievements

  • POI/ query cross-checking MVP demo

Active

  • Pipelining database interactions (in review)
  • GraphQL prefetch optimizations (in review/testing)
  • Performance testing setup
  • Ethereum firehose (with Streaming Fast)
  • Docs restructure
  • POI/query cross-checking (ongoing work)
  • graphman commands for managing block caches (fix blocks, report potentially affected subgraphs)
  • Fix for making subgraph memory allocations more robust
  • File data sources
    • Related – Availability Chain, Clock Chain
    • Immutable entities – ready for review

Next Steps

  • Make progress on availability chain/clock chain designs
  • Architecture discussions

Indexer Component Software Updates

  • Rate limiting options
  • Additional Health Endpoint
  • Fixed M1 Mac Builds
  • Management of off-chain subgraphs via CLI
  • Indexing rules by subgraph ID vs deployment ID
  • Working soon with Semiotic AI to introduce machine learning

Figment (7:40)

Achievements

  • Opened Tenderment PRs for graph-node, graph-cli and graph-ts
  • Fixed the block hash mismatch bug
  • Indexer CLI off-chain commands, indexer CLI rule instant pick-up
  • Three new engineers hired

Active

  • Troubleshooting Tendermint integration
  • Tendermint Testing (Firehose through graph-node/subgraph)
  • Working through OOM issues with Firehose
  • Updating documentation for Tendermint type generation
  • Dummy-Chain
  • Indexer Agent:
    • Filter unsupported subgraphs, add option to require supported subgraphs
    • Add max lifetime option in commands

Next Steps

  • Peer review of Tendermint PRs
  • Cosmos Hub integration
  • Collaboration with Streaming Fast on documentation

The Guild (9:58)

Risks, Problems, Help

· graph-node code reviews bottleneck

Achievements

  • Onboarded two new Guild members
  • GraphQL Validations Migration Guide
    • CLI tool
    • Bug Fixes
  • Awaiting Review
    • API versioning: Draft PR – waiting for initial review / versioning policy
    • GraphQL root __typename bug fix
    • Refactor static GraphQL
    • Filters: make comparisons case-insesitive

Active

  • · The Graph Client Architecture (with Yaniv)
  • · Relay-compliant GraphQL schema and improved pagination
  • · GraphQL Best Practices (website page)
  • · GraphQL- Codegen collaboration with Synthetix
    • o TS Query Builder – almost done
    • o AND/OR Filters

Next Steps

  • Update to latest GraphQL in graph-node
  • Introduction to the hosted service
  • The Graph Client sub-components
    • Client-side composition
    • Query Builder
    • Local/Offline query execution
  • Support GET requests

Streaming Fast (13:14)

  • Solana Work
    • Stability Testing
    • Implement fast boot mechanism
      • Near PR Ready
  • Firehose
    • Irreversibility index - done
      • Speed up linear processing
    • Starting accounts and events indexing work
  • Small bug fixes
  • NEAR protocol
  • Ethereum Testing

Firehose SDK (17:50)

The goal for the Firehose SDK is to empower development teams with the tools to kickstart their blockchain to be Firehose and The Graph ready. The SDK will be used to help new teams and team members visualize how the Firehose should be implemented for their chain. To achieve this, it utilizes a dummy chain.

The dummy chain is program that simulates a blockchain. It is non-network dependent and provides the basic information needed for testing. It will act as a lightweight end to end testing engine for development teams to simulate on-chain events without having to wait for the event to occur on mainnet.

New GraphQL API Features (25:00)

GraphQL Validations Have Been Merged

  • Needed to simplify implementation and avoid risky execution, reduces responses ambiguity
  • Almost 100% coverage
  • Migration CLI tool
    • @graphql-validate/cli is a simple CLI tool that will help validate GraphQL operations against a given schema

Upcoming Graph QL API Features

  • New Entities Filters
    • Filter by change block number
    • Filter and sort by child entity
    • AND/OR filters
    • Case-insensitive search
  • API Versioning
    • Every change in the GraphQL layer will result in a new version
  • Improved Schema & Pagination
    • Use connection types with cursors (improved pagination)
    • Use abstract “Node” interface to make it easier for GraphQL clients to implement caching

LimeChain Subgraph Debug Tool Demo (35:48)

LimeChainDemo-2

The subgraph debug tool saves developers time during the debug process. Sometimes, when a subgraph is deployed, it fails at specific point in the syncing process. Typically, a developer would need to attempt to fix the problem, redeploy and wait for syncing to complete to see if the problem was fixed.

The subgraph debug tool allows developers to run their mappings using the store from the failed subgraph. It will run locally on the block the subgraph failed, so that developers can receive feedback on their attempts to fix the problem quickly.

The tool also allows developers to easily fork and build off existing subgraphs which enables a larger use case for the tool.

A tutorial video mini-series is being developed for the debug tool and will be uploaded to The Graph’s YouTube channel and documentation for its use will be uploaded to The Graph Docs.

Graph-Node Testing Experience (44:45)

The Graph has added multiple core development teams since launch. All these teams have their own unique workflow for testing and developing features for Graph Node, which brings the need for a toolset for new developers working on The Graph. Discussions on how to improve the development experience for new teams are taking place behind the scenes.

GIP–023 Replaces GIP—0018 (50:00)

GIP–0018 Subgraph ownership transfer has been replaced by GIP–0023 due to the original GIP failing a quality audit. Technical revisions have been made to the original GIP and GIP–0023 will proceed to council voting in three days.

Stay Tuned!

Join us next month for The Graph’s Core Devs Meeting #11!

Keep up to date by joining discussions in the forum, following The Graph on Twitter or joining the Discord server.

For further discussion on the topics covered in The Graph’s Core Devs meetings, attend Indexer Office hours in the Graph Stage voice channel every Tuesday.

4 Likes