Summary
The Graph’s Core Devs Call #21 featured updates on Billing 2.0, Curation, and Layer 2 Scaling. Developers plan to offer a more flexible and predictable billing experience, implement Direct Indexer Payments, and adopt Layer 2 Arbitrum. A developer shared their positive experience with Substreams for Ethereum, resulting in an 8x improvement in syncing speed.
Topics Include:
- Billing 2.0
- Layer 2 Scaling Updates
- Developer experience with Substreams
Billing 2.0 (05:03)
BK, a product manager at Edge & Node, discussed the current billing experience for subgraph developers and consumers at The Graph, which leaves some room for improvement. The current billing system, Billing 1.0, is flexible and pay-as-you-go but has some friction and trust issues.
Edge & Node is working on a new billing system, Billing 2.0, that will offer the flexibility of pay-as-you-go alongside new monthly subscription plans. This new system aims to help users lock in a set number of queries at a fixed price, making it easier to predict costs. The system will also streamline the payment process and offer various payment options, such as GRT, USDC, and Fiat.
The goals for Billing 2.0 include:
- Flexible pay-as-you-go and monthly subscription options
- Built-in cost and query estimations
- Streamlined payment flow
- Automated and recurring payments
The roadmap for implementing Billing 2.0 includes several milestones:
- Introduction of Fiat payments
- Setting up subscriptions with an MVP for beta testers
- Managed subgraph payment plans (developer bundles)
- Auto recurring payments
Ultimately, the team aims to provide a “set it and forget it” billing experience for users, making the billing process more predictable and convenient.
Curation
Curation v1.0 was The Graph’s initial mechanism, and although it had some improvements in v1.x, it was not meant to be a long-term solution. The proposal called Direct Indexer Payments (DIP) aims to address the core issues of curation by allowing developers and consumers to directly pay Indexers for their work without converting payments to signal on a bonding curve. This would make the process more streamlined and easier for all parties involved.
However, before DIP can be implemented, indexing rewards need to be changed, as they are currently based on the amount of signal on subgraphs. This would require a Graph Improvement Proposal (GIP) on indexing rewards to be released, followed by a consensus among the community members, especially Indexers. Once consensus is reached, the DIP curation mechanism can be designed and implemented.
In the meantime, managed subgraph payment plans are being considered to mitigate some of the current pitfalls of Curation 1.0 and 1.x by abstracting away complexities from users who do not wish to engage with them. Overall, the goal is to improve the curation system for all parties involved in The Graph network, making it more efficient and user-friendly.
Question: As more blockchains are added to the decentralized network, will the billing system accept payments from those blockchains, either native token, or like USDC, which is on those chains, or always need to pay from Ethereum?
Answer: The billing system is currently on Arbitrum and can accept any ERC 20 token, but it would require a different contract for each of those tokens. The plan is to expand and accept GRT and other tokens in the future.
Question: With the back end of the coordination between consumer to Indexers, will there be a Curator stakeholder role?
Answer: In the current design, the role of Indexers and The Graph developer is prioritized, but there might still be a role for Curators. More research is needed to determine how the curation role can be more effective in attracting and compensating Indexers.
Question: When users pay in Fiat or USDT, is the currency converted to GRT before being used in the network, or is the protocol being changed to accept fees in other currencies?
Answer: It depends on the billing system. Users will have the option to pay in either GRT or Fiat. If they choose Fiat, they will jump out to a Fiat payment system and acquire GRT there, which will then go directly to their billing balance. The system is designed to accept any ERC 20 token, but the plan is to expand and accept GRT and other tokens in the future.
Question: Could there be an API access to monitor billing API key usage for a particular account?
Answer: Yes, there is a subgraph for that, and there are dashboard pages available to monitor billing API key usage.
Layer 2 Scaling Updates (30:30)
The Graph is working on scaling its network to Layer 2 Arbitrum to improve efficiency and performance. The process involves migrating the protocol contracts to Arbitrum, allowing participants to interact with the smart contracts on this platform instead of Ethereum. Currently, the contracts run on both Ethereum mainnet and Arbitrum, with the latter launched as part of stage one for the bridge. This stage allows the transfer of GRT tokens between Ethereum mainnet and Arbitrum.
Stage two of the scaling process involves enabling indexing rewards on the Arbitrum network and gradually shifting the GRT token issuance from 100% on Ethereum mainnet to 100% on Arbitrum. This stage is being approached cautiously to ensure everything works smoothly. Stage three will provide migration helpers to facilitate the easy transfer of states and GRT tokens between the mainnet and Arbitrum contracts for subgraph developers, Curators, Indexers, and Delegators.
The Graph has conducted mission in the MIPS program to test the Layer 2 contracts on Arbitrum, with around 300 participants performing various actions such as staking, delegating, and allocating to subgraphs. The MIPs helped identify bugs and gather feedback on the user interface and other aspects of the system, ultimately validating the functionality of the protocol and its surrounding components.
Recently, the stage two contracts were deployed to the testnet, allowing Indexers to receive indexing rewards. If testing continues successfully, the stage two indexing rewards will be deployed to Arbitrum One next week. The migration helper contracts for stage three are currently undergoing an audit, and if all goes well, the subgraph and curation migration helpers will be launched first, followed by the stake and delegation migration helpers a few weeks later.
As the migration process progresses, the Graph’s product team is working on UI development for the migration helpers and other improvements to ensure a smooth transition. The Graph aims to move forward with complete adoption of Layer 2 Arbitrum, with significant developments expected in the coming weeks.
Question: Is it still expected that owners with fully vested contracts will be able to use normal migration helpers, and is it important to have an assurance on this before most indexers start setting up?
Answer: Yes, that is the plan. People with fully vested vesting contracts should be able to use the normal migration process. However, approval from the entities that issued those vesting contracts is needed, and discussions are ongoing to ensure everyone is on board and that there are no unexpected issues.
Question: If I want to set up a new Indexer today on L2 with 100k GRT, will I be able to use a helper to move my existing operation on Mainnet to the address that I’m already running with different parameters on the new L2, and then eventually deal with ENS and other issues later down the line?
Answer: If you have a vesting contract that is not fully vested, then no, you will need to migrate your vesting contract first to get your L2 address for operation. If you’re using a fully vested vesting contract or just a regular multi-sig or normal address, then yes, you can start a new indexing operation on L2 and, when migrating the state from L1, specify the L2 beneficiary to be the address you’re already using on L2.
Developer Experience with Substreams (44:00)
Robin, from Messari, shared their experience as a developer working with The Graph network, specifically using Substreams for Ethereum. They mention that their subgraph, powered by Substreams, took about a week to sync, which is significantly faster than the two months it took for the equivalent subgraph without Substreams. This resulted in an eight times improvement in syncing speed.
The developer also highlights that the subgraph they created is almost identical to the original, with only a few limitations regarding the replication of certain data objects. They successfully tracked network entities and snapshots for days and hours using The Graph.
In terms of the development experience, Robin found it to be straightforward. They were able to create neat patterns and maintain a clean structure throughout the project, making it easier to track execution orders and separate file paths.
Some challenges they faced included ensuring uniqueness, tracking statistics over specific timeframes, and debugging when deploying the subgraph. However, they believe that the development of tools for automatically generating unit tests will help mitigate these issues in the future.
Overall, the developer had a positive experience working with The Graph and Substreams, and they are optimistic about improvements to the platform moving forward.
Question: Any performance numbers to share?
Answer: The speaker mentioned an 8x improvement in performance, reducing sync time from two months to a week.
Question: Are there any plans to share documentation on how to optimize the development experience?
Answer: Yes, the speaker mentioned having written some documents and patterns for working with Substreams and will share them in the working API group.
Question: Was there any functionality that couldn’t be implemented with Substreams?
Answer: The main challenge was tracking uniqueness and storing large values, but most other functionalities were achievable with Substreams.
Question: Are Substreams configured to directly write into the Postgres sync, or is there still a mappings file?
Answer: The output from the last module is returned as a set of entity changes, which get mapped into Postgres updates on The Graph side.
Stay Tuned!
Join us next month for Core Devs Call #20!
Keep up to date by joining discussions in the forum, following The Graph on Twitter or joining the Discord server.
The 19th Core Devs call is happening this Thursday 2023-03-23T15:00:00Z (2023-03-23T00:00:00Z).
Please subscribe to The Graph Foundation’s Ecosystem Calendar so you don’t miss any of the upcoming calls!
Tentative agenda:
The recording will be uploaded here.
The Graph Core R&D Workspace
Monthly Core Dev updates [NEW! 
]