Summary
During Core Devs Call #22 on July 13th, 2023, the following topics were covered:
Timeline Aggregation Protocol (Scalar TAP):
- Introduction of Scalar TAP, a unidirectional payment state-channel for query payments.
- Overview of TAP receipts and Receipt Aggregate Vouchers (RAVs).
- Gas-efficient on-chain fee collection and computational cost-effectiveness.
Geo Updates:
- Expansion of Geo into various areas, such as software, education, cryptography, psychology, and news.
- Introduction of new features, including improved search functionality and tamper-proof page history.
- Plans for social features, permissionless onboarding, and reputation-weighted voting.
Substreams.js:
- Introduction of a new JavaScript implementation for Substreams, a functional replica of the Substreams Go package.
- Details on the JS library’s features, including data consumption, interaction with Substreams packages, and debugging tools.
- Plans to build a Substreams Registry and validation of byte-to-byte identical behavior with the Go implementation.
ERC 20 Substreams Modules:
- Utilizing JavaScript libraries for manipulating the substring stack.
- Creation of a generic ERC 20 token that streams event, token transfer, and balance changes.
- Details on the tool’s functionality, including querying, filtering, and storing data.
- Plans for future features like web hook connections and WebSocket streaming service.
Detailed notes are included below!
The Timeline Aggregation Protocol (04:47)
Scalar TAP extends the existing Scalar payment system used for tracking and settling query payments within a specified allocation period. The proposed protocol, known as the Timeline Aggregation Protocol (TAP), is designed to safeguard against potentially fraudulent activities.
The Timeline Aggregation Protocol (Scalar TAP) is a unidirectional payment state-channel that provides a trustless, fast, and efficient means for query payments from untrusted gateways. It operates on the principle of ‘micro-trust’, with the stipulation that the worst-case loss for the Indexer is capped at a set amount. The protocol relies solely on elliptic curve cryptography and is periodically interactive. Most of the time, the Indexer simply passively stores TAP receipts. It also offers gas-efficient on-chain fee collection, only requiring a single signed receipt check per allocation, and is computationally cost-effective for both the Indexer and the gateway. Payments operate unidirectionally, flowing from a payer, who holds collateral, to a receiver, who can claim part of this collateral when submitting a signed aggregate.
TAP receipts serve as evidence of the value the gateway owes to the Indexer. These receipts are designed to be unique and non-duplicable. Each receipt includes an Allocation ID (a unique number for each allocation that is stored immutably on-chain and remains constant for all receipts within that allocation), a value (the payment for the query being served), a timestamp (the Unix timestamp of when the gateway generated the receipt), a random nonce (a randomly selected number concatenated with the timestamp to create a unique ID for a receipt within a batch), and a signature (an ECDSA signature of all other items in the receipt, recoverable).
The Receipt Aggregate Voucher (RAV) includes an ECDSA signed tuple containing the Allocation ID, the aggregate sum from all receipts in the batch plus the aggregate from a previous RAV if provided, and a timestamp.
An Indexer can request a new RAV at any point by submitting their most recent RAV (optional) and a batch of valid receipts. The aggregate amount on the submitted RAV is added to the new total, and its timestamp indicates which receipts are invalid. The batch of valid receipts includes all receipts with a timestamp greater than the timestamp of the first batch and less than a specified target timestamp.
Geo Updates (20:14)
An update on Geo, a web3 browser and knowledge graph app was shared, outlining its expansion into several areas such as software, education, cryptography, psychology, and news. Software developers now can distinctly categorize their knowledge, while comprehensive guides for various vocational fields like healthcare are provided. The all-encompassing content database across The Graph is now searchable, making it easier for users to find what they need.
In the psychology space, studies are documented and debates on the studies’ validity are encouraged, setting a solid foundation for policy discussions. A similar approach is being taken in the news space to combat fake news through community tagging of misleading articles. New features were introduced, such as “find or create,” which allows for quick relation creation, and the ability to view any page’s history in a tamper-proof, blockchain-anchored format. Participation is being fostered through the Geo Explorers program, which includes weekly meetings and monthly rewards for top editors.
Soon, social features for a more permissionless onboarding process will be added, including the ability to upvote and downvote content. Currently, spaces are being manually created and editors are being added. However, the implementation of governance rules is planned to ensure that each space can self-define its rules, and that changes require editor approval through voting. Reputation-weighted voting is being considered as an alternative to the one person, one vote system.
Substreams JS (33:54)
A new JavaScript (JS) implementation for Substreams, which is a functional replica of the Substreams Go package from Streaming Fast, has been introduced. The new implementation isn’t intended to copy the Command-Line Interface (CLI), but it covers all its functionality. This JS implementation is a TypeScript-based, ESM-based library that can be used in all JS environments.
The JS library allows data consumption from Substreams in all aforementioned environments and supports interaction with Substreams packages and manifests. Furthermore, it allows the submission of these packages to the Substreams gRPC service for data retrieval. There are examples in the repository for different environments, with API tokens needed for interaction. Moreover, the repo enables running the Uniswap Substream in Node.js and provides native TypeScript or JavaScript debugging tools. An example of its browser functionality can be seen here, where you can explore modules included in the .spkg file.
There are plans to build a Substreams Registry for exploring and downloading existing high-quality Substreams. This implementation is not designed to be used as a binary, rather it’s more of a library for bespoke implementations for reading specific Substreams. It is validated to have byte-to-byte identical IO behavior with the Go implementation, ensuring an identical output. It supports parsing and validating Substrate manifests, packaging manifests into .spkg files, loading existing packages, and extracting protobuf types from existing manifests, among other features.
ERC 20 Substream Modules (46:32)
Utilizing JavaScript libraries allows for the manipulation of the substring stack, enabling the creation of diverse applications. At a recent hackathon, a generic ERC 20 token was built that streamed all event, token transfer, and balance changes without needing to connect with the RPC node, utilizing state storage instead.
The tool is not yet fully synchronized, but even in its current state, it can query any transfer of any address of any account as long as Postgres keeps up with data storage. It can filter results, for instance, by address, and it can seek transfers from a specific token contract. This tool indexes every single token that matches the queried pattern.
Balance changes can also be monitored. Whenever a balance changes, the states of the balance are stored in the block, negating the need to fetch any RPC node to make a get call. Balances and state changes can be stored, fetched, and extracted easily.
A sample ERC 20 token Rust code, along with a detailed Graph QL schema and subgraph yaml is provided as an example in a repository called the Substreams Cookbook.
The code illustrates how a MapTransfer function iterates over all the block logs, matching the transfer event. The API is decoded and the event is pushed as a map module. Similarly, a MapBalance function can iterate over all the calls to map balance changes, filtering by specific methods and selectors. Once a balance change is identified, the function can store and iterate over all the storage changes and save the balances. This makes it relatively straightforward to select the right data and visualize it in real time.
There are future plans to implement features like web hook connections, and WebSocket streaming service to provide real-time feedback whenever a transaction is made. These tools are currently in the early beta/prototype stage but will soon be available to developers.
Stay Tuned!
Join us next month for Core Devs Call #22!
Keep up to date by joining discussions in the forum, following The Graph on Twitter or joining the Discord server.