Semiotic Labs June 2023 Update

Research & Development Core Team Updates
1

:woman_astronaut: Summary

In May, the team made significant progress on various projects. The core of the new query payment system, Timeline Aggregation Protocol (TAP), was further developed with the introduction of the Verification smart contract. Substreams were successfully sunk to PostgreSQL, with a focus on Ethereum Uniswap v3 data, facilitating a more accessible format for analytics use cases and advancing the World of Data Services discussions. The team also made important improvements to AutoAgora, a tool vital for indexers when selecting prices, by introducing numerous new tests. Additionally, we drafted a problem statement and requirements document for the Verifiable Firehose project, aiming to build solutions on top of existing infrastructure. Outreach activities included workshops and presentations at various forums by our heads of AI and Research. Looking ahead, the team plans to demonstrate TAP end-to-end, implement the Collateralization smart contracts, document and open source the Uniswap v3 substream, create a substream to index all Ethereum ERC-20 events, and engage with Core Devs on the Verifiable Firehose project.

:tada: Looking back (what was delivered)

Verifiable Payments

In April, we open-sourced the core of our new query payment system: TAP. In May, we finished the implementation of the TAP Manager and Gateway Receipt Aggregator - the latter being a daemon responsible for the aggregation of receipts. In both, we leverage the EIP-712 standard for serialization/checking of receipts and receipts aggregates. We also implemented the Verification smart contract which is able to verify the EIP-712 aggregated receipt created by the TAP Manager.

Development Activity

PostgreSQL from substreams

We are exploring sinking substreams to PostgreSQL (see this StreamingFast repo as a reference). The intention is to both “dogfood” the substreams product as well as bootstrap the creation of a new analytics market in The Graph. So far, we have written a substream to record the events of Ethereum Uniswap v3 (and of its clones, e.g. SushiSwap) to a PostgreSQL database. It took approximately 27 hours to index that data, and now we are up to date with the Ethereum chainhead. This is significant because we now have Uniswap v3 data that is in a format more convenient for analytics use cases. We will use the insights gained during this exercise to help drive the World of Data Services discussions.

Development Activity

AutoAgora improvements

Indexers use AutoAgora for selecting prices, so it is critical that the tool is behaving as expected. Along those lines, we have added many new tests.

Development Activity
  • AutoAgora pull requests merged:
    • PR #62: Added version to the top of the aa model
    • PR #63: build(Docker): Build whl w/ pip instead of poetry
    • PR #64: build(deps): bump pymdown-extensions from 9.10 to 10.0
    • PR #65: Added unit tests for logs_db
    • PR #66: Removed unnecessary methods
    • PR #67: Added tests for subgraph wrapper
    • PR #68: build(deps): bump requests from 2.28.2 to 2.31.0
    • PR #70: Unit tests for price multiplier
    • PR #71: Added tests for config file
    • PR #72: Added tests for misc file
    • PR #75: Moved init config to main function

Verifiable Firehose

Finished internal draft of the Verifiable Firehose problem statement and requirements document. The goal for verifiability, in general, is to build solutions on top of existing infrastructure when possible. Specifically, the solution selected for verifying Firehose should allow us to build on it to achieve Verifiable Queries for our various data services (e.g. GraphQL, gRPC, KVS, SQL).

Outreach

  • Tomasz Kornuta, our Head of AI, led an in-person workshop at the E&N House of Web3 on training neural networks for image recognition and text classification.
  • Sam Green, our Head of Research, presented at the virtual Coinbase ML and Blockchain Research Summit, where he discussed early experimental results using GPT-3.5 and GPT-4 to write GraphQL queries. The recording is here.

:rocket: Looking ahead (upcoming priorities)

Verifiable Payments

  • Demonstrate TAP end-to-end
  • Implement Collateralization smart contracts
  • Release TAP GIP

PostgreSQL from substreams

  • Document and open source our Uniswap v3 substream
  • Create a substream to index all Ethereum ERC-20 events

Verifiable Firehose

  • Engage with Core Devs on the Verifiable Firehose problem statement and requirements document.
4 Likes
2

How can I get more information on " Verifiable Firehose"?

1 Like