Semiotic Labs November 2023 Update

Research & Development Core Team Updates
1

:woman_astronaut: Summary

In October, TAP Scalar/payments saw progress, including successful integrations, smart contract audits, and deployments across Ethereum and Arbitrum test networks, as well as implementing an Escrow subgraph and a testing plan. In data verifiability and correctness, we have met with GraphOps to discuss the deployment of Verifiable Firehose; we have also solidified plans for using Verifiable Firehose as a solution for EIP-4444. We continue to progress in using substreams for analytics/SQL applications through close collaboration with StreamingFast. Semiotic has also hired a Head of IT and a Product Manager.

:tada: Looking back (what was delivered)

TAP Scalar/payments

The new Rust indexer-service is mostly done and has undergone basic integration testing with the rest of the decentralized network stack. It can currently handle basic checking of Scalar TAP receipts and save them to the Indexer’s database.

The rest of the checks will be handled by a new service in the Indexer’s stack: the TAP Agent (tentative name). It will also handle the receipt aggregate requests, which should happen continuously to avoid accumulating excessive unaggregated receipt value (not redeemable on-chain). As such, Indexers will always, for each (gateway, allocation) relationship, keep one receipt aggregate voucher (RAV) that will contain most of the receipt value and is always redeemable on-chain at any time.

The first TAP Agent PR is still in review, but when the related database schemas are agreed upon, the indexer-agent will be modified to handle redeeming RAVs on-chain. It will also enable the start of integration testing that includes the full receipt checks and the automated RAV request logic.

TAP PRs and Documentation (collaboration with GraphOps and E&N):

  • TAP smart contracts audits online (yay! link)
  • TAP smart contracts deployed to test chains:
  • Escrow subgraph implemented and deployed (GitHub)
  • End-to-end test plan formulated and being implemented (docs)
  • Updated schedule (details)

Verifiability/data correctness

This past month we met with other core devs and shared our latest updates on Verifiable Firehose, our optimistically verifiable commitment protocol, and the proof of event log protocol. Our presentation and read-ahead materials are available here. We also made our PoC code for each of these topics publicly available, here: Verifiable Firehose, PoEL, OVC. During the retreat, we agreed on an immediate next step to prepare a blog post and presentation for Devconnect detailing how Verifiable Firehose can be used as a solution for EIP-4444. Additionally, we are developing a proposal for a new data service that will enable The Graph to be used as a decentralized light client for Ethereum.

Also, we met with GraphOps to discuss using OVC to enable a Verifiable Firehose file-sharing data service.

SQL data service

Semiotic is collaborating with StreamingFast to design a SQL deployable unit manifest spec to instruct Indexers on synchronization and service specifics for SQL. The manifest will encompass hardware and software requirements, along with a substream .spkg which is versatile and will define the contracts/events to be filtered and which transformations to use. Following this, Indexers can access curated SQL manifests at substreams.dev, allowing them to synchronize contract events with their ClickHouse DB. With StreamingFast, a standard API for query submissions is being prototyped, aiming to integrate features similar to existing APIs for querying blockchain data. Once deployed to substreams.dev, developers can begin experimenting with building applications like Dune or Flipside directly on The Graph.

:rocket: Looking ahead (upcoming priorities)

TAP Scalar/payments

  • Development: The team aims to finish the development of the indexer-tap-agent and integrate TAP RAV fees redeem in the indexer-agent.
  • Smart Contracts: We plan to deploy and test TAP smart contracts on the Arbitrum Sepolia network.
  • End-to-end Testing: A “naive” end-to-end testing of all the components (smart contracts, Escrow subgraphs, Indexer with TAP, Gateway with TAP) is scheduled for the end of November.

SQL data service

Since the beginning of November, we have prototyped a REST API endpoint for serving SQL queries. During the remainder of the month, we will collaborate with StreamingFast to test this endpoint and make API calls from their ClickHouse deployment. We have also written a substream for UniswapX. We will finish testing this substream and open-source it, making it available at substreams.dev.

Verifiability/data correctness

Next month, we will productize our flat file verification code and make it available to other Indexers running Firehose; we will also publish the related EIP-4444 blog post and present it at Datapalooza (at Devconnect). We will also finish the proposal for a light client data service and begin design and prototyping.

Other notes

  • Hired a Head of IT
  • Hired a Product Manager
7 Likes