This is Kuan Huang from (www.scout.cool). I am proposing an open source software framework that enables developers to quickly build any data applications entirely on the Web3 infrastructure. Whether someone is building an analytics dashboard for a dApp/protocol or leveraging existing machine learning models to do data science work, this framework should save you lots of time.
Every blockchain data analytics company (including ourselves) that we know of is using centralized databases to index blockchain data before serving it to their users. As a result, the majority of data analytics charts or dashboards are living on centralized hosting services and databases. This approach is not only the opposite of where Web3 is heading but also creates a few real issues:
- Users are locked into a proprietary web platform due to the centralization of data indexes and hosting.
- The lack of transparency in how data is indexed and organized in a centralized database makes it very difficult to debug issues. We personally have run into numerous cases where the on-chain metrics we put together are different from other vendors.
- As faster L1 and L2 blockchains are becoming available, this approach is becoming the biggest bottleneck for every data company to scale.
The MVP of the proposed software framework should provide:
A generic module that efficiently pulls data from any subgraphs or other on-chain/off-chain data sources. This module should take care of some common challenges such as pulling from multiple subgraphs in parallel, handling “pagination” in subgraph in the background and etc. (One of the grantees from first wave, Keyko, might be working on some parts of what I am describing)
A data transformation module that does the preparation before the visualizations. There are actually some existing package can be reused such as Pandas, Tensorflow. This also opens a door for any potential machine learning applications that leverage The Graph.
Pre-built widgets (with the options to customize) to render charts and dashboards. Developer should be able to use very few lines of code to render a chart or design a dashboard without touching any front end code.
A simple mechanism to deploy and sharing through decentralized hosting/storage services. The entire community can discover and learn.
An easy to maintain structure since frequent updates of the applications are expected.
We are beginning to see what the Web3 infrastructure might look like in the near future:
- Decentralized query layers such as The Graph (700+ subgraphs and growing!)
- Decentralized storage and hosting services such as IPFS, textile.io, Fleek.co
- Decentralized code repository such as Radicle
- Decentralized database/caching layer: threadDb, Gun DB
Some inspirations: (Sorry, I am not allowed to post more than two links as a new user in this forum)