I’m all for simplifying, and removing inefficiencies.
I appreciate that units of work are more malleable than single-request-response-roundtrip, thinking of bytes and other units that could better model the business run by an indexer. It’s important we work in a direction where different business models can be expressed on the network. – So although this proposition is not perfect, it’s a step in the right direction.
(The perfect state would be to be able to get to consensus on work done behind the curtain, and to be able to bill for things that aren’t visible externally. That might prove to be an impossibility with certain target goals. The future will tell how important that is.)
This also aligns well with some of the streaming services, where having a gateway in the middle doesn’t make much sense, so being able to pay more point to point opens up those agreements to use the network.
I would actually push that forward so we learn ASAP, and to give Horizon better data to inform its final design.
From my talks with end users, latency is far far from the only concern. Simplicity of payments, easy to grasp purchasing experience, what sort of support and what sort of quality (defined in much broader terms than QoS or latency) you’ll get, they all count as much.
I think the business front is what was suffering the most for widespread adoption of the network usage, and I’m happy that this proposal addresses some of the real-world concerns from the perspective of those making purchasing decisions.
Thanks for the comment @abourget. I agree that it would be great to add flexibility to be able to express different business models. Indeed in a “World of Data Service” that is a necessity. We think indexing fees is the first step toward that, and certainly not the last step.
I think there has been a lot of very good arguments in support already. So I will just ask detractors to ponder these (and please post your conclusions):
How are indexing rewards earning you money / paying infra? Where is that money coming from?
What happens to network quality of service as the number of stale/broken/useless subgraphs with signal increases (as the dust left by developers that move on accumulates indefinitely)?
How do we avoid indexers getting rewards until the end of times on stale/broken/useless subgraphs?
What is the cost of indexing a subgraph and keeping it in storage in DB over time? What happens at the extremes, with the EIP-721 subgraph for ex.
What is the impact on quality of service and overall network operating cost as the number of indexers increases, but their size decreases? (really about where is the sweet spot)
Do we want to be competitive in the marketplace / provide an excellent service to consumers at a fair price?
Sorry if it comes off as a bit dry, but I wanted to keep it short.
I confess was initially a bit skeptical, but I’ve also come to support this proposal. I like that this proposal simplifies the interactions to get a subgraph indexed and it does seem more sustainable than the curation + rewards model.
I shared the concerns that some people expressed about the centralization risk of everyone choosing a single (or few) big Indexer(s), but I’ve come around because:
@justin gave a great talk at a core devs offsite recently, talking about factors that can cause a natural monopoly or oligopoly. What I understood is that it generally comes down to barriers to entry giving the big guys market power, and I believe there is a lot we can do to ensure this doesn’t happen in The Graph, by making it easy to set up efficient infrastructure as an Indexer. Eventually, if there ever came a need for a more active “antitrust mechanism”, we could research ways to implement them (though we’d need to be careful to not disincentivize efficiency and growth if we ever do this).
I think we can do a lot from the UI perspective so that the default way to interact with The Graph is not to choose an Indexer manually but let an algorithm (probably running on the gateway at first) choose for you. If someone wants to choose a specific Indexer, that can be an “advanced” feature, but the default algorithm could distribute load across all Indexers that are similarly efficient. I really believe most people would use the easy, simple default algorithm, which should ensure there’s enough indexing work for everyone that has a reasonable setup.
Another concern I had was how to get a subgraph indexed if there are a lot of people interested in querying it, but no dev team willing to pay the indexing fees. I think we can solve this with a good proposal for protocol rewards - hopefully there’ll be a Horizon proposal published soon and we can discuss the best way forward for that there. One approach we’ve been discussing informally, and that I like, is having a DAO that creates many of these indexing fee agreements for useful subgraphs, which would solve this and also help drive Indexer profitability until demand ramps up. This could be in parallel to protocol rewards for collateralized security on Horizon.
Thank you @Pablo . Your thoughts here as well as our numberous discussions forced me (in a good way) to think really hard about this problem.
I always really like your framing of “if there ever came a need…” I think this is the right way to approach the problem. Markets are typically considered the best option…until they’re not. Usually that can be attributed to some market failure. However, since the goal of the Graph is to remove as many instances of market failure (i.e. unlevel playing field) as possible, I like the idea of waiting to see which one we missed and then address that market failure directly.
Thank you @Alexis for your thoughts. I think your points hit on a lot of the current issues that curation doesn’t solve, and ones we didn’t express how indexing fees may solve them. And I’ll answer one question:
Do we want to be competitive in the marketplace / provide an excellent service to consumers at a fair price?
Reiterating for the umpteenth time, this proposal only makes sense within the scope of broader changes outlined in the mysterious Horizon proposal. The community cannot effectively assess or discuss this proposition without understanding the larger context. Moreover, it appears that all support for this proposal comes exclusively from Core Team members, who are likely privy to privileged information about the forthcoming Horizon proposal, maybe from informal discussions during the recent offsite.
Please share it with the community as soon as possible so that we can all participate in this excitement!
To close one of the loops @ellipfra , I can’t share the video but I did reach out to see if I can give the presentation during indexer office hours. I’ll do my best to make that happen.
EDIT: It looks like the talk will be at today’s indexer office hourse. Hope to see you there.
We are still re-evaluating this proposal as well as the mentioned “Horizon” in light of all the feedback, as well as the conversations at indexer office hourse. We recognize this proposal is a large change and want to avoid being hasty. We hope to offer a revised version soon but in the meantime continue to welcome thoughts or feedback in this forum.
The GIP has been updated to include a short but important amendment reflecting the feedback received here as well as during indexer office hours. Tagging the frequent commenters (only 10 tags allowed. Apologize for any omissions) for notification but welcome feedback from the entire community.
There will be 1/2 hour discussion of the amendment and Graph Horizon at the upcoming Indexer Office Hours on 12/5.
Information about Horizon was requested by some participants in this thread so I would like to direct your attention to one of the first structured communications we had with the indexer community about Horizon in Indexer Office Hours
